System

Abstract

A system is provided.SOLUTION: A system includes a means for receiving a request from a user, a means for presenting a generated action schedule to the user, a generation and AI means for generating the action schedule based on the request of the user, and a means for making a reservation based on the generated action schedule.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many people want to make the most of their holidays and spare time, but the hassle of thinking about what to do ultimately leads to a waste of time. To overcome this situation, there is a need for a system that automatically proposes and executes action plans that allow users to spend their time in a fulfilling way. In particular, a system that allows users to easily obtain an action plan that matches their mood and desires for that day is needed. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. Specifically, the system includes a means for receiving requests from a user, a means for presenting a generated action plan to the user, a generation AI means for generating an action plan based on the user's requests, and a means for making a reservation based on the generated action plan. This allows the user to obtain an optimal action plan with simple input, and allows the user to spend a fulfilling time simply by following the plan. Furthermore, by further including a user interface means for inputting the date, time, location, and requests, the system can propose an action plan that meets the user's specific needs. Furthermore, since the system is equipped with a communication means for transmitting and receiving data between the server and the terminal, the user can receive the action plan and reservation information in real time.

[0006] A "user" is a person who uses this system to create an activity plan for holidays and spare time.

[0007] A "request" is specific information about the user's mood and desired activities for the day that the user inputs into the system.

[0008] "Receiving means" refers to a function or device for receiving requests and other data from a user as input.

[0009] A "generative AI means" is an artificial intelligence model that automatically generates an optimal action plan based on received user requests.

[0010] An "action plan" is a set of activities or schedule that the user should perform on that day, output by the generative AI means.

[0011] "Presentation means" refers to a function or device for visually or otherwise informing the user of the generated action plan.

[0012] The "reservation means" refers to a function or device for automatically reserving necessary facilities and events based on the generated action plan.

[0013] The term "system" refers to the entirety consisting of a series of processes and functions including the above-mentioned receiving means, generating AI means, presentation means, and reservation means.

[0014] The "user interface means" refers to an interface through which the user inputs information such as date, time, location, and requests.

[0015] "Communication means" refers to the network and communication technology used to send and receive data between the server and the terminal. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that proposes action plans to users so that they can make effective use of their holidays and spare time. This system is composed of a user terminal and a server, and is characterized by generating optimal action plans using a generative AI model.

[0038] System configuration

[0039] User terminal

[0040] The user terminal refers to a mobile device such as a smartphone or tablet, and allows users to input date, time, location, and requests through a user interface. The input information is then transmitted to the server.

[0041] server

[0042] The server receives information from the user and generates an action plan based on the generative AI model. The generated action plan is sent from the server to the user's device and presented to the user. The system also has a function to make reservations for facilities and events using reservation methods as needed.

[0043] Program processing

[0044] 1. User request input

[0045] Users input a date, time, location, and request through a user interface, such as "I want to do something summer-like in Tokyo on October 10, 2023."

[0046] 2. Sending data to the server

[0047] The device sends the entered information to the server as an HTTP POST request, with the data encoded in JSON format.

[0048] 3. Generating action plans using generative AI

[0049] The server receives the request and passes the data to a generative AI model, which then generates an optimal action plan based on the user's requests. For example, it generates a specific schedule such as "visit Meiji Shrine at 9:00, have lunch in Omotesando at 12:00, and swim at the Sunshine City Pool in Toshima Ward at 15:00."

[0050] 4. Submit your proposal

[0051] The server sends the generated action plan to the user's device, where it is again encoded in JSON format.

[0052] 5. Viewing and booking your action plan

[0053] The device displays the received action plan on the user interface. The user checks the displayed action plan and requests a reservation if necessary. For example, when reserving a restaurant for lunch, the device sends a reservation request to the server again, and the server calls the reservation API to complete the reservation.

[0054] 6. User Behavior

[0055] The user follows the proposed action plan and spends their time fulfillingly. This system is extremely convenient because it automatically provides the optimal plan according to the user's requests, eliminating the need for the user to think about it.

[0056] Specific examples

[0057] Example 1: If User A inputs a request such as "I want to eat something refreshing and visit a power spot in Tokyo on October 10, 2023," the generation AI will make the following suggestions.

[0058] 09:00 Visit Meiji Shrine

[0059] 12:00 Refreshing lunch at a cafe in Omotesando

[0060] 15:00 Hiking at Mount Takao

[0061] 18:00 Sightseeing at Tokyo Tower

[0062] Example 2: If User B inputs a request such as "I want to enjoy the snow and soak in a hot spring in Sapporo on December 25, 2023," the generation AI will make the following suggestions.

[0063] 10:00 Visit the Snow Festival at Odori Park

[0064] 13:00 Lunch at a restaurant in Susukino

[0065] 16:00 Relax at Jozankei Onsen

[0066] In this way, the present invention proposes an optimal plan of action based on the user's specific needs and eliminates the need for reservations, allowing the user to spend more quality time.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] Users launch the application and input their date, time, location, and desires through the user interface, such as "I want to do something summer-like in Tokyo on October 10, 2023, and I want to go to a power spot."

[0070] Step 2:

[0071] The device collects the information entered by the user and encodes it into JSON format, which is then sent to the server as an HTTP POST request.

[0072] Step 3:

[0073] The server receives the HTTP POST request and parses the JSON data, extracting the date, time, location, and request.

[0074] Step 4:

[0075] The server passes the extracted date, time, location, and request data as input to the generative AI model, which then generates an action plan that matches the user's request.

[0076] Step 5:

[0077] The generative AI model outputs an action plan and returns the results to the server. For example, it generates a specific schedule such as "visit Meiji Shrine at 9:00, have lunch in Omotesando at 12:00, and swim at Sunshine City Pool in Toshima Ward at 15:00."

[0078] Step 6:

[0079] The server encodes the generated action plan again into JSON format and sends it to the original device as an HTTP response.

[0080] Step 7:

[0081] The device parses the JSON data received from the server and displays the action plan in the user interface, including detailed information about each activity.

[0082] Step 8:

[0083] The user reviews the proposed action plan and requests reservations for specific activities if necessary. For example, if the user wants to make a reservation for lunch, the user presses the reservation button.

[0084] Step 9:

[0085] The terminal sends the reservation request to the server again as an HTTP POST request.

[0086] Step 10:

[0087] The server receives the reservation request, calls the necessary API (e.g., restaurant reservation API) to process the reservation, and notifies the device of the reservation information after it is completed.

[0088] Step 11:

[0089] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[0090] Step 12:

[0091] The user follows the proposed plan of action and spends their time in a fulfilling manner. The device supports the user's actions by providing navigation and map links as needed.

[0092] Example 1

[0093] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0094] Conventional action plan generation systems require a lot of time and effort to process user-entered information, generate plans, and manage reservations, making it difficult to efficiently create action plans. It is also difficult to automatically provide customized plans that reflect the user's preferences and requests in detail. This has resulted in problems such as users being unable to create satisfactory plans and a lack of convenience.

[0095] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0096] In this invention, the server includes means for receiving requests from a user, input means for inputting the user's requests as date, time, location, and request information, means for transmitting the input data to the server, means for generating an action plan using a generative AI model, means for presenting the generated action plan to the user, means for making a reservation based on the generated action plan, and communication means for transmitting and receiving data between the server and a terminal. This makes it possible to efficiently and quickly provide an optimal action plan based on the information input by the user, and to automatically make a reservation based on that plan.

[0097] The "means for receiving requests from a user" refers to a device or software that collects and records the date, time, location, and individual request information provided by the user.

[0098] The "input means for inputting the user's requests as date, time, location, and request information" refers to a device or software that provides an interface that allows the user to specifically input his or her own action plans and preferences.

[0099] The "means for transmitting input data to a server" refers to a device or software that has a mechanism for transmitting information input from a user terminal to a server via communication.

[0100] A "means for generating an action plan using a generative AI model" is a device or software that uses an artificial intelligence algorithm to automatically create an optimal action plan based on data provided by a user.

[0101] The "means for presenting the generated action plan to the user" refers to a device or software that provides an interface that visualizes the generated action plan and presents it to the user, allowing the user to select the next step.

[0102] The "means for making reservations based on the generated action plan" is a device or software that automatically processes the reservation process of required resources and services based on the generated plan.

[0103] "Communication means for transmitting and receiving data between a server and a terminal" refers to a device or software that provides a communication protocol or network interface that enables data to be transferred between a server and a user terminal.

[0104] This invention is a system that proposes action plans to users so that they can make effective use of their holidays and spare time. This system is composed of a user terminal and a server, and is characterized by generating optimal action plans using a generative AI model.

[0105] User terminal

[0106] A user device refers to a mobile device such as a smartphone or tablet, and can input date, time, location, and requests through a user interface. A user uses an input form within the app to input their desired date, time, location, and specific requests. For example, if they input "I want to do something summery in Tokyo on October 10, 2023," the information is processed on the device and sent to the server.

[0107] server

[0108] The server receives the data from the device and generates an action plan based on the generative AI model. The generative AI model used automatically creates an optimal action plan based on the user's requests, taking into account past data and general information. The generated action plan is sent from the server to the user's device and presented to the user through a user interface.

[0109] Hardware and Software Use

[0110] The system uses the following hardware and software:

[0111] User devices: smartphones, tablets

[0112] Input method: Touch screen, keyboard

[0113] Communication method: Wi-Fi, 4G / 5G

[0114] Server: Cloud server, web server

[0115] Generative AI models: machine learning algorithms, natural language processing models

[0116] Data format: JSON

[0117] Specific examples

[0118] Specific examples are shown below.

[0119] Example 1: If User A inputs a request such as "I want to eat something refreshing and visit a power spot in Tokyo on October 10, 2023," the system will generate the following action plan:

[0120] 09:00 Visit Meiji Shrine

[0121] 12:00 Refreshing lunch at a cafe in Omotesando

[0122] 15:00 Hiking at Mount Takao

[0123] 18:00 Sightseeing at Tokyo Tower

[0124] Example 2: If User B inputs a request such as "I want to enjoy the snow and soak in hot springs in Sapporo on December 25, 2023," the system generates the following action plan:

[0125] 10:00 Visit the Snow Festival at Odori Park

[0126] 13:00 Lunch at a restaurant in Susukino

[0127] 16:00 Relax at Jozankei Onsen

[0128] Prompt Sentence Examples

[0129] "I want to eat something refreshing and visit a power spot in Tokyo on October 10, 2023."

[0130] In this way, the system of the present invention can efficiently and accurately generate an action plan based on the information entered by the user, and even automatically make reservations. By using this system, users can spend their time in a fulfilling manner.

[0131] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0132] Step 1:

[0133] User request input

[0134] Users launch the app on their smartphone or tablet and input the date, time, location, and requests through the user interface. For example, if a user inputs a request such as "I want to do something summer-like in Tokyo on October 10, 2023," that information is collected by the app and sent to the next step.

[0135] Input: User's date, time, location, and request information

[0136] Output: The input data (e.g., "I want to do something summery in Tokyo on October 10, 2023")

[0137] Step 2:

[0138] Sending data to the server

[0139] The device encodes the information entered by the user into JSON format (e.g., "{"date":"2023-10-10","location":"Tokyo","request":"I want to do something summery"}"). Next, it sends this encoded data to the server using an HTTP POST request. This is done over a communication network (Wi-Fi, 4G / 5G).

[0140] Input: Data entered by the user

[0141] Output: JSON data sent to the server

[0142] Step 3:

[0143] Generative AI generates action plans

[0144] The server decodes the JSON data received from the device. This decoded data is prepared as an input prompt for the generative AI model. An example prompt might be, "Please propose an action plan for doing summer-like things in Tokyo on October 10, 2023." The generative AI model generates an action plan based on this prompt. The generated plan includes specific time and activity suggestions.

[0145] Input: JSON data received by the server

[0146] Output: Generated action plan (e.g., "Visit Meiji Shrine at 09:00, have lunch at Omotesando at 12:00, and swim at Sunshine City Pool in Toshima Ward at 15:00")

[0147] Step 4:

[0148] Submit your proposal

[0149] The server re-encodes the generated action plan into JSON format (e.g., "{"schedule":"Visit Meiji Shrine at 09:00, have lunch at Omotesando at 12:00, swim at Sunshine City Pool in Toshima Ward at 15:00"}") and then sends this encoded data to the user's device.

[0150] Input: Generated action plan

[0151] Output: JSON data sent to the user's device

[0152] Step 5:

[0153] View and book your action plan

[0154] The device decodes the action plan received from the server from JSON data format and displays it on the user interface. For example, "Meiji Jingu Shrine at 9:00, lunch at Omotesando at 12:00, Sunshine City Pool at 15:00." The user confirms the displayed plan. If necessary, for example, to make a reservation at a restaurant for lunch at 12:00, the device sends a reservation request to the server again.

[0155] Input: JSON data received from the server

[0156] Output: Action plan displayed in the user interface, booking request from the user

[0157] Step 6:

[0158] Processing booking requests

[0159] The server processes the reservation request received from the device. It accesses the reservation API and completes the reservation of a specific resource or service. If the reservation is successful, it sends the result in JSON format to the user's device (e.g., "{"status":"success","message":"Lunch reservation completed."").

[0160] Input: Reservation request from user

[0161] Output: Reservation result (success or failure)

[0162] (Application example 1)

[0163] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0164] Conventional food delivery and restaurant reservation services lack the mechanisms to provide optimal suggestions based on specific user requests, requiring users to spend time searching for options on their own. They also struggle to efficiently plan meals based on specific dates, times, locations, and budgets. These challenges mean users end up spending a lot of time and effort creating their plans.

[0165] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0166] In this invention, the server includes means for receiving requests from a user, means for presenting a generated action plan to the user, generation AI means for generating an action plan based on the user's requests, means for making reservations based on the generated action plan, and means for inputting the type of meal, date and time, location, and budget range. This allows the user to automatically receive optimal meal plans and restaurants based on their detailed requests, enabling them to efficiently create an action plan.

[0167] "User" means an individual or corporation that uses the System to receive suggested activity plans and meal plans.

[0168] The "means for receiving requests" refers to an interface or communication means for inputting and receiving specific requests such as date and time, location, type of meal, budget range, etc. from the user.

[0169] The "means for presenting an action plan to a user" refers to a display means or communication means for presenting the generated action plan or meal plan to a user through a user interface.

[0170] "Generative AI means" refers to an artificial intelligence model and its processing system that automatically generates an action plan and optimal meal plan based on the user's requests.

[0171] "Means for making reservations based on an action plan" refers to API calls and communication methods for automatically making reservations at restaurants and facilities based on the generated action plan.

[0172] The "means for inputting the type of meal, date and time, location, and budget range" is a user interface for the user to input the type of meal, desired date and time, location, and budget range.

[0173] "User interface means" means an interactive display and operation means for a user to input requests and confirm proposed action plans.

[0174] "Communication means for transmitting and receiving data between a server and a terminal" refers to a network communication means and protocol for transmitting and receiving data between a server and a user terminal.

[0175] "Means for generating optimal restaurants and meal plans" refers to a processing system that uses an AI model to optimally select and generate restaurants and meal plans based on user requests.

[0176] "Means for displaying the meal plan on a user interface" refers to functionality and software for visually displaying the generated meal plan on the user's terminal.

[0177] The present invention is a system that uses a generative AI model to propose optimal action plans and meal plans based on the user's specific requests, such as the type of meal, date and time, location, and budget range, and makes reservations as needed. The system consists of a user terminal and a server.

[0178] User terminal

[0179] A user device refers to a mobile device such as a smartphone or tablet, and can input requests such as date and time, location, type of meal, and budget range through a user interface. The user device sends the input information to the server as an HTTP POST request. The data is encoded in JSON format.

[0180] server

[0181] The server receives information from the user and generates an optimal action plan or meal plan based on a generative AI model. The generated plan is sent from the server to the user's device and displayed on the device. It also has a function to make reservations at restaurants or facilities using a reservation API if necessary. The server uses a generative AI model (e.g., GPT-3 or GPT-4) to generate the optimal plan.

[0182] Program processing

[0183] Requests entered through the user interface are sent to the server. The server uses a generative AI model to generate an action plan and meal plan, which are then sent back to the user's device. The user can then review the received action plan and meal plan through the user interface and make reservations as necessary.

[0184] For example, if a user inputs a request such as "I want to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I would like a terrace seat," the generative AI will generate the following action plan:

[0185] Dine at the perfect Italian restaurant at 12:00

[0186] Dessert at a nearby cafe at 2:00 PM

[0187] Stroll around the park at 4:00 PM

[0188] Thus, we provide the generative AI model with a prompt of the following form:

[0189] I'd like to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I'd prefer a terrace seat.

[0190] Based on this, generate the best restaurant and action plan, including a schedule suggestion for 12 AM to 6 PM.

[0191] By using this invention, users can automatically receive optimal meal plans and restaurant recommendations based on their detailed requests, enabling them to plan their trips efficiently. In addition, the system is linked to a reservation system, allowing users to complete reservations hassle-free. This allows users to spend their time in a fulfilling manner.

[0192] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0193] Step 1:

[0194] User request input

[0195] The user inputs the date and time, location, type of meal, budget range, special requests, etc. through the user interface. For example, the user might input, "I would like to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I would like a terrace seat." The input data is encoded in JSON format. The input data is used to send the user's request to the server, and specific fields include the date and time (date), location (location), type of meal (meal_type), budget range (budget), and special requests (special_request).

[0196] Step 2:

[0197] Sending data to the server

[0198] The device sends the information entered in step 1 to the server as an HTTP POST request. At this time, the data is encoded in JSON format and sent to the server's API endpoint. The input data is in the format sent from the user's device to the server, for example, {"date": "2023-10-10", "location": "Tokyo", "meal_type": "Italian", "budget": "2000-5000 yen", "special_request": "Prefer terrace seat"}. In this way, the server receives the user's request.

[0199] Step 3:

[0200] Generative AI generates action plans

[0201] The server passes the user's request received in step 2 to a generative AI model. This generative AI model (e.g., GPT-3 or GPT-4) receives the user's request as input in the form of a prompt and generates an optimal action plan or meal plan. The specific prompt is as follows:

[0202] I'd like to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I'd prefer a terrace seat. Based on this, please generate the best restaurant and itinerary. Please also suggest a schedule between 12:00 AM and 6:00 PM.

[0203] The generative AI model analyzes this prompt and generates a specific action plan based on the user's request. The output data is generated in JSON format, for example:

[0204] {

[0205] "plan": [

[0206] {"time": "12:00", "activity": "Dinner at an Italian restaurant"},

[0207] {"time": "14:00", "activity": "Dessert at the cafe"},

[0208] {"time": "16:00", "activity": "Stroll around the park"}

[0209] ],

[0210] "reservation": {

[0211] "restaurant": "Italian Restaurant A",

[0212] "confirmation_number": "12345ABC"

[0213] }

[0214] }

[0215] Step 4:

[0216] Submitting and Viewing Proposals

[0217] The server sends the action plan generated in step 3 to the user's device. At this time, the generated plan is again encoded in JSON format. The user's device displays the received action plan on the user interface. The user can check the proposed action plan and make fine adjustments if necessary. This display process allows the user to visually understand the plan.

[0218] Step 5:

[0219] reservation

[0220] If the user agrees with the displayed plan of action, they request a reservation through the user interface. This request is sent back to the server. The server then calls the reservation API to reserve the necessary restaurant or facility. Specifically, it sends a request to the reservation API and receives reservation confirmation information. This reservation information is presented to the user, notifying them that the reservation has been completed. For example, when a restaurant reservation is completed, reservation confirmation information is sent to the user in the following JSON format:

[0221] {

[0222] "reservation": {

[0223] "restaurant": "Italian Restaurant A",

[0224] "confirmation_number": "12345ABC"

[0225] }

[0226] }

[0227] Through the above processing steps, the system of the present invention can provide optimal activity plans and meal plans based on the user's detailed requests and perform efficient reservation processing.

[0228] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0229] This invention is a system that proposes action plans to help users make effective use of their holidays and spare time, and by combining it with an emotion engine, provides optimal plans based on the user's emotions. This system consists of a user terminal and a server, and generates advanced action plans using the emotion engine and a generative AI model.

[0230] System configuration

[0231] User terminal

[0232] The user terminal refers to a mobile device such as a smartphone or tablet. This allows users to input date, time, location, requests, and emotional data (facial recognition data and voice data) through a user interface. The input information is then sent to the server.

[0233] server

[0234] The server receives information from the user and generates an action plan based on the emotion engine and generative AI model. The generated action plan is sent from the server to the user's device and presented to the user. The system also has a function to make reservations for facilities and events using reservation methods as needed.

[0235] Program processing

[0236] 1. User requests and emotion data input

[0237] A user launches the application and inputs the date, time, location, and desires through the user interface. In addition, the user provides emotional data (e.g., facial recognition through a camera or voice input through a microphone) to the system.

[0238] 2. Sending data to the server

[0239] The device encodes the information and emotion data entered by the user into JSON format and sends it to the server as an HTTP POST request.

[0240] 3. Emotional Data Processing and Analysis

[0241] The server passes the received emotion data to the emotion engine, which analyzes the user's emotions. For example, if the user is feeling stressed, this information is passed on to the generative AI model.

[0242] 4. Adjustment of action plans

[0243] The generative AI model generates an action plan based on data from the emotion engine, taking into account the user's desires and emotional state. For example, it suggests a relaxation plan for a stressed user.

[0244] 5. Generate a final action plan

[0245] The generative AI model then generates a new, adjusted action plan and returns it to the server. The generated action plan includes specific schedules such as "visit a hot spring at 9:00," "have lunch at a relaxing cafe at 12:00," and "take a walk in a nature park at 15:00."

[0246] 6. Submit your proposal

[0247] The server encodes the final action plan into JSON format and sends it to the originating device as an HTTP response.

[0248] 7. Viewing and booking your action plan

[0249] The terminal displays the received action plan on the user interface. The user checks the displayed action plan and requests a reservation for a specific activity if necessary. For example, if the user wants to make a reservation for a hot spring, the user presses the reservation button.

[0250] 8. Booking Processing

[0251] The device sends the reservation request back to the server as an HTTP POST request. The server receives the reservation request and processes the reservation by calling the necessary API (for example, a hot spring reservation API). After the reservation is completed, the information is notified to the device.

[0252] 9. Display of Final Action Plan

[0253] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[0254] Specific examples

[0255] Example 1: If User A inputs a request such as "I want to relax in Tokyo on October 10, 2023," and the camera recognizes a stressed expression, the generative AI will make the following suggestions:

[0256] 09:00 Take a bath in the hot springs

[0257] 12:00 Lunch at the Relaxation Cafe

[0258] 15:00 Walk in a quiet park in Tokyo

[0259] Example 2: If User B inputs a request such as "I want to spend an active time in Sapporo on December 25, 2023," and an excited state is recognized from the voice, the generation AI will make the following suggestions.

[0260] 10:00 Visit the Snow Festival at Odori Park

[0261] 13:00 Experience sledding

[0262] 16:00 Visit entertainment facilities in Susukino

[0263] As described above, the present invention proposes an action plan that takes into account the user's emotional state and also eliminates the need for reservations, thereby enabling the user to spend a more comfortable and fulfilling time.

[0264] The processing flow will be explained below.

[0265] Step 1:

[0266] A user launches the application and enters the date, time, location, and desire through a user interface, and also uses a camera and microphone to collect facial recognition data, voice data, and emotional data, including a specific request.

[0267] Step 2:

[0268] The device encodes the date, time, location, desire, and emotion data entered by the user into JSON format and sends this encoded data to the server as an HTTP POST request.

[0269] Step 3:

[0270] The server receives the HTTP POST request and parses the JSON data, extracting date, time, location, desire, and emotion data.

[0271] Step 4:

[0272] The server passes the extracted emotion data to the emotion engine, which analyzes the user's facial recognition data and voice data. Based on this data, the emotion engine identifies the user's emotional state, such as "feeling stressed" or "excited."

[0273] Step 5:

[0274] The server passes the emotional state data obtained from the emotion engine as input to the generative AI model. The generative AI model generates an optimal action plan taking into account the user's requests and emotional state. For example, it proposes a relaxing plan for a stressed user and an active plan for an excited user.

[0275] Step 6:

[0276] The generative AI model generates an action plan and returns the result to the server. The generated action plan includes a specific schedule, such as "visit a hot spring at 09:00," "have lunch at a relaxing cafe at 12:00," "take a walk in a nature park at 15:00," and "visit the hot spring again at 18:00."

[0277] Step 7:

[0278] The server encodes the generated action plan again into JSON format and sends it to the original device as an HTTP response.

[0279] Step 8:

[0280] The device parses the JSON data received from the server and displays the action plan in the user interface, including detailed information about each activity.

[0281] Step 9:

[0282] The user checks the proposed action plan and requests reservations for specific activities if necessary. For example, if the user wants to make a reservation for a hot spring, the user presses the reservation button.

[0283] Step 10:

[0284] The device sends the reservation request back to the server as an HTTP POST request. The server receives the reservation request and processes the reservation by calling the necessary API (for example, a hot spring reservation API). After the reservation is completed, the information is notified to the device.

[0285] Step 11:

[0286] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[0287] Step 12:

[0288] The user follows the proposed plan of action and spends their time in a fulfilling manner. The device supports the user's actions by providing navigation and map links as needed.

[0289] Example 2

[0290] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0291] Conventional behavior plan proposal systems generate behavior plans based on user requests, but do not consider the user's emotional state when proposing them. As a result, they are unable to provide optimal behavior plans for users when they are feeling stressed or in a specific emotional state, which can result in lower user satisfaction. Furthermore, making reservations based on the behavior plan must be done manually, which places a burden on the user.

[0292] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0293] In this invention, the server includes means for receiving requests and emotion data from a user, means for transmitting the received requests and emotion data to the server, emotion engine means for analyzing the emotion data, generation AI means for generating an action plan based on the data obtained from the emotion engine means, means for transmitting the generated action plan to a user terminal, means for displaying the generated action plan on a user interface, and means for receiving a reservation request from a user and processing the reservation. This makes it possible to automatically generate an optimal action plan that takes into account the user's emotional state, and further automate reservations based on the action plan.

[0294] The "means for receiving request and emotion data from a user" refers to a device or software function for receiving and storing the date, time, location, request and emotion data input by a user.

[0295] The "means for transmitting the received request and emotion data to the server" refers to a protocol or communication device for transmitting the request and emotion data from the user terminal to the server.

[0296] The "emotion engine means for analyzing emotion data" refers to an algorithm or software module for analyzing received emotion data and identifying the user's emotional state.

[0297] The "generative AI means for generating an action plan based on data obtained from the emotion engine means" refers to an artificial intelligence algorithm or software module for generating an optimal action plan based on the emotional state analyzed by the emotion engine and the user's requests.

[0298] "Means for transmitting the generated action plan to the user terminal" refers to a communication means or protocol for transmitting the action plan generated by the generation AI means to the user terminal.

[0299] The "means for displaying the generated action plan on a user interface" refers to a screen display device or software interface for displaying the action plan generated on the user terminal in a format that is easy for the user to view.

[0300] "Means for receiving user reservation requests and processing reservations" refers to the protocols and operational means for receiving a user's request to reserve a specific activity and calling the appropriate reservation service or API to fulfill the reservation.

[0301] This invention is a system that proposes action plans to help users make effective use of their holidays and spare time, and further provides optimal plans based on the user's emotions by combining an emotion engine. This system consists of a user terminal and a server, and generates advanced action plans using the emotion engine and a generative AI model.

[0302] User terminal

[0303] The user terminal refers to a mobile device such as a smartphone or tablet. This allows users to input date, time, location, requests, and emotional data (facial recognition data and voice data) through a user interface. This information is then sent from the terminal to the server.

[0304] server

[0305] The server receives information from the user and generates an action plan based on the emotion engine and generative AI model. The generated action plan is sent from the server to the user's device. The system also has a function to make reservations for facilities and events using reservation methods as needed.

[0306] Emotion Engine

[0307] The emotion engine is an algorithm that analyzes a user's facial recognition data and voice data to identify the user's emotional state, thereby determining whether the user is in a particular emotional state, such as stress or excitement.

[0308] Generative AI Models

[0309] The generative AI model generates an action plan based on data from the emotion engine, taking into account the user's desires and emotional state. For example, if the user is stressed, it will generate a relaxation plan, and if the user is excited, it will generate an active plan.

[0310] Specific examples

[0311] Example 1:

[0312] If a user inputs a request such as "I want to relax in Tokyo on October 10, 2023," and the camera recognizes a stressed facial expression, the generative AI model will suggest the following action plan:

[0313] 09:00 Take a bath in the hot springs

[0314] 12:00 Lunch at the Relaxation Cafe

[0315] 15:00 Walk in a quiet park in Tokyo

[0316] Example 2:

[0317] If a user inputs a request such as "I want to spend some time actively in Sapporo on December 25, 2023," and an excited state is recognized from the voice, the generative AI model will suggest the following action plan:

[0318] 10:00 Visit the Snow Festival at Odori Park

[0319] 13:00 Experience sledding

[0320] 16:00 Visit entertainment facilities in Susukino

[0321] Prompt Sentence Examples

[0322] Example prompt 1:

[0323] "I would like to relax in Tokyo on October 10, 2023. I am feeling stressed, so please tell me the best plan of action for relaxation."

[0324] Example prompt 2:

[0325] "I'm planning an active day in Sapporo on December 25, 2023. I'm excited, so please tell me about your plans, focusing on energetic activities."

[0326] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0327] Step 1:

[0328] Users launch an application on their smartphone or tablet, input the date, time, location, and request, and then use the camera to collect facial recognition data and the microphone to provide voice data.

[0329] The inputs are "date, time, location, request, facial recognition data, and voice data."

[0330] The output is "information and emotional data entered by the user."

[0331] Step 2:

[0332] The device encodes the received information and emotion data into JSON format and sends it to the server as an HTTP POST request.

[0333] The input is "information and emotional data entered by the user."

[0334] The output is the "encoded data".

[0335] Specifically, the data is encoded and an HTTP request is sent.

[0336] Step 3:

[0337] The server passes the received data to the emotion engine for analysis.

[0338] The input is the "encoded data".

[0339] The output is "data to be passed to the emotion engine."

[0340] Specific operations include decoding the data and inputting it into the emotion engine.

[0341] Step 4:

[0342] The emotion engine analyzes the facial recognition data and voice data to identify the user's emotional state, for example, "stress."

[0343] The input is "data passed to the emotion engine."

[0344] The output is the "analyzed emotional state" information.

[0345] Specifically, facial recognition and voice analysis algorithms are executed.

[0346] Step 5:

[0347] The server passes the emotional state obtained from the emotion engine to the generative AI model and instructs it to generate an action plan.

[0348] The input is the "analyzed emotional state" information.

[0349] The output is a prompt to the generative AI model.

[0350] Specific operations include generating prompts and inputting them into the generative AI model.

[0351] Step 6:

[0352] The generative AI model generates an action plan taking into account the emotional state and the user's wishes, such as "visit a hot spring at 09:00," "have lunch at a relaxing cafe at 12:00," and "take a walk in a nature park at 15:00."

[0353] The input is a "prompt to the generative AI model."

[0354] The output is a "generated action plan."

[0355] Specifically, the generative AI model calculates an action plan using an internal algorithm.

[0356] Step 7:

[0357] The server encodes the generated action plan into JSON format and sends it to the terminal as an HTTP response.

[0358] The input is the "generated action plan."

[0359] The output is "encoded action plan data."

[0360] Specifically, the data is encoded and an HTTP response is sent.

[0361] Step 8:

[0362] The terminal decodes the received action plan data and displays it on the user interface.

[0363] The input is "encoded action plan data."

[0364] The output is an action plan for display in the user interface.

[0365] As a specific operation, the data is decoded and displayed on the screen.

[0366] Step 9:

[0367] The user checks the displayed action plan and requests a reservation for a specific activity, for example, a reservation for a hot spring.

[0368] The input is the displayed action plan and booking request.

[0369] The output is "reservation request data."

[0370] As a specific operation, a request is generated and sent.

[0371] Step 10:

[0372] The terminal sends the generated reservation request to the server as an HTTP POST request.

[0373] The input is "reservation request data."

[0374] The output is "Reservation request data sent."

[0375] As a specific operation, the request data is encoded and transmitted.

[0376] Step 11:

[0377] The server receives the reservation request and processes the reservation by calling the necessary API. For example, it calls a hot spring reservation API to process the reservation. After the reservation is completed, it notifies the terminal of the information.

[0378] The input is "reservation request data sent."

[0379] The output is "reservation processing results and notification data."

[0380] Specifically, the reservation API is called and the results are obtained.

[0381] Step 12:

[0382] The terminal again displays the final action plan including the reservation information on the user interface.

[0383] The input is "reservation processing results and notification data."

[0384] The output is a "final action plan for display in the user interface."

[0385] As a specific operation, the data is decoded and displayed on the screen.

[0386] (Application example 2)

[0387] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0388] Conventional behavior plan generation systems only provide plans based on the user's requests, but are unable to provide plans that appropriately reflect the user's individual emotional state. This makes it difficult to propose an optimal behavior plan tailored to the user's emotional state. Furthermore, particularly in self-driving vehicles, behavior plans that allow users to use their time effectively while traveling are not effectively generated or executed. This makes it difficult to provide a highly satisfying experience for users.

[0389] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving request and emotion data from the user, means for presenting the generated action plan to the user, an emotion engine and generation AI means for generating an action plan based on the user's request and emotion data, means for making a reservation based on the generated action plan, and means for displaying the generated action plan on the interface of the autonomously driven vehicle. This makes it possible to propose an action plan that matches the user's request and emotional state within the autonomously driven vehicle, making effective use of time while traveling and increasing user satisfaction.

[0390] "Requests" refer to the hopes and demands that users have regarding the action plan.

[0391] "Emotion data" is data that indicates the user's emotional state, and includes facial recognition data, voice data, and the like.

[0392] An "action plan" refers to a specific schedule of activities generated based on the user's requests and emotional data.

[0393] An "emotion engine" refers to software that analyzes a user's emotional data and identifies their emotional state.

[0394] "Generative AI" is an artificial intelligence model that generates optimal action plans based on user requests and emotional data.

[0395] "User interface" refers to the interface through which a user inputs requests and emotional data into the system.

[0396] "Server" refers to a computer system that receives data from users and is responsible for generating action plans and processing reservations.

[0397] A "terminal" is a device through which a user accesses the system, including smartphones, tablets, and autonomous vehicle interfaces.

[0398] "Communication means" refers to the technology and protocols used to send and receive data between a server and a terminal.

[0399] "Reservation means" refers to a system or process for making reservations for facilities or events based on the generated action plan.

[0400] To implement this invention, a user terminal (e.g., a smartphone, tablet, or autonomous vehicle interface) and a server are required. The user terminal provides a user interface for inputting date, time, location, and requests. In addition, the user terminal uses a camera and microphone to collect emotion data such as facial recognition data and voice data.

[0401] Through the user device interface, the user inputs date, time, location, desire, and emotional data. This data is encoded in JSON format and sent to the server as an HTTP POST request. The server receives this data and uses an emotional engine to analyze the user's emotional state. For example, it identifies whether the user is feeling stressed or seeking relaxation.

[0402] The server then uses the generative AI model to generate an action plan based on the user's desires and emotional state, including specific schedules such as "visit a hot spring at 09:00," "have lunch at a relaxation cafe at 12:00," and "take a walk in a quiet park at 15:00."

[0403] The generated action plan is sent from the server to the user's device and presented to the user. The user can review this action plan and request a reservation for a specific activity if necessary. For example, if a user wants to make a reservation for a hot spring, they press the reservation button, which sends a reservation request to the server. The server then calls the necessary API (e.g., a hot spring reservation API) to process the reservation. After the reservation is completed, the information is notified to the user's device.

[0404] By using this system, users can make effective use of their time while traveling in an autonomous vehicle and enjoy optimal action plans tailored to their emotions. A specific example is shown below.

[0405] Example 1:

[0406] If User A inputs a request such as "I want to relax in Kyoto on November 3, 2023," and the camera recognizes a stressed expression, the generative AI model will suggest the following action plan:

[0407] 09:00 Take a bath in the hot springs

[0408] 12:00 Lunch at the Relaxation Cafe

[0409] 15:00 Stroll around Momiji Park

[0410] Example 2:

[0411] If User B inputs a request such as "I want to spend time actively in Sapporo on December 25, 2023," and an excited state is recognized from the voice, the generative AI model will propose the following action plan:

[0412] 10:00 Visit the Snow Festival at Odori Park

[0413] 13:00 Experience sledding

[0414] 16:00 Visit an entertainment venue

[0415] In the above example, the following prompt sentence is passed to the generative AI model to generate an action plan:

[0416] Prompt statement example 1:

[0417] "Date: 2023-11-03, Location: Kyoto, Request: Relaxation, Emotion: Stress"

[0418] Prompt statement example 2:

[0419] "Date: 2023-12-25, Location: Sapporo, Request: Active, Emotion: Excitement"

[0420] In this way, an optimal action plan that matches the user's emotions is provided, thereby increasing the user's satisfaction.

[0421] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0422] Step 1:

[0423] The user inputs the date, time, location, and request through the user interface. In addition, the device uses a camera and microphone to collect emotional data (facial recognition data and voice data). This data is encoded into JSON format by the device.

[0424] Input: Date and time, location, request, emotional data (face recognition data, voice data)

[0425] Output: User data in JSON format

[0426] Step 2:

[0427] The device sends the encoded user data to the server as an HTTP POST request, which the server receives and begins parsing the data.

[0428] Input: User data in JSON format

[0429] Output: HTTP POST request to the server

[0430] Step 3:

[0431] The server analyzes the received user data and uses an emotion engine to identify the user's emotional state, using facial recognition and voice data to determine whether the user is feeling stressed or seeking relaxation.

[0432] Input: User data (face recognition data, voice data)

[0433] Output: User's emotional state (e.g., stressed, relaxed)

[0434] Step 4:

[0435] The server uses a generative AI model to generate an optimal action plan based on the analysis results (user's emotional state) and the user's request by the emotion engine. The server passes the prompt sentence to the generative AI model and obtains a specific action plan as its output.

[0436] Input: User's emotional state, desires, prompt

[0437] Output: Action plan

[0438] Step 5:

[0439] The server encodes the generated action plan and sends it to the user device in JSON format. An action plan might include, for example, "visit a hot spring at 09:00," "have lunch at a relaxation cafe at 12:00," and "take a walk in the park at 15:00."

[0440] Input: Action Plan

[0441] Output: Action plan in JSON format

[0442] Step 6:

[0443] The terminal displays the received action plan on the user interface. The user can check the plan and request a reservation for a specific activity. For example, if the user wants to make a reservation for a hot spring, the user presses the reservation button.

[0444] Input: Action plan in JSON format

[0445] Output: Display of action plan

[0446] Step 7:

[0447] A user's reservation request is sent from the terminal to the server. The server processes the reservation by calling the necessary API (e.g., hot spring reservation API). After the reservation is completed, the information is notified to the user terminal.

[0448] Input: Booking Request

[0449] Output: Reservation completion notification

[0450] Step 8:

[0451] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[0452] Input: Reservation completion notification

[0453] Output: Display the final action plan

[0454] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0455] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0456] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0457] [Second embodiment]

[0458] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0459] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0460] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0461] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0462] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0463] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0464] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0465] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0466] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0467] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0468] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0469] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0470] This invention is a system that proposes action plans to users so that they can make effective use of their holidays and spare time. This system is composed of a user terminal and a server, and is characterized by generating optimal action plans using a generative AI model.

[0471] System configuration

[0472] User terminal

[0473] The user terminal refers to a mobile device such as a smartphone or tablet, and allows users to input date, time, location, and requests through a user interface. The input information is then transmitted to the server.

[0474] server

[0475] The server receives information from the user and generates an action plan based on the generative AI model. The generated action plan is sent from the server to the user's device and presented to the user. The system also has a function to make reservations for facilities and events using reservation methods as needed.

[0476] Program processing

[0477] 1. User request input

[0478] Users input a date, time, location, and request through a user interface, such as "I want to do something summer-like in Tokyo on October 10, 2023."

[0479] 2. Sending data to the server

[0480] The device sends the entered information to the server as an HTTP POST request, with the data encoded in JSON format.

[0481] 3. Generating action plans using generative AI

[0482] The server receives the request and passes the data to a generative AI model, which then generates an optimal action plan based on the user's requests. For example, it generates a specific schedule such as "visit Meiji Shrine at 9:00, have lunch in Omotesando at 12:00, and swim at the Sunshine City Pool in Toshima Ward at 15:00."

[0483] 4. Submit your proposal

[0484] The server sends the generated action plan to the user's device, where it is again encoded in JSON format.

[0485] 5. Viewing and booking your action plan

[0486] The device displays the received action plan on the user interface. The user checks the displayed action plan and requests a reservation if necessary. For example, when reserving a restaurant for lunch, the device sends a reservation request to the server again, and the server calls the reservation API to complete the reservation.

[0487] 6. User Behavior

[0488] The user follows the proposed action plan and spends their time fulfillingly. This system is extremely convenient because it automatically provides the optimal plan according to the user's requests, eliminating the need for the user to think about it.

[0489] Specific examples

[0490] Example 1: If User A inputs a request such as "I want to eat something refreshing and visit a power spot in Tokyo on October 10, 2023," the generation AI will make the following suggestions.

[0491] 09:00 Visit Meiji Shrine

[0492] 12:00 Refreshing lunch at a cafe in Omotesando

[0493] 15:00 Hiking at Mount Takao

[0494] 18:00 Sightseeing at Tokyo Tower

[0495] Example 2: If User B inputs a request such as "I want to enjoy the snow and soak in a hot spring in Sapporo on December 25, 2023," the generation AI will make the following suggestions.

[0496] 10:00 Visit the Snow Festival at Odori Park

[0497] 13:00 Lunch at a restaurant in Susukino

[0498] 16:00 Relax at Jozankei Onsen

[0499] In this way, the present invention proposes an optimal plan of action based on the user's specific needs and eliminates the need for reservations, allowing the user to spend more quality time.

[0500] The processing flow will be explained below.

[0501] Step 1:

[0502] Users launch the application and input their date, time, location, and desires through the user interface, such as "I want to do something summer-like in Tokyo on October 10, 2023, and I want to go to a power spot."

[0503] Step 2:

[0504] The device collects the information entered by the user and encodes it into JSON format, which is then sent to the server as an HTTP POST request.

[0505] Step 3:

[0506] The server receives the HTTP POST request and parses the JSON data, extracting the date, time, location, and request.

[0507] Step 4:

[0508] The server passes the extracted date, time, location, and request data as input to the generative AI model, which then generates an action plan that matches the user's request.

[0509] Step 5:

[0510] The generative AI model outputs an action plan and returns the results to the server. For example, it generates a specific schedule such as "visit Meiji Shrine at 9:00, have lunch in Omotesando at 12:00, and swim at Sunshine City Pool in Toshima Ward at 15:00."

[0511] Step 6:

[0512] The server encodes the generated action plan again into JSON format and sends it to the original device as an HTTP response.

[0513] Step 7:

[0514] The device parses the JSON data received from the server and displays the action plan in the user interface, including detailed information about each activity.

[0515] Step 8:

[0516] The user reviews the proposed action plan and requests reservations for specific activities if necessary. For example, if the user wants to make a reservation for lunch, the user presses the reservation button.

[0517] Step 9:

[0518] The terminal sends the reservation request to the server again as an HTTP POST request.

[0519] Step 10:

[0520] The server receives the reservation request, calls the necessary API (e.g., restaurant reservation API) to process the reservation, and notifies the device of the reservation information after it is completed.

[0521] Step 11:

[0522] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[0523] Step 12:

[0524] The user follows the proposed plan of action and spends their time in a fulfilling manner. The device supports the user's actions by providing navigation and map links as needed.

[0525] Example 1

[0526] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0527] Conventional action plan generation systems require a lot of time and effort to process user-entered information, generate plans, and manage reservations, making it difficult to efficiently create action plans. It is also difficult to automatically provide customized plans that reflect the user's preferences and requests in detail. This has resulted in problems such as users being unable to create satisfactory plans and a lack of convenience.

[0528] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0529] In this invention, the server includes means for receiving requests from a user, input means for inputting the user's requests as date, time, location, and request information, means for transmitting the input data to the server, means for generating an action plan using a generative AI model, means for presenting the generated action plan to the user, means for making a reservation based on the generated action plan, and communication means for transmitting and receiving data between the server and a terminal. This makes it possible to efficiently and quickly provide an optimal action plan based on the information input by the user, and to automatically make a reservation based on that plan.

[0530] The "means for receiving requests from a user" refers to a device or software that collects and records the date, time, location, and individual request information provided by the user.

[0531] The "input means for inputting the user's requests as date, time, location, and request information" refers to a device or software that provides an interface that allows the user to specifically input his or her own action plans and preferences.

[0532] The "means for transmitting input data to a server" refers to a device or software that has a mechanism for transmitting information input from a user terminal to a server via communication.

[0533] A "means for generating an action plan using a generative AI model" is a device or software that uses an artificial intelligence algorithm to automatically create an optimal action plan based on data provided by a user.

[0534] The "means for presenting the generated action plan to the user" refers to a device or software that provides an interface that visualizes the generated action plan and presents it to the user, allowing the user to select the next step.

[0535] The "means for making reservations based on the generated action plan" is a device or software that automatically processes the reservation process of required resources and services based on the generated plan.

[0536] "Communication means for transmitting and receiving data between a server and a terminal" refers to a device or software that provides a communication protocol or network interface that enables data to be transferred between a server and a user terminal.

[0537] This invention is a system that proposes action plans to users so that they can make effective use of their holidays and spare time. This system is composed of a user terminal and a server, and is characterized by generating optimal action plans using a generative AI model.

[0538] User terminal

[0539] A user device refers to a mobile device such as a smartphone or tablet, and can input date, time, location, and requests through a user interface. A user uses an input form within the app to input their desired date, time, location, and specific requests. For example, if they input "I want to do something summery in Tokyo on October 10, 2023," the information is processed on the device and sent to the server.

[0540] server

[0541] The server receives the data from the device and generates an action plan based on the generative AI model. The generative AI model used automatically creates an optimal action plan based on the user's requests, taking into account past data and general information. The generated action plan is sent from the server to the user's device and presented to the user through a user interface.

[0542] Hardware and Software Use

[0543] The system uses the following hardware and software:

[0544] User devices: smartphones, tablets

[0545] Input method: Touch screen, keyboard

[0546] Communication method: Wi-Fi, 4G / 5G

[0547] Server: Cloud server, web server

[0548] Generative AI models: machine learning algorithms, natural language processing models

[0549] Data format: JSON

[0550] Specific examples

[0551] Specific examples are shown below.

[0552] Example 1: If User A inputs a request such as "I want to eat something refreshing and visit a power spot in Tokyo on October 10, 2023," the system will generate the following action plan:

[0553] 09:00 Visit Meiji Shrine

[0554] 12:00 Refreshing lunch at a cafe in Omotesando

[0555] 15:00 Hiking at Mount Takao

[0556] 18:00 Sightseeing at Tokyo Tower

[0557] Example 2: If User B inputs a request such as "I want to enjoy the snow and soak in hot springs in Sapporo on December 25, 2023," the system generates the following action plan:

[0558] 10:00 Visit the Snow Festival at Odori Park

[0559] 13:00 Lunch at a restaurant in Susukino

[0560] 16:00 Relax at Jozankei Onsen

[0561] Prompt Sentence Examples

[0562] "I want to eat something refreshing and visit a power spot in Tokyo on October 10, 2023."

[0563] In this way, the system of the present invention can efficiently and accurately generate an action plan based on the information entered by the user, and even automatically make reservations. By using this system, users can spend their time in a fulfilling manner.

[0564] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0565] Step 1:

[0566] User request input

[0567] Users launch the app on their smartphone or tablet and input the date, time, location, and requests through the user interface. For example, if a user inputs a request such as "I want to do something summer-like in Tokyo on October 10, 2023," that information is collected by the app and sent to the next step.

[0568] Input: User's date, time, location, and request information

[0569] Output: The input data (e.g., "I want to do something summery in Tokyo on October 10, 2023")

[0570] Step 2:

[0571] Sending data to the server

[0572] The device encodes the information entered by the user into JSON format (e.g., "{"date":"2023-10-10","location":"Tokyo","request":"I want to do something summery"}"). Next, it sends this encoded data to the server using an HTTP POST request. This is done over a communication network (Wi-Fi, 4G / 5G).

[0573] Input: Data entered by the user

[0574] Output: JSON data sent to the server

[0575] Step 3:

[0576] Generative AI generates action plans

[0577] The server decodes the JSON data received from the device. This decoded data is prepared as an input prompt for the generative AI model. An example prompt might be, "Please propose an action plan for doing summer-like things in Tokyo on October 10, 2023." The generative AI model generates an action plan based on this prompt. The generated plan includes specific time and activity suggestions.

[0578] Input: JSON data received by the server

[0579] Output: Generated action plan (e.g., "Visit Meiji Shrine at 09:00, have lunch at Omotesando at 12:00, and swim at Sunshine City Pool in Toshima Ward at 15:00")

[0580] Step 4:

[0581] Submit your proposal

[0582] The server re-encodes the generated action plan into JSON format (e.g., "{"schedule":"Visit Meiji Shrine at 09:00, have lunch at Omotesando at 12:00, swim at Sunshine City Pool in Toshima Ward at 15:00"}") and then sends this encoded data to the user's device.

[0583] Input: Generated action plan

[0584] Output: JSON data sent to the user's device

[0585] Step 5:

[0586] View and book your action plan

[0587] The device decodes the action plan received from the server from JSON data format and displays it on the user interface. For example, "Meiji Jingu Shrine at 9:00, lunch at Omotesando at 12:00, Sunshine City Pool at 15:00." The user confirms the displayed plan. If necessary, for example, to make a reservation at a restaurant for lunch at 12:00, the device sends a reservation request to the server again.

[0588] Input: JSON data received from the server

[0589] Output: Action plan displayed in the user interface, booking request from the user

[0590] Step 6:

[0591] Processing booking requests

[0592] The server processes the reservation request received from the device. It accesses the reservation API and completes the reservation of a specific resource or service. If the reservation is successful, it sends the result in JSON format to the user's device (e.g., "{"status":"success","message":"Lunch reservation completed."").

[0593] Input: Reservation request from user

[0594] Output: Reservation result (success or failure)

[0595] (Application example 1)

[0596] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0597] Conventional food delivery and restaurant reservation services lack the mechanisms to provide optimal suggestions based on specific user requests, requiring users to spend time searching for options on their own. They also struggle to efficiently plan meals based on specific dates, times, locations, and budgets. These challenges mean users end up spending a lot of time and effort creating their plans.

[0598] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0599] In this invention, the server includes means for receiving requests from a user, means for presenting a generated action plan to the user, generation AI means for generating an action plan based on the user's requests, means for making reservations based on the generated action plan, and means for inputting the type of meal, date and time, location, and budget range. This allows the user to automatically receive optimal meal plans and restaurants based on their detailed requests, enabling them to efficiently create an action plan.

[0600] "User" means an individual or corporation that uses the System to receive suggested activity plans and meal plans.

[0601] The "means for receiving requests" refers to an interface or communication means for inputting and receiving specific requests such as date and time, location, type of meal, budget range, etc. from the user.

[0602] The "means for presenting an action plan to a user" refers to a display means or communication means for presenting the generated action plan or meal plan to a user through a user interface.

[0603] "Generative AI means" refers to an artificial intelligence model and its processing system that automatically generates an action plan and optimal meal plan based on the user's requests.

[0604] "Means for making reservations based on an action plan" refers to API calls and communication methods for automatically making reservations at restaurants and facilities based on the generated action plan.

[0605] The "means for inputting the type of meal, date and time, location, and budget range" is a user interface for the user to input the type of meal, desired date and time, location, and budget range.

[0606] "User interface means" means an interactive display and operation means for a user to input requests and confirm proposed action plans.

[0607] "Communication means for transmitting and receiving data between a server and a terminal" refers to a network communication means and protocol for transmitting and receiving data between a server and a user terminal.

[0608] "Means for generating optimal restaurants and meal plans" refers to a processing system that uses an AI model to optimally select and generate restaurants and meal plans based on user requests.

[0609] "Means for displaying the meal plan on a user interface" refers to functionality and software for visually displaying the generated meal plan on the user's terminal.

[0610] The present invention is a system that uses a generative AI model to propose optimal action plans and meal plans based on the user's specific requests, such as the type of meal, date and time, location, and budget range, and makes reservations as needed. The system consists of a user terminal and a server.

[0611] User terminal

[0612] A user device refers to a mobile device such as a smartphone or tablet, and can input requests such as date and time, location, type of meal, and budget range through a user interface. The user device sends the input information to the server as an HTTP POST request. The data is encoded in JSON format.

[0613] server

[0614] The server receives information from the user and generates an optimal action plan or meal plan based on a generative AI model. The generated plan is sent from the server to the user's device and displayed on the device. It also has a function to make reservations at restaurants or facilities using a reservation API if necessary. The server uses a generative AI model (e.g., GPT-3 or GPT-4) to generate the optimal plan.

[0615] Program processing

[0616] Requests entered through the user interface are sent to the server. The server uses a generative AI model to generate an action plan and meal plan, which are then sent back to the user's device. The user can then review the received action plan and meal plan through the user interface and make reservations as necessary.

[0617] For example, if a user inputs a request such as "I want to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I would like a terrace seat," the generative AI will generate the following action plan:

[0618] Dine at the perfect Italian restaurant at 12:00

[0619] Dessert at a nearby cafe at 2:00 PM

[0620] Stroll around the park at 4:00 PM

[0621] Thus, we provide the generative AI model with a prompt of the following form:

[0622] I'd like to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I'd prefer a terrace seat.

[0623] Based on this, generate the best restaurant and action plan, including a schedule suggestion for 12 AM to 6 PM.

[0624] By using this invention, users can automatically receive optimal meal plans and restaurant recommendations based on their detailed requests, enabling them to plan their trips efficiently. In addition, the system is linked to a reservation system, allowing users to complete reservations hassle-free. This allows users to spend their time in a fulfilling manner.

[0625] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0626] Step 1:

[0627] User request input

[0628] The user inputs the date and time, location, type of meal, budget range, special requests, etc. through the user interface. For example, the user might input, "I would like to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I would like a terrace seat." The input data is encoded in JSON format. The input data is used to send the user's request to the server, and specific fields include the date and time (date), location (location), type of meal (meal_type), budget range (budget), and special requests (special_request).

[0629] Step 2:

[0630] Sending data to the server

[0631] The device sends the information entered in step 1 to the server as an HTTP POST request. At this time, the data is encoded in JSON format and sent to the server's API endpoint. The input data is in the format sent from the user's device to the server, for example, {"date": "2023-10-10", "location": "Tokyo", "meal_type": "Italian", "budget": "2000-5000 yen", "special_request": "Prefer terrace seat"}. In this way, the server receives the user's request.

[0632] Step 3:

[0633] Generative AI generates action plans

[0634] The server passes the user's request received in step 2 to a generative AI model. This generative AI model (e.g., GPT-3 or GPT-4) receives the user's request as input in the form of a prompt and generates an optimal action plan or meal plan. The specific prompt is as follows:

[0635] I'd like to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I'd prefer a terrace seat. Based on this, please generate the best restaurant and itinerary. Please also suggest a schedule between 12:00 AM and 6:00 PM.

[0636] The generative AI model analyzes this prompt and generates a specific action plan based on the user's request. The output data is generated in JSON format, for example:

[0637] {

[0638] "plan": [

[0639] {"time": "12:00", "activity": "Dinner at an Italian restaurant"},

[0640] {"time": "14:00", "activity": "Dessert at the cafe"},

[0641] {"time": "16:00", "activity": "Stroll around the park"}

[0642] ],

[0643] "reservation": {

[0644] "restaurant": "Italian Restaurant A",

[0645] "confirmation_number": "12345ABC"

[0646] }

[0647] }

[0648] Step 4:

[0649] Submitting and Viewing Proposals

[0650] The server sends the action plan generated in step 3 to the user's device. At this time, the generated plan is again encoded in JSON format. The user's device displays the received action plan on the user interface. The user can check the proposed action plan and make fine adjustments if necessary. This display process allows the user to visually understand the plan.

[0651] Step 5:

[0652] reservation

[0653] If the user agrees with the displayed plan of action, they request a reservation through the user interface. This request is sent back to the server. The server then calls the reservation API to reserve the necessary restaurant or facility. Specifically, it sends a request to the reservation API and receives reservation confirmation information. This reservation information is presented to the user, notifying them that the reservation has been completed. For example, when a restaurant reservation is completed, reservation confirmation information is sent to the user in the following JSON format:

[0654] {

[0655] "reservation": {

[0656] "restaurant": "Italian Restaurant A",

[0657] "confirmation_number": "12345ABC"

[0658] }

[0659] }

[0660] Through the above processing steps, the system of the present invention can provide optimal activity plans and meal plans based on the user's detailed requests and perform efficient reservation processing.

[0661] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0662] This invention is a system that proposes action plans to help users make effective use of their holidays and spare time, and by combining it with an emotion engine, provides optimal plans based on the user's emotions. This system consists of a user terminal and a server, and generates advanced action plans using the emotion engine and a generative AI model.

[0663] System configuration

[0664] User terminal

[0665] The user terminal refers to a mobile device such as a smartphone or tablet. This allows users to input date, time, location, requests, and emotional data (facial recognition data and voice data) through a user interface. The input information is then sent to the server.

[0666] server

[0667] The server receives information from the user and generates an action plan based on the emotion engine and generative AI model. The generated action plan is sent from the server to the user's device and presented to the user. The system also has a function to make reservations for facilities and events using reservation methods as needed.

[0668] Program processing

[0669] 1. User requests and emotion data input

[0670] A user launches the application and inputs the date, time, location, and desires through the user interface. In addition, the user provides emotional data (e.g., facial recognition through a camera or voice input through a microphone) to the system.

[0671] 2. Sending data to the server

[0672] The device encodes the information and emotion data entered by the user into JSON format and sends it to the server as an HTTP POST request.

[0673] 3. Emotional Data Processing and Analysis

[0674] The server passes the received emotion data to the emotion engine, which analyzes the user's emotions. For example, if the user is feeling stressed, this information is passed on to the generative AI model.

[0675] 4. Adjustment of action plans

[0676] The generative AI model generates an action plan based on data from the emotion engine, taking into account the user's desires and emotional state. For example, it suggests a relaxation plan for a stressed user.

[0677] 5. Generate a final action plan

[0678] The generative AI model then generates a new, adjusted action plan and returns it to the server. The generated action plan includes specific schedules such as "visit a hot spring at 9:00," "have lunch at a relaxing cafe at 12:00," and "take a walk in a nature park at 15:00."

[0679] 6. Submit your proposal

[0680] The server encodes the final action plan into JSON format and sends it to the originating device as an HTTP response.

[0681] 7. Viewing and booking your action plan

[0682] The terminal displays the received action plan on the user interface. The user checks the displayed action plan and requests a reservation for a specific activity if necessary. For example, if the user wants to make a reservation for a hot spring, the user presses the reservation button.

[0683] 8. Booking Processing

[0684] The device sends the reservation request back to the server as an HTTP POST request. The server receives the reservation request and processes the reservation by calling the necessary API (for example, a hot spring reservation API). After the reservation is completed, the information is notified to the device.

[0685] 9. Display of Final Action Plan

[0686] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[0687] Specific examples

[0688] Example 1: If User A inputs a request such as "I want to relax in Tokyo on October 10, 2023," and the camera recognizes a stressed expression, the generative AI will make the following suggestions:

[0689] 09:00 Take a bath in the hot springs

[0690] 12:00 Lunch at the Relaxation Cafe

[0691] 15:00 Walk in a quiet park in Tokyo

[0692] Example 2: If User B inputs a request such as "I want to spend an active time in Sapporo on December 25, 2023," and an excited state is recognized from the voice, the generation AI will make the following suggestions.

[0693] 10:00 Visit the Snow Festival at Odori Park

[0694] 13:00 Experience sledding

[0695] 16:00 Visit entertainment facilities in Susukino

[0696] As described above, the present invention proposes an action plan that takes into account the user's emotional state and also eliminates the need for reservations, thereby enabling the user to spend a more comfortable and fulfilling time.

[0697] The processing flow will be explained below.

[0698] Step 1:

[0699] A user launches the application and enters the date, time, location, and desire through a user interface, and also uses a camera and microphone to collect facial recognition data, voice data, and emotional data, including a specific request.

[0700] Step 2:

[0701] The device encodes the date, time, location, desire, and emotion data entered by the user into JSON format and sends this encoded data to the server as an HTTP POST request.

[0702] Step 3:

[0703] The server receives the HTTP POST request and parses the JSON data, extracting date, time, location, desire, and emotion data.

[0704] Step 4:

[0705] The server passes the extracted emotion data to the emotion engine, which analyzes the user's facial recognition data and voice data. Based on this data, the emotion engine identifies the user's emotional state, such as "feeling stressed" or "excited."

[0706] Step 5:

[0707] The server passes the emotional state data obtained from the emotion engine as input to the generative AI model. The generative AI model generates an optimal action plan taking into account the user's requests and emotional state. For example, it proposes a relaxing plan for a stressed user and an active plan for an excited user.

[0708] Step 6:

[0709] The generative AI model generates an action plan and returns the result to the server. The generated action plan includes a specific schedule, such as "visit a hot spring at 09:00," "have lunch at a relaxing cafe at 12:00," "take a walk in a nature park at 15:00," and "visit the hot spring again at 18:00."

[0710] Step 7:

[0711] The server encodes the generated action plan again into JSON format and sends it to the original device as an HTTP response.

[0712] Step 8:

[0713] The device parses the JSON data received from the server and displays the action plan in the user interface, including detailed information about each activity.

[0714] Step 9:

[0715] The user checks the proposed action plan and requests reservations for specific activities if necessary. For example, if the user wants to make a reservation for a hot spring, the user presses the reservation button.

[0716] Step 10:

[0717] The device sends the reservation request back to the server as an HTTP POST request. The server receives the reservation request and processes the reservation by calling the necessary API (for example, a hot spring reservation API). After the reservation is completed, the information is notified to the device.

[0718] Step 11:

[0719] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[0720] Step 12:

[0721] The user follows the proposed plan of action and spends their time in a fulfilling manner. The device supports the user's actions by providing navigation and map links as needed.

[0722] Example 2

[0723] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0724] Conventional behavior plan proposal systems generate behavior plans based on user requests, but do not consider the user's emotional state when proposing them. As a result, they are unable to provide optimal behavior plans for users when they are feeling stressed or in a specific emotional state, which can result in lower user satisfaction. Furthermore, making reservations based on the behavior plan must be done manually, which places a burden on the user.

[0725] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0726] In this invention, the server includes means for receiving requests and emotion data from a user, means for transmitting the received requests and emotion data to the server, emotion engine means for analyzing the emotion data, generation AI means for generating an action plan based on the data obtained from the emotion engine means, means for transmitting the generated action plan to a user terminal, means for displaying the generated action plan on a user interface, and means for receiving a reservation request from a user and processing the reservation. This makes it possible to automatically generate an optimal action plan that takes into account the user's emotional state, and further automate reservations based on the action plan.

[0727] The "means for receiving request and emotion data from a user" refers to a device or software function for receiving and storing the date, time, location, request and emotion data input by a user.

[0728] The "means for transmitting the received request and emotion data to the server" refers to a protocol or communication device for transmitting the request and emotion data from the user terminal to the server.

[0729] The "emotion engine means for analyzing emotion data" refers to an algorithm or software module for analyzing received emotion data and identifying the user's emotional state.

[0730] The "generative AI means for generating an action plan based on data obtained from the emotion engine means" refers to an artificial intelligence algorithm or software module for generating an optimal action plan based on the emotional state analyzed by the emotion engine and the user's requests.

[0731] "Means for transmitting the generated action plan to the user terminal" refers to a communication means or protocol for transmitting the action plan generated by the generation AI means to the user terminal.

[0732] The "means for displaying the generated action plan on a user interface" refers to a screen display device or software interface for displaying the action plan generated on the user terminal in a format that is easy for the user to view.

[0733] "Means for receiving user reservation requests and processing reservations" refers to the protocols and operational means for receiving a user's request to reserve a specific activity and calling the appropriate reservation service or API to fulfill the reservation.

[0734] This invention is a system that proposes action plans to help users make effective use of their holidays and spare time, and further provides optimal plans based on the user's emotions by combining an emotion engine. This system consists of a user terminal and a server, and generates advanced action plans using the emotion engine and a generative AI model.

[0735] User terminal

[0736] The user terminal refers to a mobile device such as a smartphone or tablet. This allows users to input date, time, location, requests, and emotional data (facial recognition data and voice data) through a user interface. This information is then sent from the terminal to the server.

[0737] server

[0738] The server receives information from the user and generates an action plan based on the emotion engine and generative AI model. The generated action plan is sent from the server to the user's device. The system also has a function to make reservations for facilities and events using reservation methods as needed.

[0739] Emotion Engine

[0740] The emotion engine is an algorithm that analyzes a user's facial recognition data and voice data to identify the user's emotional state, thereby determining whether the user is in a particular emotional state, such as stress or excitement.

[0741] Generative AI Models

[0742] The generative AI model generates an action plan based on data from the emotion engine, taking into account the user's desires and emotional state. For example, if the user is stressed, it will generate a relaxation plan, and if the user is excited, it will generate an active plan.

[0743] Specific examples

[0744] Example 1:

[0745] If a user inputs a request such as "I want to relax in Tokyo on October 10, 2023," and the camera recognizes a stressed facial expression, the generative AI model will suggest the following action plan:

[0746] 09:00 Take a bath in the hot springs

[0747] 12:00 Lunch at the Relaxation Cafe

[0748] 15:00 Walk in a quiet park in Tokyo

[0749] Example 2:

[0750] If a user inputs a request such as "I want to spend some time actively in Sapporo on December 25, 2023," and an excited state is recognized from the voice, the generative AI model will suggest the following action plan:

[0751] 10:00 Visit the Snow Festival at Odori Park

[0752] 13:00 Experience sledding

[0753] 16:00 Visit entertainment facilities in Susukino

[0754] Prompt Sentence Examples

[0755] Example prompt 1:

[0756] "I would like to relax in Tokyo on October 10, 2023. I am feeling stressed, so please tell me the best plan of action for relaxation."

[0757] Example prompt 2:

[0758] "I'm planning an active day in Sapporo on December 25, 2023. I'm excited, so please tell me about your plans, focusing on energetic activities."

[0759] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0760] Step 1:

[0761] Users launch an application on their smartphone or tablet, input the date, time, location, and request, and then use the camera to collect facial recognition data and the microphone to provide voice data.

[0762] The inputs are "date, time, location, request, facial recognition data, and voice data."

[0763] The output is "information and emotional data entered by the user."

[0764] Step 2:

[0765] The device encodes the received information and emotion data into JSON format and sends it to the server as an HTTP POST request.

[0766] The input is "information and emotional data entered by the user."

[0767] The output is the "encoded data".

[0768] Specifically, the data is encoded and an HTTP request is sent.

[0769] Step 3:

[0770] The server passes the received data to the emotion engine for analysis.

[0771] The input is the "encoded data".

[0772] The output is "data to be passed to the emotion engine."

[0773] Specific operations include decoding the data and inputting it into the emotion engine.

[0774] Step 4:

[0775] The emotion engine analyzes the facial recognition data and voice data to identify the user's emotional state, for example, "stress."

[0776] The input is "data passed to the emotion engine."

[0777] The output is the "analyzed emotional state" information.

[0778] Specifically, facial recognition and voice analysis algorithms are executed.

[0779] Step 5:

[0780] The server passes the emotional state obtained from the emotion engine to the generative AI model and instructs it to generate an action plan.

[0781] The input is the "analyzed emotional state" information.

[0782] The output is a prompt to the generative AI model.

[0783] Specific operations include generating prompts and inputting them into the generative AI model.

[0784] Step 6:

[0785] The generative AI model generates an action plan taking into account the emotional state and the user's wishes, such as "visit a hot spring at 09:00," "have lunch at a relaxing cafe at 12:00," and "take a walk in a nature park at 15:00."

[0786] The input is a "prompt to the generative AI model."

[0787] The output is a "generated action plan."

[0788] Specifically, the generative AI model calculates an action plan using an internal algorithm.

[0789] Step 7:

[0790] The server encodes the generated action plan into JSON format and sends it to the terminal as an HTTP response.

[0791] The input is the "generated action plan."

[0792] The output is "encoded action plan data."

[0793] Specifically, the data is encoded and an HTTP response is sent.

[0794] Step 8:

[0795] The terminal decodes the received action plan data and displays it on the user interface.

[0796] The input is "encoded action plan data."

[0797] The output is an action plan for display in the user interface.

[0798] As a specific operation, the data is decoded and displayed on the screen.

[0799] Step 9:

[0800] The user checks the displayed action plan and requests a reservation for a specific activity, for example, a reservation for a hot spring.

[0801] The input is the displayed action plan and booking request.

[0802] The output is "reservation request data."

[0803] As a specific operation, a request is generated and sent.

[0804] Step 10:

[0805] The terminal sends the generated reservation request to the server as an HTTP POST request.

[0806] The input is "reservation request data."

[0807] The output is "Reservation request data sent."

[0808] As a specific operation, the request data is encoded and transmitted.

[0809] Step 11:

[0810] The server receives the reservation request and processes the reservation by calling the necessary API. For example, it calls a hot spring reservation API to process the reservation. After the reservation is completed, it notifies the terminal of the information.

[0811] The input is "reservation request data sent."

[0812] The output is "reservation processing results and notification data."

[0813] Specifically, the reservation API is called and the results are obtained.

[0814] Step 12:

[0815] The terminal again displays the final action plan including the reservation information on the user interface.

[0816] The input is "reservation processing results and notification data."

[0817] The output is a "final action plan for display in the user interface."

[0818] As a specific operation, the data is decoded and displayed on the screen.

[0819] (Application example 2)

[0820] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0821] Conventional behavior plan generation systems only provide plans based on the user's requests, but are unable to provide plans that appropriately reflect the user's individual emotional state. This makes it difficult to propose an optimal behavior plan tailored to the user's emotional state. Furthermore, particularly in self-driving vehicles, behavior plans that allow users to use their time effectively while traveling are not effectively generated or executed. This makes it difficult to provide a highly satisfying experience for users.

[0822] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving request and emotion data from the user, means for presenting the generated action plan to the user, an emotion engine and generation AI means for generating an action plan based on the user's request and emotion data, means for making a reservation based on the generated action plan, and means for displaying the generated action plan on the interface of the autonomously driven vehicle. This makes it possible to propose an action plan that matches the user's request and emotional state within the autonomously driven vehicle, making effective use of time while traveling and increasing user satisfaction.

[0823] "Requests" refer to the hopes and demands that users have regarding the action plan.

[0824] "Emotion data" is data that indicates the user's emotional state, and includes facial recognition data, voice data, and the like.

[0825] An "action plan" refers to a specific schedule of activities generated based on the user's requests and emotional data.

[0826] An "emotion engine" refers to software that analyzes a user's emotional data and identifies their emotional state.

[0827] "Generative AI" is an artificial intelligence model that generates optimal action plans based on user requests and emotional data.

[0828] "User interface" refers to the interface through which a user inputs requests and emotional data into the system.

[0829] "Server" refers to a computer system that receives data from users and is responsible for generating action plans and processing reservations.

[0830] A "terminal" is a device through which a user accesses the system, including smartphones, tablets, and autonomous vehicle interfaces.

[0831] "Communication means" refers to the technology and protocols used to send and receive data between a server and a terminal.

[0832] "Reservation means" refers to a system or process for making reservations for facilities or events based on the generated action plan.

[0833] To implement this invention, a user terminal (e.g., a smartphone, tablet, or autonomous vehicle interface) and a server are required. The user terminal provides a user interface for inputting date, time, location, and requests. In addition, the user terminal uses a camera and microphone to collect emotion data such as facial recognition data and voice data.

[0834] Through the user device interface, the user inputs date, time, location, desire, and emotional data. This data is encoded in JSON format and sent to the server as an HTTP POST request. The server receives this data and uses an emotional engine to analyze the user's emotional state. For example, it identifies whether the user is feeling stressed or seeking relaxation.

[0835] The server then uses the generative AI model to generate an action plan based on the user's desires and emotional state, including specific schedules such as "visit a hot spring at 09:00," "have lunch at a relaxation cafe at 12:00," and "take a walk in a quiet park at 15:00."

[0836] The generated action plan is sent from the server to the user's device and presented to the user. The user can review this action plan and request a reservation for a specific activity if necessary. For example, if a user wants to make a reservation for a hot spring, they press the reservation button, which sends a reservation request to the server. The server then calls the necessary API (e.g., a hot spring reservation API) to process the reservation. After the reservation is completed, the information is notified to the user's device.

[0837] By using this system, users can make effective use of their time while traveling in an autonomous vehicle and enjoy optimal action plans tailored to their emotions. A specific example is shown below.

[0838] Example 1:

[0839] If User A inputs a request such as "I want to relax in Kyoto on November 3, 2023," and the camera recognizes a stressed expression, the generative AI model will suggest the following action plan:

[0840] 09:00 Take a bath in the hot springs

[0841] 12:00 Lunch at the Relaxation Cafe

[0842] 15:00 Stroll around Momiji Park

[0843] Example 2:

[0844] If User B inputs a request such as "I want to spend time actively in Sapporo on December 25, 2023," and an excited state is recognized from the voice, the generative AI model will propose the following action plan:

[0845] 10:00 Visit the Snow Festival at Odori Park

[0846] 13:00 Experience sledding

[0847] 16:00 Visit an entertainment venue

[0848] In the above example, the following prompt sentence is passed to the generative AI model to generate an action plan:

[0849] Prompt statement example 1:

[0850] "Date: 2023-11-03, Location: Kyoto, Request: Relaxation, Emotion: Stress"

[0851] Prompt statement example 2:

[0852] "Date: 2023-12-25, Location: Sapporo, Request: Active, Emotion: Excitement"

[0853] In this way, an optimal action plan that matches the user's emotions is provided, thereby increasing the user's satisfaction.

[0854] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0855] Step 1:

[0856] The user inputs the date, time, location, and request through the user interface. In addition, the device uses a camera and microphone to collect emotional data (facial recognition data and voice data). This data is encoded into JSON format by the device.

[0857] Input: Date and time, location, request, emotional data (face recognition data, voice data)

[0858] Output: User data in JSON format

[0859] Step 2:

[0860] The device sends the encoded user data to the server as an HTTP POST request, which the server receives and begins parsing the data.

[0861] Input: User data in JSON format

[0862] Output: HTTP POST request to the server

[0863] Step 3:

[0864] The server analyzes the received user data and uses an emotion engine to identify the user's emotional state, using facial recognition and voice data to determine whether the user is feeling stressed or seeking relaxation.

[0865] Input: User data (face recognition data, voice data)

[0866] Output: User's emotional state (e.g., stressed, relaxed)

[0867] Step 4:

[0868] The server uses a generative AI model to generate an optimal action plan based on the analysis results (user's emotional state) and the user's request by the emotion engine. The server passes the prompt sentence to the generative AI model and obtains a specific action plan as its output.

[0869] Input: User's emotional state, desires, prompt

[0870] Output: Action plan

[0871] Step 5:

[0872] The server encodes the generated action plan and sends it to the user device in JSON format. An action plan might include, for example, "visit a hot spring at 09:00," "have lunch at a relaxation cafe at 12:00," and "take a walk in the park at 15:00."

[0873] Input: Action Plan

[0874] Output: Action plan in JSON format

[0875] Step 6:

[0876] The terminal displays the received action plan on the user interface. The user can check the plan and request a reservation for a specific activity. For example, if the user wants to make a reservation for a hot spring, the user presses the reservation button.

[0877] Input: Action plan in JSON format

[0878] Output: Display of action plan

[0879] Step 7:

[0880] A user's reservation request is sent from the terminal to the server. The server processes the reservation by calling the necessary API (e.g., hot spring reservation API). After the reservation is completed, the information is notified to the user terminal.

[0881] Input: Booking Request

[0882] Output: Reservation completion notification

[0883] Step 8:

[0884] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[0885] Input: Reservation completion notification

[0886] Output: Display the final action plan

[0887] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0888] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0889] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0890] [Third embodiment]

[0891] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0892] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0893] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0894] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0895] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0896] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0897] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0898] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0899] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0900] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0901] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0902] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0903] This invention is a system that proposes action plans to users so that they can make effective use of their holidays and spare time. This system is composed of a user terminal and a server, and is characterized by generating optimal action plans using a generative AI model.

[0904] System configuration

[0905] User terminal

[0906] The user terminal refers to a mobile device such as a smartphone or tablet, and allows users to input date, time, location, and requests through a user interface. The input information is then transmitted to the server.

[0907] server

[0908] The server receives information from the user and generates an action plan based on the generative AI model. The generated action plan is sent from the server to the user's device and presented to the user. The system also has a function to make reservations for facilities and events using reservation methods as needed.

[0909] Program processing

[0910] 1. User request input

[0911] Users input a date, time, location, and request through a user interface, such as "I want to do something summer-like in Tokyo on October 10, 2023."

[0912] 2. Sending data to the server

[0913] The device sends the entered information to the server as an HTTP POST request, with the data encoded in JSON format.

[0914] 3. Generating action plans using generative AI

[0915] The server receives the request and passes the data to a generative AI model, which then generates an optimal action plan based on the user's requests. For example, it generates a specific schedule such as "visit Meiji Shrine at 9:00, have lunch in Omotesando at 12:00, and swim at the Sunshine City Pool in Toshima Ward at 15:00."

[0916] 4. Submit your proposal

[0917] The server sends the generated action plan to the user's device, where it is again encoded in JSON format.

[0918] 5. Viewing and booking your action plan

[0919] The device displays the received action plan on the user interface. The user checks the displayed action plan and requests a reservation if necessary. For example, when reserving a restaurant for lunch, the device sends a reservation request to the server again, and the server calls the reservation API to complete the reservation.

[0920] 6. User Behavior

[0921] The user follows the proposed action plan and spends their time fulfillingly. This system is extremely convenient because it automatically provides the optimal plan according to the user's requests, eliminating the need for the user to think about it.

[0922] Specific examples

[0923] Example 1: If User A inputs a request such as "I want to eat something refreshing and visit a power spot in Tokyo on October 10, 2023," the generation AI will make the following suggestions.

[0924] 09:00 Visit Meiji Shrine

[0925] 12:00 Refreshing lunch at a cafe in Omotesando

[0926] 15:00 Hiking at Mount Takao

[0927] 18:00 Sightseeing at Tokyo Tower

[0928] Example 2: If User B inputs a request such as "I want to enjoy the snow and soak in a hot spring in Sapporo on December 25, 2023," the generation AI will make the following suggestions.

[0929] 10:00 Visit the Snow Festival at Odori Park

[0930] 13:00 Lunch at a restaurant in Susukino

[0931] 16:00 Relax at Jozankei Onsen

[0932] In this way, the present invention proposes an optimal plan of action based on the user's specific needs and eliminates the need for reservations, allowing the user to spend more quality time.

[0933] The processing flow will be explained below.

[0934] Step 1:

[0935] Users launch the application and input their date, time, location, and desires through the user interface, such as "I want to do something summer-like in Tokyo on October 10, 2023, and I want to go to a power spot."

[0936] Step 2:

[0937] The device collects the information entered by the user and encodes it into JSON format, which is then sent to the server as an HTTP POST request.

[0938] Step 3:

[0939] The server receives the HTTP POST request and parses the JSON data, extracting the date, time, location, and request.

[0940] Step 4:

[0941] The server passes the extracted date, time, location, and request data as input to the generative AI model, which then generates an action plan that matches the user's request.

[0942] Step 5:

[0943] The generative AI model outputs an action plan and returns the results to the server. For example, it generates a specific schedule such as "visit Meiji Shrine at 9:00, have lunch in Omotesando at 12:00, and swim at Sunshine City Pool in Toshima Ward at 15:00."

[0944] Step 6:

[0945] The server encodes the generated action plan again into JSON format and sends it to the original device as an HTTP response.

[0946] Step 7:

[0947] The device parses the JSON data received from the server and displays the action plan in the user interface, including detailed information about each activity.

[0948] Step 8:

[0949] The user reviews the proposed action plan and requests reservations for specific activities if necessary. For example, if the user wants to make a reservation for lunch, the user presses the reservation button.

[0950] Step 9:

[0951] The terminal sends the reservation request to the server again as an HTTP POST request.

[0952] Step 10:

[0953] The server receives the reservation request, calls the necessary API (e.g., restaurant reservation API) to process the reservation, and notifies the device of the reservation information after it is completed.

[0954] Step 11:

[0955] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[0956] Step 12:

[0957] The user follows the proposed plan of action and spends their time in a fulfilling manner. The device supports the user's actions by providing navigation and map links as needed.

[0958] Example 1

[0959] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0960] Conventional action plan generation systems require a lot of time and effort to process user-entered information, generate plans, and manage reservations, making it difficult to efficiently create action plans. It is also difficult to automatically provide customized plans that reflect the user's preferences and requests in detail. This has resulted in problems such as users being unable to create satisfactory plans and a lack of convenience.

[0961] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0962] In this invention, the server includes means for receiving requests from a user, input means for inputting the user's requests as date, time, location, and request information, means for transmitting the input data to the server, means for generating an action plan using a generative AI model, means for presenting the generated action plan to the user, means for making a reservation based on the generated action plan, and communication means for transmitting and receiving data between the server and a terminal. This makes it possible to efficiently and quickly provide an optimal action plan based on the information input by the user, and to automatically make a reservation based on that plan.

[0963] The "means for receiving requests from a user" refers to a device or software that collects and records the date, time, location, and individual request information provided by the user.

[0964] The "input means for inputting the user's requests as date, time, location, and request information" refers to a device or software that provides an interface that allows the user to specifically input his or her own action plans and preferences.

[0965] The "means for transmitting input data to a server" refers to a device or software that has a mechanism for transmitting information input from a user terminal to a server via communication.

[0966] A "means for generating an action plan using a generative AI model" is a device or software that uses an artificial intelligence algorithm to automatically create an optimal action plan based on data provided by a user.

[0967] The "means for presenting the generated action plan to the user" refers to a device or software that provides an interface that visualizes the generated action plan and presents it to the user, allowing the user to select the next step.

[0968] The "means for making reservations based on the generated action plan" is a device or software that automatically processes the reservation process of required resources and services based on the generated plan.

[0969] "Communication means for transmitting and receiving data between a server and a terminal" refers to a device or software that provides a communication protocol or network interface that enables data to be transferred between a server and a user terminal.

[0970] This invention is a system that proposes action plans to users so that they can make effective use of their holidays and spare time. This system is composed of a user terminal and a server, and is characterized by generating optimal action plans using a generative AI model.

[0971] User terminal

[0972] A user device refers to a mobile device such as a smartphone or tablet, and can input date, time, location, and requests through a user interface. A user uses an input form within the app to input their desired date, time, location, and specific requests. For example, if they input "I want to do something summery in Tokyo on October 10, 2023," the information is processed on the device and sent to the server.

[0973] server

[0974] The server receives the data from the device and generates an action plan based on the generative AI model. The generative AI model used automatically creates an optimal action plan based on the user's requests, taking into account past data and general information. The generated action plan is sent from the server to the user's device and presented to the user through a user interface.

[0975] Hardware and Software Use

[0976] The system uses the following hardware and software:

[0977] User devices: smartphones, tablets

[0978] Input method: Touch screen, keyboard

[0979] Communication method: Wi-Fi, 4G / 5G

[0980] Server: Cloud server, web server

[0981] Generative AI models: machine learning algorithms, natural language processing models

[0982] Data format: JSON

[0983] Specific examples

[0984] Specific examples are shown below.

[0985] Example 1: If User A inputs a request such as "I want to eat something refreshing and visit a power spot in Tokyo on October 10, 2023," the system will generate the following action plan:

[0986] 09:00 Visit Meiji Shrine

[0987] 12:00 Refreshing lunch at a cafe in Omotesando

[0988] 15:00 Hiking at Mount Takao

[0989] 18:00 Sightseeing at Tokyo Tower

[0990] Example 2: If User B inputs a request such as "I want to enjoy the snow and soak in hot springs in Sapporo on December 25, 2023," the system generates the following action plan:

[0991] 10:00 Visit the Snow Festival at Odori Park

[0992] 13:00 Lunch at a restaurant in Susukino

[0993] 16:00 Relax at Jozankei Onsen

[0994] Prompt Sentence Examples

[0995] "I want to eat something refreshing and visit a power spot in Tokyo on October 10, 2023."

[0996] In this way, the system of the present invention can efficiently and accurately generate an action plan based on the information entered by the user, and even automatically make reservations. By using this system, users can spend their time in a fulfilling manner.

[0997] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0998] Step 1:

[0999] User request input

[1000] Users launch the app on their smartphone or tablet and input the date, time, location, and requests through the user interface. For example, if a user inputs a request such as "I want to do something summer-like in Tokyo on October 10, 2023," that information is collected by the app and sent to the next step.

[1001] Input: User's date, time, location, and request information

[1002] Output: The input data (e.g., "I want to do something summery in Tokyo on October 10, 2023")

[1003] Step 2:

[1004] Sending data to the server

[1005] The device encodes the information entered by the user into JSON format (e.g., "{"date":"2023-10-10","location":"Tokyo","request":"I want to do something summery"}"). Next, it sends this encoded data to the server using an HTTP POST request. This is done over a communication network (Wi-Fi, 4G / 5G).

[1006] Input: Data entered by the user

[1007] Output: JSON data sent to the server

[1008] Step 3:

[1009] Generative AI generates action plans

[1010] The server decodes the JSON data received from the device. This decoded data is prepared as an input prompt for the generative AI model. An example prompt might be, "Please propose an action plan for doing summer-like things in Tokyo on October 10, 2023." The generative AI model generates an action plan based on this prompt. The generated plan includes specific time and activity suggestions.

[1011] Input: JSON data received by the server

[1012] Output: Generated action plan (e.g., "Visit Meiji Shrine at 09:00, have lunch at Omotesando at 12:00, and swim at Sunshine City Pool in Toshima Ward at 15:00")

[1013] Step 4:

[1014] Submit your proposal

[1015] The server re-encodes the generated action plan into JSON format (e.g., "{"schedule":"Visit Meiji Shrine at 09:00, have lunch at Omotesando at 12:00, swim at Sunshine City Pool in Toshima Ward at 15:00"}") and then sends this encoded data to the user's device.

[1016] Input: Generated action plan

[1017] Output: JSON data sent to the user's device

[1018] Step 5:

[1019] View and book your action plan

[1020] The device decodes the action plan received from the server from JSON data format and displays it on the user interface. For example, "Meiji Jingu Shrine at 9:00, lunch at Omotesando at 12:00, Sunshine City Pool at 15:00." The user confirms the displayed plan. If necessary, for example, to make a reservation at a restaurant for lunch at 12:00, the device sends a reservation request to the server again.

[1021] Input: JSON data received from the server

[1022] Output: Action plan displayed in the user interface, booking request from the user

[1023] Step 6:

[1024] Processing booking requests

[1025] The server processes the reservation request received from the device. It accesses the reservation API and completes the reservation of a specific resource or service. If the reservation is successful, it sends the result in JSON format to the user's device (e.g., "{"status":"success","message":"Lunch reservation completed."").

[1026] Input: Reservation request from user

[1027] Output: Reservation result (success or failure)

[1028] (Application example 1)

[1029] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1030] Conventional food delivery and restaurant reservation services lack the mechanisms to provide optimal suggestions based on specific user requests, requiring users to spend time searching for options on their own. They also struggle to efficiently plan meals based on specific dates, times, locations, and budgets. These challenges mean users end up spending a lot of time and effort creating their plans.

[1031] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1032] In this invention, the server includes means for receiving requests from a user, means for presenting a generated action plan to the user, generation AI means for generating an action plan based on the user's requests, means for making reservations based on the generated action plan, and means for inputting the type of meal, date and time, location, and budget range. This allows the user to automatically receive optimal meal plans and restaurants based on their detailed requests, enabling them to efficiently create an action plan.

[1033] "User" means an individual or corporation that uses the System to receive suggested activity plans and meal plans.

[1034] The "means for receiving requests" refers to an interface or communication means for inputting and receiving specific requests such as date and time, location, type of meal, budget range, etc. from the user.

[1035] The "means for presenting an action plan to a user" refers to a display means or communication means for presenting the generated action plan or meal plan to a user through a user interface.

[1036] "Generative AI means" refers to an artificial intelligence model and its processing system that automatically generates an action plan and optimal meal plan based on the user's requests.

[1037] "Means for making reservations based on an action plan" refers to API calls and communication methods for automatically making reservations at restaurants and facilities based on the generated action plan.

[1038] The "means for inputting the type of meal, date and time, location, and budget range" is a user interface for the user to input the type of meal, desired date and time, location, and budget range.

[1039] "User interface means" means an interactive display and operation means for a user to input requests and confirm proposed action plans.

[1040] "Communication means for transmitting and receiving data between a server and a terminal" refers to a network communication means and protocol for transmitting and receiving data between a server and a user terminal.

[1041] "Means for generating optimal restaurants and meal plans" refers to a processing system that uses an AI model to optimally select and generate restaurants and meal plans based on user requests.

[1042] "Means for displaying the meal plan on a user interface" refers to functionality and software for visually displaying the generated meal plan on the user's terminal.

[1043] The present invention is a system that uses a generative AI model to propose optimal action plans and meal plans based on the user's specific requests, such as the type of meal, date and time, location, and budget range, and makes reservations as needed. The system consists of a user terminal and a server.

[1044] User terminal

[1045] A user device refers to a mobile device such as a smartphone or tablet, and can input requests such as date and time, location, type of meal, and budget range through a user interface. The user device sends the input information to the server as an HTTP POST request. The data is encoded in JSON format.

[1046] server

[1047] The server receives information from the user and generates an optimal action plan or meal plan based on a generative AI model. The generated plan is sent from the server to the user's device and displayed on the device. It also has a function to make reservations at restaurants or facilities using a reservation API if necessary. The server uses a generative AI model (e.g., GPT-3 or GPT-4) to generate the optimal plan.

[1048] Program processing

[1049] Requests entered through the user interface are sent to the server. The server uses a generative AI model to generate an action plan and meal plan, which are then sent back to the user's device. The user can then review the received action plan and meal plan through the user interface and make reservations as necessary.

[1050] For example, if a user inputs a request such as "I want to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I would like a terrace seat," the generative AI will generate the following action plan:

[1051] Dine at the perfect Italian restaurant at 12:00

[1052] Dessert at a nearby cafe at 2:00 PM

[1053] Stroll around the park at 4:00 PM

[1054] Thus, we provide the generative AI model with a prompt of the following form:

[1055] I'd like to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I'd prefer a terrace seat.

[1056] Based on this, generate the best restaurant and action plan, including a schedule suggestion for 12 AM to 6 PM.

[1057] By using this invention, users can automatically receive optimal meal plans and restaurant recommendations based on their detailed requests, enabling them to plan their trips efficiently. In addition, the system is linked to a reservation system, allowing users to complete reservations hassle-free. This allows users to spend their time in a fulfilling manner.

[1058] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1059] Step 1:

[1060] User request input

[1061] The user inputs the date and time, location, type of meal, budget range, special requests, etc. through the user interface. For example, the user might input, "I would like to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I would like a terrace seat." The input data is encoded in JSON format. The input data is used to send the user's request to the server, and specific fields include the date and time (date), location (location), type of meal (meal_type), budget range (budget), and special requests (special_request).

[1062] Step 2:

[1063] Sending data to the server

[1064] The device sends the information entered in step 1 to the server as an HTTP POST request. At this time, the data is encoded in JSON format and sent to the server's API endpoint. The input data is in the format sent from the user's device to the server, for example, {"date": "2023-10-10", "location": "Tokyo", "meal_type": "Italian", "budget": "2000-5000 yen", "special_request": "Prefer terrace seat"}. In this way, the server receives the user's request.

[1065] Step 3:

[1066] Generative AI generates action plans

[1067] The server passes the user's request received in step 2 to a generative AI model. This generative AI model (e.g., GPT-3 or GPT-4) receives the user's request as input in the form of a prompt and generates an optimal action plan or meal plan. The specific prompt is as follows:

[1068] I'd like to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I'd prefer a terrace seat. Based on this, please generate the best restaurant and itinerary. Please also suggest a schedule between 12:00 AM and 6:00 PM.

[1069] The generative AI model analyzes this prompt and generates a specific action plan based on the user's request. The output data is generated in JSON format, for example:

[1070] {

[1071] "plan": [

[1072] {"time": "12:00", "activity": "Dinner at an Italian restaurant"},

[1073] {"time": "14:00", "activity": "Dessert at the cafe"},

[1074] {"time": "16:00", "activity": "Stroll around the park"}

[1075] ],

[1076] "reservation": {

[1077] "restaurant": "Italian Restaurant A",

[1078] "confirmation_number": "12345ABC"

[1079] }

[1080] }

[1081] Step 4:

[1082] Submitting and Viewing Proposals

[1083] The server sends the action plan generated in step 3 to the user's device. At this time, the generated plan is again encoded in JSON format. The user's device displays the received action plan on the user interface. The user can check the proposed action plan and make fine adjustments if necessary. This display process allows the user to visually understand the plan.

[1084] Step 5:

[1085] reservation

[1086] If the user agrees with the displayed plan of action, they request a reservation through the user interface. This request is sent back to the server. The server then calls the reservation API to reserve the necessary restaurant or facility. Specifically, it sends a request to the reservation API and receives reservation confirmation information. This reservation information is presented to the user, notifying them that the reservation has been completed. For example, when a restaurant reservation is completed, reservation confirmation information is sent to the user in the following JSON format:

[1087] {

[1088] "reservation": {

[1089] "restaurant": "Italian Restaurant A",

[1090] "confirmation_number": "12345ABC"

[1091] }

[1092] }

[1093] Through the above processing steps, the system of the present invention can provide optimal activity plans and meal plans based on the user's detailed requests and perform efficient reservation processing.

[1094] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1095] This invention is a system that proposes action plans to help users make effective use of their holidays and spare time, and by combining it with an emotion engine, provides optimal plans based on the user's emotions. This system consists of a user terminal and a server, and generates advanced action plans using the emotion engine and a generative AI model.

[1096] System configuration

[1097] User terminal

[1098] The user terminal refers to a mobile device such as a smartphone or tablet. This allows users to input date, time, location, requests, and emotional data (facial recognition data and voice data) through a user interface. The input information is then sent to the server.

[1099] server

[1100] The server receives information from the user and generates an action plan based on the emotion engine and generative AI model. The generated action plan is sent from the server to the user's device and presented to the user. The system also has a function to make reservations for facilities and events using reservation methods as needed.

[1101] Program processing

[1102] 1. User requests and emotion data input

[1103] A user launches the application and inputs the date, time, location, and desires through the user interface. In addition, the user provides emotional data (e.g., facial recognition through a camera or voice input through a microphone) to the system.

[1104] 2. Sending data to the server

[1105] The device encodes the information and emotion data entered by the user into JSON format and sends it to the server as an HTTP POST request.

[1106] 3. Emotional Data Processing and Analysis

[1107] The server passes the received emotion data to the emotion engine, which analyzes the user's emotions. For example, if the user is feeling stressed, this information is passed on to the generative AI model.

[1108] 4. Adjustment of action plans

[1109] The generative AI model generates an action plan based on data from the emotion engine, taking into account the user's desires and emotional state. For example, it suggests a relaxation plan for a stressed user.

[1110] 5. Generate a final action plan

[1111] The generative AI model then generates a new, adjusted action plan and returns it to the server. The generated action plan includes specific schedules such as "visit a hot spring at 9:00," "have lunch at a relaxing cafe at 12:00," and "take a walk in a nature park at 15:00."

[1112] 6. Submit your proposal

[1113] The server encodes the final action plan into JSON format and sends it to the originating device as an HTTP response.

[1114] 7. Viewing and booking your action plan

[1115] The terminal displays the received action plan on the user interface. The user checks the displayed action plan and requests a reservation for a specific activity if necessary. For example, if the user wants to make a reservation for a hot spring, the user presses the reservation button.

[1116] 8. Booking Processing

[1117] The device sends the reservation request back to the server as an HTTP POST request. The server receives the reservation request and processes the reservation by calling the necessary API (for example, a hot spring reservation API). After the reservation is completed, the information is notified to the device.

[1118] 9. Display of Final Action Plan

[1119] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[1120] Specific examples

[1121] Example 1: If User A inputs a request such as "I want to relax in Tokyo on October 10, 2023," and the camera recognizes a stressed expression, the generative AI will make the following suggestions:

[1122] 09:00 Take a bath in the hot springs

[1123] 12:00 Lunch at the Relaxation Cafe

[1124] 15:00 Walk in a quiet park in Tokyo

[1125] Example 2: If User B inputs a request such as "I want to spend an active time in Sapporo on December 25, 2023," and an excited state is recognized from the voice, the generation AI will make the following suggestions.

[1126] 10:00 Visit the Snow Festival at Odori Park

[1127] 13:00 Experience sledding

[1128] 16:00 Visit entertainment facilities in Susukino

[1129] As described above, the present invention proposes an action plan that takes into account the user's emotional state and also eliminates the need for reservations, thereby enabling the user to spend a more comfortable and fulfilling time.

[1130] The processing flow will be explained below.

[1131] Step 1:

[1132] A user launches the application and enters the date, time, location, and desire through a user interface, and also uses a camera and microphone to collect facial recognition data, voice data, and emotional data, including a specific request.

[1133] Step 2:

[1134] The device encodes the date, time, location, desire, and emotion data entered by the user into JSON format and sends this encoded data to the server as an HTTP POST request.

[1135] Step 3:

[1136] The server receives the HTTP POST request and parses the JSON data, extracting date, time, location, desire, and emotion data.

[1137] Step 4:

[1138] The server passes the extracted emotion data to the emotion engine, which analyzes the user's facial recognition data and voice data. Based on this data, the emotion engine identifies the user's emotional state, such as "feeling stressed" or "excited."

[1139] Step 5:

[1140] The server passes the emotional state data obtained from the emotion engine as input to the generative AI model. The generative AI model generates an optimal action plan taking into account the user's requests and emotional state. For example, it proposes a relaxing plan for a stressed user and an active plan for an excited user.

[1141] Step 6:

[1142] The generative AI model generates an action plan and returns the result to the server. The generated action plan includes a specific schedule, such as "visit a hot spring at 09:00," "have lunch at a relaxing cafe at 12:00," "take a walk in a nature park at 15:00," and "visit the hot spring again at 18:00."

[1143] Step 7:

[1144] The server encodes the generated action plan again into JSON format and sends it to the original device as an HTTP response.

[1145] Step 8:

[1146] The device parses the JSON data received from the server and displays the action plan in the user interface, including detailed information about each activity.

[1147] Step 9:

[1148] The user checks the proposed action plan and requests reservations for specific activities if necessary. For example, if the user wants to make a reservation for a hot spring, the user presses the reservation button.

[1149] Step 10:

[1150] The device sends the reservation request back to the server as an HTTP POST request. The server receives the reservation request and processes the reservation by calling the necessary API (for example, a hot spring reservation API). After the reservation is completed, the information is notified to the device.

[1151] Step 11:

[1152] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[1153] Step 12:

[1154] The user follows the proposed plan of action and spends their time in a fulfilling manner. The device supports the user's actions by providing navigation and map links as needed.

[1155] Example 2

[1156] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1157] Conventional behavior plan proposal systems generate behavior plans based on user requests, but do not consider the user's emotional state when proposing them. As a result, they are unable to provide optimal behavior plans for users when they are feeling stressed or in a specific emotional state, which can result in lower user satisfaction. Furthermore, making reservations based on the behavior plan must be done manually, which places a burden on the user.

[1158] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1159] In this invention, the server includes means for receiving requests and emotion data from a user, means for transmitting the received requests and emotion data to the server, emotion engine means for analyzing the emotion data, generation AI means for generating an action plan based on the data obtained from the emotion engine means, means for transmitting the generated action plan to a user terminal, means for displaying the generated action plan on a user interface, and means for receiving a reservation request from a user and processing the reservation. This makes it possible to automatically generate an optimal action plan that takes into account the user's emotional state, and further automate reservations based on the action plan.

[1160] The "means for receiving request and emotion data from a user" refers to a device or software function for receiving and storing the date, time, location, request and emotion data input by a user.

[1161] The "means for transmitting the received request and emotion data to the server" refers to a protocol or communication device for transmitting the request and emotion data from the user terminal to the server.

[1162] The "emotion engine means for analyzing emotion data" refers to an algorithm or software module for analyzing received emotion data and identifying the user's emotional state.

[1163] The "generative AI means for generating an action plan based on data obtained from the emotion engine means" refers to an artificial intelligence algorithm or software module for generating an optimal action plan based on the emotional state analyzed by the emotion engine and the user's requests.

[1164] "Means for transmitting the generated action plan to the user terminal" refers to a communication means or protocol for transmitting the action plan generated by the generation AI means to the user terminal.

[1165] The "means for displaying the generated action plan on a user interface" refers to a screen display device or software interface for displaying the action plan generated on the user terminal in a format that is easy for the user to view.

[1166] "Means for receiving user reservation requests and processing reservations" refers to the protocols and operational means for receiving a user's request to reserve a specific activity and calling the appropriate reservation service or API to fulfill the reservation.

[1167] This invention is a system that proposes action plans to help users make effective use of their holidays and spare time, and further provides optimal plans based on the user's emotions by combining an emotion engine. This system consists of a user terminal and a server, and generates advanced action plans using the emotion engine and a generative AI model.

[1168] User terminal

[1169] The user terminal refers to a mobile device such as a smartphone or tablet. This allows users to input date, time, location, requests, and emotional data (facial recognition data and voice data) through a user interface. This information is then sent from the terminal to the server.

[1170] server

[1171] The server receives information from the user and generates an action plan based on the emotion engine and generative AI model. The generated action plan is sent from the server to the user's device. The system also has a function to make reservations for facilities and events using reservation methods as needed.

[1172] Emotion Engine

[1173] The emotion engine is an algorithm that analyzes a user's facial recognition data and voice data to identify the user's emotional state, thereby determining whether the user is in a particular emotional state, such as stress or excitement.

[1174] Generative AI Models

[1175] The generative AI model generates an action plan based on data from the emotion engine, taking into account the user's desires and emotional state. For example, if the user is stressed, it will generate a relaxation plan, and if the user is excited, it will generate an active plan.

[1176] Specific examples

[1177] Example 1:

[1178] If a user inputs a request such as "I want to relax in Tokyo on October 10, 2023," and the camera recognizes a stressed facial expression, the generative AI model will suggest the following action plan:

[1179] 09:00 Take a bath in the hot springs

[1180] 12:00 Lunch at the Relaxation Cafe

[1181] 15:00 Walk in a quiet park in Tokyo

[1182] Example 2:

[1183] If a user inputs a request such as "I want to spend some time actively in Sapporo on December 25, 2023," and an excited state is recognized from the voice, the generative AI model will suggest the following action plan:

[1184] 10:00 Visit the Snow Festival at Odori Park

[1185] 13:00 Experience sledding

[1186] 16:00 Visit entertainment facilities in Susukino

[1187] Prompt Sentence Examples

[1188] Example prompt 1:

[1189] "I would like to relax in Tokyo on October 10, 2023. I am feeling stressed, so please tell me the best plan of action for relaxation."

[1190] Example prompt 2:

[1191] "I'm planning an active day in Sapporo on December 25, 2023. I'm excited, so please tell me about your plans, focusing on energetic activities."

[1192] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1193] Step 1:

[1194] Users launch an application on their smartphone or tablet, input the date, time, location, and request, and then use the camera to collect facial recognition data and the microphone to provide voice data.

[1195] The inputs are "date, time, location, request, facial recognition data, and voice data."

[1196] The output is "information and emotional data entered by the user."

[1197] Step 2:

[1198] The device encodes the received information and emotion data into JSON format and sends it to the server as an HTTP POST request.

[1199] The input is "information and emotional data entered by the user."

[1200] The output is the "encoded data".

[1201] Specifically, the data is encoded and an HTTP request is sent.

[1202] Step 3:

[1203] The server passes the received data to the emotion engine for analysis.

[1204] The input is the "encoded data".

[1205] The output is "data to be passed to the emotion engine."

[1206] Specific operations include decoding the data and inputting it into the emotion engine.

[1207] Step 4:

[1208] The emotion engine analyzes the facial recognition data and voice data to identify the user's emotional state, for example, "stress."

[1209] The input is "data passed to the emotion engine."

[1210] The output is the "analyzed emotional state" information.

[1211] Specifically, facial recognition and voice analysis algorithms are executed.

[1212] Step 5:

[1213] The server passes the emotional state obtained from the emotion engine to the generative AI model and instructs it to generate an action plan.

[1214] The input is the "analyzed emotional state" information.

[1215] The output is a prompt to the generative AI model.

[1216] Specific operations include generating prompts and inputting them into the generative AI model.

[1217] Step 6:

[1218] The generative AI model generates an action plan taking into account the emotional state and the user's wishes, such as "visit a hot spring at 09:00," "have lunch at a relaxing cafe at 12:00," and "take a walk in a nature park at 15:00."

[1219] The input is a "prompt to the generative AI model."

[1220] The output is a "generated action plan."

[1221] Specifically, the generative AI model calculates an action plan using an internal algorithm.

[1222] Step 7:

[1223] The server encodes the generated action plan into JSON format and sends it to the terminal as an HTTP response.

[1224] The input is the "generated action plan."

[1225] The output is "encoded action plan data."

[1226] Specifically, the data is encoded and an HTTP response is sent.

[1227] Step 8:

[1228] The terminal decodes the received action plan data and displays it on the user interface.

[1229] The input is "encoded action plan data."

[1230] The output is an action plan for display in the user interface.

[1231] As a specific operation, the data is decoded and displayed on the screen.

[1232] Step 9:

[1233] The user checks the displayed action plan and requests a reservation for a specific activity, for example, a reservation for a hot spring.

[1234] The input is the displayed action plan and booking request.

[1235] The output is "reservation request data."

[1236] As a specific operation, a request is generated and sent.

[1237] Step 10:

[1238] The terminal sends the generated reservation request to the server as an HTTP POST request.

[1239] The input is "reservation request data."

[1240] The output is "Reservation request data sent."

[1241] As a specific operation, the request data is encoded and transmitted.

[1242] Step 11:

[1243] The server receives the reservation request and processes the reservation by calling the necessary API. For example, it calls a hot spring reservation API to process the reservation. After the reservation is completed, it notifies the terminal of the information.

[1244] The input is "reservation request data sent."

[1245] The output is "reservation processing results and notification data."

[1246] Specifically, the reservation API is called and the results are obtained.

[1247] Step 12:

[1248] The terminal again displays the final action plan including the reservation information on the user interface.

[1249] The input is "reservation processing results and notification data."

[1250] The output is a "final action plan for display in the user interface."

[1251] As a specific operation, the data is decoded and displayed on the screen.

[1252] (Application example 2)

[1253] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1254] Conventional behavior plan generation systems only provide plans based on the user's requests, but are unable to provide plans that appropriately reflect the user's individual emotional state. This makes it difficult to propose an optimal behavior plan tailored to the user's emotional state. Furthermore, particularly in self-driving vehicles, behavior plans that allow users to use their time effectively while traveling are not effectively generated or executed. This makes it difficult to provide a highly satisfying experience for users.

[1255] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving request and emotion data from the user, means for presenting the generated action plan to the user, an emotion engine and generation AI means for generating an action plan based on the user's request and emotion data, means for making a reservation based on the generated action plan, and means for displaying the generated action plan on the interface of the autonomously driven vehicle. This makes it possible to propose an action plan that matches the user's request and emotional state within the autonomously driven vehicle, making effective use of time while traveling and increasing user satisfaction.

[1256] "Requests" refer to the hopes and demands that users have regarding the action plan.

[1257] "Emotion data" is data that indicates the user's emotional state, and includes facial recognition data, voice data, and the like.

[1258] An "action plan" refers to a specific schedule of activities generated based on the user's requests and emotional data.

[1259] An "emotion engine" refers to software that analyzes a user's emotional data and identifies their emotional state.

[1260] "Generative AI" is an artificial intelligence model that generates optimal action plans based on user requests and emotional data.

[1261] "User interface" refers to the interface through which a user inputs requests and emotional data into the system.

[1262] "Server" refers to a computer system that receives data from users and is responsible for generating action plans and processing reservations.

[1263] A "terminal" is a device through which a user accesses the system, including smartphones, tablets, and autonomous vehicle interfaces.

[1264] "Communication means" refers to the technology and protocols used to send and receive data between a server and a terminal.

[1265] "Reservation means" refers to a system or process for making reservations for facilities or events based on the generated action plan.

[1266] To implement this invention, a user terminal (e.g., a smartphone, tablet, or autonomous vehicle interface) and a server are required. The user terminal provides a user interface for inputting date, time, location, and requests. In addition, the user terminal uses a camera and microphone to collect emotion data such as facial recognition data and voice data.

[1267] Through the user device interface, the user inputs date, time, location, desire, and emotional data. This data is encoded in JSON format and sent to the server as an HTTP POST request. The server receives this data and uses an emotional engine to analyze the user's emotional state. For example, it identifies whether the user is feeling stressed or seeking relaxation.

[1268] The server then uses the generative AI model to generate an action plan based on the user's desires and emotional state, including specific schedules such as "visit a hot spring at 09:00," "have lunch at a relaxation cafe at 12:00," and "take a walk in a quiet park at 15:00."

[1269] The generated action plan is sent from the server to the user's device and presented to the user. The user can review this action plan and request a reservation for a specific activity if necessary. For example, if a user wants to make a reservation for a hot spring, they press the reservation button, which sends a reservation request to the server. The server then calls the necessary API (e.g., a hot spring reservation API) to process the reservation. After the reservation is completed, the information is notified to the user's device.

[1270] By using this system, users can make effective use of their time while traveling in an autonomous vehicle and enjoy optimal action plans tailored to their emotions. A specific example is shown below.

[1271] Example 1:

[1272] If User A inputs a request such as "I want to relax in Kyoto on November 3, 2023," and the camera recognizes a stressed expression, the generative AI model will suggest the following action plan:

[1273] 09:00 Take a bath in the hot springs

[1274] 12:00 Lunch at the Relaxation Cafe

[1275] 15:00 Stroll around Momiji Park

[1276] Example 2:

[1277] If User B inputs a request such as "I want to spend time actively in Sapporo on December 25, 2023," and an excited state is recognized from the voice, the generative AI model will propose the following action plan:

[1278] 10:00 Visit the Snow Festival at Odori Park

[1279] 13:00 Experience sledding

[1280] 16:00 Visit an entertainment venue

[1281] In the above example, the following prompt sentence is passed to the generative AI model to generate an action plan:

[1282] Prompt statement example 1:

[1283] "Date: 2023-11-03, Location: Kyoto, Request: Relaxation, Emotion: Stress"

[1284] Prompt statement example 2:

[1285] "Date: 2023-12-25, Location: Sapporo, Request: Active, Emotion: Excitement"

[1286] In this way, an optimal action plan that matches the user's emotions is provided, thereby increasing the user's satisfaction.

[1287] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1288] Step 1:

[1289] The user inputs the date, time, location, and request through the user interface. In addition, the device uses a camera and microphone to collect emotional data (facial recognition data and voice data). This data is encoded into JSON format by the device.

[1290] Input: Date and time, location, request, emotional data (face recognition data, voice data)

[1291] Output: User data in JSON format

[1292] Step 2:

[1293] The device sends the encoded user data to the server as an HTTP POST request, which the server receives and begins parsing the data.

[1294] Input: User data in JSON format

[1295] Output: HTTP POST request to the server

[1296] Step 3:

[1297] The server analyzes the received user data and uses an emotion engine to identify the user's emotional state, using facial recognition and voice data to determine whether the user is feeling stressed or seeking relaxation.

[1298] Input: User data (face recognition data, voice data)

[1299] Output: User's emotional state (e.g., stressed, relaxed)

[1300] Step 4:

[1301] The server uses a generative AI model to generate an optimal action plan based on the analysis results (user's emotional state) and the user's request by the emotion engine. The server passes the prompt sentence to the generative AI model and obtains a specific action plan as its output.

[1302] Input: User's emotional state, desires, prompt

[1303] Output: Action plan

[1304] Step 5:

[1305] The server encodes the generated action plan and sends it to the user device in JSON format. An action plan might include, for example, "visit a hot spring at 09:00," "have lunch at a relaxation cafe at 12:00," and "take a walk in the park at 15:00."

[1306] Input: Action Plan

[1307] Output: Action plan in JSON format

[1308] Step 6:

[1309] The terminal displays the received action plan on the user interface. The user can check the plan and request a reservation for a specific activity. For example, if the user wants to make a reservation for a hot spring, the user presses the reservation button.

[1310] Input: Action plan in JSON format

[1311] Output: Display of action plan

[1312] Step 7:

[1313] A user's reservation request is sent from the terminal to the server. The server processes the reservation by calling the necessary API (e.g., hot spring reservation API). After the reservation is completed, the information is notified to the user terminal.

[1314] Input: Booking Request

[1315] Output: Reservation completion notification

[1316] Step 8:

[1317] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[1318] Input: Reservation completion notification

[1319] Output: Display the final action plan

[1320] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1321] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1322] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1323] [Fourth embodiment]

[1324] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1325] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1326] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1327] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1328] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1329] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1330] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1331] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1332] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1333] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1334] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1335] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1336] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1337] This invention is a system that proposes action plans to users so that they can make effective use of their holidays and spare time. This system is composed of a user terminal and a server, and is characterized by generating optimal action plans using a generative AI model.

[1338] System configuration

[1339] User terminal

[1340] The user terminal refers to a mobile device such as a smartphone or tablet, and allows users to input date, time, location, and requests through a user interface. The input information is then transmitted to the server.

[1341] server

[1342] The server receives information from the user and generates an action plan based on the generative AI model. The generated action plan is sent from the server to the user's device and presented to the user. The system also has a function to make reservations for facilities and events using reservation methods as needed.

[1343] Program processing

[1344] 1. User request input

[1345] Users input a date, time, location, and request through a user interface, such as "I want to do something summer-like in Tokyo on October 10, 2023."

[1346] 2. Sending data to the server

[1347] The device sends the entered information to the server as an HTTP POST request, with the data encoded in JSON format.

[1348] 3. Generating action plans using generative AI

[1349] The server receives the request and passes the data to a generative AI model, which then generates an optimal action plan based on the user's requests. For example, it generates a specific schedule such as "visit Meiji Shrine at 9:00, have lunch in Omotesando at 12:00, and swim at the Sunshine City Pool in Toshima Ward at 15:00."

[1350] 4. Submit your proposal

[1351] The server sends the generated action plan to the user's device, where it is again encoded in JSON format.

[1352] 5. Viewing and booking your action plan

[1353] The device displays the received action plan on the user interface. The user checks the displayed action plan and requests a reservation if necessary. For example, when reserving a restaurant for lunch, the device sends a reservation request to the server again, and the server calls the reservation API to complete the reservation.

[1354] 6. User Behavior

[1355] The user follows the proposed action plan and spends their time fulfillingly. This system is extremely convenient because it automatically provides the optimal plan according to the user's requests, eliminating the need for the user to think about it.

[1356] Specific examples

[1357] Example 1: If User A inputs a request such as "I want to eat something refreshing and visit a power spot in Tokyo on October 10, 2023," the generation AI will make the following suggestions.

[1358] 09:00 Visit Meiji Shrine

[1359] 12:00 Refreshing lunch at a cafe in Omotesando

[1360] 15:00 Hiking at Mount Takao

[1361] 18:00 Sightseeing at Tokyo Tower

[1362] Example 2: If User B inputs a request such as "I want to enjoy the snow and soak in a hot spring in Sapporo on December 25, 2023," the generation AI will make the following suggestions.

[1363] 10:00 Visit the Snow Festival at Odori Park

[1364] 13:00 Lunch at a restaurant in Susukino

[1365] 16:00 Relax at Jozankei Onsen

[1366] In this way, the present invention proposes an optimal plan of action based on the user's specific needs and eliminates the need for reservations, allowing the user to spend more quality time.

[1367] The processing flow will be explained below.

[1368] Step 1:

[1369] Users launch the application and input their date, time, location, and desires through the user interface, such as "I want to do something summer-like in Tokyo on October 10, 2023, and I want to go to a power spot."

[1370] Step 2:

[1371] The device collects the information entered by the user and encodes it into JSON format, which is then sent to the server as an HTTP POST request.

[1372] Step 3:

[1373] The server receives the HTTP POST request and parses the JSON data, extracting the date, time, location, and request.

[1374] Step 4:

[1375] The server passes the extracted date, time, location, and request data as input to the generative AI model, which then generates an action plan that matches the user's request.

[1376] Step 5:

[1377] The generative AI model outputs an action plan and returns the results to the server. For example, it generates a specific schedule such as "visit Meiji Shrine at 9:00, have lunch in Omotesando at 12:00, and swim at Sunshine City Pool in Toshima Ward at 15:00."

[1378] Step 6:

[1379] The server encodes the generated action plan again into JSON format and sends it to the original device as an HTTP response.

[1380] Step 7:

[1381] The device parses the JSON data received from the server and displays the action plan in the user interface, including detailed information about each activity.

[1382] Step 8:

[1383] The user reviews the proposed action plan and requests reservations for specific activities if necessary. For example, if the user wants to make a reservation for lunch, the user presses the reservation button.

[1384] Step 9:

[1385] The terminal sends the reservation request to the server again as an HTTP POST request.

[1386] Step 10:

[1387] The server receives the reservation request, calls the necessary API (e.g., restaurant reservation API) to process the reservation, and notifies the device of the reservation information after it is completed.

[1388] Step 11:

[1389] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[1390] Step 12:

[1391] The user follows the proposed plan of action and spends their time in a fulfilling manner. The device supports the user's actions by providing navigation and map links as needed.

[1392] Example 1

[1393] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1394] Conventional action plan generation systems require a lot of time and effort to process user-entered information, generate plans, and manage reservations, making it difficult to efficiently create action plans. It is also difficult to automatically provide customized plans that reflect the user's preferences and requests in detail. This has resulted in problems such as users being unable to create satisfactory plans and a lack of convenience.

[1395] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1396] In this invention, the server includes means for receiving requests from a user, input means for inputting the user's requests as date, time, location, and request information, means for transmitting the input data to the server, means for generating an action plan using a generative AI model, means for presenting the generated action plan to the user, means for making a reservation based on the generated action plan, and communication means for transmitting and receiving data between the server and a terminal. This makes it possible to efficiently and quickly provide an optimal action plan based on the information input by the user, and to automatically make a reservation based on that plan.

[1397] The "means for receiving requests from a user" refers to a device or software that collects and records the date, time, location, and individual request information provided by the user.

[1398] The "input means for inputting the user's requests as date, time, location, and request information" refers to a device or software that provides an interface that allows the user to specifically input his or her own action plans and preferences.

[1399] The "means for transmitting input data to a server" refers to a device or software that has a mechanism for transmitting information input from a user terminal to a server via communication.

[1400] A "means for generating an action plan using a generative AI model" is a device or software that uses an artificial intelligence algorithm to automatically create an optimal action plan based on data provided by a user.

[1401] The "means for presenting the generated action plan to the user" refers to a device or software that provides an interface that visualizes the generated action plan and presents it to the user, allowing the user to select the next step.

[1402] The "means for making reservations based on the generated action plan" is a device or software that automatically processes the reservation process of required resources and services based on the generated plan.

[1403] "Communication means for transmitting and receiving data between a server and a terminal" refers to a device or software that provides a communication protocol or network interface that enables data to be transferred between a server and a user terminal.

[1404] This invention is a system that proposes action plans to users so that they can make effective use of their holidays and spare time. This system is composed of a user terminal and a server, and is characterized by generating optimal action plans using a generative AI model.

[1405] User terminal

[1406] A user device refers to a mobile device such as a smartphone or tablet, and can input date, time, location, and requests through a user interface. A user uses an input form within the app to input their desired date, time, location, and specific requests. For example, if they input "I want to do something summery in Tokyo on October 10, 2023," the information is processed on the device and sent to the server.

[1407] server

[1408] The server receives the data from the device and generates an action plan based on the generative AI model. The generative AI model used automatically creates an optimal action plan based on the user's requests, taking into account past data and general information. The generated action plan is sent from the server to the user's device and presented to the user through a user interface.

[1409] Hardware and Software Use

[1410] The system uses the following hardware and software:

[1411] User devices: smartphones, tablets

[1412] Input method: Touch screen, keyboard

[1413] Communication method: Wi-Fi, 4G / 5G

[1414] Server: Cloud server, web server

[1415] Generative AI models: machine learning algorithms, natural language processing models

[1416] Data format: JSON

[1417] Specific examples

[1418] Specific examples are shown below.

[1419] Example 1: If User A inputs a request such as "I want to eat something refreshing and visit a power spot in Tokyo on October 10, 2023," the system will generate the following action plan:

[1420] 09:00 Visit Meiji Shrine

[1421] 12:00 Refreshing lunch at a cafe in Omotesando

[1422] 15:00 Hiking at Mount Takao

[1423] 18:00 Sightseeing at Tokyo Tower

[1424] Example 2: If User B inputs a request such as "I want to enjoy the snow and soak in hot springs in Sapporo on December 25, 2023," the system generates the following action plan:

[1425] 10:00 Visit the Snow Festival at Odori Park

[1426] 13:00 Lunch at a restaurant in Susukino

[1427] 16:00 Relax at Jozankei Onsen

[1428] Prompt Sentence Examples

[1429] "I want to eat something refreshing and visit a power spot in Tokyo on October 10, 2023."

[1430] In this way, the system of the present invention can efficiently and accurately generate an action plan based on the information entered by the user, and even automatically make reservations. By using this system, users can spend their time in a fulfilling manner.

[1431] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1432] Step 1:

[1433] User request input

[1434] Users launch the app on their smartphone or tablet and input the date, time, location, and requests through the user interface. For example, if a user inputs a request such as "I want to do something summer-like in Tokyo on October 10, 2023," that information is collected by the app and sent to the next step.

[1435] Input: User's date, time, location, and request information

[1436] Output: The input data (e.g., "I want to do something summery in Tokyo on October 10, 2023")

[1437] Step 2:

[1438] Sending data to the server

[1439] The device encodes the information entered by the user into JSON format (e.g., "{"date":"2023-10-10","location":"Tokyo","request":"I want to do something summery"}"). Next, it sends this encoded data to the server using an HTTP POST request. This is done over a communication network (Wi-Fi, 4G / 5G).

[1440] Input: Data entered by the user

[1441] Output: JSON data sent to the server

[1442] Step 3:

[1443] Generative AI generates action plans

[1444] The server decodes the JSON data received from the device. This decoded data is prepared as an input prompt for the generative AI model. An example prompt might be, "Please propose an action plan for doing summer-like things in Tokyo on October 10, 2023." The generative AI model generates an action plan based on this prompt. The generated plan includes specific time and activity suggestions.

[1445] Input: JSON data received by the server

[1446] Output: Generated action plan (e.g., "Visit Meiji Shrine at 09:00, have lunch at Omotesando at 12:00, and swim at Sunshine City Pool in Toshima Ward at 15:00")

[1447] Step 4:

[1448] Submit your proposal

[1449] The server re-encodes the generated action plan into JSON format (e.g., "{"schedule":"Visit Meiji Shrine at 09:00, have lunch at Omotesando at 12:00, swim at Sunshine City Pool in Toshima Ward at 15:00"}") and then sends this encoded data to the user's device.

[1450] Input: Generated action plan

[1451] Output: JSON data sent to the user's device

[1452] Step 5:

[1453] View and book your action plan

[1454] The device decodes the action plan received from the server from JSON data format and displays it on the user interface. For example, "Meiji Jingu Shrine at 9:00, lunch at Omotesando at 12:00, Sunshine City Pool at 15:00." The user confirms the displayed plan. If necessary, for example, to make a reservation at a restaurant for lunch at 12:00, the device sends a reservation request to the server again.

[1455] Input: JSON data received from the server

[1456] Output: Action plan displayed in the user interface, booking request from the user

[1457] Step 6:

[1458] Processing booking requests

[1459] The server processes the reservation request received from the device. It accesses the reservation API and completes the reservation of a specific resource or service. If the reservation is successful, it sends the result in JSON format to the user's device (e.g., "{"status":"success","message":"Lunch reservation completed."").

[1460] Input: Reservation request from user

[1461] Output: Reservation result (success or failure)

[1462] (Application example 1)

[1463] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1464] Conventional food delivery and restaurant reservation services lack the mechanisms to provide optimal suggestions based on specific user requests, requiring users to spend time searching for options on their own. They also struggle to efficiently plan meals based on specific dates, times, locations, and budgets. These challenges mean users end up spending a lot of time and effort creating their plans.

[1465] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1466] In this invention, the server includes means for receiving requests from a user, means for presenting a generated action plan to the user, generation AI means for generating an action plan based on the user's requests, means for making reservations based on the generated action plan, and means for inputting the type of meal, date and time, location, and budget range. This allows the user to automatically receive optimal meal plans and restaurants based on their detailed requests, enabling them to efficiently create an action plan.

[1467] "User" means an individual or corporation that uses the System to receive suggested activity plans and meal plans.

[1468] The "means for receiving requests" refers to an interface or communication means for inputting and receiving specific requests such as date and time, location, type of meal, budget range, etc. from the user.

[1469] The "means for presenting an action plan to a user" refers to a display means or communication means for presenting the generated action plan or meal plan to a user through a user interface.

[1470] "Generative AI means" refers to an artificial intelligence model and its processing system that automatically generates an action plan and optimal meal plan based on the user's requests.

[1471] "Means for making reservations based on an action plan" refers to API calls and communication methods for automatically making reservations at restaurants and facilities based on the generated action plan.

[1472] The "means for inputting the type of meal, date and time, location, and budget range" is a user interface for the user to input the type of meal, desired date and time, location, and budget range.

[1473] "User interface means" means an interactive display and operation means for a user to input requests and confirm proposed action plans.

[1474] "Communication means for transmitting and receiving data between a server and a terminal" refers to a network communication means and protocol for transmitting and receiving data between a server and a user terminal.

[1475] "Means for generating optimal restaurants and meal plans" refers to a processing system that uses an AI model to optimally select and generate restaurants and meal plans based on user requests.

[1476] "Means for displaying the meal plan on a user interface" refers to functionality and software for visually displaying the generated meal plan on the user's terminal.

[1477] The present invention is a system that uses a generative AI model to propose optimal action plans and meal plans based on the user's specific requests, such as the type of meal, date and time, location, and budget range, and makes reservations as needed. The system consists of a user terminal and a server.

[1478] User terminal

[1479] A user device refers to a mobile device such as a smartphone or tablet, and can input requests such as date and time, location, type of meal, and budget range through a user interface. The user device sends the input information to the server as an HTTP POST request. The data is encoded in JSON format.

[1480] server

[1481] The server receives information from the user and generates an optimal action plan or meal plan based on a generative AI model. The generated plan is sent from the server to the user's device and displayed on the device. It also has a function to make reservations at restaurants or facilities using a reservation API if necessary. The server uses a generative AI model (e.g., GPT-3 or GPT-4) to generate the optimal plan.

[1482] Program processing

[1483] Requests entered through the user interface are sent to the server. The server uses a generative AI model to generate an action plan and meal plan, which are then sent back to the user's device. The user can then review the received action plan and meal plan through the user interface and make reservations as necessary.

[1484] For example, if a user inputs a request such as "I want to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I would like a terrace seat," the generative AI will generate the following action plan:

[1485] Dine at the perfect Italian restaurant at 12:00

[1486] Dessert at a nearby cafe at 2:00 PM

[1487] Stroll around the park at 4:00 PM

[1488] Thus, we provide the generative AI model with a prompt of the following form:

[1489] I'd like to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I'd prefer a terrace seat.

[1490] Based on this, generate the best restaurant and action plan, including a schedule suggestion for 12 AM to 6 PM.

[1491] By using this invention, users can automatically receive optimal meal plans and restaurant recommendations based on their detailed requests, enabling them to plan their trips efficiently. In addition, the system is linked to a reservation system, allowing users to complete reservations hassle-free. This allows users to spend their time in a fulfilling manner.

[1492] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1493] Step 1:

[1494] User request input

[1495] The user inputs the date and time, location, type of meal, budget range, special requests, etc. through the user interface. For example, the user might input, "I would like to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I would like a terrace seat." The input data is encoded in JSON format. The input data is used to send the user's request to the server, and specific fields include the date and time (date), location (location), type of meal (meal_type), budget range (budget), and special requests (special_request).

[1496] Step 2:

[1497] Sending data to the server

[1498] The device sends the information entered in step 1 to the server as an HTTP POST request. At this time, the data is encoded in JSON format and sent to the server's API endpoint. The input data is in the format sent from the user's device to the server, for example, {"date": "2023-10-10", "location": "Tokyo", "meal_type": "Italian", "budget": "2000-5000 yen", "special_request": "Prefer terrace seat"}. In this way, the server receives the user's request.

[1499] Step 3:

[1500] Generative AI generates action plans

[1501] The server passes the user's request received in step 2 to a generative AI model. This generative AI model (e.g., GPT-3 or GPT-4) receives the user's request as input in the form of a prompt and generates an optimal action plan or meal plan. The specific prompt is as follows:

[1502] I'd like to eat Italian food in Tokyo on October 10, 2023. My budget is between 2,000 and 5,000 yen, and I'd prefer a terrace seat. Based on this, please generate the best restaurant and itinerary. Please also suggest a schedule between 12:00 AM and 6:00 PM.

[1503] The generative AI model analyzes this prompt and generates a specific action plan based on the user's request. The output data is generated in JSON format, for example:

[1504] {

[1505] "plan": [

[1506] {"time": "12:00", "activity": "Dinner at an Italian restaurant"},

[1507] {"time": "14:00", "activity": "Dessert at the cafe"},

[1508] {"time": "16:00", "activity": "Stroll around the park"}

[1509] ],

[1510] "reservation": {

[1511] "restaurant": "Italian Restaurant A",

[1512] "confirmation_number": "12345ABC"

[1513] }

[1514] }

[1515] Step 4:

[1516] Submitting and Viewing Proposals

[1517] The server sends the action plan generated in step 3 to the user's device. At this time, the generated plan is again encoded in JSON format. The user's device displays the received action plan on the user interface. The user can check the proposed action plan and make fine adjustments if necessary. This display process allows the user to visually understand the plan.

[1518] Step 5:

[1519] reservation

[1520] If the user agrees with the displayed plan of action, they request a reservation through the user interface. This request is sent back to the server. The server then calls the reservation API to reserve the necessary restaurant or facility. Specifically, it sends a request to the reservation API and receives reservation confirmation information. This reservation information is presented to the user, notifying them that the reservation has been completed. For example, when a restaurant reservation is completed, reservation confirmation information is sent to the user in the following JSON format:

[1521] {

[1522] "reservation": {

[1523] "restaurant": "Italian Restaurant A",

[1524] "confirmation_number": "12345ABC"

[1525] }

[1526] }

[1527] Through the above processing steps, the system of the present invention can provide optimal activity plans and meal plans based on the user's detailed requests and perform efficient reservation processing.

[1528] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1529] This invention is a system that proposes action plans to help users make effective use of their holidays and spare time, and by combining it with an emotion engine, provides optimal plans based on the user's emotions. This system consists of a user terminal and a server, and generates advanced action plans using the emotion engine and a generative AI model.

[1530] System configuration

[1531] User terminal

[1532] The user terminal refers to a mobile device such as a smartphone or tablet. This allows users to input date, time, location, requests, and emotional data (facial recognition data and voice data) through a user interface. The input information is then sent to the server.

[1533] server

[1534] The server receives information from the user and generates an action plan based on the emotion engine and generative AI model. The generated action plan is sent from the server to the user's device and presented to the user. The system also has a function to make reservations for facilities and events using reservation methods as needed.

[1535] Program processing

[1536] 1. User requests and emotion data input

[1537] A user launches the application and inputs the date, time, location, and desires through the user interface. In addition, the user provides emotional data (e.g., facial recognition through a camera or voice input through a microphone) to the system.

[1538] 2. Sending data to the server

[1539] The device encodes the information and emotion data entered by the user into JSON format and sends it to the server as an HTTP POST request.

[1540] 3. Emotional Data Processing and Analysis

[1541] The server passes the received emotion data to the emotion engine, which analyzes the user's emotions. For example, if the user is feeling stressed, this information is passed on to the generative AI model.

[1542] 4. Adjustment of action plans

[1543] The generative AI model generates an action plan based on data from the emotion engine, taking into account the user's desires and emotional state. For example, it suggests a relaxation plan for a stressed user.

[1544] 5. Generate a final action plan

[1545] The generative AI model then generates a new, adjusted action plan and returns it to the server. The generated action plan includes specific schedules such as "visit a hot spring at 9:00," "have lunch at a relaxing cafe at 12:00," and "take a walk in a nature park at 15:00."

[1546] 6. Submit your proposal

[1547] The server encodes the final action plan into JSON format and sends it to the originating device as an HTTP response.

[1548] 7. Viewing and booking your action plan

[1549] The terminal displays the received action plan on the user interface. The user checks the displayed action plan and requests a reservation for a specific activity if necessary. For example, if the user wants to make a reservation for a hot spring, the user presses the reservation button.

[1550] 8. Booking Processing

[1551] The device sends the reservation request back to the server as an HTTP POST request. The server receives the reservation request and processes the reservation by calling the necessary API (for example, a hot spring reservation API). After the reservation is completed, the information is notified to the device.

[1552] 9. Display of Final Action Plan

[1553] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[1554] Specific examples

[1555] Example 1: If User A inputs a request such as "I want to relax in Tokyo on October 10, 2023," and the camera recognizes a stressed expression, the generative AI will make the following suggestions:

[1556] 09:00 Take a bath in the hot springs

[1557] 12:00 Lunch at the Relaxation Cafe

[1558] 15:00 Walk in a quiet park in Tokyo

[1559] Example 2: If User B inputs a request such as "I want to spend an active time in Sapporo on December 25, 2023," and an excited state is recognized from the voice, the generation AI will make the following suggestions.

[1560] 10:00 Visit the Snow Festival at Odori Park

[1561] 13:00 Experience sledding

[1562] 16:00 Visit entertainment facilities in Susukino

[1563] As described above, the present invention proposes an action plan that takes into account the user's emotional state and also eliminates the need for reservations, thereby enabling the user to spend a more comfortable and fulfilling time.

[1564] The processing flow will be explained below.

[1565] Step 1:

[1566] A user launches the application and enters the date, time, location, and desire through a user interface, and also uses a camera and microphone to collect facial recognition data, voice data, and emotional data, including a specific request.

[1567] Step 2:

[1568] The device encodes the date, time, location, desire, and emotion data entered by the user into JSON format and sends this encoded data to the server as an HTTP POST request.

[1569] Step 3:

[1570] The server receives the HTTP POST request and parses the JSON data, extracting date, time, location, desire, and emotion data.

[1571] Step 4:

[1572] The server passes the extracted emotion data to the emotion engine, which analyzes the user's facial recognition data and voice data. Based on this data, the emotion engine identifies the user's emotional state, such as "feeling stressed" or "excited."

[1573] Step 5:

[1574] The server passes the emotional state data obtained from the emotion engine as input to the generative AI model. The generative AI model generates an optimal action plan taking into account the user's requests and emotional state. For example, it proposes a relaxing plan for a stressed user and an active plan for an excited user.

[1575] Step 6:

[1576] The generative AI model generates an action plan and returns the result to the server. The generated action plan includes a specific schedule, such as "visit a hot spring at 09:00," "have lunch at a relaxing cafe at 12:00," "take a walk in a nature park at 15:00," and "visit the hot spring again at 18:00."

[1577] Step 7:

[1578] The server encodes the generated action plan again into JSON format and sends it to the original device as an HTTP response.

[1579] Step 8:

[1580] The device parses the JSON data received from the server and displays the action plan in the user interface, including detailed information about each activity.

[1581] Step 9:

[1582] The user checks the proposed action plan and requests reservations for specific activities if necessary. For example, if the user wants to make a reservation for a hot spring, the user presses the reservation button.

[1583] Step 10:

[1584] The device sends the reservation request back to the server as an HTTP POST request. The server receives the reservation request and processes the reservation by calling the necessary API (for example, a hot spring reservation API). After the reservation is completed, the information is notified to the device.

[1585] Step 11:

[1586] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[1587] Step 12:

[1588] The user follows the proposed plan of action and spends their time in a fulfilling manner. The device supports the user's actions by providing navigation and map links as needed.

[1589] Example 2

[1590] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1591] Conventional behavior plan proposal systems generate behavior plans based on user requests, but do not consider the user's emotional state when proposing them. As a result, they are unable to provide optimal behavior plans for users when they are feeling stressed or in a specific emotional state, which can result in lower user satisfaction. Furthermore, making reservations based on the behavior plan must be done manually, which places a burden on the user.

[1592] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1593] In this invention, the server includes means for receiving requests and emotion data from a user, means for transmitting the received requests and emotion data to the server, emotion engine means for analyzing the emotion data, generation AI means for generating an action plan based on the data obtained from the emotion engine means, means for transmitting the generated action plan to a user terminal, means for displaying the generated action plan on a user interface, and means for receiving a reservation request from a user and processing the reservation. This makes it possible to automatically generate an optimal action plan that takes into account the user's emotional state, and further automate reservations based on the action plan.

[1594] The "means for receiving request and emotion data from a user" refers to a device or software function for receiving and storing the date, time, location, request and emotion data input by a user.

[1595] The "means for transmitting the received request and emotion data to the server" refers to a protocol or communication device for transmitting the request and emotion data from the user terminal to the server.

[1596] The "emotion engine means for analyzing emotion data" refers to an algorithm or software module for analyzing received emotion data and identifying the user's emotional state.

[1597] The "generative AI means for generating an action plan based on data obtained from the emotion engine means" refers to an artificial intelligence algorithm or software module for generating an optimal action plan based on the emotional state analyzed by the emotion engine and the user's requests.

[1598] "Means for transmitting the generated action plan to the user terminal" refers to a communication means or protocol for transmitting the action plan generated by the generation AI means to the user terminal.

[1599] The "means for displaying the generated action plan on a user interface" refers to a screen display device or software interface for displaying the action plan generated on the user terminal in a format that is easy for the user to view.

[1600] "Means for receiving user reservation requests and processing reservations" refers to the protocols and operational means for receiving a user's request to reserve a specific activity and calling the appropriate reservation service or API to fulfill the reservation.

[1601] This invention is a system that proposes action plans to help users make effective use of their holidays and spare time, and further provides optimal plans based on the user's emotions by combining an emotion engine. This system consists of a user terminal and a server, and generates advanced action plans using the emotion engine and a generative AI model.

[1602] User terminal

[1603] The user terminal refers to a mobile device such as a smartphone or tablet. This allows users to input date, time, location, requests, and emotional data (facial recognition data and voice data) through a user interface. This information is then sent from the terminal to the server.

[1604] server

[1605] The server receives information from the user and generates an action plan based on the emotion engine and generative AI model. The generated action plan is sent from the server to the user's device. The system also has a function to make reservations for facilities and events using reservation methods as needed.

[1606] Emotion Engine

[1607] The emotion engine is an algorithm that analyzes a user's facial recognition data and voice data to identify the user's emotional state, thereby determining whether the user is in a particular emotional state, such as stress or excitement.

[1608] Generative AI Models

[1609] The generative AI model generates an action plan based on data from the emotion engine, taking into account the user's desires and emotional state. For example, if the user is stressed, it will generate a relaxation plan, and if the user is excited, it will generate an active plan.

[1610] Specific examples

[1611] Example 1:

[1612] If a user inputs a request such as "I want to relax in Tokyo on October 10, 2023," and the camera recognizes a stressed facial expression, the generative AI model will suggest the following action plan:

[1613] 09:00 Take a bath in the hot springs

[1614] 12:00 Lunch at the Relaxation Cafe

[1615] 15:00 Walk in a quiet park in Tokyo

[1616] Example 2:

[1617] If a user inputs a request such as "I want to spend some time actively in Sapporo on December 25, 2023," and an excited state is recognized from the voice, the generative AI model will suggest the following action plan:

[1618] 10:00 Visit the Snow Festival at Odori Park

[1619] 13:00 Experience sledding

[1620] 16:00 Visit entertainment facilities in Susukino

[1621] Prompt Sentence Examples

[1622] Example prompt 1:

[1623] "I would like to relax in Tokyo on October 10, 2023. I am feeling stressed, so please tell me the best plan of action for relaxation."

[1624] Example prompt 2:

[1625] "I'm planning an active day in Sapporo on December 25, 2023. I'm excited, so please tell me about your plans, focusing on energetic activities."

[1626] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1627] Step 1:

[1628] Users launch an application on their smartphone or tablet, input the date, time, location, and request, and then use the camera to collect facial recognition data and the microphone to provide voice data.

[1629] The inputs are "date, time, location, request, facial recognition data, and voice data."

[1630] The output is "information and emotional data entered by the user."

[1631] Step 2:

[1632] The device encodes the received information and emotion data into JSON format and sends it to the server as an HTTP POST request.

[1633] The input is "information and emotional data entered by the user."

[1634] The output is the "encoded data".

[1635] Specifically, the data is encoded and an HTTP request is sent.

[1636] Step 3:

[1637] The server passes the received data to the emotion engine for analysis.

[1638] The input is the "encoded data".

[1639] The output is "data to be passed to the emotion engine."

[1640] Specific operations include decoding the data and inputting it into the emotion engine.

[1641] Step 4:

[1642] The emotion engine analyzes the facial recognition data and voice data to identify the user's emotional state, for example, "stress."

[1643] The input is "data passed to the emotion engine."

[1644] The output is the "analyzed emotional state" information.

[1645] Specifically, facial recognition and voice analysis algorithms are executed.

[1646] Step 5:

[1647] The server passes the emotional state obtained from the emotion engine to the generative AI model and instructs it to generate an action plan.

[1648] The input is the "analyzed emotional state" information.

[1649] The output is a prompt to the generative AI model.

[1650] Specific operations include generating prompts and inputting them into the generative AI model.

[1651] Step 6:

[1652] The generative AI model generates an action plan taking into account the emotional state and the user's wishes, such as "visit a hot spring at 09:00," "have lunch at a relaxing cafe at 12:00," and "take a walk in a nature park at 15:00."

[1653] The input is a "prompt to the generative AI model."

[1654] The output is a "generated action plan."

[1655] Specifically, the generative AI model calculates an action plan using an internal algorithm.

[1656] Step 7:

[1657] The server encodes the generated action plan into JSON format and sends it to the terminal as an HTTP response.

[1658] The input is the "generated action plan."

[1659] The output is "encoded action plan data."

[1660] Specifically, the data is encoded and an HTTP response is sent.

[1661] Step 8:

[1662] The terminal decodes the received action plan data and displays it on the user interface.

[1663] The input is "encoded action plan data."

[1664] The output is an action plan for display in the user interface.

[1665] As a specific operation, the data is decoded and displayed on the screen.

[1666] Step 9:

[1667] The user checks the displayed action plan and requests a reservation for a specific activity, for example, a reservation for a hot spring.

[1668] The input is the displayed action plan and booking request.

[1669] The output is "reservation request data."

[1670] As a specific operation, a request is generated and sent.

[1671] Step 10:

[1672] The terminal sends the generated reservation request to the server as an HTTP POST request.

[1673] The input is "reservation request data."

[1674] The output is "Reservation request data sent."

[1675] As a specific operation, the request data is encoded and transmitted.

[1676] Step 11:

[1677] The server receives the reservation request and processes the reservation by calling the necessary API. For example, it calls a hot spring reservation API to process the reservation. After the reservation is completed, it notifies the terminal of the information.

[1678] The input is "reservation request data sent."

[1679] The output is "reservation processing results and notification data."

[1680] Specifically, the reservation API is called and the results are obtained.

[1681] Step 12:

[1682] The terminal again displays the final action plan including the reservation information on the user interface.

[1683] The input is "reservation processing results and notification data."

[1684] The output is a "final action plan for display in the user interface."

[1685] As a specific operation, the data is decoded and displayed on the screen.

[1686] (Application example 2)

[1687] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1688] Conventional behavior plan generation systems only provide plans based on the user's requests, but are unable to provide plans that appropriately reflect the user's individual emotional state. This makes it difficult to propose an optimal behavior plan tailored to the user's emotional state. Furthermore, particularly in self-driving vehicles, behavior plans that allow users to use their time effectively while traveling are not effectively generated or executed. This makes it difficult to provide a highly satisfying experience for users.

[1689] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving request and emotion data from the user, means for presenting the generated action plan to the user, an emotion engine and generation AI means for generating an action plan based on the user's request and emotion data, means for making a reservation based on the generated action plan, and means for displaying the generated action plan on the interface of the autonomously driven vehicle. This makes it possible to propose an action plan that matches the user's request and emotional state within the autonomously driven vehicle, making effective use of time while traveling and increasing user satisfaction.

[1690] "Requests" refer to the hopes and demands that users have regarding the action plan.

[1691] "Emotion data" is data that indicates the user's emotional state, and includes facial recognition data, voice data, and the like.

[1692] An "action plan" refers to a specific schedule of activities generated based on the user's requests and emotional data.

[1693] An "emotion engine" refers to software that analyzes a user's emotional data and identifies their emotional state.

[1694] "Generative AI" is an artificial intelligence model that generates optimal action plans based on user requests and emotional data.

[1695] "User interface" refers to the interface through which a user inputs requests and emotional data into the system.

[1696] "Server" refers to a computer system that receives data from users and is responsible for generating action plans and processing reservations.

[1697] A "terminal" is a device through which a user accesses the system, including smartphones, tablets, and autonomous vehicle interfaces.

[1698] "Communication means" refers to the technology and protocols used to send and receive data between a server and a terminal.

[1699] "Reservation means" refers to a system or process for making reservations for facilities or events based on the generated action plan.

[1700] To implement this invention, a user terminal (e.g., a smartphone, tablet, or autonomous vehicle interface) and a server are required. The user terminal provides a user interface for inputting date, time, location, and requests. In addition, the user terminal uses a camera and microphone to collect emotion data such as facial recognition data and voice data.

[1701] Through the user device interface, the user inputs date, time, location, desire, and emotional data. This data is encoded in JSON format and sent to the server as an HTTP POST request. The server receives this data and uses an emotional engine to analyze the user's emotional state. For example, it identifies whether the user is feeling stressed or seeking relaxation.

[1702] The server then uses the generative AI model to generate an action plan based on the user's desires and emotional state, including specific schedules such as "visit a hot spring at 09:00," "have lunch at a relaxation cafe at 12:00," and "take a walk in a quiet park at 15:00."

[1703] The generated action plan is sent from the server to the user's device and presented to the user. The user can review this action plan and request a reservation for a specific activity if necessary. For example, if a user wants to make a reservation for a hot spring, they press the reservation button, which sends a reservation request to the server. The server then calls the necessary API (e.g., a hot spring reservation API) to process the reservation. After the reservation is completed, the information is notified to the user's device.

[1704] By using this system, users can make effective use of their time while traveling in an autonomous vehicle and enjoy optimal action plans tailored to their emotions. A specific example is shown below.

[1705] Example 1:

[1706] If User A inputs a request such as "I want to relax in Kyoto on November 3, 2023," and the camera recognizes a stressed expression, the generative AI model will suggest the following action plan:

[1707] 09:00 Take a bath in the hot springs

[1708] 12:00 Lunch at the Relaxation Cafe

[1709] 15:00 Stroll around Momiji Park

[1710] Example 2:

[1711] If User B inputs a request such as "I want to spend time actively in Sapporo on December 25, 2023," and an excited state is recognized from the voice, the generative AI model will propose the following action plan:

[1712] 10:00 Visit the Snow Festival at Odori Park

[1713] 13:00 Experience sledding

[1714] 16:00 Visit an entertainment venue

[1715] In the above example, the following prompt sentence is passed to the generative AI model to generate an action plan:

[1716] Prompt statement example 1:

[1717] "Date: 2023-11-03, Location: Kyoto, Request: Relaxation, Emotion: Stress"

[1718] Prompt statement example 2:

[1719] "Date: 2023-12-25, Location: Sapporo, Request: Active, Emotion: Excitement"

[1720] In this way, an optimal action plan that matches the user's emotions is provided, thereby increasing the user's satisfaction.

[1721] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1722] Step 1:

[1723] The user inputs the date, time, location, and request through the user interface. In addition, the device uses a camera and microphone to collect emotional data (facial recognition data and voice data). This data is encoded into JSON format by the device.

[1724] Input: Date and time, location, request, emotional data (face recognition data, voice data)

[1725] Output: User data in JSON format

[1726] Step 2:

[1727] The device sends the encoded user data to the server as an HTTP POST request, which the server receives and begins parsing the data.

[1728] Input: User data in JSON format

[1729] Output: HTTP POST request to the server

[1730] Step 3:

[1731] The server analyzes the received user data and uses an emotion engine to identify the user's emotional state, using facial recognition and voice data to determine whether the user is feeling stressed or seeking relaxation.

[1732] Input: User data (face recognition data, voice data)

[1733] Output: User's emotional state (e.g., stressed, relaxed)

[1734] Step 4:

[1735] The server uses a generative AI model to generate an optimal action plan based on the analysis results (user's emotional state) and the user's request by the emotion engine. The server passes the prompt sentence to the generative AI model and obtains a specific action plan as its output.

[1736] Input: User's emotional state, desires, prompt

[1737] Output: Action plan

[1738] Step 5:

[1739] The server encodes the generated action plan and sends it to the user device in JSON format. An action plan might include, for example, "visit a hot spring at 09:00," "have lunch at a relaxation cafe at 12:00," and "take a walk in the park at 15:00."

[1740] Input: Action Plan

[1741] Output: Action plan in JSON format

[1742] Step 6:

[1743] The terminal displays the received action plan on the user interface. The user can check the plan and request a reservation for a specific activity. For example, if the user wants to make a reservation for a hot spring, the user presses the reservation button.

[1744] Input: Action plan in JSON format

[1745] Output: Display of action plan

[1746] Step 7:

[1747] A user's reservation request is sent from the terminal to the server. The server processes the reservation by calling the necessary API (e.g., hot spring reservation API). After the reservation is completed, the information is notified to the user terminal.

[1748] Input: Booking Request

[1749] Output: Reservation completion notification

[1750] Step 8:

[1751] The terminal displays the final action plan including the reservation information again on the user interface, allowing the user to start an action based on the latest action plan.

[1752] Input: Reservation completion notification

[1753] Output: Display the final action plan

[1754] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1755] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1756] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1757] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1758] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1759] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1760] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1761] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1762] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1763] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1764] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1765] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1766] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1767] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1768] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1769] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1770] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1771] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1772] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1773] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1774] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1775] The following is further disclosed regarding the above embodiment.

[1776] (Claim 1)

[1777] means for receiving a request from a user;

[1778] means for presenting the generated action plan to a user;

[1779] A generating AI means for generating an action plan based on a user's request;

[1780] means for making reservations based on the generated action plan;

[1781] A system including:

[1782] (Claim 2)

[1783] 10. The system of claim 1, further comprising a user interface means for inputting date, time, location, and desires.

[1784] (Claim 3)

[1785] 2. The system according to claim 1, further comprising a communication means for transmitting and receiving data between the server and the terminal.

[1786] "Example 1"

[1787] (Claim 1)

[1788] means for receiving a request from a user;

[1789] an input means for inputting a user's request as date, time, location, and request information;

[1790] means for transmitting the input data to a server;

[1791] a means for generating an action plan using the generative AI model;

[1792] means for presenting the generated action plan to a user;

[1793] means for making reservations based on the generated action plan;

[1794] A communication means for transmitting and receiving data between the server and the terminal;

[1795] A system including:

[1796] (Claim 2)

[1797] 10. The system of claim 1, further comprising processing means for encoding information entered by the user and transmitting the encoded information to the server.

[1798] (Claim 3)

[1799] 10. The system of claim 1, further comprising a processing means for constructing the data received by the server as input prompts for a generative AI model to generate an action plan.

[1800] "Application Example 1"

[1801] (Claim 1)

[1802] means for receiving a request from a user;

[1803] means for presenting the generated action plan to a user;

[1804] A generating AI means for generating an action plan based on a user's request;

[1805] means for making reservations based on the generated action plan;

[1806] A means to input meal type, date, time, location, and budget range;

[1807] A system including:

[1808] (Claim 2)

[1809] 10. The system of claim 1, further comprising a user interface means for inputting date, time, location, and desires.

[1810] (Claim 3)

[1811] 2. The system according to claim 1, further comprising a communication means for transmitting and receiving data between the server and the terminal.

[1812] (Claim 4)

[1813] 10. The system of claim 1, further comprising means for generating optimal restaurant and meal plans based on the generative AI model.

[1814] (Claim 5)

[1815] 10. The system of claim 1, further comprising means for displaying the generated meal plan in a user interface.

[1816] "Example 2: Combining Emotion Engines"

[1817] (Claim 1)

[1818] means for receiving request and emotion data from a user;

[1819] means for transmitting the received request and emotion data to a server;

[1820] emotion engine means for analyzing emotion data;

[1821] a generation AI means for generating an action plan based on the data obtained from the emotion engine means;

[1822] means for transmitting the generated action plan to a user terminal;

[1823] means for displaying the generated action plan on a user interface;

[1824] means for receiving user reservation requests and processing reservations;

[1825] A system including:

[1826] (Claim 2)

[1827] 10. The system of claim 1, including a user interface for inputting date, time, location, and desires.

[1828] (Claim 3)

[1829] 2. The system according to claim 1, further comprising a communication means for transmitting and receiving data between the server and the terminal.

[1830] "Application example 2 when combining emotion engines"

[1831] (Claim 1)

[1832] means for receiving request and emotion data from a user;

[1833] means for presenting the generated action plan to a user;

[1834] an emotion engine and generation AI means for generating an action plan based on a user's request and emotion data;

[1835] means for making reservations based on the generated action plan;

[1836] means for displaying the generated action plan on an interface of the autonomous vehicle;

[1837] A system including:

[1838] (Claim 2)

[1839] 2. The system according to claim 1, further comprising a user interface means for inputting date, time, location, and desire, and for acquiring emotion data.

[1840] (Claim 3)

[1841] 2. The system according to claim 1, further comprising a communication means for transmitting and receiving data between the server and the terminal. [Explanation of symbols]

[1842] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving a request from a user; means for presenting the generated action plan to a user; A generating AI means for generating an action plan based on a user's request; means for making reservations based on the generated action plan; A system including:

2. 2. The system of claim 1, further comprising a user interface means for inputting date, time, location, and desires.

3. 2. The system according to claim 1, further comprising a communication means for transmitting and receiving data between the server and the terminal.

Images