system

Abstract

Provide a system. A terminal means for a user to input travel information; A server receives the input travel information and converts it into a data format; A means for the server to request travel information from the chat generation AI, A means for generating a travel scenario based on the travel information received by the chat generation AI; a means for the server to transmit the generated travel scenario to a user; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, creating individually customized travel plans is extremely time-consuming and labor-intensive. Furthermore, the limited means for easily sharing travel scenarios hinders smooth sharing and collaboration of travel plans. There is a need for a system that can solve these problems and enable users to easily and efficiently create and share travel scenarios. [Means for solving the problem]

[0005] The present invention solves the above problem by providing a system including a terminal means for a user to input travel information, a means for a server to receive the input travel information and convert it into a data format, a means for the server to request the travel information from a chat generation AI, a means for the chat generation AI to generate a travel scenario based on the received travel information, a means for the server to send the generated travel scenario to a user, a terminal means for a user to input a request to modify the scenario, a means for the server to resend the modification request to the chat generation AI, a means for the chat generation AI to modify the scenario based on the modification request, a means for the server to send the modified scenario to the user, a terminal means for a user to input a request to save or share the generated scenario, a means for the server to save the scenario in a database, and a means for the server to share the scenario using the LINE API.

[0006] A "user" is an individual or end user who inputs travel information and creates, modifies, saves, and shares travel scenarios.

[0007] "Terminal" refers to a device used by a user, such as a smartphone, tablet, or PC, and is an apparatus for inputting travel information and viewing scenarios.

[0008] "Server" refers to a computer system responsible for receiving input from users and processing, storing, and transmitting data.

[0009] "Converting to data format" refers to converting information entered by a user into a format that is easy for AI and other systems to process.

[0010] "Chat generation AI" refers to artificial intelligence that generates dialogue and text based on input information, and in this case, it plays a role in generating travel scenarios.

[0011] "Travel information" refers to information entered by the user, such as travel destination, date, budget, and tourist spots.

[0012] A "travel scenario" refers to a plan that includes travel itineraries, tourist spots, accommodations, etc., generated based on the user's travel information.

[0013] A "modification request" refers to a request for a change or addition made by a user to a generated travel scenario.

[0014] "Database" refers to a system for systematically collecting, storing, and managing information, and is used here to store travel scenarios.

[0015] "LINE API" refers to an interface that allows the functions of the communication application LINE to be used by external applications and systems.

[0016] "Means for saving" refers to the technology and process for saving the generated travel scenario in a storage device such as a database.

[0017] "Means for sharing" refers to a function or process for transmitting the generated travel scenario to other users or groups designated by the user. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] This invention provides a system that allows users to easily create, edit, save, and share travel scenarios. This system is realized using devices, a server, chat generation AI, a database, and the LINE API.

[0040] 1. User registration and authentication

[0041] overview:

[0042] To access the system, users first register, which creates a dedicated account for them.

[0043] What happens:

[0044] Device: The user enters their name, email address, and password into the registration form. For example, the user enters their name "Taro Tanaka," their email address "taro.tanaka@example.com," and their password "securepassword," and submits the form.

[0045] Server: Receives registration information and stores it in a database. A hashed password is also generated and stored.

[0046] Server: Notify the user that registration was successful.

[0047] 2. Requesting and Generating Travel Scenarios

[0048] overview:

[0049] Users enter travel details (destination, date, budget, etc.), and the chat generation AI generates a travel scenario based on that information.

[0050] What happens:

[0051] Device: The user enters the travel destination "Kyoto," the date "December 1, 2023 - December 3, 2023," and the budget "50,000 yen," and submits.

[0052] Server: Receives travel information and converts it into an internal data format.

[0053] Server: The travel information converted into a data format is sent as a request to the chat generation AI.

[0054] Chat generation AI: Generates travel scenarios based on the received travel information. For example, it generates scenarios such as "Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion," "Day 2: Visit Kiyomizu-dera Temple and Kinkaku-ji Temple," and "Day 3: Visit Arashiyama, depart from Kyoto Station."

[0055] Server: Sends the generated travel scenario to the user.

[0056] 3. Customizing the scenario

[0057] overview:

[0058] The user can request changes or additions to the generated travel scenario.

[0059] What happens:

[0060] Device: The user enters "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" and submits a correction request.

[0061] Server: Receives the correction request and resends it to the chat generation AI.

[0062] Chat generation AI: Regenerates scenarios based on revision requests. For example, it generates a new scenario called "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[0063] Server: Sends the modified scenario to the user.

[0064] 4. Share and save your scenarios

[0065] overview:

[0066] Users can save the scenarios they create and share them with other users via LINE.

[0067] What happens:

[0068] Device: The user selects whether to save the scenario or share it on LINE and sends a request.

[0069] server:

[0070] For save: Save the scenario to the database.

[0071] For sharing: Send the scenario to the specified contact via the LINE API.

[0072] LINE API: Send the received scenario information to a specified group or contact. For example, send a message to the "Family" group saying, "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[0073] In this way, users can easily create, save, and share customized travel scenarios.The system that supports users' travel planning is realized by the collaboration of the server, terminals, chat generation AI, database, and LINE API.

[0074] The processing flow will be explained below.

[0075] 1. User registration and authentication

[0076] Step 1:

[0077] On your device: The user enters their name, email address, and password into the provided registration form.

[0078] Step 2:

[0079] Terminal: The user clicks the submit button, sending the input data to the server.

[0080] Step 3:

[0081] Server: Receives input data and checks whether it is entered correctly (validation check).

[0082] Step 4:

[0083] Server: Creates SQL queries to store registration information in a database and saves the data. Passwords are hashed for security purposes.

[0084] Step 5:

[0085] Server: Generates and sends back a message to the user indicating successful registration.

[0086] 2. Requesting and Generating Travel Scenarios

[0087] Step 1:

[0088] Device: The user enters travel destination "Kyoto," date "December 1, 2023 - December 3, 2023," and budget "50,000 yen," and clicks the send button.

[0089] Step 2:

[0090] Server: Receives travel information and converts the data into a format that can be processed by AI.

[0091] Step 3:

[0092] Server: Sends the converted travel information as a request to the chat generation AI.

[0093] Step 4:

[0094] Chat generation AI: Generates a travel scenario (e.g., Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion) based on the received travel information.

[0095] Step 5:

[0096] Server: Receives travel scenarios generated by the AI ​​and sends them to the user.

[0097] 3. Customizing the scenario

[0098] Step 1:

[0099] Terminal: The user inputs "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" to an existing scenario and submits a modification request.

[0100] Step 2:

[0101] Server: Receives the correction request and sends it back to the chat generation AI.

[0102] Step 3:

[0103] Chat generation AI: Based on modification requests, it adjusts existing scenarios and generates new ones (e.g., Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine).

[0104] Step 4:

[0105] Server: Receives modified scenarios from the AI ​​and sends them to the user.

[0106] 4. Share and save your scenarios

[0107] Step 1:

[0108] Device: The user selects whether to save the scenario or share it on LINE, and sends a request to the server.

[0109] Step 2:

[0110] Server: If a save request is received, it saves the scenario to the database.

[0111] Step 3:

[0112] Server: When a sharing request is received, the server uses the LINE API to send the scenario to the specified contacts.

[0113] Step 4:

[0114] LINE API: Send the received scenario to the specified contacts. For example, send the scenario to the "Family" group with the message "Taro Tanaka shared a travel scenario."

[0115] In this way, each processing step works together, allowing users to efficiently create, modify, save, and share travel scenarios.

[0116] Example 1

[0117] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0118] Conventional travel planning systems have made it difficult for users to efficiently generate, modify, and share travel scenarios. Specifically, it is difficult to generate an appropriate scenario based on the travel information entered by the user, and customizing and modifying the scenario takes time. In addition, there is a lack of a way to easily share the generated scenario with other users.

[0119] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0120] In this invention, the server includes terminal means for a user to input travel information, means for the server to receive the input travel information and convert it into a data format, means for the server to request the travel information from the generative AI model, means for the generative AI model to generate a travel scenario based on the received travel information, means for the server to send the generated travel scenario to the user, terminal means for a user to input a request to modify the scenario, means for the server to resend the modification request to the generative AI model, means for the generative AI model to modify the scenario based on the modification request, means for the server to send the modified scenario to the user, terminal means for a user to input a request to save or share the generated scenario, means for the server to save the scenario in a database, and means for the server to share the scenario using a communication API. This enables users to easily generate travel scenarios, modify them as needed, and efficiently share them with other users.

[0121] "Terminal means" refers to a device or interface through which a user inputs travel information and modification requests and transmits them to a server.

[0122] The "server" is a central computer that processes travel information and modification requests received from users, converts them into data formats, and sends requests to the generative AI model.

[0123] "Means for converting into data format" refers to the process or algorithm for converting travel information received from a user into a format that is easy to process internally (e.g., JSON format).

[0124] The "means for making a request to the generative AI model" is a process for passing the converted travel information to the generative AI model and sending a request as a prompt sentence.

[0125] A "generative AI model" is an artificial intelligence system that generates travel scenarios based on input prompts.

[0126] The "means for generating a travel scenario" is a function that enables the generative AI model to create a specific travel plan based on travel information.

[0127] The "means for resending a modification request to the generative AI model" is a process for receiving a scenario modification request from a user and sending it again as a prompt to the generative AI model.

[0128] A "communication API" is a communication protocol or interface used to share generated travel scenarios with other users, and is a means for connecting to external systems (e.g., SNS applications).

[0129] This invention provides a system that allows users to easily create, modify, save, and share travel scenarios. This system is realized using terminals, servers, generative AI models, databases, and communication APIs.

[0130] Hardware and software used

[0131] Device: User device such as a smartphone or computer

[0132] Server: A central computer for information processing and database management.

[0133] Generative AI model: Chat generation AI (e.g., GPT-3 (registered trademark))

[0134] Database: Data storage for storing and managing information (e.g., SQL database)

[0135] Communication API: Interface for data communication with external systems (e.g., RESTful API)

[0136] User Registration and Authentication

[0137] User: Enter your name, email address, and password, and send them from your device to the server. For example, enter the name "Taro Tanaka," the email address "taro.tanaka@example.com," and the password "securepassword."

[0138] Server: Stores the received information in a database, hashes the password, and notifies the user that registration was successful.

[0139] Requesting and generating travel scenarios

[0140] User: Enters details such as travel destination, date, and budget, and sends them to the server from the device. For example, enters "Kyoto," "December 1, 2023 - December 3, 2023," and "50,000 yen."

[0141] Server: Converts received travel information into a data format and sends it to the generative AI model.

[0142] Generative AI model: Generates travel scenarios based on received prompts.

[0143] Example prompt: "I would like to create a scenario for traveling to Kyoto. The travel dates are from December 1, 2023 to December 3, 2023, and the budget is 50,000 yen. Please suggest a scenario."

[0144] Server: Sends the generated travel scenario to the user.

[0145] Customizing travel scenarios

[0146] User: Enter any corrections or additions to the scenario and send them to the server from the device. For example, you might enter, "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[0147] Server: Resubmits the correction request to the generative AI model.

[0148] Generative AI model: Regenerates scenarios based on modification requests.

[0149] Server: Sends the modified scenario to the user.

[0150] Save and share your travel scenarios

[0151] User: Selects to save or share the scenario and sends a request from the device to the server.

[0152] server:

[0153] For save: Save the scenario to the database.

[0154] For sharing: Use a communication API to send the scenario to an external system. For example, send a message saying "Taro Tanaka has shared a travel scenario" to a specified group or contact.

[0155] This allows users to easily create, modify, save, and share travel scenarios. Furthermore, by linking together the server, terminal, generative AI model, database, and communication API, a system that supports users in planning their trips is realized.

[0156] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0157] Step 1:

[0158] The user enters travel information. Specifically, the user enters their name, email address, and password into the terminal. For example, the user enters the name "Taro Tanaka," the email address "taro.tanaka@example.com," and the password "securepassword." This is the input data.

[0159] Step 2:

[0160] The terminal sends the entered information to the server. The entered data is sent to the server as is. For example, {"name": "Taro Tanaka", "email": "taro.tanaka@example.com", "password": "securepassword"}.

[0161] Step 3:

[0162] The server processes the received information and hashes the password. Specifically, it hashes "securepassword" using a hashing algorithm (e.g., SHA-256). This is the data processing step. For example, "securepassword" -> "5e884898da28047151d0e56f8dc6292773603d0d6aabbdd5d029d6ad1db1c5f7".

[0163] Step 4:

[0164] The server stores the hashed password along with other user information in a database, such as {"name": "Taro Tanaka", "email": "taro.tanaka@example.com", "password_hash": "5e884898da28047151d0e56f8dc6292773603d0d6aabbdd5d029d6ad1db1c5f7"} .

[0165] Step 5:

[0166] The server generates a message indicating that registration has been completed successfully and sends it to the terminal. The message is simple text data, such as "Registration completed."

[0167] Step 6:

[0168] The terminal displays the received message to the user, displaying the text "Registration complete."

[0169] Step 7:

[0170] The user inputs travel information such as destination, date, budget, etc. For example, the user inputs the destination "Kyoto," the date "December 1, 2023 - December 3, 2023," and the budget "50,000 yen." This becomes new input data.

[0171] Step 8:

[0172] The device sends the entered travel information to the server. This information is passed to the server as is. For example, {"destination": "Kyoto", "dates": "December 1, 2023 - December 3, 2023", "budget": "50,000 yen"}.

[0173] Step 9:

[0174] The server converts the received travel information into an internal data format. For example, it converts it into the JSON format used internally. The internal data becomes {"destination": "Kyoto", "dates": {"start": "2023-12-01", "end": "2023-12-03"}, "budget": 50000}.

[0175] Step 10:

[0176] The server sends the converted travel information to the generative AI model as a prompt. For example, the prompt might read, "I would like to create a scenario for traveling to Kyoto. The travel dates are from December 1, 2023 to December 3, 2023, and the budget is 50,000 yen. Please suggest a scenario."

[0177] Step 11:

[0178] The generative AI model generates a travel scenario based on the received prompt. For example, the generated scenario might be "Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion," "Day 2: Visit Kiyomizu-dera Temple and Kinkaku-ji Temple," or "Day 3: Visit Arashiyama, depart from Kyoto Station."

[0179] Step 12:

[0180] The generative AI model returns the generated travel scenario to the server, which then sends the received scenario to the user. The scenario text is sent to the user.

[0181] Step 13:

[0182] The device displays the generated travel scenario to the user. Specifically, the scenario text is displayed on the device screen. For example, "Day 1: Arrive at Kyoto Station, check in at hotel, stroll around Gion..."

[0183] Step 14:

[0184] The user inputs a modification request for the generated scenario. For example, the user might input, "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day." This becomes the input data for the modification request.

[0185] Step 15:

[0186] The device sends a modification request to the server, which passes the request to the server as is. For example, {"modification": "I also want to visit Fushimi Inari Taisha Shrine in the afternoon of the second day"}.

[0187] Step 16:

[0188] The server resubmits the correction request to the generative AI model. For example, the correction request might be, "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[0189] Step 17:

[0190] The generative AI model regenerates the scenario based on the modification request. For example, the regenerated scenario might be "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[0191] Step 18:

[0192] The generative AI model sends the modified scenario back to the server, which then sends the received scenario to the user. The user receives the regenerated scenario text.

[0193] Step 19:

[0194] The terminal displays the revised scenario to the user. Specifically, the text of the regenerated scenario is displayed on the terminal screen. For example, "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[0195] Step 20:

[0196] The user inputs a request to save or share a scenario. For example, they select from the options "Save scenario" or "Share on LINE." This becomes the input data for the save or share request.

[0197] Step 21:

[0198] The device sends a request to the server. The request content is passed to the server as is. For example, for "save" it is {"action": "save"}, and for sharing it is {"action": "share"}.

[0199] Step 22:

[0200] The server processes the request accordingly. If save is selected, the scenario is saved to the database. For example, the following data is saved to the database: {"scenario": "Day 1: Arrive at Kyoto Station, check in at hotel, stroll around Gion..."}

[0201] Step 23:

[0202] If sharing is selected, the server uses a communication API to send the scenario to an external system. Specifically, it sends the scenario information to the specified group or contact. For example, it sends a message saying, "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[0203] Step 24:

[0204] The communication API sends the received scenario information to the specified group or contact, allowing users to share the scenarios they have created with other users.

[0205] Step 25:

[0206] The server notifies the user that the saving or sharing is complete, for example, "The scenario has been saved" or "The scenario has been shared."

[0207] Step 26:

[0208] The device displays a notification message to the user, specifically, a message such as "Scenario saved" or "Scenario shared" on the device screen.

[0209] (Application example 1)

[0210] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0211] Conventional food delivery services often lack the flexibility to accommodate specific user meal requests. Furthermore, when users customize their meal menus or have complex dietary restrictions, it is difficult to quickly provide suggestions that meet those requests. Furthermore, they lack the ability to easily share or save generated menus with other users. There is a need for a food delivery system that can meet these user needs.

[0212] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0213] In this invention, the server includes a terminal means for a user to input meal information, a means for the server to receive the input meal information and convert it into a data format, a means for the server to request the meal information from a generative AI model, a means for the generative AI model to generate a meal menu based on the received meal information, a means for the server to send the generated meal menu to the user, and a means for customizing the meal menu. This allows users to quickly and flexibly receive menu suggestions based on specific meal requests. Furthermore, the generated menu can be easily customized, saved, and shared.

[0214] A "user" is an individual who utilizes the system to input meal information and receive or customize the generated menu.

[0215] "Meal information" refers to information including the details of the meal desired by the user, such as the type of meal, budget, and specific dietary restrictions.

[0216] "Terminal means" refers to a device used by a user to input meal information, and specifically refers to a smartphone, tablet, etc.

[0217] A "server" is a computer system that receives and processes input information from a user.

[0218] "Means for converting into data format" refers to the function of converting the dietary information entered by the user into a format that can be understood by the generative AI model.

[0219] A "generative AI model" is an artificial intelligence model that generates optimal meal menus based on input meal information.

[0220] "Means for requesting" refers to the function by which the server sends the meal information received from the user to the generative AI model.

[0221] A "meal menu" is a specific meal suggestion generated by a generative AI model based on dietary information.

[0222] "Customizable means" refers to a function that allows a user to request changes or additions to the meal menu they receive.

[0223] The "means for saving" is a function that allows a user to save the menu created in a database within the system.

[0224] "Means for sharing" is a function for transmitting the generated menu using a communication means in order to share it with other users.

[0225] This invention provides a food delivery system that allows users to input their desired meal details, and a generative AI model proposes and customizes meal menus based on that information, which can then be saved and shared. The system is configured using the following hardware and software:

[0226] Hardware

[0227] Server: A computer system for receiving and processing meal information from users.

[0228] Terminal: The device through which the user inputs meal information. Examples include smartphones and tablets.

[0229] software

[0230] Python (Django, Flask, etc.): A framework for performing server-side processing.

[0231] MySQL (registered trademark): Used as a database management system.

[0232] LINE API: Communication method for sharing menus.

[0233] OpenAI® generative AI model: Used to generate menus based on meal information.

[0234] Program processing

[0235] The server receives the meal information sent by the user and converts it into a data format. The converted data is sent as a request to the generative AI model, which then generates a meal menu based on the meal information. For example, based on the information that "the user requested pizza as the meal type and their budget is 2,000 yen," the generative AI model would suggest the optimal pizza menu.

[0236] The generated meal menu is sent to the user by the server. If the user subsequently wishes to customize the menu, they can input a modification request from their device, which is then sent again to the generative AI model via the server. The generative AI model then generates a new menu based on the modification request and sends it to the user via the server.

[0237] If the user saves the created menu, the server saves it in the database. If the user wishes to share the menu, the server uses the LINE API to send the menu to the specified contacts or group. This allows users to easily customize, save, and share their desired meal menu.

[0238] Specific examples

[0239] If a user inputs a request such as "vegan pizza, budget 3,000 yen," the generative AI model will receive the following prompts and generate the optimal menu based on that request:

[0240] Example prompt sentence:

[0241] Based on user requests, please suggest a vegan pizza menu within a budget of 3,000 yen.

[0242] When a user receives the generated menu and sends a modification request, such as "I'd like to add an olive topping," the generative AI model reprocesses it and generates the following modification prompt:

[0243] Example of a modified prompt:

[0244] Add olive topping to your current menu and regenerate.

[0245] The system allows users to efficiently obtain customized meal menus that can be easily saved and shared with other users.

[0246] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0247] Step 1:

[0248] The user inputs meal information. Using a device such as a smartphone or tablet, the user inputs detailed information such as meal type (e.g., pizza) and budget (e.g., 2,000 yen) and submits it. The input data is sent to the server.

[0249] Step 2:

[0250] The server receives the meal information and converts it into a data format. The server analyzes the raw data received from the user and converts it into a format that the generative AI model can understand, for example, JSON format. The input is the user's meal information, and the output is the converted data format.

[0251] Step 3:

[0252] The server sends a request to the generative AI model. The converted data format is used to send a prompt to the generative AI model (for example, OpenAI's GPT model). The specific prompt generated is, "Based on the user's request, please suggest a pizza menu item within a budget of 2,000 yen."

[0253] Step 4:

[0254] The generative AI model generates a meal menu. Based on the prompt received, the generative AI model generates the optimal meal menu. The input is the prompt, and the output is the generated meal menu. For example, a menu such as "Margherita pizza, topped with basil and cheese, priced at 1,800 yen" is generated.

[0255] Step 5:

[0256] The server sends the generated meal menu to the user. The server then sends the meal menu received from the generative AI model back to the user's device. The user can then view the proposed menu on their device.

[0257] Step 6:

[0258] The user inputs a menu customization request. For example, the user sends a request to change the proposed menu, such as "I would like to add olives to the toppings," to the server via the terminal.

[0259] Step 7:

[0260] The server sends the customization request to the generative AI model. The server receives the modification request from the user, reformats it, and sends it back to the generative AI model. The specific prompt generated is "Please add olive topping to the current menu and regenerate it."

[0261] Step 8:

[0262] The generative AI model generates a modified meal menu. The generative AI model generates a new menu based on the modification request. The input is the modification prompt, and the output is the modified meal menu.

[0263] Step 9:

[0264] The server sends the revised menu to the user, and upon receiving the revised menu, the server sends it again to the user's terminal.

[0265] Step 10:

[0266] When a user wants to save a generated menu, he sends a save request to the server through the terminal. The server saves this menu in the database. The input is the menu save request and the output is the saved menu.

[0267] Step 11:

[0268] When a user wants to share a created menu, they send a share request to the server via their device. The server then shares this menu with other users using the LINE API, etc. The input is the share request, and the output is the shared menu.

[0269] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0270] This invention provides a system that allows users to create, modify, save, and share travel scenarios while improving the user experience by combining an emotion engine that recognizes user emotions. This system is realized using devices, a server, chat generation AI, a database, the LINE API, and an emotion engine.

[0271] 1. User registration and authentication

[0272] overview:

[0273] To access the system, users first register, which creates a dedicated account for them.

[0274] What happens:

[0275] Terminal: The user enters their name, email address, and password into the registration form. For example, they enter the username "Taro Tanaka," the email address "taro.tanaka@example.com," and the password "securepassword," and submit.

[0276] Server: Receives registration information and stores it in a database. Passwords are hashed for security purposes.

[0277] Server: Generates and sends a message to the user indicating successful registration.

[0278] 2. Requesting and Generating Travel Scenarios

[0279] overview:

[0280] Users input their travel details (destination, date, budget, etc.), and the chat generation AI generates a travel scenario based on that information. At this time, the emotion engine recognizes the user's emotions and reflects them in customizing the travel scenario.

[0281] What happens:

[0282] Device: The user enters travel destination "Kyoto," date "December 1, 2023 - December 3, 2023," and budget "50,000 yen," and clicks the send button.

[0283] Emotion engine: Analyzes facial expressions and voice data while the user is typing to recognize emotions. For example, if the user is excited, it will suggest an active sightseeing plan.

[0284] Server: Receives travel information and emotion information and converts it into a data format.

[0285] Server: Sends the converted travel information and emotion information as a request to the chat generation AI.

[0286] Chat generation AI: Generates a travel scenario (e.g., Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion) based on received travel information and emotional information.

[0287] Server: Sends the generated travel scenario to the user.

[0288] 3. Customizing the scenario

[0289] overview:

[0290] The user can request changes or additions to the generated travel scenario, and the emotion engine will recognize the user's emotions and reflect them in the modification request.

[0291] What happens:

[0292] Terminal: The user inputs "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" to an existing scenario and submits a modification request.

[0293] Emotion engine: Analyzes facial and voice data while the user is entering a correction request to recognize emotions. For example, if the user is feeling stressed, the engine can suggest relaxation plans.

[0294] Server: Receives the modification request and emotion information, and sends the request again to the chat generation AI.

[0295] Chat generation AI: Based on modification requests and emotional information, it adjusts existing scenarios and generates new ones (e.g., Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine).

[0296] Server: Sends the modified scenario to the user.

[0297] 4. Share and save your scenarios

[0298] overview:

[0299] Users can save the scenarios they create or share them with other users via LINE. In this case, the emotion engine also recognizes the user's emotions and suggests appropriate ways to share them.

[0300] What happens:

[0301] Device: The user selects whether to save the scenario or share it on LINE, and sends a request to the server.

[0302] Emotion engine: Analyzes facial expressions and voice data while the user is operating the app to recognize emotions. For example, if the user is excited, the engine will suggest ways to share the app.

[0303] Server: If a save request is received, it saves the scenario to the database.

[0304] Server: When a sharing request is received, the server uses the LINE API to send the scenario to the specified contacts.

[0305] LINE API: Send the received scenario to the specified contacts. For example, to the "Family" group, send the scenario with the message "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[0306] In this way, each processing step works together, allowing users to efficiently create, modify, save, and share customized travel scenarios based on emotion recognition.The collaboration of each element—server, device, chat generation AI, database, LINE API, and emotion engine—creates a system that improves the user experience.

[0307] The processing flow will be explained below.

[0308] 1. User registration and authentication

[0309] Step 1:

[0310] On your device: The user enters their name, email address, and password into the provided registration form.

[0311] Step 2:

[0312] Terminal: The user clicks the submit button, sending the input data to the server.

[0313] Step 3:

[0314] Server: Receives input data and checks whether it is entered correctly (validation check).

[0315] Step 4:

[0316] Server: Creates SQL queries to store registration information in a database and saves the data. Passwords are hashed for security purposes.

[0317] Step 5:

[0318] Server: Generates and sends back a message to the user indicating successful registration.

[0319] 2. Requesting and Generating Travel Scenarios

[0320] Step 1:

[0321] Device: The user enters travel destination "Kyoto," date "December 1, 2023 - December 3, 2023," and budget "50,000 yen," and clicks the send button.

[0322] Step 2:

[0323] Emotion engine: Analyzes facial expressions and voice data in real time while the user is entering travel information to extract emotional information. For example, if the user is excited, it determines that the user prefers an active scenario.

[0324] Step 3:

[0325] Server: Receives travel information and emotion information and converts it into an internal data format.

[0326] Step 4:

[0327] Server: Sends the converted travel information and emotion information as a request to the chat generation AI.

[0328] Step 5:

[0329] Chat generation AI: Generates travel scenarios based on received travel information and emotional data. For example, it generates a scenario including "Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion, Day 2: Sightseeing at Kiyomizu-dera Temple and Kinkaku-ji Temple, and activities to boost your mood."

[0330] Step 6:

[0331] Server: Receives travel scenarios generated by the AI ​​and sends them to the user.

[0332] 3. Customizing the scenario

[0333] Step 1:

[0334] Terminal: The user submits a modification request to an existing scenario by entering "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[0335] Step 2:

[0336] Emotion engine: Analyzes facial expressions and voice data in real time while the user is entering correction requests to extract emotional information. For example, if the user is feeling stressed, the engine will suggest plans to help them relax.

[0337] Step 3:

[0338] Server: Receives modification requests and emotion information and converts them into an internal data format.

[0339] Step 4:

[0340] Server: Resubmit the correction request and emotion information to the chat generation AI.

[0341] Step 5:

[0342] Chat generation AI: Based on revision requests and emotional information, it adjusts existing scenarios and generates new ones. For example, it generates a scenario that includes "Day 2: Sightseeing at Kiyomizu-dera Temple and Kinkaku-ji Temple, visiting Fushimi Inari Taisha Shrine, and experiencing a tea ceremony to relax."

[0343] Step 6:

[0344] Server: Receives modified scenarios from the AI ​​and sends them to the user.

[0345] 4. Share and save your scenarios

[0346] Step 1:

[0347] Device: The user selects whether to save the scenario or share it on LINE, and sends a request to the server.

[0348] Step 2:

[0349] Emotion engine: Analyzes facial expressions and voice data in real time while the user is operating the app, extracting emotional information. For example, if the user is excited, the engine suggests ways to share the information.

[0350] Step 3:

[0351] Server: If a save request is received, save the scenario to the database. If a share request is received, send the scenario to the specified contacts using the LINE API.

[0352] Step 4:

[0353] LINE API: Send the received scenario to the specified contacts. For example, send the scenario to the "Family" group with the message "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[0354] In this way, each processing step works together, allowing users to efficiently create, modify, save, and share customized travel scenarios based on emotion recognition.The collaboration of each element—server, device, chat generation AI, database, LINE API, and emotion engine—creates a system that improves the user experience.

[0355] Example 2

[0356] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0357] Conventional travel scenario generation systems have difficulty in properly reflecting users' emotions and individual needs. Furthermore, they lack the means to detect changes in users' emotions and optimize the scenarios based on those emotions during the process of generating, modifying, saving, and sharing the scenarios. As a result, user satisfaction cannot be sufficiently increased. Furthermore, the methods for sharing scenarios are limited, making it difficult to smoothly collaborate with other users.

[0358] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes terminal means for the user to input travel information, means for the terminal means to transmit the travel information, means for the server to receive the input travel information and convert it into a data format, means for the server to request the travel information from the chat generation AI, means for the chat generation AI to generate a travel scenario based on the received travel information, means for the emotion engine to recognize the user's emotions and generate data to reflect them in the travel scenario, and means for the server to transmit the generated travel scenario to the user. This makes it possible to generate, modify, save, and share customized travel scenarios that take the user's emotions into consideration.

[0359] "Terminal means" refers to a device that allows a user to input information and receive information from the system.

[0360] "Server" means a computing device that receives, processes, and stores information sent from terminal means over a network, and transmits the data to other systems or devices as needed.

[0361] The "means for converting into a data format" is a function for converting information received from a user into a format that can be used by other systems or processing modules.

[0362] "Chat generation AI" is an artificial intelligence system that generates natural language based on user input and creates specific scenarios and suggestions.

[0363] The "means for generating a travel scenario" is the process by which the chat generation AI formulates a specific travel plan based on the information it receives.

[0364] The "emotion engine" is an analytical engine that recognizes the user's emotions, generates them as data, and provides them to other systems.

[0365] "Means for generating data to be reflected in travel scenarios" refers to the function in which the emotion engine analyzes the user's emotions and creates data that can be used to generate or modify scenarios.

[0366] A "modification request" is a request made by a user to make changes or additions to an existing travel scenario.

[0367] The "terminal means for inputting a modification request" is a device that allows a user to input a request for a change or addition to a scenario.

[0368] The "means for sending a correction request and emotional information to the chat generation AI" is a function in which the server sends the input correction request and recognized emotional information to the chat generation AI, requesting that the scenario be corrected.

[0369] The "means for saving a scenario in a database" is a function for storing a generated or modified scenario in a database in order to save it permanently.

[0370] "Means for sharing scenarios using external APIs" is a function that allows the server to work with other applications and services to share scenarios.

[0371] This invention is a system that allows users to create, modify, save, and share travel scenarios while improving the user experience by combining an emotion engine that recognizes user emotions. This system is realized using devices, a server, chat generation AI, a database, an external API, and an emotion engine.

[0372] 1. User registration and authentication

[0373] overview:

[0374] To access the system, users first register, which creates a dedicated account for them.

[0375] Hardware and software used:

[0376] Terminal: The device through which the user enters information (PC, smartphone, etc.)

[0377] Server: A server that receives registration information and stores it in a database

[0378] Database: Data storage for storing registration information

[0379] Emotion engine: facial recognition and voice analysis software

[0380] 2. Requesting and Generating Travel Scenarios

[0381] overview:

[0382] Users input their travel details (destination, date, budget, etc.), and the chat generation AI generates a travel scenario based on that information. At this time, the emotion engine recognizes the user's emotions and reflects them in customizing the travel scenario.

[0383] Hardware and software used:

[0384] Terminal: A device through which a user enters information.

[0385] Server: A server for receiving travel information and emotion information and converting them into data formats.

[0386] Chat Generation AI: Artificial Intelligence for Generating Travel Scenarios

[0387] Emotion Engine: Software for recognizing user emotions

[0388] Database: Data storage for travel scenarios and user information

[0389] Examples:

[0390] Let's say a user inputs their travel destination "Kyoto," the dates "December 1, 2023 - December 3, 2023," and a budget of "50,000 yen" and sends them to the system. At this time, the emotion engine analyzes the user's facial expressions and voice data and recognizes emotions such as excitement. This allows the system to suggest an active sightseeing plan.

[0391] 3. Customizing the scenario

[0392] overview:

[0393] The user can request changes or additions to the generated travel scenario, and the emotion engine will recognize the user's emotions and reflect them in the modification request.

[0394] Hardware and software used:

[0395] Terminal: A device through which a user enters information.

[0396] Server: A server for receiving modification requests and emotion information and converting them into data formats.

[0397] Chat Generation AI: Artificial intelligence that generates revised scenarios

[0398] Emotion Engine: Software for recognizing user emotions

[0399] Database: Data storage for generated scenarios

[0400] Examples:

[0401] Let's say a user inputs "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" into an existing scenario and sends it to the system. At this time, the emotion engine analyzes the facial expressions and voice data while the user is inputting, and if the user is feeling stressed, it will suggest a plan that will help them relax.

[0402] 4. Share and save your scenarios

[0403] overview:

[0404] Users can save the scenarios they create or share them with other users through external APIs. In this case, the emotion engine also recognizes the user's emotions and suggests appropriate solutions.

[0405] Hardware and software used:

[0406] Terminal: A device through which a user enters information.

[0407] Server: The server that receives the share / save request.

[0408] Database: Data storage for saving scenarios

[0409] External API: Interface for connecting with external services such as the LINE API

[0410] Examples:

[0411] Let's say a user chooses to save the scenario or share it via the LINE API and sends a request to the system. If the emotion engine recognizes high tension, it will suggest a way to share it.

[0412] Example prompt sentence:

[0413] A specific prompt for a user to enter travel information might be "Destination: Kyoto, Date: December 1, 2023 - December 3, 2023, Budget: 50,000 yen." In addition, for a revision request, the user could enter "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[0414] As described above, the present invention realizes a system that takes into account the user's emotions and provides a more personalized travel scenario.

[0415] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0416] Step 1: User enters trip information

[0417] User: Enter details such as travel destination, date, and budget into the device. For example, enter "Kyoto," "December 1, 2023 - December 3, 2023," and "50,000 yen."

[0418] Terminal: Converts the input travel information into a data format and generates a request to send it to the server.

[0419] Input: Trip information entered by the user.

[0420] Output: A travel information request converted into a data format.

[0421] Step 2: The server receives the trip information

[0422] Server: Receives the travel information request sent from the device.

[0423] Input: A travel information request converted into a data format.

[0424] Output: Received trip information.

[0425] Step 3: The emotion engine recognizes the user's emotion

[0426] Emotion engine: Analyzes facial expressions and voice data when users input, and recognizes emotions. Uses emotion recognition APIs and voice analysis software.

[0427] Input: User's facial expression and voice data.

[0428] Output: Recognized emotion information.

[0429] Step 4: The server integrates the emotion information and the travel information.

[0430] Server: Integrates the received travel information and emotional information and converts it into a data format for sending to the chat generation AI.

[0431] Input: Received travel information and recognized emotion information.

[0432] Output: Data integrating travel information and emotional information.

[0433] Step 5: The server sends the integrated data to the chat generation AI.

[0434] Server: Sends data that integrates travel information and emotional information as a request to the chat generation AI.

[0435] Input: Data integrating travel information and emotional information.

[0436] Output: A request to send to the chat generation AI.

[0437] Step 6: Chat generation AI generates travel scenarios

[0438] Chat generation AI: Generates a travel scenario based on the received travel information and emotional information. For example, it creates a plan such as "Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion."

[0439] Input: Travel information and emotional information sent to the chat generation AI.

[0440] Output: The generated travel scenario.

[0441] Step 7: The server receives the generated travel scenario and sends it to the user.

[0442] Server: Receives the generated travel scenarios and converts them into a data format for transmission to the user.

[0443] Input: Generated travel scenarios.

[0444] Output: The travel scenario converted into a data format to be sent to the user.

[0445] Step 8: User receives and displays the travel scenario

[0446] Terminal: Receives the travel scenario sent from the server and displays it to the user.

[0447] Input: A travel scenario converted into a data format.

[0448] Output: The travel scenario displayed on the user's screen.

[0449] Step 9: User enters a request to modify the scenario

[0450] User: Request a change or addition to an existing scenario. For example, enter "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[0451] Terminal: Converts the modification request into a data format and generates a request to send to the server.

[0452] Input: The correction request entered by the user.

[0453] Output: The request for the modification request converted into data format.

[0454] Step 10: Server receives the modification request

[0455] Server: Receives the modification request sent from the device.

[0456] Input: The request for the correction request converted into data format.

[0457] Output: The correction request received.

[0458] Step 11: The emotion engine recognizes the user's emotion again

[0459] Emotion engine: Re-analyzes the user's facial and voice data while the correction request is being entered to recognize emotions.

[0460] Input: User's facial expression and voice data.

[0461] Output: Re-recognized emotion information.

[0462] Step 12: The server integrates the modification request with the emotion information.

[0463] Server: Integrates received correction requests and emotional information, and converts them into a data format for retransmission to the chat generation AI.

[0464] Input: Received correction request and re-recognized emotion information.

[0465] Output: Data integrating correction requests and emotional information.

[0466] Step 13: The server resends the integrated data to the chat generation AI.

[0467] Server: The data that integrates the correction request and emotional information is sent to the chat generation AI as a re-request.

[0468] Input: Data integrating correction requests and emotional information.

[0469] Output: A request to resend to the chat generation AI.

[0470] Step 14: Chat generation AI generates revised scenarios

[0471] Chat generation AI: Based on the received revision requests and emotional information, the AI ​​modifies the scenario. For example, it generates a revised plan such as "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[0472] Input: Edit request and sentiment sent to the chat generation AI.

[0473] Output: The modified travel scenario.

[0474] Step 15: The server receives the modified travel scenario and sends it to the user.

[0475] Server: Receives the modified travel scenario and converts it into a data format for transmission to the user.

[0476] Input: The modified travel scenario.

[0477] Output: The modified trip scenario converted into a data format for sending to the user.

[0478] Step 16: User receives and displays the revised travel scenario

[0479] Terminal: Receives the modified travel scenario sent from the server and displays it to the user.

[0480] Input: A modified travel scenario converted into a data format.

[0481] Output: The revised trip scenario displayed on the user's screen.

[0482] Step 17: User saves or shares the scenario

[0483] User: Choose whether to store the generated scenario in a database or use an external API to share the scenario with other users.

[0484] Device: Generates a save or share request to convert it into a data format and send it to the server.

[0485] Input: The save or share request selected by the user.

[0486] Output: The save or share request converted into a data format.

[0487] Step 18: Server receives save request

[0488] Server: Receives the save request sent from the device.

[0489] Input: The save request converted to a data format.

[0490] Output: The query to save to the database.

[0491] Step 19: The server saves the scenario to the database

[0492] Server: Saves the scenario to the database based on the save request.

[0493] Input: The query to save to the database.

[0494] Output: The scenario stored in the database.

[0495] Step 20: Share your scenario using an external API

[0496] Server: When a sharing request is received, it uses an external API to send the scenario to the specified contacts.

[0497] External API: Use LINE API, etc., to send the scenario as a message to the specified contact.

[0498] Input: The share request converted into data format.

[0499] Output: The scenario shared with the specified contact.

[0500] In this way, at each processing step, the input data is processed and the process moves to the next step, making it possible to generate, modify, save, and share customized travel scenarios that take the user's emotions into account.

[0501] (Application example 2)

[0502] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0503] Conventional advertising systems have difficulty targeting users based on their real-time emotions. This can result in the inability to provide ads that best fit the user's interests and emotions, resulting in insufficient advertising effectiveness. Another issue is that there is no established method for accurately capturing user emotions and generating ads based on them. Furthermore, the process of generating emotion-based ads is complicated, making effective ad delivery difficult.

[0504] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means for a user to input information, means for the server to receive the input information and convert it into a data format, means for the server to request information from a chat generation AI, means for the chat generation AI to generate a scenario based on the received information, means for the server to send the generated scenario to the user, and means for the emotion engine to recognize the user's emotions and reflect them in the input data. This makes it possible to generate and deliver personalized advertisements based on the user's real-time emotions.

[0505] The "terminal means through which the user inputs information" refers to an electronic device that is operated by the user to input information into the system.

[0506] "Means by which the server receives input information and converts it into a data format" refers to the process by which the server receives information sent by the user and converts that information into an appropriate format.

[0507] "Means for the server to request information from the chat generation AI" refers to a method in which the server uses the converted information to request processing from the chat generation AI.

[0508] "Means for generating a scenario based on information received by the chat generation AI" refers to the function by which the chat generation AI utilizes input information to automatically generate a scenario that suits a specific purpose.

[0509] "Means by which the server sends the generated scenario to the user" refers to the method by which the server delivers the scenario generated by the chat generation AI to the user.

[0510] "Means by which the emotion engine recognizes the user's emotions and reflects them in the input data" refers to the process of analyzing the user's emotions using emotion recognition technology and reflecting the results in the input data and scenario generation.

[0511] The "terminal means for the user to input a request to modify a scenario" is a device for the user to input information for modifying or adding to an existing scenario.

[0512] "Means for the server to resend the correction request to the chat generation AI" refers to a method in which the server resends the correction request received from the user to the chat generation AI.

[0513] "Means for the chat generation AI to modify the scenario based on a modification request" refers to a function that allows the chat generation AI to modify an existing scenario in response to a user's modification request.

[0514] The "means by which the server sends the modified scenario to the user" refers to the method by which the server sends the modified scenario to the user.

[0515] "Means for the emotion engine to recognize the user's emotions and reflect them in the modification request" is a process of understanding the user's emotions using emotion recognition technology and applying them to the scenario modification request.

[0516] The "terminal means for inputting a request from a user to save or share a generated scenario" is a device for a user to select and instruct whether to save a generated scenario or share it with other users.

[0517] The "means by which the server stores the scenario in the database" refers to the method by which the server records the scenario in the database for long-term storage.

[0518] "Means for the server to share a scenario using a communication platform API" refers to a method in which the server uses an API to share a scenario with other users via a communication network.

[0519] "Means for the emotion engine to recognize the user's emotions and reflect them in the sharing method" is a process in which the emotion engine determines the user's emotions and reflects the results in the sharing method of the scenario.

[0520] To realize the present invention, the following system configuration and processing procedures are required.

[0521] System Configuration

[0522] 1. User Device:

[0523] This is a device that users use to input information. This device can be a smartphone, tablet, or PC, and is used by installing a dedicated application.

[0524] 2. Server:

[0525] It receives information and converts it into a data format. It also has the function of requesting information from the chat generation AI and sending the generated scenario to the user. The server is a high-performance computer and can use cloud services that can process large amounts of data.

[0526] 3. Chat generation AI:

[0527] AI models with advanced natural language generation capabilities, such as OpenAI's ChatGPT® model, generate scenarios based on the information they receive and modify them according to user requests.

[0528] 4. Emotion Engine:

[0529] Using emotion recognition technologies such as Emotibot and AFFDEX SDK, the system analyzes user emotions in real time, and this emotional information is reflected in travel scenarios and advertisement generation.

[0530] 5. Database:

[0531] This is a database for storing user information, scenarios, emotional data, etc. A general relational database management system (RDBMS) can be used for this.

[0532] 6. Communication Platform API:

[0533] A communication platform API, such as the LINE API, for sharing generated scenarios with users.

[0534] Program Description

[0535] 1. User registration and authentication:

[0536] A user installs the application and registers by entering their name, email address, and password. The server receives this information, stores it in a database, and the password is hashed for security. A registration completion message is sent to the user.

[0537] Examples:

[0538] User "Taro Tanaka" registers in the app by entering his name "Taro Tanaka", email address "taro.tanaka@example.com", and password "securepassword".

[0539] 2. Emotion recognition and information input:

[0540] When users input travel destination or advertisement information, the emotion engine analyzes facial expressions and voice to recognize emotions, and this information is sent to the server in real time.

[0541] Examples:

[0542] When a user enters information for a trip to "Kyoto," the system recognizes the emotion of excitement and suggests an active sightseeing plan based on the emotion.

[0543] 3. Scenario generation:

[0544] The server sends a request to the chat generation AI based on the user information and emotion information. The AI ​​generates a scenario based on this information, and the server sends the generated scenario to the user.

[0545] Example prompt sentence:

[0546] "User information: Taro Tanaka, Interest: Travel, Emotion: Excitement. Please generate an appropriate travel scenario."

[0547] 4. Modify the scenario:

[0548] When a user inputs a request to modify a scenario, the emotion engine recognizes the emotion at that time and reflects it in the modification request. The server resends this information to the AI, and a modified scenario is generated.

[0549] Examples:

[0550] If a user inputs "I want to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" and the system recognizes that the user is feeling relaxed, it will add plans that allow for relaxation.

[0551] 5. Save and share your scenarios:

[0552] When a user inputs a request to save or share the generated scenario, the emotion engine analyzes the emotion at that time and suggests the best way to save or share it. The server saves the scenario in a database and shares it using the communication platform API.

[0553] Examples:

[0554] When a user requests to share a generated scenario with a "contact group," if excitement is recognized, a sharing message will be sent according to that emotion.

[0555] This will realize a system that effectively creates, modifies, saves, and shares travel scenarios using an advanced emotion recognition system based on user experience.

[0556] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0557] Step 1:

[0558] The user accesses a terminal means for entering information and enters their name, email address, and password into the registration form. The input data is submitted, and the server receives this information. The password is hashed for security purposes, and all information is stored in a database. The server generates a message indicating that registration is complete and sends it to the terminal.

[0559] Step 2:

[0560] The user inputs travel destination and advertisement information into the device. At this time, the emotion engine analyzes the user's facial expressions and voice in real time to recognize emotions. The device then sends the input information and recognized emotion information to the server, which converts it into a data format.

[0561] Step 3:

[0562] The server sends the converted travel information and emotion information as a request to the chat generation AI. The chat generation AI processes the information based on the prompt and generates an appropriate scenario. This scenario is then sent back to the server.

[0563] Step 4:

[0564] Once the server receives the generated scenario, it sends it to the user, who can review it and enter correction requests as needed.

[0565] Step 5:

[0566] The user inputs a revision request and sends it from their device to the server. The emotion engine again recognizes the user's emotions and reflects them in the revision request. The server again sends the revision request and emotion information to the chat generation AI and requests that the scenario be revised.

[0567] Step 6:

[0568] The chat generation AI regenerates the scenario based on the modification request and emotional information, and sends the modified scenario back to the server, which then sends it to the user.

[0569] Step 7:

[0570] The user inputs a request to save or share the generated scenario, which is sent from the device to the server. The emotion engine recognizes the user's emotions and suggests appropriate ways to save or share them. The server then saves the scenario to a database or shares it using the communication platform API.

[0571] Example input and output:

[0572] Step 1 input:

[0573] Name: Taro Tanaka

[0574] Email address: taro.tanaka@example.com

[0575] Password: securepassword

[0576] Step 1 output:

[0577] Registration completion message

[0578] Step 2 input:

[0579] Travel destination: Kyoto

[0580] Dates: December 1, 2023 - December 3, 2023

[0581] Budget: 50,000 yen

[0582] Perceived Emotion: Excitement

[0583] Step 2 output:

[0584] Travel information and sentiment information format conversion data

[0585] Step 3 Input:

[0586] Formatted travel and sentiment information

[0587] Prompt: "User information: Taro Tanaka, Travel destination: Kyoto, Date: December 1, 2023 - December 3, 2023, Budget: ¥50,000, Emotion: Excitement. Please generate an appropriate travel scenario."

[0588] Step 3 output:

[0589] Generated travel scenario (e.g., Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion)

[0590] Step 4 Input:

[0591] Generated scenarios

[0592] Step 4 output:

[0593] Providing scenarios to users

[0594] Input for step 5:

[0595] Request an amendment (e.g., visit Fushimi Inari Shrine in the afternoon of the second day)

[0596] Perceived Emotion: Relaxed

[0597] Step 5 output:

[0598] Format conversion data for correction requests and emotional information

[0599] Input for step 6:

[0600] Formatted correction requests and sentiment information

[0601] Prompt: "Please revise the scenario based on the revision request. Revision request: Visit to Fushimi Inari Shrine in the afternoon of the second day, Emotion: Relaxation"

[0602] Output from step 6:

[0603] Modified scenario (e.g. Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine)

[0604] Input for step 7:

[0605] Request to save or share

[0606] Recognized emotion: Excitement

[0607] Output from step 7:

[0608] Saved scenario data

[0609] Shared scenario messages

[0610] This allows users to create, modify, save, and share personalized scenarios based on their emotions.

[0611] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0612] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0613] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0614] [Second embodiment]

[0615] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0616] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0617] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0618] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0619] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0620] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0621] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0622] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0623] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0624] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0625] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0626] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0627] This invention provides a system that allows users to easily create, edit, save, and share travel scenarios. This system is realized using devices, a server, chat generation AI, a database, and the LINE API.

[0628] 1. User registration and authentication

[0629] overview:

[0630] To access the system, users first register, which creates a dedicated account for them.

[0631] What happens:

[0632] Device: The user enters their name, email address, and password into the registration form. For example, the user enters their name "Taro Tanaka," their email address "taro.tanaka@example.com," and their password "securepassword," and submits the form.

[0633] Server: Receives registration information and stores it in a database. A hashed password is also generated and stored.

[0634] Server: Notify the user that registration was successful.

[0635] 2. Requesting and Generating Travel Scenarios

[0636] overview:

[0637] Users enter travel details (destination, date, budget, etc.), and the chat generation AI generates a travel scenario based on that information.

[0638] What happens:

[0639] Device: The user enters the travel destination "Kyoto," the date "December 1, 2023 - December 3, 2023," and the budget "50,000 yen," and submits.

[0640] Server: Receives travel information and converts it into an internal data format.

[0641] Server: The travel information converted into a data format is sent as a request to the chat generation AI.

[0642] Chat generation AI: Generates travel scenarios based on the received travel information. For example, it generates scenarios such as "Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion," "Day 2: Visit Kiyomizu-dera Temple and Kinkaku-ji Temple," and "Day 3: Visit Arashiyama, depart from Kyoto Station."

[0643] Server: Sends the generated travel scenario to the user.

[0644] 3. Customizing the scenario

[0645] overview:

[0646] The user can request changes or additions to the generated travel scenario.

[0647] What happens:

[0648] Device: The user enters "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" and submits a correction request.

[0649] Server: Receives the correction request and resends it to the chat generation AI.

[0650] Chat generation AI: Regenerates scenarios based on revision requests. For example, it generates a new scenario called "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[0651] Server: Sends the modified scenario to the user.

[0652] 4. Share and save your scenarios

[0653] overview:

[0654] Users can save the scenarios they create and share them with other users via LINE.

[0655] What happens:

[0656] Device: The user selects whether to save the scenario or share it on LINE and sends a request.

[0657] server:

[0658] For save: Save the scenario to the database.

[0659] For sharing: Send the scenario to the specified contact via the LINE API.

[0660] LINE API: Send the received scenario information to a specified group or contact. For example, send a message to the "Family" group saying, "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[0661] In this way, users can easily create, save, and share customized travel scenarios.The system that supports users' travel planning is realized by the collaboration of the server, terminals, chat generation AI, database, and LINE API.

[0662] The processing flow will be explained below.

[0663] 1. User registration and authentication

[0664] Step 1:

[0665] On your device: The user enters their name, email address, and password into the provided registration form.

[0666] Step 2:

[0667] Terminal: The user clicks the submit button, sending the input data to the server.

[0668] Step 3:

[0669] Server: Receives input data and checks whether it is entered correctly (validation check).

[0670] Step 4:

[0671] Server: Creates SQL queries to store registration information in a database and saves the data. Passwords are hashed for security purposes.

[0672] Step 5:

[0673] Server: Generates and sends back a message to the user indicating successful registration.

[0674] 2. Requesting and Generating Travel Scenarios

[0675] Step 1:

[0676] Device: The user enters travel destination "Kyoto," date "December 1, 2023 - December 3, 2023," and budget "50,000 yen," and clicks the send button.

[0677] Step 2:

[0678] Server: Receives travel information and converts the data into a format that can be processed by AI.

[0679] Step 3:

[0680] Server: Sends the converted travel information as a request to the chat generation AI.

[0681] Step 4:

[0682] Chat generation AI: Generates a travel scenario (e.g., Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion) based on the received travel information.

[0683] Step 5:

[0684] Server: Receives travel scenarios generated by the AI ​​and sends them to the user.

[0685] 3. Customizing the scenario

[0686] Step 1:

[0687] Terminal: The user inputs "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" to an existing scenario and submits a modification request.

[0688] Step 2:

[0689] Server: Receives the correction request and sends it back to the chat generation AI.

[0690] Step 3:

[0691] Chat generation AI: Based on modification requests, it adjusts existing scenarios and generates new ones (e.g., Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine).

[0692] Step 4:

[0693] Server: Receives modified scenarios from the AI ​​and sends them to the user.

[0694] 4. Share and save your scenarios

[0695] Step 1:

[0696] Device: The user selects whether to save the scenario or share it on LINE, and sends a request to the server.

[0697] Step 2:

[0698] Server: If a save request is received, it saves the scenario to the database.

[0699] Step 3:

[0700] Server: When a sharing request is received, the server uses the LINE API to send the scenario to the specified contacts.

[0701] Step 4:

[0702] LINE API: Send the received scenario to the specified contacts. For example, send the scenario to the "Family" group with the message "Taro Tanaka shared a travel scenario."

[0703] In this way, each processing step works together, allowing users to efficiently create, modify, save, and share travel scenarios.

[0704] Example 1

[0705] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0706] Conventional travel planning systems have made it difficult for users to efficiently generate, modify, and share travel scenarios. Specifically, it is difficult to generate an appropriate scenario based on the travel information entered by the user, and customizing and modifying the scenario takes time. In addition, there is a lack of a way to easily share the generated scenario with other users.

[0707] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0708] In this invention, the server includes terminal means for a user to input travel information, means for the server to receive the input travel information and convert it into a data format, means for the server to request the travel information from the generative AI model, means for the generative AI model to generate a travel scenario based on the received travel information, means for the server to send the generated travel scenario to the user, terminal means for a user to input a request to modify the scenario, means for the server to resend the modification request to the generative AI model, means for the generative AI model to modify the scenario based on the modification request, means for the server to send the modified scenario to the user, terminal means for a user to input a request to save or share the generated scenario, means for the server to save the scenario in a database, and means for the server to share the scenario using a communication API. This enables users to easily generate travel scenarios, modify them as needed, and efficiently share them with other users.

[0709] "Terminal means" refers to a device or interface through which a user inputs travel information and modification requests and transmits them to a server.

[0710] The "server" is a central computer that processes travel information and modification requests received from users, converts them into data formats, and sends requests to the generative AI model.

[0711] "Means for converting into data format" refers to the process or algorithm for converting travel information received from a user into a format that is easy to process internally (e.g., JSON format).

[0712] The "means for making a request to the generative AI model" is a process for passing the converted travel information to the generative AI model and sending a request as a prompt sentence.

[0713] A "generative AI model" is an artificial intelligence system that generates travel scenarios based on input prompts.

[0714] The "means for generating a travel scenario" is a function that enables the generative AI model to create a specific travel plan based on travel information.

[0715] The "means for resending a modification request to the generative AI model" is a process for receiving a scenario modification request from a user and sending it again as a prompt to the generative AI model.

[0716] A "communication API" is a communication protocol or interface used to share generated travel scenarios with other users, and is a means for connecting to external systems (e.g., SNS applications).

[0717] This invention provides a system that allows users to easily create, modify, save, and share travel scenarios. This system is realized using terminals, servers, generative AI models, databases, and communication APIs.

[0718] Hardware and software used

[0719] Device: User device such as a smartphone or computer

[0720] Server: A central computer for information processing and database management.

[0721] Generative AI model: Chat generation AI (e.g. GPT-3)

[0722] Database: Data storage for storing and managing information (e.g., SQL database)

[0723] Communication API: Interface for data communication with external systems (e.g., RESTful API)

[0724] User Registration and Authentication

[0725] User: Enter your name, email address, and password, and send them from your device to the server. For example, enter the name "Taro Tanaka," the email address "taro.tanaka@example.com," and the password "securepassword."

[0726] Server: Stores the received information in a database, hashes the password, and notifies the user that registration was successful.

[0727] Requesting and generating travel scenarios

[0728] User: Enters details such as travel destination, date, and budget, and sends them to the server from the device. For example, enters "Kyoto," "December 1, 2023 - December 3, 2023," and "50,000 yen."

[0729] Server: Converts received travel information into a data format and sends it to the generative AI model.

[0730] Generative AI model: Generates travel scenarios based on received prompts.

[0731] Example prompt: "I would like to create a scenario for traveling to Kyoto. The travel dates are from December 1, 2023 to December 3, 2023, and the budget is 50,000 yen. Please suggest a scenario."

[0732] Server: Sends the generated travel scenario to the user.

[0733] Customizing travel scenarios

[0734] User: Enter any corrections or additions to the scenario and send them to the server from the device. For example, you might enter, "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[0735] Server: Resubmits the correction request to the generative AI model.

[0736] Generative AI model: Regenerates scenarios based on modification requests.

[0737] Server: Sends the modified scenario to the user.

[0738] Save and share your travel scenarios

[0739] User: Selects to save or share the scenario and sends a request from the device to the server.

[0740] server:

[0741] For save: Save the scenario to the database.

[0742] For sharing: Use a communication API to send the scenario to an external system. For example, send a message saying "Taro Tanaka has shared a travel scenario" to a specified group or contact.

[0743] This allows users to easily create, modify, save, and share travel scenarios. Furthermore, by linking together the server, terminal, generative AI model, database, and communication API, a system that supports users in planning their trips is realized.

[0744] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0745] Step 1:

[0746] The user enters travel information. Specifically, the user enters their name, email address, and password into the terminal. For example, the user enters the name "Taro Tanaka," the email address "taro.tanaka@example.com," and the password "securepassword." This is the input data.

[0747] Step 2:

[0748] The terminal sends the entered information to the server. The entered data is sent to the server as is. For example, {"name": "Taro Tanaka", "email": "taro.tanaka@example.com", "password": "securepassword"}.

[0749] Step 3:

[0750] The server processes the received information and hashes the password. Specifically, it hashes "securepassword" using a hashing algorithm (e.g., SHA-256). This is the data processing step. For example, "securepassword" -> "5e884898da28047151d0e56f8dc6292773603d0d6aabbdd5d029d6ad1db1c5f7".

[0751] Step 4:

[0752] The server stores the hashed password along with other user information in a database, such as {"name": "Taro Tanaka", "email": "taro.tanaka@example.com", "password_hash": "5e884898da28047151d0e56f8dc6292773603d0d6aabbdd5d029d6ad1db1c5f7"} .

[0753] Step 5:

[0754] The server generates a message indicating that registration has been completed successfully and sends it to the terminal. The message is simple text data, such as "Registration completed."

[0755] Step 6:

[0756] The terminal displays the received message to the user, displaying the text "Registration complete."

[0757] Step 7:

[0758] The user inputs travel information such as destination, date, budget, etc. For example, the user inputs the destination "Kyoto," the date "December 1, 2023 - December 3, 2023," and the budget "50,000 yen." This becomes new input data.

[0759] Step 8:

[0760] The device sends the entered travel information to the server. This information is passed to the server as is. For example, {"destination": "Kyoto", "dates": "December 1, 2023 - December 3, 2023", "budget": "50,000 yen"}.

[0761] Step 9:

[0762] The server converts the received travel information into an internal data format. For example, it converts it into the JSON format used internally. The internal data becomes {"destination": "Kyoto", "dates": {"start": "2023-12-01", "end": "2023-12-03"}, "budget": 50000}.

[0763] Step 10:

[0764] The server sends the converted travel information to the generative AI model as a prompt. For example, the prompt might read, "I would like to create a scenario for traveling to Kyoto. The travel dates are from December 1, 2023 to December 3, 2023, and the budget is 50,000 yen. Please suggest a scenario."

[0765] Step 11:

[0766] The generative AI model generates a travel scenario based on the received prompt. For example, the generated scenario might be "Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion," "Day 2: Visit Kiyomizu-dera Temple and Kinkaku-ji Temple," or "Day 3: Visit Arashiyama, depart from Kyoto Station."

[0767] Step 12:

[0768] The generative AI model returns the generated travel scenario to the server, which then sends the received scenario to the user. The scenario text is sent to the user.

[0769] Step 13:

[0770] The device displays the generated travel scenario to the user. Specifically, the scenario text is displayed on the device screen. For example, "Day 1: Arrive at Kyoto Station, check in at hotel, stroll around Gion..."

[0771] Step 14:

[0772] The user inputs a modification request for the generated scenario. For example, the user might input, "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day." This becomes the input data for the modification request.

[0773] Step 15:

[0774] The device sends a modification request to the server, which passes the request to the server as is. For example, {"modification": "I also want to visit Fushimi Inari Taisha Shrine in the afternoon of the second day"}.

[0775] Step 16:

[0776] The server resubmits the correction request to the generative AI model. For example, the correction request might be, "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[0777] Step 17:

[0778] The generative AI model regenerates the scenario based on the modification request. For example, the regenerated scenario might be "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[0779] Step 18:

[0780] The generative AI model sends the modified scenario back to the server, which then sends the received scenario to the user. The user receives the regenerated scenario text.

[0781] Step 19:

[0782] The terminal displays the revised scenario to the user. Specifically, the text of the regenerated scenario is displayed on the terminal screen. For example, "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[0783] Step 20:

[0784] The user inputs a request to save or share a scenario. For example, they select from the options "Save scenario" or "Share on LINE." This becomes the input data for the save or share request.

[0785] Step 21:

[0786] The device sends a request to the server. The request content is passed to the server as is. For example, for "save" it is {"action": "save"}, and for sharing it is {"action": "share"}.

[0787] Step 22:

[0788] The server processes the request accordingly. If save is selected, the scenario is saved to the database. For example, the following data is saved to the database: {"scenario": "Day 1: Arrive at Kyoto Station, check in at hotel, stroll around Gion..."}

[0789] Step 23:

[0790] If sharing is selected, the server uses a communication API to send the scenario to an external system. Specifically, it sends the scenario information to the specified group or contact. For example, it sends a message saying, "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[0791] Step 24:

[0792] The communication API sends the received scenario information to the specified group or contact, allowing users to share the scenarios they have created with other users.

[0793] Step 25:

[0794] The server notifies the user that the saving or sharing is complete, for example, "The scenario has been saved" or "The scenario has been shared."

[0795] Step 26:

[0796] The device displays a notification message to the user, specifically, a message such as "Scenario saved" or "Scenario shared" on the device screen.

[0797] (Application example 1)

[0798] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0799] Conventional food delivery services often lack the flexibility to accommodate specific user meal requests. Furthermore, when users customize their meal menus or have complex dietary restrictions, it is difficult to quickly provide suggestions that meet those requests. Furthermore, they lack the ability to easily share or save generated menus with other users. There is a need for a food delivery system that can meet these user needs.

[0800] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0801] In this invention, the server includes a terminal means for a user to input meal information, a means for the server to receive the input meal information and convert it into a data format, a means for the server to request the meal information from a generative AI model, a means for the generative AI model to generate a meal menu based on the received meal information, a means for the server to send the generated meal menu to the user, and a means for customizing the meal menu. This allows users to quickly and flexibly receive menu suggestions based on specific meal requests. Furthermore, the generated menu can be easily customized, saved, and shared.

[0802] A "user" is an individual who utilizes the system to input meal information and receive or customize the generated menu.

[0803] "Meal information" refers to information including the details of the meal desired by the user, such as the type of meal, budget, and specific dietary restrictions.

[0804] "Terminal means" refers to a device used by a user to input meal information, and specifically refers to a smartphone, tablet, etc.

[0805] A "server" is a computer system that receives and processes input information from a user.

[0806] "Means for converting into data format" refers to the function of converting the dietary information entered by the user into a format that can be understood by the generative AI model.

[0807] A "generative AI model" is an artificial intelligence model that generates optimal meal menus based on input meal information.

[0808] "Means for requesting" refers to the function by which the server sends the meal information received from the user to the generative AI model.

[0809] A "meal menu" is a specific meal suggestion generated by a generative AI model based on dietary information.

[0810] "Customizable means" refers to a function that allows a user to request changes or additions to the meal menu they receive.

[0811] The "means for saving" is a function that allows a user to save the menu created in a database within the system.

[0812] "Means for sharing" is a function for transmitting the generated menu using a communication means in order to share it with other users.

[0813] This invention provides a food delivery system that allows users to input their desired meal details, and a generative AI model proposes and customizes meal menus based on that information, which can then be saved and shared. The system is configured using the following hardware and software:

[0814] Hardware

[0815] Server: A computer system for receiving and processing meal information from users.

[0816] Terminal: The device through which the user inputs meal information. Examples include smartphones and tablets.

[0817] software

[0818] Python (Django, Flask, etc.): A framework for performing server-side processing.

[0819] MySQL: Used as a database management system.

[0820] LINE API: Communication method for sharing menus.

[0821] OpenAI's generative AI model: Used to generate menus based on meal information.

[0822] Program processing

[0823] The server receives the meal information sent by the user and converts it into a data format. The converted data is sent as a request to the generative AI model, which then generates a meal menu based on the meal information. For example, based on the information that "the user requested pizza as the meal type and their budget is 2,000 yen," the generative AI model would suggest the optimal pizza menu.

[0824] The generated meal menu is sent to the user by the server. If the user subsequently wishes to customize the menu, they can input a modification request from their device, which is then sent again to the generative AI model via the server. The generative AI model then generates a new menu based on the modification request and sends it to the user via the server.

[0825] If the user saves the created menu, the server saves it in the database. If the user wishes to share the menu, the server uses the LINE API to send the menu to the specified contacts or group. This allows users to easily customize, save, and share their desired meal menu.

[0826] Specific examples

[0827] If a user inputs a request such as "vegan pizza, budget 3,000 yen," the generative AI model will receive the following prompts and generate the optimal menu based on that request:

[0828] Example prompt sentence:

[0829] Based on user requests, please suggest a vegan pizza menu within a budget of 3,000 yen.

[0830] When a user receives the generated menu and sends a modification request, such as "I'd like to add an olive topping," the generative AI model reprocesses it and generates the following modification prompt:

[0831] Example of a modified prompt:

[0832] Add olive topping to your current menu and regenerate.

[0833] The system allows users to efficiently obtain customized meal menus that can be easily saved and shared with other users.

[0834] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0835] Step 1:

[0836] The user inputs meal information. Using a device such as a smartphone or tablet, the user inputs detailed information such as meal type (e.g., pizza) and budget (e.g., 2,000 yen) and submits it. The input data is sent to the server.

[0837] Step 2:

[0838] The server receives the meal information and converts it into a data format. The server analyzes the raw data received from the user and converts it into a format that the generative AI model can understand, for example, JSON format. The input is the user's meal information, and the output is the converted data format.

[0839] Step 3:

[0840] The server sends a request to the generative AI model. The converted data format is used to send a prompt to the generative AI model (for example, OpenAI's GPT model). The specific prompt generated is, "Based on the user's request, please suggest a pizza menu item within a budget of 2,000 yen."

[0841] Step 4:

[0842] The generative AI model generates a meal menu. Based on the prompt received, the generative AI model generates the optimal meal menu. The input is the prompt, and the output is the generated meal menu. For example, a menu such as "Margherita pizza, topped with basil and cheese, priced at 1,800 yen" is generated.

[0843] Step 5:

[0844] The server sends the generated meal menu to the user. The server then sends the meal menu received from the generative AI model back to the user's device. The user can then view the proposed menu on their device.

[0845] Step 6:

[0846] The user inputs a menu customization request. For example, the user sends a request to change the proposed menu, such as "I would like to add olives to the toppings," to the server via the terminal.

[0847] Step 7:

[0848] The server sends the customization request to the generative AI model. The server receives the modification request from the user, reformats it, and sends it back to the generative AI model. The specific prompt generated is "Please add olive topping to the current menu and regenerate it."

[0849] Step 8:

[0850] The generative AI model generates a modified meal menu. The generative AI model generates a new menu based on the modification request. The input is the modification prompt, and the output is the modified meal menu.

[0851] Step 9:

[0852] The server sends the revised menu to the user, and upon receiving the revised menu, the server sends it again to the user's terminal.

[0853] Step 10:

[0854] When a user wants to save a generated menu, he sends a save request to the server through the terminal. The server saves this menu in the database. The input is the menu save request and the output is the saved menu.

[0855] Step 11:

[0856] When a user wants to share a created menu, they send a share request to the server via their device. The server then shares this menu with other users using the LINE API, etc. The input is the share request, and the output is the shared menu.

[0857] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0858] This invention provides a system that allows users to create, modify, save, and share travel scenarios while improving the user experience by combining an emotion engine that recognizes user emotions. This system is realized using devices, a server, chat generation AI, a database, the LINE API, and an emotion engine.

[0859] 1. User registration and authentication

[0860] overview:

[0861] To access the system, users first register, which creates a dedicated account for them.

[0862] What happens:

[0863] Terminal: The user enters their name, email address, and password into the registration form. For example, they enter the username "Taro Tanaka," the email address "taro.tanaka@example.com," and the password "securepassword," and submit.

[0864] Server: Receives registration information and stores it in a database. Passwords are hashed for security purposes.

[0865] Server: Generates and sends a message to the user indicating successful registration.

[0866] 2. Requesting and Generating Travel Scenarios

[0867] overview:

[0868] Users input their travel details (destination, date, budget, etc.), and the chat generation AI generates a travel scenario based on that information. At this time, the emotion engine recognizes the user's emotions and reflects them in customizing the travel scenario.

[0869] What happens:

[0870] Device: The user enters travel destination "Kyoto," date "December 1, 2023 - December 3, 2023," and budget "50,000 yen," and clicks the send button.

[0871] Emotion engine: Analyzes facial expressions and voice data while the user is typing to recognize emotions. For example, if the user is excited, it will suggest an active sightseeing plan.

[0872] Server: Receives travel information and emotion information and converts it into a data format.

[0873] Server: Sends the converted travel information and emotion information as a request to the chat generation AI.

[0874] Chat generation AI: Generates a travel scenario (e.g., Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion) based on received travel information and emotional information.

[0875] Server: Sends the generated travel scenario to the user.

[0876] 3. Customizing the scenario

[0877] overview:

[0878] The user can request changes or additions to the generated travel scenario, and the emotion engine will recognize the user's emotions and reflect them in the modification request.

[0879] What happens:

[0880] Terminal: The user inputs "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" to an existing scenario and submits a modification request.

[0881] Emotion engine: Analyzes facial and voice data while the user is entering a correction request to recognize emotions. For example, if the user is feeling stressed, the engine can suggest relaxation plans.

[0882] Server: Receives the modification request and emotion information, and sends the request again to the chat generation AI.

[0883] Chat generation AI: Based on modification requests and emotional information, it adjusts existing scenarios and generates new ones (e.g., Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine).

[0884] Server: Sends the modified scenario to the user.

[0885] 4. Share and save your scenarios

[0886] overview:

[0887] Users can save the scenarios they create or share them with other users via LINE. In this case, the emotion engine also recognizes the user's emotions and suggests appropriate ways to share them.

[0888] What happens:

[0889] Device: The user selects whether to save the scenario or share it on LINE, and sends a request to the server.

[0890] Emotion engine: Analyzes facial expressions and voice data while the user is operating the app to recognize emotions. For example, if the user is excited, the engine will suggest ways to share the app.

[0891] Server: If a save request is received, it saves the scenario to the database.

[0892] Server: When a sharing request is received, the server uses the LINE API to send the scenario to the specified contacts.

[0893] LINE API: Send the received scenario to the specified contacts. For example, to the "Family" group, send the scenario with the message "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[0894] In this way, each processing step works together, allowing users to efficiently create, modify, save, and share customized travel scenarios based on emotion recognition.The collaboration of each element—server, device, chat generation AI, database, LINE API, and emotion engine—creates a system that improves the user experience.

[0895] The processing flow will be explained below.

[0896] 1. User registration and authentication

[0897] Step 1:

[0898] On your device: The user enters their name, email address, and password into the provided registration form.

[0899] Step 2:

[0900] Terminal: The user clicks the submit button, sending the input data to the server.

[0901] Step 3:

[0902] Server: Receives input data and checks whether it is entered correctly (validation check).

[0903] Step 4:

[0904] Server: Creates SQL queries to store registration information in a database and saves the data. Passwords are hashed for security purposes.

[0905] Step 5:

[0906] Server: Generates and sends back a message to the user indicating successful registration.

[0907] 2. Requesting and Generating Travel Scenarios

[0908] Step 1:

[0909] Device: The user enters travel destination "Kyoto," date "December 1, 2023 - December 3, 2023," and budget "50,000 yen," and clicks the send button.

[0910] Step 2:

[0911] Emotion engine: Analyzes facial expressions and voice data in real time while the user is entering travel information to extract emotional information. For example, if the user is excited, it determines that the user prefers an active scenario.

[0912] Step 3:

[0913] Server: Receives travel information and emotion information and converts it into an internal data format.

[0914] Step 4:

[0915] Server: Sends the converted travel information and emotion information as a request to the chat generation AI.

[0916] Step 5:

[0917] Chat generation AI: Generates travel scenarios based on received travel information and emotional data. For example, it generates a scenario including "Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion, Day 2: Sightseeing at Kiyomizu-dera Temple and Kinkaku-ji Temple, and activities to boost your mood."

[0918] Step 6:

[0919] Server: Receives travel scenarios generated by the AI ​​and sends them to the user.

[0920] 3. Customizing the scenario

[0921] Step 1:

[0922] Terminal: The user submits a modification request to an existing scenario by entering "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[0923] Step 2:

[0924] Emotion engine: Analyzes facial expressions and voice data in real time while the user is entering correction requests to extract emotional information. For example, if the user is feeling stressed, the engine will suggest plans to help them relax.

[0925] Step 3:

[0926] Server: Receives modification requests and emotion information and converts them into an internal data format.

[0927] Step 4:

[0928] Server: Resubmit the correction request and emotion information to the chat generation AI.

[0929] Step 5:

[0930] Chat generation AI: Based on revision requests and emotional information, it adjusts existing scenarios and generates new ones. For example, it generates a scenario that includes "Day 2: Sightseeing at Kiyomizu-dera Temple and Kinkaku-ji Temple, visiting Fushimi Inari Taisha Shrine, and experiencing a tea ceremony to relax."

[0931] Step 6:

[0932] Server: Receives modified scenarios from the AI ​​and sends them to the user.

[0933] 4. Share and save your scenarios

[0934] Step 1:

[0935] Device: The user selects whether to save the scenario or share it on LINE, and sends a request to the server.

[0936] Step 2:

[0937] Emotion engine: Analyzes facial expressions and voice data in real time while the user is operating the app, extracting emotional information. For example, if the user is excited, the engine suggests ways to share the information.

[0938] Step 3:

[0939] Server: If a save request is received, save the scenario to the database. If a share request is received, send the scenario to the specified contacts using the LINE API.

[0940] Step 4:

[0941] LINE API: Send the received scenario to the specified contacts. For example, send the scenario to the "Family" group with the message "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[0942] In this way, each processing step works together, allowing users to efficiently create, modify, save, and share customized travel scenarios based on emotion recognition.The collaboration of each element—server, device, chat generation AI, database, LINE API, and emotion engine—creates a system that improves the user experience.

[0943] Example 2

[0944] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0945] Conventional travel scenario generation systems have difficulty in properly reflecting users' emotions and individual needs. Furthermore, they lack the means to detect changes in users' emotions and optimize the scenarios based on those emotions during the process of generating, modifying, saving, and sharing the scenarios. As a result, user satisfaction cannot be sufficiently increased. Furthermore, the methods for sharing scenarios are limited, making it difficult to smoothly collaborate with other users.

[0946] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes terminal means for the user to input travel information, means for the terminal means to transmit the travel information, means for the server to receive the input travel information and convert it into a data format, means for the server to request the travel information from the chat generation AI, means for the chat generation AI to generate a travel scenario based on the received travel information, means for the emotion engine to recognize the user's emotions and generate data to reflect them in the travel scenario, and means for the server to transmit the generated travel scenario to the user. This makes it possible to generate, modify, save, and share customized travel scenarios that take the user's emotions into consideration.

[0947] "Terminal means" refers to a device that allows a user to input information and receive information from the system.

[0948] "Server" means a computing device that receives, processes, and stores information sent from terminal means over a network, and transmits the data to other systems or devices as needed.

[0949] The "means for converting into a data format" is a function for converting information received from a user into a format that can be used by other systems or processing modules.

[0950] "Chat generation AI" is an artificial intelligence system that generates natural language based on user input and creates specific scenarios and suggestions.

[0951] The "means for generating a travel scenario" is the process by which the chat generation AI formulates a specific travel plan based on the information it receives.

[0952] The "emotion engine" is an analytical engine that recognizes the user's emotions, generates them as data, and provides them to other systems.

[0953] "Means for generating data to be reflected in travel scenarios" refers to the function in which the emotion engine analyzes the user's emotions and creates data that can be used to generate or modify scenarios.

[0954] A "modification request" is a request made by a user to make changes or additions to an existing travel scenario.

[0955] The "terminal means for inputting a modification request" is a device that allows a user to input a request for a change or addition to a scenario.

[0956] The "means for sending a correction request and emotional information to the chat generation AI" is a function in which the server sends the input correction request and recognized emotional information to the chat generation AI, requesting that the scenario be corrected.

[0957] The "means for saving a scenario in a database" is a function for storing a generated or modified scenario in a database in order to save it permanently.

[0958] "Means for sharing scenarios using external APIs" is a function that allows the server to work with other applications and services to share scenarios.

[0959] This invention is a system that allows users to create, modify, save, and share travel scenarios while improving the user experience by combining an emotion engine that recognizes user emotions. This system is realized using devices, a server, chat generation AI, a database, an external API, and an emotion engine.

[0960] 1. User registration and authentication

[0961] overview:

[0962] To access the system, users first register, which creates a dedicated account for them.

[0963] Hardware and software used:

[0964] Terminal: The device through which the user enters information (PC, smartphone, etc.)

[0965] Server: A server that receives registration information and stores it in a database

[0966] Database: Data storage for storing registration information

[0967] Emotion engine: facial recognition and voice analysis software

[0968] 2. Requesting and Generating Travel Scenarios

[0969] overview:

[0970] Users input their travel details (destination, date, budget, etc.), and the chat generation AI generates a travel scenario based on that information. At this time, the emotion engine recognizes the user's emotions and reflects them in customizing the travel scenario.

[0971] Hardware and software used:

[0972] Terminal: A device through which a user enters information.

[0973] Server: A server for receiving travel information and emotion information and converting them into data formats.

[0974] Chat Generation AI: Artificial Intelligence for Generating Travel Scenarios

[0975] Emotion Engine: Software for recognizing user emotions

[0976] Database: Data storage for travel scenarios and user information

[0977] Examples:

[0978] Let's say a user inputs their travel destination "Kyoto," the dates "December 1, 2023 - December 3, 2023," and a budget of "50,000 yen" and sends them to the system. At this time, the emotion engine analyzes the user's facial expressions and voice data and recognizes emotions such as excitement. This allows the system to suggest an active sightseeing plan.

[0979] 3. Customizing the scenario

[0980] overview:

[0981] The user can request changes or additions to the generated travel scenario, and the emotion engine will recognize the user's emotions and reflect them in the modification request.

[0982] Hardware and software used:

[0983] Terminal: A device through which a user enters information.

[0984] Server: A server for receiving modification requests and emotion information and converting them into data formats.

[0985] Chat Generation AI: Artificial intelligence that generates revised scenarios

[0986] Emotion Engine: Software for recognizing user emotions

[0987] Database: Data storage for generated scenarios

[0988] Examples:

[0989] Let's say a user inputs "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" into an existing scenario and sends it to the system. At this time, the emotion engine analyzes the facial expressions and voice data while the user is inputting, and if the user is feeling stressed, it will suggest a plan that will help them relax.

[0990] 4. Share and save your scenarios

[0991] overview:

[0992] Users can save the scenarios they create or share them with other users through external APIs. In this case, the emotion engine also recognizes the user's emotions and suggests appropriate solutions.

[0993] Hardware and software used:

[0994] Terminal: A device through which a user enters information.

[0995] Server: The server that receives the share / save request.

[0996] Database: Data storage for saving scenarios

[0997] External API: Interface for connecting with external services such as the LINE API

[0998] Examples:

[0999] Let's say a user chooses to save the scenario or share it via the LINE API and sends a request to the system. If the emotion engine recognizes high tension, it will suggest a way to share it.

[1000] Example prompt sentence:

[1001] A specific prompt for a user to enter travel information might be "Destination: Kyoto, Date: December 1, 2023 - December 3, 2023, Budget: 50,000 yen." In addition, for a revision request, the user could enter "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[1002] As described above, the present invention realizes a system that takes into account the user's emotions and provides a more personalized travel scenario.

[1003] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1004] Step 1: User enters trip information

[1005] User: Enter details such as travel destination, date, and budget into the device. For example, enter "Kyoto," "December 1, 2023 - December 3, 2023," and "50,000 yen."

[1006] Terminal: Converts the input travel information into a data format and generates a request to send it to the server.

[1007] Input: Trip information entered by the user.

[1008] Output: A travel information request converted into a data format.

[1009] Step 2: The server receives the trip information

[1010] Server: Receives the travel information request sent from the device.

[1011] Input: A travel information request converted into a data format.

[1012] Output: Received trip information.

[1013] Step 3: The emotion engine recognizes the user's emotion

[1014] Emotion engine: Analyzes facial expressions and voice data when users input, and recognizes emotions. Uses emotion recognition APIs and voice analysis software.

[1015] Input: User's facial expression and voice data.

[1016] Output: Recognized emotion information.

[1017] Step 4: The server integrates the emotion information and the travel information.

[1018] Server: Integrates the received travel information and emotional information and converts it into a data format for sending to the chat generation AI.

[1019] Input: Received travel information and recognized emotion information.

[1020] Output: Data integrating travel information and emotional information.

[1021] Step 5: The server sends the integrated data to the chat generation AI.

[1022] Server: Sends data that integrates travel information and emotional information as a request to the chat generation AI.

[1023] Input: Data integrating travel information and emotional information.

[1024] Output: A request to send to the chat generation AI.

[1025] Step 6: Chat generation AI generates travel scenarios

[1026] Chat generation AI: Generates a travel scenario based on the received travel information and emotional information. For example, it creates a plan such as "Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion."

[1027] Input: Travel information and emotional information sent to the chat generation AI.

[1028] Output: The generated travel scenario.

[1029] Step 7: The server receives the generated travel scenario and sends it to the user.

[1030] Server: Receives the generated travel scenarios and converts them into a data format for transmission to the user.

[1031] Input: Generated travel scenarios.

[1032] Output: The travel scenario converted into a data format to be sent to the user.

[1033] Step 8: User receives and displays the travel scenario

[1034] Terminal: Receives the travel scenario sent from the server and displays it to the user.

[1035] Input: A travel scenario converted into a data format.

[1036] Output: The travel scenario displayed on the user's screen.

[1037] Step 9: User enters a request to modify the scenario

[1038] User: Request a change or addition to an existing scenario. For example, enter "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[1039] Terminal: Converts the modification request into a data format and generates a request to send to the server.

[1040] Input: The correction request entered by the user.

[1041] Output: The request for the modification request converted into data format.

[1042] Step 10: Server receives the modification request

[1043] Server: Receives the modification request sent from the device.

[1044] Input: The request for the correction request converted into data format.

[1045] Output: The correction request received.

[1046] Step 11: The emotion engine recognizes the user's emotion again

[1047] Emotion engine: Re-analyzes the user's facial and voice data while the correction request is being entered to recognize emotions.

[1048] Input: User's facial expression and voice data.

[1049] Output: Re-recognized emotion information.

[1050] Step 12: The server integrates the modification request with the emotion information.

[1051] Server: Integrates received correction requests and emotional information, and converts them into a data format for retransmission to the chat generation AI.

[1052] Input: Received correction request and re-recognized emotion information.

[1053] Output: Data integrating correction requests and emotional information.

[1054] Step 13: The server resends the integrated data to the chat generation AI.

[1055] Server: The data that integrates the correction request and emotional information is sent to the chat generation AI as a re-request.

[1056] Input: Data integrating correction requests and emotional information.

[1057] Output: A request to resend to the chat generation AI.

[1058] Step 14: Chat generation AI generates revised scenarios

[1059] Chat generation AI: Based on the received revision requests and emotional information, the AI ​​modifies the scenario. For example, it generates a revised plan such as "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[1060] Input: Edit request and sentiment sent to the chat generation AI.

[1061] Output: The modified travel scenario.

[1062] Step 15: The server receives the modified travel scenario and sends it to the user.

[1063] Server: Receives the modified travel scenario and converts it into a data format for transmission to the user.

[1064] Input: The modified travel scenario.

[1065] Output: The modified trip scenario converted into a data format for sending to the user.

[1066] Step 16: User receives and displays the revised travel scenario

[1067] Terminal: Receives the modified travel scenario sent from the server and displays it to the user.

[1068] Input: A modified travel scenario converted into a data format.

[1069] Output: The revised trip scenario displayed on the user's screen.

[1070] Step 17: User saves or shares the scenario

[1071] User: Choose whether to store the generated scenario in a database or use an external API to share the scenario with other users.

[1072] Device: Generates a save or share request to convert it into a data format and send it to the server.

[1073] Input: The save or share request selected by the user.

[1074] Output: The save or share request converted into a data format.

[1075] Step 18: Server receives save request

[1076] Server: Receives the save request sent from the device.

[1077] Input: The save request converted to a data format.

[1078] Output: The query to save to the database.

[1079] Step 19: The server saves the scenario to the database

[1080] Server: Saves the scenario to the database based on the save request.

[1081] Input: The query to save to the database.

[1082] Output: The scenario stored in the database.

[1083] Step 20: Share your scenario using an external API

[1084] Server: When a sharing request is received, it uses an external API to send the scenario to the specified contacts.

[1085] External API: Use LINE API, etc., to send the scenario as a message to the specified contact.

[1086] Input: The share request converted into data format.

[1087] Output: The scenario shared with the specified contact.

[1088] In this way, at each processing step, the input data is processed and the process moves to the next step, making it possible to generate, modify, save, and share customized travel scenarios that take the user's emotions into account.

[1089] (Application example 2)

[1090] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1091] Conventional advertising systems have difficulty targeting users based on their real-time emotions. This can result in the inability to provide ads that best fit the user's interests and emotions, resulting in insufficient advertising effectiveness. Another issue is that there is no established method for accurately capturing user emotions and generating ads based on them. Furthermore, the process of generating emotion-based ads is complicated, making effective ad delivery difficult.

[1092] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means for a user to input information, means for the server to receive the input information and convert it into a data format, means for the server to request information from a chat generation AI, means for the chat generation AI to generate a scenario based on the received information, means for the server to send the generated scenario to the user, and means for the emotion engine to recognize the user's emotions and reflect them in the input data. This makes it possible to generate and deliver personalized advertisements based on the user's real-time emotions.

[1093] The "terminal means through which the user inputs information" refers to an electronic device that is operated by the user to input information into the system.

[1094] "Means by which the server receives input information and converts it into a data format" refers to the process by which the server receives information sent by the user and converts that information into an appropriate format.

[1095] "Means for the server to request information from the chat generation AI" refers to a method in which the server uses the converted information to request processing from the chat generation AI.

[1096] "Means for generating a scenario based on information received by the chat generation AI" refers to the function by which the chat generation AI utilizes input information to automatically generate a scenario that suits a specific purpose.

[1097] "Means by which the server sends the generated scenario to the user" refers to the method by which the server delivers the scenario generated by the chat generation AI to the user.

[1098] "Means by which the emotion engine recognizes the user's emotions and reflects them in the input data" refers to the process of analyzing the user's emotions using emotion recognition technology and reflecting the results in the input data and scenario generation.

[1099] The "terminal means for the user to input a request to modify a scenario" is a device for the user to input information for modifying or adding to an existing scenario.

[1100] "Means for the server to resend the correction request to the chat generation AI" refers to a method in which the server resends the correction request received from the user to the chat generation AI.

[1101] "Means for the chat generation AI to modify the scenario based on a modification request" refers to a function that allows the chat generation AI to modify an existing scenario in response to a user's modification request.

[1102] The "means by which the server sends the modified scenario to the user" refers to the method by which the server sends the modified scenario to the user.

[1103] "Means for the emotion engine to recognize the user's emotions and reflect them in the modification request" is a process of understanding the user's emotions using emotion recognition technology and applying them to the scenario modification request.

[1104] The "terminal means for inputting a request from a user to save or share a generated scenario" is a device for a user to select and instruct whether to save a generated scenario or share it with other users.

[1105] The "means by which the server stores the scenario in the database" refers to the method by which the server records the scenario in the database for long-term storage.

[1106] "Means for the server to share a scenario using a communication platform API" refers to a method in which the server uses an API to share a scenario with other users via a communication network.

[1107] "Means for the emotion engine to recognize the user's emotions and reflect them in the sharing method" is a process in which the emotion engine determines the user's emotions and reflects the results in the sharing method of the scenario.

[1108] To realize the present invention, the following system configuration and processing procedures are required.

[1109] System Configuration

[1110] 1. User Device:

[1111] This is a device that users use to input information. This device can be a smartphone, tablet, or PC, and is used by installing a dedicated application.

[1112] 2. Server:

[1113] It receives information and converts it into a data format. It also has the function of requesting information from the chat generation AI and sending the generated scenario to the user. The server is a high-performance computer and can use cloud services that can process large amounts of data.

[1114] 3. Chat generation AI:

[1115] AI models with advanced natural language generation capabilities, such as OpenAI's ChatGPT model, generate scenarios based on the information they receive and modify them according to user requests.

[1116] 4. Emotion Engine:

[1117] Using emotion recognition technologies such as Emotibot and AFFDEX SDK, the system analyzes user emotions in real time, and this emotional information is reflected in travel scenarios and advertisement generation.

[1118] 5. Database:

[1119] This is a database for storing user information, scenarios, emotional data, etc. A general relational database management system (RDBMS) can be used for this.

[1120] 6. Communication Platform API:

[1121] A communication platform API, such as the LINE API, for sharing generated scenarios with users.

[1122] Program Description

[1123] 1. User registration and authentication:

[1124] A user installs the application and registers by entering their name, email address, and password. The server receives this information, stores it in a database, and the password is hashed for security. A registration completion message is sent to the user.

[1125] Examples:

[1126] User "Taro Tanaka" registers in the app by entering his name "Taro Tanaka", email address "taro.tanaka@example.com", and password "securepassword".

[1127] 2. Emotion recognition and information input:

[1128] When users input travel destination or advertisement information, the emotion engine analyzes facial expressions and voice to recognize emotions, and this information is sent to the server in real time.

[1129] Examples:

[1130] When a user enters information for a trip to "Kyoto," the system recognizes the emotion of excitement and suggests an active sightseeing plan based on the emotion.

[1131] 3. Scenario generation:

[1132] The server sends a request to the chat generation AI based on the user information and emotion information. The AI ​​generates a scenario based on this information, and the server sends the generated scenario to the user.

[1133] Example prompt sentence:

[1134] "User information: Taro Tanaka, Interest: Travel, Emotion: Excitement. Please generate an appropriate travel scenario."

[1135] 4. Modify the scenario:

[1136] When a user inputs a request to modify a scenario, the emotion engine recognizes the emotion at that time and reflects it in the modification request. The server resends this information to the AI, and a modified scenario is generated.

[1137] Examples:

[1138] If a user inputs "I want to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" and the system recognizes that the user is feeling relaxed, it will add plans that allow for relaxation.

[1139] 5. Save and share your scenarios:

[1140] When a user inputs a request to save or share the generated scenario, the emotion engine analyzes the emotion at that time and suggests the best way to save or share it. The server saves the scenario in a database and shares it using the communication platform API.

[1141] Examples:

[1142] When a user requests to share a generated scenario with a "contact group," if excitement is recognized, a sharing message will be sent according to that emotion.

[1143] This will realize a system that effectively creates, modifies, saves, and shares travel scenarios using an advanced emotion recognition system based on user experience.

[1144] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1145] Step 1:

[1146] The user accesses a terminal means for entering information and enters their name, email address, and password into the registration form. The input data is submitted, and the server receives this information. The password is hashed for security purposes, and all information is stored in a database. The server generates a message indicating that registration is complete and sends it to the terminal.

[1147] Step 2:

[1148] The user inputs travel destination and advertisement information into the device. At this time, the emotion engine analyzes the user's facial expressions and voice in real time to recognize emotions. The device then sends the input information and recognized emotion information to the server, which converts it into a data format.

[1149] Step 3:

[1150] The server sends the converted travel information and emotion information as a request to the chat generation AI. The chat generation AI processes the information based on the prompt and generates an appropriate scenario. This scenario is then sent back to the server.

[1151] Step 4:

[1152] Once the server receives the generated scenario, it sends it to the user, who can review it and enter correction requests as needed.

[1153] Step 5:

[1154] The user inputs a revision request and sends it from their device to the server. The emotion engine again recognizes the user's emotions and reflects them in the revision request. The server again sends the revision request and emotion information to the chat generation AI and requests that the scenario be revised.

[1155] Step 6:

[1156] The chat generation AI regenerates the scenario based on the modification request and emotional information, and sends the modified scenario back to the server, which then sends it to the user.

[1157] Step 7:

[1158] The user inputs a request to save or share the generated scenario, which is sent from the device to the server. The emotion engine recognizes the user's emotions and suggests appropriate ways to save or share them. The server then saves the scenario to a database or shares it using the communication platform API.

[1159] Example input and output:

[1160] Step 1 input:

[1161] Name: Taro Tanaka

[1162] Email address: taro.tanaka@example.com

[1163] Password: securepassword

[1164] Step 1 output:

[1165] Registration completion message

[1166] Step 2 input:

[1167] Travel destination: Kyoto

[1168] Dates: December 1, 2023 - December 3, 2023

[1169] Budget: 50,000 yen

[1170] Perceived Emotion: Excitement

[1171] Step 2 output:

[1172] Travel information and sentiment information format conversion data

[1173] Step 3 Input:

[1174] Formatted travel and sentiment information

[1175] Prompt: "User information: Taro Tanaka, Travel destination: Kyoto, Date: December 1, 2023 - December 3, 2023, Budget: ¥50,000, Emotion: Excitement. Please generate an appropriate travel scenario."

[1176] Step 3 output:

[1177] Generated travel scenario (e.g., Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion)

[1178] Step 4 Input:

[1179] Generated scenarios

[1180] Step 4 output:

[1181] Providing scenarios to users

[1182] Input for step 5:

[1183] Request an amendment (e.g., visit Fushimi Inari Shrine in the afternoon of the second day)

[1184] Perceived Emotion: Relaxed

[1185] Step 5 output:

[1186] Format conversion data for correction requests and emotional information

[1187] Input for step 6:

[1188] Formatted correction requests and sentiment information

[1189] Prompt: "Please revise the scenario based on the revision request. Revision request: Visit to Fushimi Inari Shrine in the afternoon of the second day, Emotion: Relaxation"

[1190] Output from step 6:

[1191] Modified scenario (e.g. Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine)

[1192] Input for step 7:

[1193] Request to save or share

[1194] Recognized emotion: Excitement

[1195] Output from step 7:

[1196] Saved scenario data

[1197] Shared scenario messages

[1198] This allows users to create, modify, save, and share personalized scenarios based on their emotions.

[1199] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1200] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1201] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1202] [Third embodiment]

[1203] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1204] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1205] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1206] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1207] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1208] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1209] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1210] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1211] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1212] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1213] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1214] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1215] This invention provides a system that allows users to easily create, edit, save, and share travel scenarios. This system is realized using devices, a server, chat generation AI, a database, and the LINE API.

[1216] 1. User registration and authentication

[1217] overview:

[1218] To access the system, users first register, which creates a dedicated account for them.

[1219] What happens:

[1220] Device: The user enters their name, email address, and password into the registration form. For example, the user enters their name "Taro Tanaka," their email address "taro.tanaka@example.com," and their password "securepassword," and submits the form.

[1221] Server: Receives registration information and stores it in a database. A hashed password is also generated and stored.

[1222] Server: Notify the user that registration was successful.

[1223] 2. Requesting and Generating Travel Scenarios

[1224] overview:

[1225] Users enter travel details (destination, date, budget, etc.), and the chat generation AI generates a travel scenario based on that information.

[1226] What happens:

[1227] Device: The user enters the travel destination "Kyoto," the date "December 1, 2023 - December 3, 2023," and the budget "50,000 yen," and submits.

[1228] Server: Receives travel information and converts it into an internal data format.

[1229] Server: The travel information converted into a data format is sent as a request to the chat generation AI.

[1230] Chat generation AI: Generates travel scenarios based on the received travel information. For example, it generates scenarios such as "Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion," "Day 2: Visit Kiyomizu-dera Temple and Kinkaku-ji Temple," and "Day 3: Visit Arashiyama, depart from Kyoto Station."

[1231] Server: Sends the generated travel scenario to the user.

[1232] 3. Customizing the scenario

[1233] overview:

[1234] The user can request changes or additions to the generated travel scenario.

[1235] What happens:

[1236] Device: The user enters "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" and submits a correction request.

[1237] Server: Receives the correction request and resends it to the chat generation AI.

[1238] Chat generation AI: Regenerates scenarios based on revision requests. For example, it generates a new scenario called "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[1239] Server: Sends the modified scenario to the user.

[1240] 4. Share and save your scenarios

[1241] overview:

[1242] Users can save the scenarios they create and share them with other users via LINE.

[1243] What happens:

[1244] Device: The user selects whether to save the scenario or share it on LINE and sends a request.

[1245] server:

[1246] For save: Save the scenario to the database.

[1247] For sharing: Send the scenario to the specified contact via the LINE API.

[1248] LINE API: Send the received scenario information to a specified group or contact. For example, send a message to the "Family" group saying, "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[1249] In this way, users can easily create, save, and share customized travel scenarios.The system that supports users' travel planning is realized by the collaboration of the server, terminals, chat generation AI, database, and LINE API.

[1250] The processing flow will be explained below.

[1251] 1. User registration and authentication

[1252] Step 1:

[1253] On your device: The user enters their name, email address, and password into the provided registration form.

[1254] Step 2:

[1255] Terminal: The user clicks the submit button, sending the input data to the server.

[1256] Step 3:

[1257] Server: Receives input data and checks whether it is entered correctly (validation check).

[1258] Step 4:

[1259] Server: Creates SQL queries to store registration information in a database and saves the data. Passwords are hashed for security purposes.

[1260] Step 5:

[1261] Server: Generates and sends back a message to the user indicating successful registration.

[1262] 2. Requesting and Generating Travel Scenarios

[1263] Step 1:

[1264] Device: The user enters travel destination "Kyoto," date "December 1, 2023 - December 3, 2023," and budget "50,000 yen," and clicks the send button.

[1265] Step 2:

[1266] Server: Receives travel information and converts the data into a format that can be processed by AI.

[1267] Step 3:

[1268] Server: Sends the converted travel information as a request to the chat generation AI.

[1269] Step 4:

[1270] Chat generation AI: Generates a travel scenario (e.g., Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion) based on the received travel information.

[1271] Step 5:

[1272] Server: Receives travel scenarios generated by the AI ​​and sends them to the user.

[1273] 3. Customizing the scenario

[1274] Step 1:

[1275] Terminal: The user inputs "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" to an existing scenario and submits a modification request.

[1276] Step 2:

[1277] Server: Receives the correction request and sends it back to the chat generation AI.

[1278] Step 3:

[1279] Chat generation AI: Based on modification requests, it adjusts existing scenarios and generates new ones (e.g., Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine).

[1280] Step 4:

[1281] Server: Receives modified scenarios from the AI ​​and sends them to the user.

[1282] 4. Share and save your scenarios

[1283] Step 1:

[1284] Device: The user selects whether to save the scenario or share it on LINE, and sends a request to the server.

[1285] Step 2:

[1286] Server: If a save request is received, it saves the scenario to the database.

[1287] Step 3:

[1288] Server: When a sharing request is received, the server uses the LINE API to send the scenario to the specified contacts.

[1289] Step 4:

[1290] LINE API: Send the received scenario to the specified contacts. For example, send the scenario to the "Family" group with the message "Taro Tanaka shared a travel scenario."

[1291] In this way, each processing step works together, allowing users to efficiently create, modify, save, and share travel scenarios.

[1292] Example 1

[1293] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1294] Conventional travel planning systems have made it difficult for users to efficiently generate, modify, and share travel scenarios. Specifically, it is difficult to generate an appropriate scenario based on the travel information entered by the user, and customizing and modifying the scenario takes time. In addition, there is a lack of a way to easily share the generated scenario with other users.

[1295] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1296] In this invention, the server includes terminal means for a user to input travel information, means for the server to receive the input travel information and convert it into a data format, means for the server to request the travel information from the generative AI model, means for the generative AI model to generate a travel scenario based on the received travel information, means for the server to send the generated travel scenario to the user, terminal means for a user to input a request to modify the scenario, means for the server to resend the modification request to the generative AI model, means for the generative AI model to modify the scenario based on the modification request, means for the server to send the modified scenario to the user, terminal means for a user to input a request to save or share the generated scenario, means for the server to save the scenario in a database, and means for the server to share the scenario using a communication API. This enables users to easily generate travel scenarios, modify them as needed, and efficiently share them with other users.

[1297] "Terminal means" refers to a device or interface through which a user inputs travel information and modification requests and transmits them to a server.

[1298] The "server" is a central computer that processes travel information and modification requests received from users, converts them into data formats, and sends requests to the generative AI model.

[1299] "Means for converting into data format" refers to the process or algorithm for converting travel information received from a user into a format that is easy to process internally (e.g., JSON format).

[1300] The "means for making a request to the generative AI model" is a process for passing the converted travel information to the generative AI model and sending a request as a prompt sentence.

[1301] A "generative AI model" is an artificial intelligence system that generates travel scenarios based on input prompts.

[1302] The "means for generating a travel scenario" is a function that enables the generative AI model to create a specific travel plan based on travel information.

[1303] The "means for resending a modification request to the generative AI model" is a process for receiving a scenario modification request from a user and sending it again as a prompt to the generative AI model.

[1304] A "communication API" is a communication protocol or interface used to share generated travel scenarios with other users, and is a means for connecting to external systems (e.g., SNS applications).

[1305] This invention provides a system that allows users to easily create, modify, save, and share travel scenarios. This system is realized using terminals, servers, generative AI models, databases, and communication APIs.

[1306] Hardware and software used

[1307] Device: User device such as a smartphone or computer

[1308] Server: A central computer for information processing and database management.

[1309] Generative AI model: Chat generation AI (e.g. GPT-3)

[1310] Database: Data storage for storing and managing information (e.g., SQL database)

[1311] Communication API: Interface for data communication with external systems (e.g., RESTful API)

[1312] User Registration and Authentication

[1313] User: Enter your name, email address, and password, and send them from your device to the server. For example, enter the name "Taro Tanaka," the email address "taro.tanaka@example.com," and the password "securepassword."

[1314] Server: Stores the received information in a database, hashes the password, and notifies the user that registration was successful.

[1315] Requesting and generating travel scenarios

[1316] User: Enters details such as travel destination, date, and budget, and sends them to the server from the device. For example, enters "Kyoto," "December 1, 2023 - December 3, 2023," and "50,000 yen."

[1317] Server: Converts received travel information into a data format and sends it to the generative AI model.

[1318] Generative AI model: Generates travel scenarios based on received prompts.

[1319] Example prompt: "I would like to create a scenario for traveling to Kyoto. The travel dates are from December 1, 2023 to December 3, 2023, and the budget is 50,000 yen. Please suggest a scenario."

[1320] Server: Sends the generated travel scenario to the user.

[1321] Customizing travel scenarios

[1322] User: Enter any corrections or additions to the scenario and send them to the server from the device. For example, you might enter, "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[1323] Server: Resubmits the correction request to the generative AI model.

[1324] Generative AI model: Regenerates scenarios based on modification requests.

[1325] Server: Sends the modified scenario to the user.

[1326] Save and share your travel scenarios

[1327] User: Selects to save or share the scenario and sends a request from the device to the server.

[1328] server:

[1329] For save: Save the scenario to the database.

[1330] For sharing: Use a communication API to send the scenario to an external system. For example, send a message saying "Taro Tanaka has shared a travel scenario" to a specified group or contact.

[1331] This allows users to easily create, modify, save, and share travel scenarios. Furthermore, by linking together the server, terminal, generative AI model, database, and communication API, a system that supports users in planning their trips is realized.

[1332] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1333] Step 1:

[1334] The user enters travel information. Specifically, the user enters their name, email address, and password into the terminal. For example, the user enters the name "Taro Tanaka," the email address "taro.tanaka@example.com," and the password "securepassword." This is the input data.

[1335] Step 2:

[1336] The terminal sends the entered information to the server. The entered data is sent to the server as is. For example, {"name": "Taro Tanaka", "email": "taro.tanaka@example.com", "password": "securepassword"}.

[1337] Step 3:

[1338] The server processes the received information and hashes the password. Specifically, it hashes "securepassword" using a hashing algorithm (e.g., SHA-256). This is the data processing step. For example, "securepassword" -> "5e884898da28047151d0e56f8dc6292773603d0d6aabbdd5d029d6ad1db1c5f7".

[1339] Step 4:

[1340] The server stores the hashed password along with other user information in a database, such as {"name": "Taro Tanaka", "email": "taro.tanaka@example.com", "password_hash": "5e884898da28047151d0e56f8dc6292773603d0d6aabbdd5d029d6ad1db1c5f7"} .

[1341] Step 5:

[1342] The server generates a message indicating that registration has been completed successfully and sends it to the terminal. The message is simple text data, such as "Registration completed."

[1343] Step 6:

[1344] The terminal displays the received message to the user, displaying the text "Registration complete."

[1345] Step 7:

[1346] The user inputs travel information such as destination, date, budget, etc. For example, the user inputs the destination "Kyoto," the date "December 1, 2023 - December 3, 2023," and the budget "50,000 yen." This becomes new input data.

[1347] Step 8:

[1348] The device sends the entered travel information to the server. This information is passed to the server as is. For example, {"destination": "Kyoto", "dates": "December 1, 2023 - December 3, 2023", "budget": "50,000 yen"}.

[1349] Step 9:

[1350] The server converts the received travel information into an internal data format. For example, it converts it into the JSON format used internally. The internal data becomes {"destination": "Kyoto", "dates": {"start": "2023-12-01", "end": "2023-12-03"}, "budget": 50000}.

[1351] Step 10:

[1352] The server sends the converted travel information to the generative AI model as a prompt. For example, the prompt might read, "I would like to create a scenario for traveling to Kyoto. The travel dates are from December 1, 2023 to December 3, 2023, and the budget is 50,000 yen. Please suggest a scenario."

[1353] Step 11:

[1354] The generative AI model generates a travel scenario based on the received prompt. For example, the generated scenario might be "Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion," "Day 2: Visit Kiyomizu-dera Temple and Kinkaku-ji Temple," or "Day 3: Visit Arashiyama, depart from Kyoto Station."

[1355] Step 12:

[1356] The generative AI model returns the generated travel scenario to the server, which then sends the received scenario to the user. The scenario text is sent to the user.

[1357] Step 13:

[1358] The device displays the generated travel scenario to the user. Specifically, the scenario text is displayed on the device screen. For example, "Day 1: Arrive at Kyoto Station, check in at hotel, stroll around Gion..."

[1359] Step 14:

[1360] The user inputs a modification request for the generated scenario. For example, the user might input, "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day." This becomes the input data for the modification request.

[1361] Step 15:

[1362] The device sends a modification request to the server, which passes the request to the server as is. For example, {"modification": "I also want to visit Fushimi Inari Taisha Shrine in the afternoon of the second day"}.

[1363] Step 16:

[1364] The server resubmits the correction request to the generative AI model. For example, the correction request might be, "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[1365] Step 17:

[1366] The generative AI model regenerates the scenario based on the modification request. For example, the regenerated scenario might be "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[1367] Step 18:

[1368] The generative AI model sends the modified scenario back to the server, which then sends the received scenario to the user. The user receives the regenerated scenario text.

[1369] Step 19:

[1370] The terminal displays the revised scenario to the user. Specifically, the text of the regenerated scenario is displayed on the terminal screen. For example, "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[1371] Step 20:

[1372] The user inputs a request to save or share a scenario. For example, they select from the options "Save scenario" or "Share on LINE." This becomes the input data for the save or share request.

[1373] Step 21:

[1374] The device sends a request to the server. The request content is passed to the server as is. For example, for "save" it is {"action": "save"}, and for sharing it is {"action": "share"}.

[1375] Step 22:

[1376] The server processes the request accordingly. If save is selected, the scenario is saved to the database. For example, the following data is saved to the database: {"scenario": "Day 1: Arrive at Kyoto Station, check in at hotel, stroll around Gion..."}

[1377] Step 23:

[1378] If sharing is selected, the server uses a communication API to send the scenario to an external system. Specifically, it sends the scenario information to the specified group or contact. For example, it sends a message saying, "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[1379] Step 24:

[1380] The communication API sends the received scenario information to the specified group or contact, allowing users to share the scenarios they have created with other users.

[1381] Step 25:

[1382] The server notifies the user that the saving or sharing is complete, for example, "The scenario has been saved" or "The scenario has been shared."

[1383] Step 26:

[1384] The device displays a notification message to the user, specifically, a message such as "Scenario saved" or "Scenario shared" on the device screen.

[1385] (Application example 1)

[1386] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1387] Conventional food delivery services often lack the flexibility to accommodate specific user meal requests. Furthermore, when users customize their meal menus or have complex dietary restrictions, it is difficult to quickly provide suggestions that meet those requests. Furthermore, they lack the ability to easily share or save generated menus with other users. There is a need for a food delivery system that can meet these user needs.

[1388] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1389] In this invention, the server includes a terminal means for a user to input meal information, a means for the server to receive the input meal information and convert it into a data format, a means for the server to request the meal information from a generative AI model, a means for the generative AI model to generate a meal menu based on the received meal information, a means for the server to send the generated meal menu to the user, and a means for customizing the meal menu. This allows users to quickly and flexibly receive menu suggestions based on specific meal requests. Furthermore, the generated menu can be easily customized, saved, and shared.

[1390] A "user" is an individual who utilizes the system to input meal information and receive or customize the generated menu.

[1391] "Meal information" refers to information including the details of the meal desired by the user, such as the type of meal, budget, and specific dietary restrictions.

[1392] "Terminal means" refers to a device used by a user to input meal information, and specifically refers to a smartphone, tablet, etc.

[1393] A "server" is a computer system that receives and processes input information from a user.

[1394] "Means for converting into data format" refers to the function of converting the dietary information entered by the user into a format that can be understood by the generative AI model.

[1395] A "generative AI model" is an artificial intelligence model that generates optimal meal menus based on input meal information.

[1396] "Means for requesting" refers to the function by which the server sends the meal information received from the user to the generative AI model.

[1397] A "meal menu" is a specific meal suggestion generated by a generative AI model based on dietary information.

[1398] "Customizable means" refers to a function that allows a user to request changes or additions to the meal menu they receive.

[1399] The "means for saving" is a function that allows a user to save the menu created in a database within the system.

[1400] "Means for sharing" is a function for transmitting the generated menu using a communication means in order to share it with other users.

[1401] This invention provides a food delivery system that allows users to input their desired meal details, and a generative AI model proposes and customizes meal menus based on that information, which can then be saved and shared. The system is configured using the following hardware and software:

[1402] Hardware

[1403] Server: A computer system for receiving and processing meal information from users.

[1404] Terminal: The device through which the user inputs meal information. Examples include smartphones and tablets.

[1405] software

[1406] Python (Django, Flask, etc.): A framework for performing server-side processing.

[1407] MySQL: Used as a database management system.

[1408] LINE API: Communication method for sharing menus.

[1409] OpenAI's generative AI model: Used to generate menus based on meal information.

[1410] Program processing

[1411] The server receives the meal information sent by the user and converts it into a data format. The converted data is sent as a request to the generative AI model, which then generates a meal menu based on the meal information. For example, based on the information that "the user requested pizza as the meal type and their budget is 2,000 yen," the generative AI model would suggest the optimal pizza menu.

[1412] The generated meal menu is sent to the user by the server. If the user subsequently wishes to customize the menu, they can input a modification request from their device, which is then sent again to the generative AI model via the server. The generative AI model then generates a new menu based on the modification request and sends it to the user via the server.

[1413] If the user saves the created menu, the server saves it in the database. If the user wishes to share the menu, the server uses the LINE API to send the menu to the specified contacts or group. This allows users to easily customize, save, and share their desired meal menu.

[1414] Specific examples

[1415] If a user inputs a request such as "vegan pizza, budget 3,000 yen," the generative AI model will receive the following prompts and generate the optimal menu based on that request:

[1416] Example prompt sentence:

[1417] Based on user requests, please suggest a vegan pizza menu within a budget of 3,000 yen.

[1418] When a user receives the generated menu and sends a modification request, such as "I'd like to add an olive topping," the generative AI model reprocesses it and generates the following modification prompt:

[1419] Example of a modified prompt:

[1420] Add olive topping to your current menu and regenerate.

[1421] The system allows users to efficiently obtain customized meal menus that can be easily saved and shared with other users.

[1422] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1423] Step 1:

[1424] The user inputs meal information. Using a device such as a smartphone or tablet, the user inputs detailed information such as meal type (e.g., pizza) and budget (e.g., 2,000 yen) and submits it. The input data is sent to the server.

[1425] Step 2:

[1426] The server receives the meal information and converts it into a data format. The server analyzes the raw data received from the user and converts it into a format that the generative AI model can understand, for example, JSON format. The input is the user's meal information, and the output is the converted data format.

[1427] Step 3:

[1428] The server sends a request to the generative AI model. The converted data format is used to send a prompt to the generative AI model (for example, OpenAI's GPT model). The specific prompt generated is, "Based on the user's request, please suggest a pizza menu item within a budget of 2,000 yen."

[1429] Step 4:

[1430] The generative AI model generates a meal menu. Based on the prompt received, the generative AI model generates the optimal meal menu. The input is the prompt, and the output is the generated meal menu. For example, a menu such as "Margherita pizza, topped with basil and cheese, priced at 1,800 yen" is generated.

[1431] Step 5:

[1432] The server sends the generated meal menu to the user. The server then sends the meal menu received from the generative AI model back to the user's device. The user can then view the proposed menu on their device.

[1433] Step 6:

[1434] The user inputs a menu customization request. For example, the user sends a request to change the proposed menu, such as "I would like to add olives to the toppings," to the server via the terminal.

[1435] Step 7:

[1436] The server sends the customization request to the generative AI model. The server receives the modification request from the user, reformats it, and sends it back to the generative AI model. The specific prompt generated is "Please add olive topping to the current menu and regenerate it."

[1437] Step 8:

[1438] The generative AI model generates a modified meal menu. The generative AI model generates a new menu based on the modification request. The input is the modification prompt, and the output is the modified meal menu.

[1439] Step 9:

[1440] The server sends the revised menu to the user, and upon receiving the revised menu, the server sends it again to the user's terminal.

[1441] Step 10:

[1442] When a user wants to save a generated menu, he sends a save request to the server through the terminal. The server saves this menu in the database. The input is the menu save request and the output is the saved menu.

[1443] Step 11:

[1444] When a user wants to share a created menu, they send a share request to the server via their device. The server then shares this menu with other users using the LINE API, etc. The input is the share request, and the output is the shared menu.

[1445] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1446] This invention provides a system that allows users to create, modify, save, and share travel scenarios while improving the user experience by combining an emotion engine that recognizes user emotions. This system is realized using devices, a server, chat generation AI, a database, the LINE API, and an emotion engine.

[1447] 1. User registration and authentication

[1448] overview:

[1449] To access the system, users first register, which creates a dedicated account for them.

[1450] What happens:

[1451] Terminal: The user enters their name, email address, and password into the registration form. For example, they enter the username "Taro Tanaka," the email address "taro.tanaka@example.com," and the password "securepassword," and submit.

[1452] Server: Receives registration information and stores it in a database. Passwords are hashed for security purposes.

[1453] Server: Generates and sends a message to the user indicating successful registration.

[1454] 2. Requesting and Generating Travel Scenarios

[1455] overview:

[1456] Users input their travel details (destination, date, budget, etc.), and the chat generation AI generates a travel scenario based on that information. At this time, the emotion engine recognizes the user's emotions and reflects them in customizing the travel scenario.

[1457] What happens:

[1458] Device: The user enters travel destination "Kyoto," date "December 1, 2023 - December 3, 2023," and budget "50,000 yen," and clicks the send button.

[1459] Emotion engine: Analyzes facial expressions and voice data while the user is typing to recognize emotions. For example, if the user is excited, it will suggest an active sightseeing plan.

[1460] Server: Receives travel information and emotion information and converts it into a data format.

[1461] Server: Sends the converted travel information and emotion information as a request to the chat generation AI.

[1462] Chat generation AI: Generates a travel scenario (e.g., Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion) based on received travel information and emotional information.

[1463] Server: Sends the generated travel scenario to the user.

[1464] 3. Customizing the scenario

[1465] overview:

[1466] The user can request changes or additions to the generated travel scenario, and the emotion engine will recognize the user's emotions and reflect them in the modification request.

[1467] What happens:

[1468] Terminal: The user inputs "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" to an existing scenario and submits a modification request.

[1469] Emotion engine: Analyzes facial and voice data while the user is entering a correction request to recognize emotions. For example, if the user is feeling stressed, the engine can suggest relaxation plans.

[1470] Server: Receives the modification request and emotion information, and sends the request again to the chat generation AI.

[1471] Chat generation AI: Based on modification requests and emotional information, it adjusts existing scenarios and generates new ones (e.g., Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine).

[1472] Server: Sends the modified scenario to the user.

[1473] 4. Share and save your scenarios

[1474] overview:

[1475] Users can save the scenarios they create or share them with other users via LINE. In this case, the emotion engine also recognizes the user's emotions and suggests appropriate ways to share them.

[1476] What happens:

[1477] Device: The user selects whether to save the scenario or share it on LINE, and sends a request to the server.

[1478] Emotion engine: Analyzes facial expressions and voice data while the user is operating the app to recognize emotions. For example, if the user is excited, the engine will suggest ways to share the app.

[1479] Server: If a save request is received, it saves the scenario to the database.

[1480] Server: When a sharing request is received, the server uses the LINE API to send the scenario to the specified contacts.

[1481] LINE API: Send the received scenario to the specified contacts. For example, to the "Family" group, send the scenario with the message "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[1482] In this way, each processing step works together, allowing users to efficiently create, modify, save, and share customized travel scenarios based on emotion recognition.The collaboration of each element—server, device, chat generation AI, database, LINE API, and emotion engine—creates a system that improves the user experience.

[1483] The processing flow will be explained below.

[1484] 1. User registration and authentication

[1485] Step 1:

[1486] On your device: The user enters their name, email address, and password into the provided registration form.

[1487] Step 2:

[1488] Terminal: The user clicks the submit button, sending the input data to the server.

[1489] Step 3:

[1490] Server: Receives input data and checks whether it is entered correctly (validation check).

[1491] Step 4:

[1492] Server: Creates SQL queries to store registration information in a database and saves the data. Passwords are hashed for security purposes.

[1493] Step 5:

[1494] Server: Generates and sends back a message to the user indicating successful registration.

[1495] 2. Requesting and Generating Travel Scenarios

[1496] Step 1:

[1497] Device: The user enters travel destination "Kyoto," date "December 1, 2023 - December 3, 2023," and budget "50,000 yen," and clicks the send button.

[1498] Step 2:

[1499] Emotion engine: Analyzes facial expressions and voice data in real time while the user is entering travel information to extract emotional information. For example, if the user is excited, it determines that the user prefers an active scenario.

[1500] Step 3:

[1501] Server: Receives travel information and emotion information and converts it into an internal data format.

[1502] Step 4:

[1503] Server: Sends the converted travel information and emotion information as a request to the chat generation AI.

[1504] Step 5:

[1505] Chat generation AI: Generates travel scenarios based on received travel information and emotional data. For example, it generates a scenario including "Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion, Day 2: Sightseeing at Kiyomizu-dera Temple and Kinkaku-ji Temple, and activities to boost your mood."

[1506] Step 6:

[1507] Server: Receives travel scenarios generated by the AI ​​and sends them to the user.

[1508] 3. Customizing the scenario

[1509] Step 1:

[1510] Terminal: The user submits a modification request to an existing scenario by entering "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[1511] Step 2:

[1512] Emotion engine: Analyzes facial expressions and voice data in real time while the user is entering correction requests to extract emotional information. For example, if the user is feeling stressed, the engine will suggest plans to help them relax.

[1513] Step 3:

[1514] Server: Receives modification requests and emotion information and converts them into an internal data format.

[1515] Step 4:

[1516] Server: Resubmit the correction request and emotion information to the chat generation AI.

[1517] Step 5:

[1518] Chat generation AI: Based on revision requests and emotional information, it adjusts existing scenarios and generates new ones. For example, it generates a scenario that includes "Day 2: Sightseeing at Kiyomizu-dera Temple and Kinkaku-ji Temple, visiting Fushimi Inari Taisha Shrine, and experiencing a tea ceremony to relax."

[1519] Step 6:

[1520] Server: Receives modified scenarios from the AI ​​and sends them to the user.

[1521] 4. Share and save your scenarios

[1522] Step 1:

[1523] Device: The user selects whether to save the scenario or share it on LINE, and sends a request to the server.

[1524] Step 2:

[1525] Emotion engine: Analyzes facial expressions and voice data in real time while the user is operating the app, extracting emotional information. For example, if the user is excited, the engine suggests ways to share the information.

[1526] Step 3:

[1527] Server: If a save request is received, save the scenario to the database. If a share request is received, send the scenario to the specified contacts using the LINE API.

[1528] Step 4:

[1529] LINE API: Send the received scenario to the specified contacts. For example, send the scenario to the "Family" group with the message "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[1530] In this way, each processing step works together, allowing users to efficiently create, modify, save, and share customized travel scenarios based on emotion recognition.The collaboration of each element—server, device, chat generation AI, database, LINE API, and emotion engine—creates a system that improves the user experience.

[1531] Example 2

[1532] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1533] Conventional travel scenario generation systems have difficulty in properly reflecting users' emotions and individual needs. Furthermore, they lack the means to detect changes in users' emotions and optimize the scenarios based on those emotions during the process of generating, modifying, saving, and sharing the scenarios. As a result, user satisfaction cannot be sufficiently increased. Furthermore, the methods for sharing scenarios are limited, making it difficult to smoothly collaborate with other users.

[1534] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes terminal means for the user to input travel information, means for the terminal means to transmit the travel information, means for the server to receive the input travel information and convert it into a data format, means for the server to request the travel information from the chat generation AI, means for the chat generation AI to generate a travel scenario based on the received travel information, means for the emotion engine to recognize the user's emotions and generate data to reflect them in the travel scenario, and means for the server to transmit the generated travel scenario to the user. This makes it possible to generate, modify, save, and share customized travel scenarios that take the user's emotions into consideration.

[1535] "Terminal means" refers to a device that allows a user to input information and receive information from the system.

[1536] "Server" means a computing device that receives, processes, and stores information sent from terminal means over a network, and transmits the data to other systems or devices as needed.

[1537] The "means for converting into a data format" is a function for converting information received from a user into a format that can be used by other systems or processing modules.

[1538] "Chat generation AI" is an artificial intelligence system that generates natural language based on user input and creates specific scenarios and suggestions.

[1539] The "means for generating a travel scenario" is the process by which the chat generation AI formulates a specific travel plan based on the information it receives.

[1540] The "emotion engine" is an analytical engine that recognizes the user's emotions, generates them as data, and provides them to other systems.

[1541] "Means for generating data to be reflected in travel scenarios" refers to the function in which the emotion engine analyzes the user's emotions and creates data that can be used to generate or modify scenarios.

[1542] A "modification request" is a request made by a user to make changes or additions to an existing travel scenario.

[1543] The "terminal means for inputting a modification request" is a device that allows a user to input a request for a change or addition to a scenario.

[1544] The "means for sending a correction request and emotional information to the chat generation AI" is a function in which the server sends the input correction request and recognized emotional information to the chat generation AI, requesting that the scenario be corrected.

[1545] The "means for saving a scenario in a database" is a function for storing a generated or modified scenario in a database in order to save it permanently.

[1546] "Means for sharing scenarios using external APIs" is a function that allows the server to work with other applications and services to share scenarios.

[1547] This invention is a system that allows users to create, modify, save, and share travel scenarios while improving the user experience by combining an emotion engine that recognizes user emotions. This system is realized using devices, a server, chat generation AI, a database, an external API, and an emotion engine.

[1548] 1. User registration and authentication

[1549] overview:

[1550] To access the system, users first register, which creates a dedicated account for them.

[1551] Hardware and software used:

[1552] Terminal: The device through which the user enters information (PC, smartphone, etc.)

[1553] Server: A server that receives registration information and stores it in a database

[1554] Database: Data storage for storing registration information

[1555] Emotion engine: facial recognition and voice analysis software

[1556] 2. Requesting and Generating Travel Scenarios

[1557] overview:

[1558] Users input their travel details (destination, date, budget, etc.), and the chat generation AI generates a travel scenario based on that information. At this time, the emotion engine recognizes the user's emotions and reflects them in customizing the travel scenario.

[1559] Hardware and software used:

[1560] Terminal: A device through which a user enters information.

[1561] Server: A server for receiving travel information and emotion information and converting them into data formats.

[1562] Chat Generation AI: Artificial Intelligence for Generating Travel Scenarios

[1563] Emotion Engine: Software for recognizing user emotions

[1564] Database: Data storage for travel scenarios and user information

[1565] Examples:

[1566] Let's say a user inputs their travel destination "Kyoto," the dates "December 1, 2023 - December 3, 2023," and a budget of "50,000 yen" and sends them to the system. At this time, the emotion engine analyzes the user's facial expressions and voice data and recognizes emotions such as excitement. This allows the system to suggest an active sightseeing plan.

[1567] 3. Customizing the scenario

[1568] overview:

[1569] The user can request changes or additions to the generated travel scenario, and the emotion engine will recognize the user's emotions and reflect them in the modification request.

[1570] Hardware and software used:

[1571] Terminal: A device through which a user enters information.

[1572] Server: A server for receiving modification requests and emotion information and converting them into data formats.

[1573] Chat Generation AI: Artificial intelligence that generates revised scenarios

[1574] Emotion Engine: Software for recognizing user emotions

[1575] Database: Data storage for generated scenarios

[1576] Examples:

[1577] Let's say a user inputs "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" into an existing scenario and sends it to the system. At this time, the emotion engine analyzes the facial expressions and voice data while the user is inputting, and if the user is feeling stressed, it will suggest a plan that will help them relax.

[1578] 4. Share and save your scenarios

[1579] overview:

[1580] Users can save the scenarios they create or share them with other users through external APIs. In this case, the emotion engine also recognizes the user's emotions and suggests appropriate solutions.

[1581] Hardware and software used:

[1582] Terminal: A device through which a user enters information.

[1583] Server: The server that receives the share / save request.

[1584] Database: Data storage for saving scenarios

[1585] External API: Interface for connecting with external services such as the LINE API

[1586] Examples:

[1587] Let's say a user chooses to save the scenario or share it via the LINE API and sends a request to the system. If the emotion engine recognizes high tension, it will suggest a way to share it.

[1588] Example prompt sentence:

[1589] A specific prompt for a user to enter travel information might be "Destination: Kyoto, Date: December 1, 2023 - December 3, 2023, Budget: 50,000 yen." In addition, for a revision request, the user could enter "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[1590] As described above, the present invention realizes a system that takes into account the user's emotions and provides a more personalized travel scenario.

[1591] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1592] Step 1: User enters trip information

[1593] User: Enter details such as travel destination, date, and budget into the device. For example, enter "Kyoto," "December 1, 2023 - December 3, 2023," and "50,000 yen."

[1594] Terminal: Converts the input travel information into a data format and generates a request to send it to the server.

[1595] Input: Trip information entered by the user.

[1596] Output: A travel information request converted into a data format.

[1597] Step 2: The server receives the trip information

[1598] Server: Receives the travel information request sent from the device.

[1599] Input: A travel information request converted into a data format.

[1600] Output: Received trip information.

[1601] Step 3: The emotion engine recognizes the user's emotion

[1602] Emotion engine: Analyzes facial expressions and voice data when users input, and recognizes emotions. Uses emotion recognition APIs and voice analysis software.

[1603] Input: User's facial expression and voice data.

[1604] Output: Recognized emotion information.

[1605] Step 4: The server integrates the emotion information and the travel information.

[1606] Server: Integrates the received travel information and emotional information and converts it into a data format for sending to the chat generation AI.

[1607] Input: Received travel information and recognized emotion information.

[1608] Output: Data integrating travel information and emotional information.

[1609] Step 5: The server sends the integrated data to the chat generation AI.

[1610] Server: Sends data that integrates travel information and emotional information as a request to the chat generation AI.

[1611] Input: Data integrating travel information and emotional information.

[1612] Output: A request to send to the chat generation AI.

[1613] Step 6: Chat generation AI generates travel scenarios

[1614] Chat generation AI: Generates a travel scenario based on the received travel information and emotional information. For example, it creates a plan such as "Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion."

[1615] Input: Travel information and emotional information sent to the chat generation AI.

[1616] Output: The generated travel scenario.

[1617] Step 7: The server receives the generated travel scenario and sends it to the user.

[1618] Server: Receives the generated travel scenarios and converts them into a data format for transmission to the user.

[1619] Input: Generated travel scenarios.

[1620] Output: The travel scenario converted into a data format to be sent to the user.

[1621] Step 8: User receives and displays the travel scenario

[1622] Terminal: Receives the travel scenario sent from the server and displays it to the user.

[1623] Input: A travel scenario converted into a data format.

[1624] Output: The travel scenario displayed on the user's screen.

[1625] Step 9: User enters a request to modify the scenario

[1626] User: Request a change or addition to an existing scenario. For example, enter "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[1627] Terminal: Converts the modification request into a data format and generates a request to send to the server.

[1628] Input: The correction request entered by the user.

[1629] Output: The request for the modification request converted into data format.

[1630] Step 10: Server receives the modification request

[1631] Server: Receives the modification request sent from the device.

[1632] Input: The request for the correction request converted into data format.

[1633] Output: The correction request received.

[1634] Step 11: The emotion engine recognizes the user's emotion again

[1635] Emotion engine: Re-analyzes the user's facial and voice data while the correction request is being entered to recognize emotions.

[1636] Input: User's facial expression and voice data.

[1637] Output: Re-recognized emotion information.

[1638] Step 12: The server integrates the modification request with the emotion information.

[1639] Server: Integrates received correction requests and emotional information, and converts them into a data format for retransmission to the chat generation AI.

[1640] Input: Received correction request and re-recognized emotion information.

[1641] Output: Data integrating correction requests and emotional information.

[1642] Step 13: The server resends the integrated data to the chat generation AI.

[1643] Server: The data that integrates the correction request and emotional information is sent to the chat generation AI as a re-request.

[1644] Input: Data integrating correction requests and emotional information.

[1645] Output: A request to resend to the chat generation AI.

[1646] Step 14: Chat generation AI generates revised scenarios

[1647] Chat generation AI: Based on the received revision requests and emotional information, the AI ​​modifies the scenario. For example, it generates a revised plan such as "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[1648] Input: Edit request and sentiment sent to the chat generation AI.

[1649] Output: The modified travel scenario.

[1650] Step 15: The server receives the modified travel scenario and sends it to the user.

[1651] Server: Receives the modified travel scenario and converts it into a data format for transmission to the user.

[1652] Input: The modified travel scenario.

[1653] Output: The modified trip scenario converted into a data format for sending to the user.

[1654] Step 16: User receives and displays the revised travel scenario

[1655] Terminal: Receives the modified travel scenario sent from the server and displays it to the user.

[1656] Input: A modified travel scenario converted into a data format.

[1657] Output: The revised trip scenario displayed on the user's screen.

[1658] Step 17: User saves or shares the scenario

[1659] User: Choose whether to store the generated scenario in a database or use an external API to share the scenario with other users.

[1660] Device: Generates a save or share request to convert it into a data format and send it to the server.

[1661] Input: The save or share request selected by the user.

[1662] Output: The save or share request converted into a data format.

[1663] Step 18: Server receives save request

[1664] Server: Receives the save request sent from the device.

[1665] Input: The save request converted to a data format.

[1666] Output: The query to save to the database.

[1667] Step 19: The server saves the scenario to the database

[1668] Server: Saves the scenario to the database based on the save request.

[1669] Input: The query to save to the database.

[1670] Output: The scenario stored in the database.

[1671] Step 20: Share your scenario using an external API

[1672] Server: When a sharing request is received, it uses an external API to send the scenario to the specified contacts.

[1673] External API: Use LINE API, etc., to send the scenario as a message to the specified contact.

[1674] Input: The share request converted into data format.

[1675] Output: The scenario shared with the specified contact.

[1676] In this way, at each processing step, the input data is processed and the process moves to the next step, making it possible to generate, modify, save, and share customized travel scenarios that take the user's emotions into account.

[1677] (Application example 2)

[1678] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1679] Conventional advertising systems have difficulty targeting users based on their real-time emotions. This can result in the inability to provide ads that best fit the user's interests and emotions, resulting in insufficient advertising effectiveness. Another issue is that there is no established method for accurately capturing user emotions and generating ads based on them. Furthermore, the process of generating emotion-based ads is complicated, making effective ad delivery difficult.

[1680] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means for a user to input information, means for the server to receive the input information and convert it into a data format, means for the server to request information from a chat generation AI, means for the chat generation AI to generate a scenario based on the received information, means for the server to send the generated scenario to the user, and means for the emotion engine to recognize the user's emotions and reflect them in the input data. This makes it possible to generate and deliver personalized advertisements based on the user's real-time emotions.

[1681] The "terminal means through which the user inputs information" refers to an electronic device that is operated by the user to input information into the system.

[1682] "Means by which the server receives input information and converts it into a data format" refers to the process by which the server receives information sent by the user and converts that information into an appropriate format.

[1683] "Means for the server to request information from the chat generation AI" refers to a method in which the server uses the converted information to request processing from the chat generation AI.

[1684] "Means for generating a scenario based on information received by the chat generation AI" refers to the function by which the chat generation AI utilizes input information to automatically generate a scenario that suits a specific purpose.

[1685] "Means by which the server sends the generated scenario to the user" refers to the method by which the server delivers the scenario generated by the chat generation AI to the user.

[1686] "Means by which the emotion engine recognizes the user's emotions and reflects them in the input data" refers to the process of analyzing the user's emotions using emotion recognition technology and reflecting the results in the input data and scenario generation.

[1687] The "terminal means for the user to input a request to modify a scenario" is a device for the user to input information for modifying or adding to an existing scenario.

[1688] "Means for the server to resend the correction request to the chat generation AI" refers to a method in which the server resends the correction request received from the user to the chat generation AI.

[1689] "Means for the chat generation AI to modify the scenario based on a modification request" refers to a function that allows the chat generation AI to modify an existing scenario in response to a user's modification request.

[1690] The "means by which the server sends the modified scenario to the user" refers to the method by which the server sends the modified scenario to the user.

[1691] "Means for the emotion engine to recognize the user's emotions and reflect them in the modification request" is a process of understanding the user's emotions using emotion recognition technology and applying them to the scenario modification request.

[1692] The "terminal means for inputting a request from a user to save or share a generated scenario" is a device for a user to select and instruct whether to save a generated scenario or share it with other users.

[1693] The "means by which the server stores the scenario in the database" refers to the method by which the server records the scenario in the database for long-term storage.

[1694] "Means for the server to share a scenario using a communication platform API" refers to a method in which the server uses an API to share a scenario with other users via a communication network.

[1695] "Means for the emotion engine to recognize the user's emotions and reflect them in the sharing method" is a process in which the emotion engine determines the user's emotions and reflects the results in the sharing method of the scenario.

[1696] To realize the present invention, the following system configuration and processing procedures are required.

[1697] System Configuration

[1698] 1. User Device:

[1699] This is a device that users use to input information. This device can be a smartphone, tablet, or PC, and is used by installing a dedicated application.

[1700] 2. Server:

[1701] It receives information and converts it into a data format. It also has the function of requesting information from the chat generation AI and sending the generated scenario to the user. The server is a high-performance computer and can use cloud services that can process large amounts of data.

[1702] 3. Chat generation AI:

[1703] AI models with advanced natural language generation capabilities, such as OpenAI's ChatGPT model, generate scenarios based on the information they receive and modify them according to user requests.

[1704] 4. Emotion Engine:

[1705] Using emotion recognition technologies such as Emotibot and AFFDEX SDK, the system analyzes user emotions in real time, and this emotional information is reflected in travel scenarios and advertisement generation.

[1706] 5. Database:

[1707] This is a database for storing user information, scenarios, emotional data, etc. A general relational database management system (RDBMS) can be used for this.

[1708] 6. Communication Platform API:

[1709] A communication platform API, such as the LINE API, for sharing generated scenarios with users.

[1710] Program Description

[1711] 1. User registration and authentication:

[1712] A user installs the application and registers by entering their name, email address, and password. The server receives this information, stores it in a database, and the password is hashed for security. A registration completion message is sent to the user.

[1713] Examples:

[1714] User "Taro Tanaka" registers in the app by entering his name "Taro Tanaka", email address "taro.tanaka@example.com", and password "securepassword".

[1715] 2. Emotion recognition and information input:

[1716] When users input travel destination or advertisement information, the emotion engine analyzes facial expressions and voice to recognize emotions, and this information is sent to the server in real time.

[1717] Examples:

[1718] When a user enters information for a trip to "Kyoto," the system recognizes the emotion of excitement and suggests an active sightseeing plan based on the emotion.

[1719] 3. Scenario generation:

[1720] The server sends a request to the chat generation AI based on the user information and emotion information. The AI ​​generates a scenario based on this information, and the server sends the generated scenario to the user.

[1721] Example prompt sentence:

[1722] "User information: Taro Tanaka, Interest: Travel, Emotion: Excitement. Please generate an appropriate travel scenario."

[1723] 4. Modify the scenario:

[1724] When a user inputs a request to modify a scenario, the emotion engine recognizes the emotion at that time and reflects it in the modification request. The server resends this information to the AI, and a modified scenario is generated.

[1725] Examples:

[1726] If a user inputs "I want to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" and the system recognizes that the user is feeling relaxed, it will add plans that allow for relaxation.

[1727] 5. Save and share your scenarios:

[1728] When a user inputs a request to save or share the generated scenario, the emotion engine analyzes the emotion at that time and suggests the best way to save or share it. The server saves the scenario in a database and shares it using the communication platform API.

[1729] Examples:

[1730] When a user requests to share a generated scenario with a "contact group," if excitement is recognized, a sharing message will be sent according to that emotion.

[1731] This will realize a system that effectively creates, modifies, saves, and shares travel scenarios using an advanced emotion recognition system based on user experience.

[1732] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1733] Step 1:

[1734] The user accesses a terminal means for entering information and enters their name, email address, and password into the registration form. The input data is submitted, and the server receives this information. The password is hashed for security purposes, and all information is stored in a database. The server generates a message indicating that registration is complete and sends it to the terminal.

[1735] Step 2:

[1736] The user inputs travel destination and advertisement information into the device. At this time, the emotion engine analyzes the user's facial expressions and voice in real time to recognize emotions. The device then sends the input information and recognized emotion information to the server, which converts it into a data format.

[1737] Step 3:

[1738] The server sends the converted travel information and emotion information as a request to the chat generation AI. The chat generation AI processes the information based on the prompt and generates an appropriate scenario. This scenario is then sent back to the server.

[1739] Step 4:

[1740] Once the server receives the generated scenario, it sends it to the user, who can review it and enter correction requests as needed.

[1741] Step 5:

[1742] The user inputs a revision request and sends it from their device to the server. The emotion engine again recognizes the user's emotions and reflects them in the revision request. The server again sends the revision request and emotion information to the chat generation AI and requests that the scenario be revised.

[1743] Step 6:

[1744] The chat generation AI regenerates the scenario based on the modification request and emotional information, and sends the modified scenario back to the server, which then sends it to the user.

[1745] Step 7:

[1746] The user inputs a request to save or share the generated scenario, which is sent from the device to the server. The emotion engine recognizes the user's emotions and suggests appropriate ways to save or share them. The server then saves the scenario to a database or shares it using the communication platform API.

[1747] Example input and output:

[1748] Step 1 input:

[1749] Name: Taro Tanaka

[1750] Email address: taro.tanaka@example.com

[1751] Password: securepassword

[1752] Step 1 output:

[1753] Registration completion message

[1754] Step 2 input:

[1755] Travel destination: Kyoto

[1756] Dates: December 1, 2023 - December 3, 2023

[1757] Budget: 50,000 yen

[1758] Perceived Emotion: Excitement

[1759] Step 2 output:

[1760] Travel information and sentiment information format conversion data

[1761] Step 3 Input:

[1762] Formatted travel and sentiment information

[1763] Prompt: "User information: Taro Tanaka, Travel destination: Kyoto, Date: December 1, 2023 - December 3, 2023, Budget: ¥50,000, Emotion: Excitement. Please generate an appropriate travel scenario."

[1764] Step 3 output:

[1765] Generated travel scenario (e.g., Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion)

[1766] Step 4 Input:

[1767] Generated scenarios

[1768] Step 4 output:

[1769] Providing scenarios to users

[1770] Input for step 5:

[1771] Request an amendment (e.g., visit Fushimi Inari Shrine in the afternoon of the second day)

[1772] Perceived Emotion: Relaxed

[1773] Step 5 output:

[1774] Format conversion data for correction requests and emotional information

[1775] Input for step 6:

[1776] Formatted correction requests and sentiment information

[1777] Prompt: "Please revise the scenario based on the revision request. Revision request: Visit to Fushimi Inari Shrine in the afternoon of the second day, Emotion: Relaxation"

[1778] Output from step 6:

[1779] Modified scenario (e.g. Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine)

[1780] Input for step 7:

[1781] Request to save or share

[1782] Recognized emotion: Excitement

[1783] Output from step 7:

[1784] Saved scenario data

[1785] Shared scenario messages

[1786] This allows users to create, modify, save, and share personalized scenarios based on their emotions.

[1787] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1788] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1789] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1790] [Fourth embodiment]

[1791] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1792] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1793] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1794] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1795] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1796] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1797] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1798] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1799] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1800] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1801] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1802] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1803] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1804] This invention provides a system that allows users to easily create, edit, save, and share travel scenarios. This system is realized using devices, a server, chat generation AI, a database, and the LINE API.

[1805] 1. User registration and authentication

[1806] overview:

[1807] To access the system, users first register, which creates a dedicated account for them.

[1808] What happens:

[1809] Device: The user enters their name, email address, and password into the registration form. For example, the user enters their name "Taro Tanaka," their email address "taro.tanaka@example.com," and their password "securepassword," and submits the form.

[1810] Server: Receives registration information and stores it in a database. A hashed password is also generated and stored.

[1811] Server: Notify the user that registration was successful.

[1812] 2. Requesting and Generating Travel Scenarios

[1813] overview:

[1814] Users enter travel details (destination, date, budget, etc.), and the chat generation AI generates a travel scenario based on that information.

[1815] What happens:

[1816] Device: The user enters the travel destination "Kyoto," the date "December 1, 2023 - December 3, 2023," and the budget "50,000 yen," and submits.

[1817] Server: Receives travel information and converts it into an internal data format.

[1818] Server: The travel information converted into a data format is sent as a request to the chat generation AI.

[1819] Chat generation AI: Generates travel scenarios based on the received travel information. For example, it generates scenarios such as "Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion," "Day 2: Visit Kiyomizu-dera Temple and Kinkaku-ji Temple," and "Day 3: Visit Arashiyama, depart from Kyoto Station."

[1820] Server: Sends the generated travel scenario to the user.

[1821] 3. Customizing the scenario

[1822] overview:

[1823] The user can request changes or additions to the generated travel scenario.

[1824] What happens:

[1825] Device: The user enters "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" and submits a correction request.

[1826] Server: Receives the correction request and resends it to the chat generation AI.

[1827] Chat generation AI: Regenerates scenarios based on revision requests. For example, it generates a new scenario called "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[1828] Server: Sends the modified scenario to the user.

[1829] 4. Share and save your scenarios

[1830] overview:

[1831] Users can save the scenarios they create and share them with other users via LINE.

[1832] What happens:

[1833] Device: The user selects whether to save the scenario or share it on LINE and sends a request.

[1834] server:

[1835] For save: Save the scenario to the database.

[1836] For sharing: Send the scenario to the specified contact via the LINE API.

[1837] LINE API: Send the received scenario information to a specified group or contact. For example, send a message to the "Family" group saying, "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[1838] In this way, users can easily create, save, and share customized travel scenarios.The system that supports users' travel planning is realized by the collaboration of the server, terminals, chat generation AI, database, and LINE API.

[1839] The processing flow will be explained below.

[1840] 1. User registration and authentication

[1841] Step 1:

[1842] On your device: The user enters their name, email address, and password into the provided registration form.

[1843] Step 2:

[1844] Terminal: The user clicks the submit button, sending the input data to the server.

[1845] Step 3:

[1846] Server: Receives input data and checks whether it is entered correctly (validation check).

[1847] Step 4:

[1848] Server: Creates SQL queries to store registration information in a database and saves the data. Passwords are hashed for security purposes.

[1849] Step 5:

[1850] Server: Generates and sends back a message to the user indicating successful registration.

[1851] 2. Requesting and Generating Travel Scenarios

[1852] Step 1:

[1853] Device: The user enters travel destination "Kyoto," date "December 1, 2023 - December 3, 2023," and budget "50,000 yen," and clicks the send button.

[1854] Step 2:

[1855] Server: Receives travel information and converts the data into a format that can be processed by AI.

[1856] Step 3:

[1857] Server: Sends the converted travel information as a request to the chat generation AI.

[1858] Step 4:

[1859] Chat generation AI: Generates a travel scenario (e.g., Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion) based on the received travel information.

[1860] Step 5:

[1861] Server: Receives travel scenarios generated by the AI ​​and sends them to the user.

[1862] 3. Customizing the scenario

[1863] Step 1:

[1864] Terminal: The user inputs "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" to an existing scenario and submits a modification request.

[1865] Step 2:

[1866] Server: Receives the correction request and sends it back to the chat generation AI.

[1867] Step 3:

[1868] Chat generation AI: Based on modification requests, it adjusts existing scenarios and generates new ones (e.g., Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine).

[1869] Step 4:

[1870] Server: Receives modified scenarios from the AI ​​and sends them to the user.

[1871] 4. Share and save your scenarios

[1872] Step 1:

[1873] Device: The user selects whether to save the scenario or share it on LINE, and sends a request to the server.

[1874] Step 2:

[1875] Server: If a save request is received, it saves the scenario to the database.

[1876] Step 3:

[1877] Server: When a sharing request is received, the server uses the LINE API to send the scenario to the specified contacts.

[1878] Step 4:

[1879] LINE API: Send the received scenario to the specified contacts. For example, send the scenario to the "Family" group with the message "Taro Tanaka shared a travel scenario."

[1880] In this way, each processing step works together, allowing users to efficiently create, modify, save, and share travel scenarios.

[1881] Example 1

[1882] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1883] Conventional travel planning systems have made it difficult for users to efficiently generate, modify, and share travel scenarios. Specifically, it is difficult to generate an appropriate scenario based on the travel information entered by the user, and customizing and modifying the scenario takes time. In addition, there is a lack of a way to easily share the generated scenario with other users.

[1884] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1885] In this invention, the server includes terminal means for a user to input travel information, means for the server to receive the input travel information and convert it into a data format, means for the server to request the travel information from the generative AI model, means for the generative AI model to generate a travel scenario based on the received travel information, means for the server to send the generated travel scenario to the user, terminal means for a user to input a request to modify the scenario, means for the server to resend the modification request to the generative AI model, means for the generative AI model to modify the scenario based on the modification request, means for the server to send the modified scenario to the user, terminal means for a user to input a request to save or share the generated scenario, means for the server to save the scenario in a database, and means for the server to share the scenario using a communication API. This enables users to easily generate travel scenarios, modify them as needed, and efficiently share them with other users.

[1886] "Terminal means" refers to a device or interface through which a user inputs travel information and modification requests and transmits them to a server.

[1887] The "server" is a central computer that processes travel information and modification requests received from users, converts them into data formats, and sends requests to the generative AI model.

[1888] "Means for converting into data format" refers to the process or algorithm for converting travel information received from a user into a format that is easy to process internally (e.g., JSON format).

[1889] The "means for making a request to the generative AI model" is a process for passing the converted travel information to the generative AI model and sending a request as a prompt sentence.

[1890] A "generative AI model" is an artificial intelligence system that generates travel scenarios based on input prompts.

[1891] The "means for generating a travel scenario" is a function that enables the generative AI model to create a specific travel plan based on travel information.

[1892] The "means for resending a modification request to the generative AI model" is a process for receiving a scenario modification request from a user and sending it again as a prompt to the generative AI model.

[1893] A "communication API" is a communication protocol or interface used to share generated travel scenarios with other users, and is a means for connecting to external systems (e.g., SNS applications).

[1894] This invention provides a system that allows users to easily create, modify, save, and share travel scenarios. This system is realized using terminals, servers, generative AI models, databases, and communication APIs.

[1895] Hardware and software used

[1896] Device: User device such as a smartphone or computer

[1897] Server: A central computer for information processing and database management.

[1898] Generative AI model: Chat generation AI (e.g. GPT-3)

[1899] Database: Data storage for storing and managing information (e.g., SQL database)

[1900] Communication API: Interface for data communication with external systems (e.g., RESTful API)

[1901] User Registration and Authentication

[1902] User: Enter your name, email address, and password, and send them from your device to the server. For example, enter the name "Taro Tanaka," the email address "taro.tanaka@example.com," and the password "securepassword."

[1903] Server: Stores the received information in a database, hashes the password, and notifies the user that registration was successful.

[1904] Requesting and generating travel scenarios

[1905] User: Enters details such as travel destination, date, and budget, and sends them to the server from the device. For example, enters "Kyoto," "December 1, 2023 - December 3, 2023," and "50,000 yen."

[1906] Server: Converts received travel information into a data format and sends it to the generative AI model.

[1907] Generative AI model: Generates travel scenarios based on received prompts.

[1908] Example prompt: "I would like to create a scenario for traveling to Kyoto. The travel dates are from December 1, 2023 to December 3, 2023, and the budget is 50,000 yen. Please suggest a scenario."

[1909] Server: Sends the generated travel scenario to the user.

[1910] Customizing travel scenarios

[1911] User: Enter any corrections or additions to the scenario and send them to the server from the device. For example, you might enter, "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[1912] Server: Resubmits the correction request to the generative AI model.

[1913] Generative AI model: Regenerates scenarios based on modification requests.

[1914] Server: Sends the modified scenario to the user.

[1915] Save and share your travel scenarios

[1916] User: Selects to save or share the scenario and sends a request from the device to the server.

[1917] server:

[1918] For save: Save the scenario to the database.

[1919] For sharing: Use a communication API to send the scenario to an external system. For example, send a message saying "Taro Tanaka has shared a travel scenario" to a specified group or contact.

[1920] This allows users to easily create, modify, save, and share travel scenarios. Furthermore, by linking together the server, terminal, generative AI model, database, and communication API, a system that supports users in planning their trips is realized.

[1921] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1922] Step 1:

[1923] The user enters travel information. Specifically, the user enters their name, email address, and password into the terminal. For example, the user enters the name "Taro Tanaka," the email address "taro.tanaka@example.com," and the password "securepassword." This is the input data.

[1924] Step 2:

[1925] The terminal sends the entered information to the server. The entered data is sent to the server as is. For example, {"name": "Taro Tanaka", "email": "taro.tanaka@example.com", "password": "securepassword"}.

[1926] Step 3:

[1927] The server processes the received information and hashes the password. Specifically, it hashes "securepassword" using a hashing algorithm (e.g., SHA-256). This is the data processing step. For example, "securepassword" -> "5e884898da28047151d0e56f8dc6292773603d0d6aabbdd5d029d6ad1db1c5f7".

[1928] Step 4:

[1929] The server stores the hashed password along with other user information in a database, such as {"name": "Taro Tanaka", "email": "taro.tanaka@example.com", "password_hash": "5e884898da28047151d0e56f8dc6292773603d0d6aabbdd5d029d6ad1db1c5f7"} .

[1930] Step 5:

[1931] The server generates a message indicating that registration has been completed successfully and sends it to the terminal. The message is simple text data, such as "Registration completed."

[1932] Step 6:

[1933] The terminal displays the received message to the user, displaying the text "Registration complete."

[1934] Step 7:

[1935] The user inputs travel information such as destination, date, budget, etc. For example, the user inputs the destination "Kyoto," the date "December 1, 2023 - December 3, 2023," and the budget "50,000 yen." This becomes new input data.

[1936] Step 8:

[1937] The device sends the entered travel information to the server. This information is passed to the server as is. For example, {"destination": "Kyoto", "dates": "December 1, 2023 - December 3, 2023", "budget": "50,000 yen"}.

[1938] Step 9:

[1939] The server converts the received travel information into an internal data format. For example, it converts it into the JSON format used internally. The internal data becomes {"destination": "Kyoto", "dates": {"start": "2023-12-01", "end": "2023-12-03"}, "budget": 50000}.

[1940] Step 10:

[1941] The server sends the converted travel information to the generative AI model as a prompt. For example, the prompt might read, "I would like to create a scenario for traveling to Kyoto. The travel dates are from December 1, 2023 to December 3, 2023, and the budget is 50,000 yen. Please suggest a scenario."

[1942] Step 11:

[1943] The generative AI model generates a travel scenario based on the received prompt. For example, the generated scenario might be "Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion," "Day 2: Visit Kiyomizu-dera Temple and Kinkaku-ji Temple," or "Day 3: Visit Arashiyama, depart from Kyoto Station."

[1944] Step 12:

[1945] The generative AI model returns the generated travel scenario to the server, which then sends the received scenario to the user. The scenario text is sent to the user.

[1946] Step 13:

[1947] The device displays the generated travel scenario to the user. Specifically, the scenario text is displayed on the device screen. For example, "Day 1: Arrive at Kyoto Station, check in at hotel, stroll around Gion..."

[1948] Step 14:

[1949] The user inputs a modification request for the generated scenario. For example, the user might input, "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day." This becomes the input data for the modification request.

[1950] Step 15:

[1951] The device sends a modification request to the server, which passes the request to the server as is. For example, {"modification": "I also want to visit Fushimi Inari Taisha Shrine in the afternoon of the second day"}.

[1952] Step 16:

[1953] The server resubmits the correction request to the generative AI model. For example, the correction request might be, "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[1954] Step 17:

[1955] The generative AI model regenerates the scenario based on the modification request. For example, the regenerated scenario might be "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[1956] Step 18:

[1957] The generative AI model sends the modified scenario back to the server, which then sends the received scenario to the user. The user receives the regenerated scenario text.

[1958] Step 19:

[1959] The terminal displays the revised scenario to the user. Specifically, the text of the regenerated scenario is displayed on the terminal screen. For example, "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[1960] Step 20:

[1961] The user inputs a request to save or share a scenario. For example, they select from the options "Save scenario" or "Share on LINE." This becomes the input data for the save or share request.

[1962] Step 21:

[1963] The device sends a request to the server. The request content is passed to the server as is. For example, for "save" it is {"action": "save"}, and for sharing it is {"action": "share"}.

[1964] Step 22:

[1965] The server processes the request accordingly. If save is selected, the scenario is saved to the database. For example, the following data is saved to the database: {"scenario": "Day 1: Arrive at Kyoto Station, check in at hotel, stroll around Gion..."}

[1966] Step 23:

[1967] If sharing is selected, the server uses a communication API to send the scenario to an external system. Specifically, it sends the scenario information to the specified group or contact. For example, it sends a message saying, "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[1968] Step 24:

[1969] The communication API sends the received scenario information to the specified group or contact, allowing users to share the scenarios they have created with other users.

[1970] Step 25:

[1971] The server notifies the user that the saving or sharing is complete, for example, "The scenario has been saved" or "The scenario has been shared."

[1972] Step 26:

[1973] The device displays a notification message to the user, specifically, a message such as "Scenario saved" or "Scenario shared" on the device screen.

[1974] (Application example 1)

[1975] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1976] Conventional food delivery services often lack the flexibility to accommodate specific user meal requests. Furthermore, when users customize their meal menus or have complex dietary restrictions, it is difficult to quickly provide suggestions that meet those requests. Furthermore, they lack the ability to easily share or save generated menus with other users. There is a need for a food delivery system that can meet these user needs.

[1977] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1978] In this invention, the server includes a terminal means for a user to input meal information, a means for the server to receive the input meal information and convert it into a data format, a means for the server to request the meal information from a generative AI model, a means for the generative AI model to generate a meal menu based on the received meal information, a means for the server to send the generated meal menu to the user, and a means for customizing the meal menu. This allows users to quickly and flexibly receive menu suggestions based on specific meal requests. Furthermore, the generated menu can be easily customized, saved, and shared.

[1979] A "user" is an individual who utilizes the system to input meal information and receive or customize the generated menu.

[1980] "Meal information" refers to information including the details of the meal desired by the user, such as the type of meal, budget, and specific dietary restrictions.

[1981] "Terminal means" refers to a device used by a user to input meal information, and specifically refers to a smartphone, tablet, etc.

[1982] A "server" is a computer system that receives and processes input information from a user.

[1983] "Means for converting into data format" refers to the function of converting the dietary information entered by the user into a format that can be understood by the generative AI model.

[1984] A "generative AI model" is an artificial intelligence model that generates optimal meal menus based on input meal information.

[1985] "Means for requesting" refers to the function by which the server sends the meal information received from the user to the generative AI model.

[1986] A "meal menu" is a specific meal suggestion generated by a generative AI model based on dietary information.

[1987] "Customizable means" refers to a function that allows a user to request changes or additions to the meal menu they receive.

[1988] The "means for saving" is a function that allows a user to save the menu created in a database within the system.

[1989] "Means for sharing" is a function for transmitting the generated menu using a communication means in order to share it with other users.

[1990] This invention provides a food delivery system that allows users to input their desired meal details, and a generative AI model proposes and customizes meal menus based on that information, which can then be saved and shared. The system is configured using the following hardware and software:

[1991] Hardware

[1992] Server: A computer system for receiving and processing meal information from users.

[1993] Terminal: The device through which the user inputs meal information. Examples include smartphones and tablets.

[1994] software

[1995] Python (Django, Flask, etc.): A framework for performing server-side processing.

[1996] MySQL: Used as a database management system.

[1997] LINE API: Communication method for sharing menus.

[1998] OpenAI's generative AI model: Used to generate menus based on meal information.

[1999] Program processing

[2000] The server receives the meal information sent by the user and converts it into a data format. The converted data is sent as a request to the generative AI model, which then generates a meal menu based on the meal information. For example, based on the information that "the user requested pizza as the meal type and their budget is 2,000 yen," the generative AI model would suggest the optimal pizza menu.

[2001] The generated meal menu is sent to the user by the server. If the user subsequently wishes to customize the menu, they can input a modification request from their device, which is then sent again to the generative AI model via the server. The generative AI model then generates a new menu based on the modification request and sends it to the user via the server.

[2002] If the user saves the created menu, the server saves it in the database. If the user wishes to share the menu, the server uses the LINE API to send the menu to the specified contacts or group. This allows users to easily customize, save, and share their desired meal menu.

[2003] Specific examples

[2004] If a user inputs a request such as "vegan pizza, budget 3,000 yen," the generative AI model will receive the following prompts and generate the optimal menu based on that request:

[2005] Example prompt sentence:

[2006] Based on user requests, please suggest a vegan pizza menu within a budget of 3,000 yen.

[2007] When a user receives the generated menu and sends a modification request, such as "I'd like to add an olive topping," the generative AI model reprocesses it and generates the following modification prompt:

[2008] Example of a modified prompt:

[2009] Add olive topping to your current menu and regenerate.

[2010] The system allows users to efficiently obtain customized meal menus that can be easily saved and shared with other users.

[2011] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2012] Step 1:

[2013] The user inputs meal information. Using a device such as a smartphone or tablet, the user inputs detailed information such as meal type (e.g., pizza) and budget (e.g., 2,000 yen) and submits it. The input data is sent to the server.

[2014] Step 2:

[2015] The server receives the meal information and converts it into a data format. The server analyzes the raw data received from the user and converts it into a format that the generative AI model can understand, for example, JSON format. The input is the user's meal information, and the output is the converted data format.

[2016] Step 3:

[2017] The server sends a request to the generative AI model. The converted data format is used to send a prompt to the generative AI model (for example, OpenAI's GPT model). The specific prompt generated is, "Based on the user's request, please suggest a pizza menu item within a budget of 2,000 yen."

[2018] Step 4:

[2019] The generative AI model generates a meal menu. Based on the prompt received, the generative AI model generates the optimal meal menu. The input is the prompt, and the output is the generated meal menu. For example, a menu such as "Margherita pizza, topped with basil and cheese, priced at 1,800 yen" is generated.

[2020] Step 5:

[2021] The server sends the generated meal menu to the user. The server then sends the meal menu received from the generative AI model back to the user's device. The user can then view the proposed menu on their device.

[2022] Step 6:

[2023] The user inputs a menu customization request. For example, the user sends a request to change the proposed menu, such as "I would like to add olives to the toppings," to the server via the terminal.

[2024] Step 7:

[2025] The server sends the customization request to the generative AI model. The server receives the modification request from the user, reformats it, and sends it back to the generative AI model. The specific prompt generated is "Please add olive topping to the current menu and regenerate it."

[2026] Step 8:

[2027] The generative AI model generates a modified meal menu. The generative AI model generates a new menu based on the modification request. The input is the modification prompt, and the output is the modified meal menu.

[2028] Step 9:

[2029] The server sends the revised menu to the user, and upon receiving the revised menu, the server sends it again to the user's terminal.

[2030] Step 10:

[2031] When a user wants to save a generated menu, he sends a save request to the server through the terminal. The server saves this menu in the database. The input is the menu save request and the output is the saved menu.

[2032] Step 11:

[2033] When a user wants to share a created menu, they send a share request to the server via their device. The server then shares this menu with other users using the LINE API, etc. The input is the share request, and the output is the shared menu.

[2034] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2035] This invention provides a system that allows users to create, modify, save, and share travel scenarios while improving the user experience by combining an emotion engine that recognizes user emotions. This system is realized using devices, a server, chat generation AI, a database, the LINE API, and an emotion engine.

[2036] 1. User registration and authentication

[2037] overview:

[2038] To access the system, users first register, which creates a dedicated account for them.

[2039] What happens:

[2040] Terminal: The user enters their name, email address, and password into the registration form. For example, they enter the username "Taro Tanaka," the email address "taro.tanaka@example.com," and the password "securepassword," and submit.

[2041] Server: Receives registration information and stores it in a database. Passwords are hashed for security purposes.

[2042] Server: Generates and sends a message to the user indicating successful registration.

[2043] 2. Requesting and Generating Travel Scenarios

[2044] overview:

[2045] Users input their travel details (destination, date, budget, etc.), and the chat generation AI generates a travel scenario based on that information. At this time, the emotion engine recognizes the user's emotions and reflects them in customizing the travel scenario.

[2046] What happens:

[2047] Device: The user enters travel destination "Kyoto," date "December 1, 2023 - December 3, 2023," and budget "50,000 yen," and clicks the send button.

[2048] Emotion engine: Analyzes facial expressions and voice data while the user is typing to recognize emotions. For example, if the user is excited, it will suggest an active sightseeing plan.

[2049] Server: Receives travel information and emotion information and converts it into a data format.

[2050] Server: Sends the converted travel information and emotion information as a request to the chat generation AI.

[2051] Chat generation AI: Generates a travel scenario (e.g., Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion) based on received travel information and emotional information.

[2052] Server: Sends the generated travel scenario to the user.

[2053] 3. Customizing the scenario

[2054] overview:

[2055] The user can request changes or additions to the generated travel scenario, and the emotion engine will recognize the user's emotions and reflect them in the modification request.

[2056] What happens:

[2057] Terminal: The user inputs "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" to an existing scenario and submits a modification request.

[2058] Emotion engine: Analyzes facial and voice data while the user is entering a correction request to recognize emotions. For example, if the user is feeling stressed, the engine can suggest relaxation plans.

[2059] Server: Receives the modification request and emotion information, and sends the request again to the chat generation AI.

[2060] Chat generation AI: Based on modification requests and emotional information, it adjusts existing scenarios and generates new ones (e.g., Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine).

[2061] Server: Sends the modified scenario to the user.

[2062] 4. Share and save your scenarios

[2063] overview:

[2064] Users can save the scenarios they create or share them with other users via LINE. In this case, the emotion engine also recognizes the user's emotions and suggests appropriate ways to share them.

[2065] What happens:

[2066] Device: The user selects whether to save the scenario or share it on LINE, and sends a request to the server.

[2067] Emotion engine: Analyzes facial expressions and voice data while the user is operating the app to recognize emotions. For example, if the user is excited, the engine will suggest ways to share the app.

[2068] Server: If a save request is received, it saves the scenario to the database.

[2069] Server: When a sharing request is received, the server uses the LINE API to send the scenario to the specified contacts.

[2070] LINE API: Send the received scenario to the specified contacts. For example, to the "Family" group, send the scenario with the message "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[2071] In this way, each processing step works together, allowing users to efficiently create, modify, save, and share customized travel scenarios based on emotion recognition.The collaboration of each element—server, device, chat generation AI, database, LINE API, and emotion engine—creates a system that improves the user experience.

[2072] The processing flow will be explained below.

[2073] 1. User registration and authentication

[2074] Step 1:

[2075] On your device: The user enters their name, email address, and password into the provided registration form.

[2076] Step 2:

[2077] Terminal: The user clicks the submit button, sending the input data to the server.

[2078] Step 3:

[2079] Server: Receives input data and checks whether it is entered correctly (validation check).

[2080] Step 4:

[2081] Server: Creates SQL queries to store registration information in a database and saves the data. Passwords are hashed for security purposes.

[2082] Step 5:

[2083] Server: Generates and sends back a message to the user indicating successful registration.

[2084] 2. Requesting and Generating Travel Scenarios

[2085] Step 1:

[2086] Device: The user enters travel destination "Kyoto," date "December 1, 2023 - December 3, 2023," and budget "50,000 yen," and clicks the send button.

[2087] Step 2:

[2088] Emotion engine: Analyzes facial expressions and voice data in real time while the user is entering travel information to extract emotional information. For example, if the user is excited, it determines that the user prefers an active scenario.

[2089] Step 3:

[2090] Server: Receives travel information and emotion information and converts it into an internal data format.

[2091] Step 4:

[2092] Server: Sends the converted travel information and emotion information as a request to the chat generation AI.

[2093] Step 5:

[2094] Chat generation AI: Generates travel scenarios based on received travel information and emotional data. For example, it generates a scenario including "Day 1: Arrival at Kyoto Station, check-in at hotel, stroll around Gion, Day 2: Sightseeing at Kiyomizu-dera Temple and Kinkaku-ji Temple, and activities to boost your mood."

[2095] Step 6:

[2096] Server: Receives travel scenarios generated by the AI ​​and sends them to the user.

[2097] 3. Customizing the scenario

[2098] Step 1:

[2099] Terminal: The user submits a modification request to an existing scenario by entering "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[2100] Step 2:

[2101] Emotion engine: Analyzes facial expressions and voice data in real time while the user is entering correction requests to extract emotional information. For example, if the user is feeling stressed, the engine will suggest plans to help them relax.

[2102] Step 3:

[2103] Server: Receives modification requests and emotion information and converts them into an internal data format.

[2104] Step 4:

[2105] Server: Resubmit the correction request and emotion information to the chat generation AI.

[2106] Step 5:

[2107] Chat generation AI: Based on revision requests and emotional information, it adjusts existing scenarios and generates new ones. For example, it generates a scenario that includes "Day 2: Sightseeing at Kiyomizu-dera Temple and Kinkaku-ji Temple, visiting Fushimi Inari Taisha Shrine, and experiencing a tea ceremony to relax."

[2108] Step 6:

[2109] Server: Receives modified scenarios from the AI ​​and sends them to the user.

[2110] 4. Share and save your scenarios

[2111] Step 1:

[2112] Device: The user selects whether to save the scenario or share it on LINE, and sends a request to the server.

[2113] Step 2:

[2114] Emotion engine: Analyzes facial expressions and voice data in real time while the user is operating the app, extracting emotional information. For example, if the user is excited, the engine suggests ways to share the information.

[2115] Step 3:

[2116] Server: If a save request is received, save the scenario to the database. If a share request is received, send the scenario to the specified contacts using the LINE API.

[2117] Step 4:

[2118] LINE API: Send the received scenario to the specified contacts. For example, send the scenario to the "Family" group with the message "Taro Tanaka shared a travel scenario: Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion..."

[2119] In this way, each processing step works together, allowing users to efficiently create, modify, save, and share customized travel scenarios based on emotion recognition.The collaboration of each element—server, device, chat generation AI, database, LINE API, and emotion engine—creates a system that improves the user experience.

[2120] Example 2

[2121] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2122] Conventional travel scenario generation systems have difficulty in properly reflecting users' emotions and individual needs. Furthermore, they lack the means to detect changes in users' emotions and optimize the scenarios based on those emotions during the process of generating, modifying, saving, and sharing the scenarios. As a result, user satisfaction cannot be sufficiently increased. Furthermore, the methods for sharing scenarios are limited, making it difficult to smoothly collaborate with other users.

[2123] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes terminal means for the user to input travel information, means for the terminal means to transmit the travel information, means for the server to receive the input travel information and convert it into a data format, means for the server to request the travel information from the chat generation AI, means for the chat generation AI to generate a travel scenario based on the received travel information, means for the emotion engine to recognize the user's emotions and generate data to reflect them in the travel scenario, and means for the server to transmit the generated travel scenario to the user. This makes it possible to generate, modify, save, and share customized travel scenarios that take the user's emotions into consideration.

[2124] "Terminal means" refers to a device that allows a user to input information and receive information from the system.

[2125] "Server" means a computing device that receives, processes, and stores information sent from terminal means over a network, and transmits the data to other systems or devices as needed.

[2126] The "means for converting into a data format" is a function for converting information received from a user into a format that can be used by other systems or processing modules.

[2127] "Chat generation AI" is an artificial intelligence system that generates natural language based on user input and creates specific scenarios and suggestions.

[2128] The "means for generating a travel scenario" is the process by which the chat generation AI formulates a specific travel plan based on the information it receives.

[2129] The "emotion engine" is an analytical engine that recognizes the user's emotions, generates them as data, and provides them to other systems.

[2130] "Means for generating data to be reflected in travel scenarios" refers to the function in which the emotion engine analyzes the user's emotions and creates data that can be used to generate or modify scenarios.

[2131] A "modification request" is a request made by a user to make changes or additions to an existing travel scenario.

[2132] The "terminal means for inputting a modification request" is a device that allows a user to input a request for a change or addition to a scenario.

[2133] The "means for sending a correction request and emotional information to the chat generation AI" is a function in which the server sends the input correction request and recognized emotional information to the chat generation AI, requesting that the scenario be corrected.

[2134] The "means for saving a scenario in a database" is a function for storing a generated or modified scenario in a database in order to save it permanently.

[2135] "Means for sharing scenarios using external APIs" is a function that allows the server to work with other applications and services to share scenarios.

[2136] This invention is a system that allows users to create, modify, save, and share travel scenarios while improving the user experience by combining an emotion engine that recognizes user emotions. This system is realized using devices, a server, chat generation AI, a database, an external API, and an emotion engine.

[2137] 1. User registration and authentication

[2138] overview:

[2139] To access the system, users first register, which creates a dedicated account for them.

[2140] Hardware and software used:

[2141] Terminal: The device through which the user enters information (PC, smartphone, etc.)

[2142] Server: A server that receives registration information and stores it in a database

[2143] Database: Data storage for storing registration information

[2144] Emotion engine: facial recognition and voice analysis software

[2145] 2. Requesting and Generating Travel Scenarios

[2146] overview:

[2147] Users input their travel details (destination, date, budget, etc.), and the chat generation AI generates a travel scenario based on that information. At this time, the emotion engine recognizes the user's emotions and reflects them in customizing the travel scenario.

[2148] Hardware and software used:

[2149] Terminal: A device through which a user enters information.

[2150] Server: A server for receiving travel information and emotion information and converting them into data formats.

[2151] Chat Generation AI: Artificial Intelligence for Generating Travel Scenarios

[2152] Emotion Engine: Software for recognizing user emotions

[2153] Database: Data storage for travel scenarios and user information

[2154] Examples:

[2155] Let's say a user inputs their travel destination "Kyoto," the dates "December 1, 2023 - December 3, 2023," and a budget of "50,000 yen" and sends them to the system. At this time, the emotion engine analyzes the user's facial expressions and voice data and recognizes emotions such as excitement. This allows the system to suggest an active sightseeing plan.

[2156] 3. Customizing the scenario

[2157] overview:

[2158] The user can request changes or additions to the generated travel scenario, and the emotion engine will recognize the user's emotions and reflect them in the modification request.

[2159] Hardware and software used:

[2160] Terminal: A device through which a user enters information.

[2161] Server: A server for receiving modification requests and emotion information and converting them into data formats.

[2162] Chat Generation AI: Artificial intelligence that generates revised scenarios

[2163] Emotion Engine: Software for recognizing user emotions

[2164] Database: Data storage for generated scenarios

[2165] Examples:

[2166] Let's say a user inputs "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day" into an existing scenario and sends it to the system. At this time, the emotion engine analyzes the facial expressions and voice data while the user is inputting, and if the user is feeling stressed, it will suggest a plan that will help them relax.

[2167] 4. Share and save your scenarios

[2168] overview:

[2169] Users can save the scenarios they create or share them with other users through external APIs. In this case, the emotion engine also recognizes the user's emotions and suggests appropriate solutions.

[2170] Hardware and software used:

[2171] Terminal: A device through which a user enters information.

[2172] Server: The server that receives the share / save request.

[2173] Database: Data storage for saving scenarios

[2174] External API: Interface for connecting with external services such as the LINE API

[2175] Examples:

[2176] Let's say a user chooses to save the scenario or share it via the LINE API and sends a request to the system. If the emotion engine recognizes high tension, it will suggest a way to share it.

[2177] Example prompt sentence:

[2178] A specific prompt for a user to enter travel information might be "Destination: Kyoto, Date: December 1, 2023 - December 3, 2023, Budget: 50,000 yen." In addition, for a revision request, the user could enter "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[2179] As described above, the present invention realizes a system that takes into account the user's emotions and provides a more personalized travel scenario.

[2180] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2181] Step 1: User enters trip information

[2182] User: Enter details such as travel destination, date, and budget into the device. For example, enter "Kyoto," "December 1, 2023 - December 3, 2023," and "50,000 yen."

[2183] Terminal: Converts the input travel information into a data format and generates a request to send it to the server.

[2184] Input: Trip information entered by the user.

[2185] Output: A travel information request converted into a data format.

[2186] Step 2: The server receives the trip information

[2187] Server: Receives the travel information request sent from the device.

[2188] Input: A travel information request converted into a data format.

[2189] Output: Received trip information.

[2190] Step 3: The emotion engine recognizes the user's emotion

[2191] Emotion engine: Analyzes facial expressions and voice data when users input, and recognizes emotions. Uses emotion recognition APIs and voice analysis software.

[2192] Input: User's facial expression and voice data.

[2193] Output: Recognized emotion information.

[2194] Step 4: The server integrates the emotion information and the travel information.

[2195] Server: Integrates the received travel information and emotional information and converts it into a data format for sending to the chat generation AI.

[2196] Input: Received travel information and recognized emotion information.

[2197] Output: Data integrating travel information and emotional information.

[2198] Step 5: The server sends the integrated data to the chat generation AI.

[2199] Server: Sends data that integrates travel information and emotional information as a request to the chat generation AI.

[2200] Input: Data integrating travel information and emotional information.

[2201] Output: A request to send to the chat generation AI.

[2202] Step 6: Chat generation AI generates travel scenarios

[2203] Chat generation AI: Generates a travel scenario based on the received travel information and emotional information. For example, it creates a plan such as "Day 1: Arrive at Kyoto Station, check into hotel, stroll around Gion."

[2204] Input: Travel information and emotional information sent to the chat generation AI.

[2205] Output: The generated travel scenario.

[2206] Step 7: The server receives the generated travel scenario and sends it to the user.

[2207] Server: Receives the generated travel scenarios and converts them into a data format for transmission to the user.

[2208] Input: Generated travel scenarios.

[2209] Output: The travel scenario converted into a data format to be sent to the user.

[2210] Step 8: User receives and displays the travel scenario

[2211] Terminal: Receives the travel scenario sent from the server and displays it to the user.

[2212] Input: A travel scenario converted into a data format.

[2213] Output: The travel scenario displayed on the user's screen.

[2214] Step 9: User enters a request to modify the scenario

[2215] User: Request a change or addition to an existing scenario. For example, enter "I would also like to visit Fushimi Inari Taisha Shrine in the afternoon of the second day."

[2216] Terminal: Converts the modification request into a data format and generates a request to send to the server.

[2217] Input: The correction request entered by the user.

[2218] Output: The request for the modification request converted into data format.

[2219] Step 10: Server receives the modification request

[2220] Server: Receives the modification request sent from the device.

[2221] Input: The request for the correction request converted into data format.

[2222] Output: The correction request received.

[2223] Step 11: The emotion engine recognizes the user's emotion again

[2224] Emotion engine: Re-analyzes the user's facial and voice data while the correction request is being entered to recognize emotions.

[2225] Input: User's facial expression and voice data.

[2226] Output: Re-recognized emotion information.

[2227] Step 12: The server integrates the modification request with the emotion information.

[2228] Server: Integrates received correction requests and emotional information, and converts them into a data format for retransmission to the chat generation AI.

[2229] Input: Received correction request and re-recognized emotion information.

[2230] Output: Data integrating correction requests and emotional information.

[2231] Step 13: The server resends the integrated data to the chat generation AI.

[2232] Server: The data that integrates the correction request and emotional information is sent to the chat generation AI as a re-request.

[2233] Input: Data integrating correction requests and emotional information.

[2234] Output: A request to resend to the chat generation AI.

[2235] Step 14: Chat generation AI generates revised scenarios

[2236] Chat generation AI: Based on the received revision requests and emotional information, the AI ​​modifies the scenario. For example, it generates a revised plan such as "Day 2: Visit Kiyomizu-dera Temple, Kinkaku-ji Temple, and Fushimi Inari Taisha Shrine."

[2237] Input: Edit request and sentiment sent to the chat generation AI.

[2238] Output: The modified travel scenario.

[2239] Step 15: The server receives the modified travel scenario and sends it to the user.

[2240] Server: Receives the modified travel scenario and converts it into a data format for transmission to the user.

[2241] Input: The modified travel scenario.

[2242] Output: The modified trip scenario converted into a data format for sending to the user.

[2243] Step 16: User receives and displays the revised travel scenario

[2244] Terminal: Receives the modified travel scenario sent from the server and displays it to the user.

[2245] Input: A modified travel scenario converted into a data format.

[2246] Output: The revised trip scenario displayed on the user's screen.

[2247] Step 17: User saves or shares the scenario

[2248] User: Choose whether to store the generated scenario in a database or use an external API to share the scenario with other users.

[2249] Device: Generates a save or share request to convert it into a data format and send it to the server.

[2250] Input: The save or share request selected by the user.

[2251] Output: The save or share request converted into a data format.

[2252] Step 18: Server receives save request

[2253] Server: Receives the save request sent from the device.

[2254] Input: The save request converted to a data format.

[2255] Output: The query to save to the database.

[2256] Step 19: The server saves the scenario to the database

[2257] Server: Saves the scenario to the database based on the save request.

[2258] Input: The query to save to the database.

[2259] Output: The scenario stored in the database.

[2260] Step 20: Share your scenario using an external API

[2261] Server: When a sharing request is received, it uses an external API to send the scenario to the specified contacts.

[2262] External API: Use LINE API, etc., to send the scenario as a message to the specified contact.

[2263] Input: The share request converted into data format.

[2264] Output: The scenario shared with the specified contact.

[2265] In this way, at each processing step, the input data is processed and the process moves to the next step, making it possible to generate, modify, save, and share customized travel scenarios that take the user's emotions into account.

[2266] (Application example 2)

[2267] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2268] Conventional advertising systems have difficulty targeting users based on their real-time emotions. This can result in the inability to provide ads that best fit the user's interests and emotions, resulting in insufficient advertising effectiveness. Another issue is that there is no established method for accurately capturing user emotions and generating ads based on them. Furthermore, the process of generating emotion-based ads is complicated, making effective ad delivery difficult.

[2269] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes terminal means for a user to input information, means for the server to receive the input information and convert it into a data format, means for the server to request information from a chat generation AI, means for the chat generation AI to generate a scenario based on the received information, means for the server to send the generated scenario to the user, and means for the emotion engine to recognize the user's emotions and reflect them in the input data. This makes it possible to generate and deliver personalized advertisements based on the user's real-time emotions.

[2270] The "terminal means through which the user inputs information" refers to an electronic device that is operated by the user to input information into the system.

[2271] "Means by which the server receives input information and converts it into a data format" refers to the process by which the server receives information sent by the user and converts that information into an appropriate format.

[2272] "Means for the server to request information from the chat generation AI" refers to a method in which the server uses the converted information to request processing from the chat generation AI.

[2273] "Means for generating a scenario based on information received by the chat generation AI" refers to the function by which the chat generation AI utilizes input information to automatically generate a scenario that suits a specific purpose.

[2274] "Means by which the server sends the generated scenario to the user" refers to the method by which the server delivers the scenario generated by the chat generation AI to the user.

[2275] "Means by which the emotion engine recognizes the user's emotions and reflects them in the input data" refers to the process of analyzing the user's emotions using emotion recognition technology and reflecting the results in the input data and scenario generation.

[2276] The "terminal means for the user to input a reque...

Claims

1. a terminal means for a user to input travel information; A server receives the input travel information and converts it into a data format; A means for the server to request travel information from the chat generation AI, A means for generating a travel scenario based on the travel information received by the chat generation AI; a means for the server to transmit the generated travel scenario to a user; A system including:

2. a terminal means for a user to input a request to modify a scenario; A means for the server to retransmit the correction request to the chat generation AI; A means for the chat generation AI to modify the scenario based on the modification request, and means for the server to send the modified scenario to the user; The system of claim 1 , comprising:

3. a terminal means for a user to input a request to save or share the generated scenario; a means for the server to store the scenario in a database; A means for the server to share a scenario using the LINE API; The system of claim 1 , comprising:

Images