system

Abstract

Provide a system. A means for a user to input a travel destination and itinerary; means for transmitting the input information to a server; A means for a server to generate a plurality of optimal travel schedules using a generation AI model based on the input information; means for transmitting the generated schedule to a terminal and displaying it; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Travelers face the problem of having to spend a lot of time and effort when planning their trips. Furthermore, changes in weather and traffic conditions can make it difficult to enjoy sightseeing as planned. Furthermore, managing one's health while traveling and checking parking availability are also important issues. There is a need for a way to solve these problems and enjoy travel more efficiently and safely. [Means for solving the problem]

[0005] The present invention provides a system including a means for a user to input a travel destination and itinerary, a means for transmitting the input information to a server, a means for the server to generate multiple optimal travel schedules based on the input information using a generative AI model, and a means for transmitting and displaying the generated schedules to a terminal. The system further includes a means for the server to periodically obtain weather and traffic information, evaluate the impact of the obtained information on the current schedule, generate a new schedule, and propose it to the terminal. The system also includes a means for the terminal to use a camera to monitor the user's facial recognition and physical condition, detect abnormalities based on the monitoring results, display a warning, and suggest rest areas and preventive measures. The system also includes a means for the terminal to query the server for nearby parking availability information based on the user's current location and destination, collect parking availability information in real time, select the optimal parking lot, and transmit the selected parking lot information to the terminal for proposal. In this way, the present invention achieves efficient travel planning, flexible travel arrangements, health management, and parking management, thereby improving convenience and safety for travelers.

[0006] "User" refers to an individual using the Travel Application.

[0007] "Destination" refers to a geographic location that a user plans to visit.

[0008] "Dates" refers to a range of dates and times indicating the duration of the user's trip.

[0009] "Terminal" refers to the electronic device that a user uses to run the travel application.

[0010] "Server" refers to a computing system that processes data to generate and manage travel schedules.

[0011] A "generative AI model" refers to an artificial intelligence algorithm that automatically generates optimal schedules and proposals based on input information.

[0012] A "schedule" refers to a timetable showing planned places to visit and activities during a trip.

[0013] "Weather Information" means information that includes current weather data and forecasts for a particular geographic area.

[0014] "Traffic Information" refers to information that includes data about traffic conditions during a trip.

[0015] "Monitoring" refers to the process by which a device observes and monitors the user's situation through sensors such as cameras.

[0016] "Warning" refers to an alert that notifies the user of some abnormality or situation that requires attention.

[0017] "Rest Stop" refers to a specific location suggested for the user to take a rest during the journey.

[0018] "Parking information" refers to information that includes data regarding the availability and location of a particular parking space.

[0019] "Real-time" refers to the immediate reflection of ongoing conditions and events. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] overview

[0042] The smart tour guide app of this invention is a system that improves the efficiency of travel planning and provides flexible support during the trip. This system allows users to input their travel destinations and itineraries, and generates an optimal schedule using a generative AI model based on that information. It also obtains real-time weather and traffic information during the trip, and suggests or changes the schedule depending on the situation. Furthermore, the device also has the function of managing the user's health and checking the availability of parking spaces.

[0043] The user inputs the travel destination and dates

[0044] 1. The user launches the Smart Tour Guide app and enters their travel destination and dates.

[0045] 2. The terminal sends the entered information to the server.

[0046] The server generates the schedule

[0047] 1. The server calls the generative AI model based on the received travel destination and itinerary.

[0048] 2. The server collects information on tourist attractions, restaurants, events, etc. at the travel destination and generates several optimal schedules based on this information.

[0049] 3. The generated schedule is sent to the terminal and displayed to the user, who can then select the best one from the multiple schedules.

[0050] Get real-time information and schedule changes while traveling

[0051] 1. The device periodically collects weather and traffic information and sends it to the server.

[0052] 2. The server analyzes the real-time information and generates a new schedule if it affects the current schedule.

[0053] 3. If a new schedule is required, the server sends it to the terminal, which notifies the user, who then checks the new schedule and approves the changes.

[0054] User health management

[0055] 1. The device uses a camera to recognize the user's face and monitor their physical condition.

[0056] 2. If an abnormality or drowsiness is detected, the device will display a warning to the user and suggest rest or preventive measures.

[0057] Check parking availability

[0058] 1. When a user searches for a parking space, the device queries the server for nearby parking information based on the current location and destination.

[0059] 2. The server collects real-time information on available parking spaces in the surrounding area and selects the most suitable parking space.

[0060] 3. The selected parking lot information is sent to the terminal and suggested to the user, who then parks the car in the designated parking lot.

[0061] Specific examples

[0062] Suppose a user launches a travel app and enters "Tokyo" and "May 1st to 3rd." The device sends this information to a server, which uses a generative AI model to create multiple schedules. For example, a schedule might be generated in which the user visits tourist spot A on day 1, tourist spot B and restaurant C on day 2, and so on. This schedule is then sent to the device and displayed to the user.

[0063] If it starts raining while the user is traveling to tourist spot A, the device automatically updates the weather information and reports it to the server. The server generates a new schedule that changes outdoor activities to indoor sightseeing and sends it to the device. The user can then review and select the new schedule.

[0064] Furthermore, if a user begins to feel drowsy while driving, the device's camera will detect frequent blinking and display a warning, allowing the user to take a break at a nearby rest spot.

[0065] When searching for a parking space, the user searches for a parking space around Tokyo Station, and the server finds a suitable parking space and sends information about its availability to the terminal. Based on this information, the user can park in the most optimal location.

[0066] In this way, the present invention allows users to enjoy traveling efficiently and comfortably.

[0067] The processing flow will be explained below.

[0068] A flow where the user inputs the travel destination and dates and generates a schedule

[0069] Step 1:

[0070] The user launches the Smart Tour Consulting app on their device.

[0071] Step 2:

[0072] The user inputs the travel destination and dates.

[0073] Step 3:

[0074] The terminal transmits the travel destination and itinerary input by the user to the server.

[0075] Step 4:

[0076] The server receives the travel destination and itinerary information.

[0077] Step 5:

[0078] The server calls the generated AI model and collects information on tourist attractions, restaurants, events, etc. at the travel destination.

[0079] Step 6:

[0080] Based on the information collected by the server, multiple optimal schedules are generated.

[0081] Step 7:

[0082] The server transmits the generated schedule to the terminal.

[0083] Step 8:

[0084] The terminal displays the generated schedule to the user.

[0085] Step 9:

[0086] The user selects the best schedule from among multiple schedules.

[0087] Get real-time information and schedule changes while traveling

[0088] Step 1:

[0089] The device periodically collects weather and traffic information.

[0090] Step 2:

[0091] The device sends the collected weather and traffic information to the server.

[0092] Step 3:

[0093] The server receives and analyzes the real-time information.

[0094] Step 4:

[0095] The server evaluates whether the current schedule is affected.

[0096] Step 5:

[0097] The server generates new schedules as needed.

[0098] Step 6:

[0099] The server sends the new schedule to the terminal.

[0100] Step 7:

[0101] The terminal notifies the user of the new schedule.

[0102] Step 8:

[0103] The user reviews and approves the proposed schedule changes.

[0104] User health management and warning function

[0105] Step 1:

[0106] The device uses a camera to recognize the user's face and monitor their physical condition.

[0107] Step 2:

[0108] The device uses facial recognition technology to detect signs of drowsiness in the user (e.g., frequent blinking, rubbing of eyes, etc.).

[0109] Step 3:

[0110] If the device detects an abnormality, it displays a warning to the user.

[0111] Step 4:

[0112] The device will suggest appropriate rest areas and preventive measures to the user.

[0113] Check parking availability

[0114] Step 1:

[0115] A user searches for parking near a destination.

[0116] Step 2:

[0117] The terminal queries the server for information about nearby parking lots based on the user's current location and destination.

[0118] Step 3:

[0119] The server collects real-time information on available parking spaces in the surrounding area.

[0120] Step 4:

[0121] The server selects the best parking space.

[0122] Step 5:

[0123] The server transmits the selected parking lot information to the terminal.

[0124] Step 6:

[0125] The terminal proposes optimal parking information to the user.

[0126] Through the above steps, the present invention provides users with more efficient travel planning, flexible travel arrangements, health management, and parking lot management.

[0127] Example 1

[0128] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0129] In today's world, planning and executing a trip involves many variables, so a flexible and efficient travel planning system is necessary to ensure a comfortable and stress-free trip. Conventional travel planning systems have had difficulty adequately meeting diverse needs, such as optimizing schedules, responding to unexpected changes in circumstances, managing the user's health, and checking parking availability. The present invention aims to solve these problems and provide users with a fulfilling travel experience.

[0130] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0131] In this invention, the server includes a means for a user to input travel destinations and itineraries, a means for transmitting the input information to the server, a means for the server to generate multiple optimal travel schedules using a generative AI model based on the input information, and a means for transmitting the generated schedules to a terminal and displaying them, thereby enabling users to easily create optimal travel plans.

[0132] Furthermore, the server includes means for periodically acquiring weather and traffic information, means for evaluating the impact on the current schedule based on the acquired information, and means for generating a new schedule and proposing it to the terminal, thereby enabling the system to flexibly respond to unexpected changes in circumstances.

[0133] Furthermore, the terminal includes a means for recognizing the user's face and monitoring their physical condition using a camera, a means for detecting abnormalities based on the monitoring results and displaying a warning, and a means for suggesting rest areas and preventive measures, thereby protecting the user's safety and health.

[0134] Furthermore, the system includes a means for the terminal to search for available parking spaces and inquire of the server, a means for the server to select an appropriate parking space based on the current location and destination, and a means for transmitting information about the selected parking space to the terminal and suggesting it to the user, thereby efficiently solving parking problems.

[0135] In addition, the system includes a means for generating a prompt sentence for the server to call the generative AI model, a means for generating a schedule using the generative AI model based on the prompt sentence, and a means for presenting the schedule to the user as multiple options, thereby providing the user with a variety of options and realizing more satisfying travel plans.

[0136] "User" refers to a person who uses the system to provide input to plan a trip.

[0137] "Destination" refers to a geographic location that a user plans to visit.

[0138] "Schedule" refers to the start and end dates of the trip planned by the user, as well as the duration of the trip.

[0139] "Terminal" refers to the digital device that a user uses to enter their travel plans and receive system results.

[0140] "Server" refers to a remote computer system that receives user input, processes it, and sends the results to the terminal.

[0141] "Generative AI Model" refers to an artificial intelligence model used to generate travel itineraries based on input information.

[0142] "Schedule" refers to the plan of the itinerary and activities for each day at the travel destination.

[0143] "Weather information" refers to data about the weather around your travel destination.

[0144] "Traffic information" refers to data regarding traffic conditions around the travel destination.

[0145] "Facial recognition" refers to technology that uses a camera to identify a user's face.

[0146] "Physical condition" refers to information for monitoring the user's health condition and fatigue level.

[0147] "Parking Availability" refers to information about available parking spaces within a specified area.

[0148] "Prompt" refers to a textual instruction used to invoke a generative AI model.

[0149] overview

[0150] The smart tour guide system of the present invention is a system that improves the efficiency of travel planning and provides flexible support during the trip. The system allows users to input their travel destinations and itineraries, and generates an optimal schedule based on that information using a generative AI model on the server. It also obtains weather and traffic information during the trip in real time, and proposes or changes the schedule depending on the situation. Furthermore, the terminal also has the function of managing the user's health and checking the availability of parking spaces.

[0151] Hardware and software used

[0152] The main hardware and software used to implement this system are as follows:

[0153] User device: Mobile device such as a smartphone or tablet (e.g., iOS, ANDROID (registered trademark) device)

[0154] Server: Cloud-based computing resources (e.g., Amazon Web Services, Google® Cloud Platform)

[0155] Generative AI model: a generative AI algorithm (e.g., GPT-4 (registered trademark))

[0156] External API: Weather API, Google Maps API, Yelp API, Parkopedia API

[0157] Facial recognition algorithm: TENSORFLOW(R) Lite, Apple Core ML

[0158] Explanation of program processing

[0159] User inputs travel destination and dates

[0160] The user launches the smart tour guide app and enters their travel destination and itinerary in the form displayed on the UI. For example, they enter "Tokyo" and "May 1st to 3rd." The device then sends this information in JSON format to the server.

[0161] Server-based schedule generation

[0162] The server generates a prompt to call the generative AI model based on the received travel destination and itinerary information. For example, the prompt might look like this:

[0163] Travel Planner

[0164] Travel destination: Tokyo

[0165] Dates: May 1st to 3rd

[0166] The server inputs this prompt into a generative AI model and generates multiple optimal schedules. For example, the generated schedules are as follows:

[0167] Day 1: Visit Sensoji Temple in the morning, Ueno Zoo in the afternoon

[0168] Day 2: Visit Sensoji Temple in the morning and Ueno Zoo in the afternoon

[0169] The generated schedule is sent from the server in JSON format to the device and displayed on the device's UI. The user can then select the most suitable schedule from multiple options.

[0170] Get real-time information and schedule changes while traveling

[0171] The device periodically collects weather and traffic information through APIs and sends that information to the server. For example, weather information is obtained from the Weather API, and traffic information is obtained from the Google Maps API. The server analyzes the real-time information and generates a new schedule if it affects the current schedule. If a change is determined to be necessary, the new schedule is sent to the device and the user is notified.

[0172] User health management

[0173] The device monitors the user's face using a built-in camera. For example, it uses a camera sensor and a facial recognition algorithm (e.g., TensorFlow Lite). It estimates the user's physical condition from the user's complexion, blinking frequency, and facial expression, and displays an alert if an abnormality is detected. For example, it displays a message such as, "Are you feeling sleepy? There is a rest area nearby."

[0174] Check parking availability

[0175] The user uses the app's "Parking Lot Search" function to input their current location and destination information. The device sends this information to the server, which then uses a parking API (e.g., Parkopedia API) to obtain information about available parking lots in the area. Based on this information, the server selects the most suitable parking lot and sends it back to the device.

[0176] Specific examples

[0177] Suppose a user launches a travel app and enters "Tokyo" and "May 1st to 3rd." The device sends this information in JSON format to the server, which then uses a generative AI model to create multiple schedules. For example, a schedule might be generated in which the user visits tourist spot A on the first day, tourist spot B and restaurant C on the second day, and so on. This schedule is then sent to the device and displayed to the user.

[0178] If it starts raining while the user is traveling to tourist spot A, the device automatically updates the weather information and reports it to the server. The server generates a new schedule that changes outdoor activities to indoor sightseeing and sends it to the device. The user can then review and select the new schedule.

[0179] Furthermore, if the user begins to feel drowsy while driving, the device's camera will detect frequent blinking and display a warning. This warning will allow the user to take a rest at a nearby rest spot. When searching for parking, if the user searches for a parking lot near Tokyo Station, the server will find a suitable parking lot and send information about its availability to the device. The user can then park in the most optimal location based on this information.

[0180] In this way, the present invention allows users to enjoy traveling efficiently and comfortably.

[0181] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0182] Step 1:

[0183] The user enters the travel destination and itinerary. The user launches the smart tour guide app and enters the travel destination (e.g., Tokyo) and travel dates (e.g., May 1st to 3rd) in the form displayed in the UI. The input data is temporarily saved in the device's memory.

[0184] Input: Travel destination and itinerary information

[0185] Output: Travel destination and itinerary data in memory

[0186] Step 2:

[0187] The device sends the input information to the server, which converts the input data of the travel destination and itinerary into JSON format and sends it to the server using the HTTPS protocol, using Wi-Fi or mobile data connection.

[0188] Input: Travel destination and itinerary data in memory

[0189] Output: Data sent to the server in JSON format

[0190] Step 3:

[0191] The server generates the schedule. Based on the received travel destination and schedule information, the server generates a prompt to call the generative AI model. For example, the prompt might say, "The travel destination is Tokyo, and the dates are three days from May 1st to 3rd. Please generate a schedule including tourist spots, restaurants, and events." This prompt is input into the generative AI model to generate the schedule.

[0192] Input: Received travel destination and itinerary information

[0193] Output: Multiple schedules generated by the generative AI model

[0194] Step 4:

[0195] The server sends the generated schedule to the terminal. The generated schedules are converted into JSON format and sent to the terminal using the HTTPS protocol.

[0196] Input: Multiple generated schedules

[0197] Output: Schedule data sent to the device in JSON format

[0198] Step 5:

[0199] The device displays the schedule to the user. The received schedule is displayed on the UI, and the user can select the most appropriate one from multiple options. The schedule is displayed in a visually easy-to-understand list or calendar format.

[0200] Input: Received schedule data

[0201] Output: The schedule displayed on the user interface

[0202] Step 6:

[0203] Obtaining real-time information and changing schedules during travel. The device periodically obtains weather and traffic information via API (for example, using the Weather API or Google Maps API). The collected information is sent to the server, which analyzes the real-time information. If the current schedule is affected, a new schedule is generated and sent to the device. The user can review the new schedule and approve the changes.

[0204] Input: Weather and traffic information

[0205] Output: New schedule as needed

[0206] Step 7:

[0207] User health management. The device uses a built-in camera to recognize the user's face and monitor their physical condition. For example, it estimates their physical condition from factors such as facial color, blinking frequency, and facial expressions. If an abnormality is detected, it displays a warning and suggests rest areas and preventive measures.

[0208] Input: Face data captured from the camera

[0209] Output: Warning messages and suggestions for the user

[0210] Step 8:

[0211] Checking parking availability. The user uses the app's "parking search" function to input their current location and destination information. The device sends this information to the server, which then uses the parking API to obtain information on available parking in the surrounding area. The server then returns the optimal parking information to the device and makes a suggestion to the user.

[0212] Input: Current location and destination information

[0213] Output: Appropriate parking information

[0214] (Application example 1)

[0215] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0216] For food delivery services, improving delivery efficiency and managing the safety of delivery personnel are important. However, challenges include dynamically changing delivery schedules based on real-time weather and traffic information, managing the health of delivery personnel, and checking the availability of parking spaces around the destination. These factors can lead to delivery delays and a decline in service quality. Furthermore, if delivery personnel's fatigue and poor health are not adequately managed, the risk of accidents and delivery errors increases.

[0217] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0218] In this invention, the server includes a means for a user to input a delivery destination and delivery date and time, a means for transmitting the input information to the server, a means for the server to generate multiple optimal delivery schedules using a generative AI model based on the input information, and a means for transmitting the generated schedules to a terminal and displaying them, thereby enabling efficient and flexible generation of delivery schedules.

[0219] The server also includes a means for periodically acquiring weather and traffic information, a means for evaluating the impact of the acquired information on the current schedule, and a means for generating a new schedule and proposing it to the terminal, thereby enabling schedule adjustments in response to real-time changes in the situation.

[0220] The terminal also has a means for using a camera to recognize the face and monitor the physical condition of the delivery person, a means for detecting abnormalities and displaying a warning based on the monitoring results, and a means for suggesting rest areas and preventive measures, which will enable the health management and safety of delivery people to be ensured.

[0221] Furthermore, the server acquires the availability of parking spaces around the destination in real time, and includes a means for suggesting the most suitable parking space to the terminal based on the availability, thereby enabling the delivery person to find a parking space efficiently and quickly.

[0222] "User" refers to the person who uses or operates the delivery service.

[0223] "Delivery Destination" means the destination to which goods or items are to be delivered.

[0224] "Delivery Date and Time" means the date and time when the goods or items are to be delivered to their destination.

[0225] "Server" refers to a central computer system that processes data and collects and processes various types of information.

[0226] A "generative AI model" refers to an algorithm that uses artificial intelligence to suggest optimal schedules and solutions.

[0227] "Terminal" refers to a device or equipment used by a user, such as a smartphone or tablet.

[0228] A "schedule" is a timetable showing the order and plan of deliveries.

[0229] "Weather Information" means data regarding current and future weather conditions.

[0230] "Traffic information" refers to the latest data on road traffic, such as road congestion and traffic regulations.

[0231] "Facial recognition" refers to a technology that uses a camera to identify a person's face and then processes that information.

[0232] Monitoring "health status" refers to the process of monitoring the health status of delivery personnel in real time and detecting any abnormalities.

[0233] "Warning" refers to a notification that alerts the user when the system detects an abnormality.

[0234] "Rest Area" means a place where a delivery person temporarily stops to take rest.

[0235] "Preventive measures" refer to measures taken to prevent delivery personnel from becoming ill or having accidents.

[0236] "Parking Space" means a location available for a Delivery Person to temporarily park their vehicle.

[0237] "Availability" refers to information about whether a parking space is available.

[0238] "Real-time" means that the latest information is obtained and updated almost instantly.

[0239] The present invention relates to a system for improving delivery efficiency and managing the safety of delivery personnel in a food delivery service. Detailed embodiments of the present invention will be described below.

[0240] System Program Overview

[0241] This system includes a means for the user to input the delivery destination and delivery date and time, a means for transmitting the input information to a server, a means for generating an optimal delivery schedule using a generative AI model, and a means for transmitting and displaying the generated schedule to a terminal. The server also has a means for periodically obtaining weather and traffic information, evaluating the impact on the current schedule, generating a new schedule, and proposing it to the terminal. The terminal also has a means for recognizing the delivery person's face and monitoring their physical condition, detecting abnormalities, displaying a warning, and suggesting rest areas and preventive measures. It also includes a means for obtaining real-time information on available parking spaces around the destination and proposing the optimal parking space to the terminal.

[0242] Program processing and the hardware and software used

[0243] 1. User Input and Information Submission

[0244] Hardware: Smartphone or tablet

[0245] Software: User Interface Application

[0246] Processing: The user inputs the delivery address and delivery date and time. The input information is sent from the terminal to the server.

[0247] 2. Schedule generation using generative AI models

[0248] Hardware: Server

[0249] Software: Generative AI models (e.g., OpenAI® GPT-3®)

[0250] Processing: The server calls the generative AI model based on the received delivery destination and date and time information. The prompt "Generate the optimal delivery schedule for arriving at Tokyo Station at 12:00 on May 1, 2023" is entered, and the AI ​​model generates the optimal schedule. The generated schedule is sent to the device and displayed to the user.

[0251] 3. Real-time information acquisition and schedule updates

[0252] Hardware: Server

[0253] Software: Weather information API, traffic information API

[0254] Processing: The server periodically obtains weather and traffic information using an API. Based on the obtained information, it evaluates the impact on the current schedule and generates a new schedule. An example of a prompt sentence in this case is "Please update the delivery route taking into account the current weather (rain) and traffic conditions (congestion)." The new schedule is then resent to the device.

[0255] 4. Managing the health of delivery personnel

[0256] Hardware: Smartphone or tablet with a camera

[0257] Software: OpenCV (open source computer vision library)

[0258] Processing: The device uses a camera to recognize the delivery person's face and monitor their physical condition. If an abnormality is detected, it displays a warning and suggests resting places and preventive measures. For example, it displays a notification saying, "Frequent blinking has been detected. For safety reasons, we suggest you take a break at a nearby rest facility."

[0259] 5. Check availability of parking spaces

[0260] Hardware: Server

[0261] Software: Parking Information API

[0262] Processing: The server obtains the availability of parking spaces near the delivery destination in real time and suggests the most suitable parking location to the terminal. An example of a prompt sentence in this case is "Please search for available parking spaces around Tokyo Station in real time and suggest the most suitable parking location."

[0263] Specific examples

[0264] When a user uses their smartphone to enter "Tokyo Station" and "May 1, 2023, 12:00," the device sends this information to the server. The server uses a generative AI model to generate an optimal delivery schedule and sends this schedule to the device. If the weather worsens and traffic congestion occurs during the trip, the server obtains new real-time information, updates the schedule, and resends it to the device.

[0265] Furthermore, if a delivery person feels unwell during a delivery, the device's camera will use facial recognition to detect the abnormality and display a warning. This warning allows the delivery person to take a break safely. If the delivery person needs to find a parking space around Tokyo Station, the server will obtain the most suitable parking space in real time and send that information to the device, helping the delivery person park efficiently.

[0266] In this way, the present invention improves the delivery efficiency and safety of delivery personnel of food delivery services.

[0267] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0268] Step 1:

[0269] The user enters the delivery address and delivery date and time. The user uses a smartphone or tablet to enter the delivery address and delivery date and time into the application. This input information is sent from the device to the server. Input: Delivery address and delivery date and time. Output: Delivery information sent to the server.

[0270] Step 2:

[0271] The server uses the generative AI model to generate an optimal delivery schedule. Based on the delivery destination and date and time information received, the server sends a prompt to the generative AI model to generate an optimal schedule. An example of a specific prompt is, "Please generate an optimal delivery schedule that arrives at Tokyo Station at 12:00 on May 1, 2023." Input: Delivery information and prompt. Output: Optimal delivery schedule.

[0272] Step 3:

[0273] The generated schedule is sent to the terminal and displayed to the user. The server sends the generated schedule to the terminal, and the terminal displays it to the user. Input: Generated schedule. Output: Schedule displayed on the terminal.

[0274] Step 4:

[0275] The server periodically obtains weather and traffic information. The server obtains weather and traffic information in real time using the weather information API and traffic information API. Input: Requests to the weather information API and traffic information API. Output: Real-time weather and traffic information.

[0276] Step 5:

[0277] The impact on the current schedule is evaluated based on the acquired information. The server analyzes the acquired weather and traffic information and evaluates the impact on the current schedule. For example, if it starts to rain or traffic congestion occurs, it evaluates whether the current schedule will be delayed. Input: Real-time weather information and traffic information. Output: Result of the impact evaluation on the schedule.

[0278] Step 6:

[0279] If a new schedule needs to be generated and proposed to the device, the server again sends a prompt to the generation AI model to generate a new schedule. An example of a specific prompt is, "Please update the delivery route taking into account the current weather (rain) and traffic conditions (traffic jam)." The new schedule generated is sent to the device and proposed to the user. Input: Schedule impact assessment results and prompt. Output: New schedule.

[0280] Step 7:

[0281] The device uses a camera to recognize the delivery person's face and monitor their physical condition. The device uses a camera to recognize the delivery person's face and detects frequent blinking and signs of fatigue. Input: Camera image. Output: Results of facial recognition and physical condition monitoring.

[0282] Step 8:

[0283] If an abnormality is detected, the device will display a warning and suggest rest areas or preventive measures. For example, it may display a notification saying, "Frequent blinking has been detected. For safety reasons, we suggest you take a rest at a nearby rest area." Input: Monitoring results. Output: Warning display and suggested rest areas.

[0284] Step 9:

[0285] The server obtains the availability of parking spaces around the destination in real time and suggests the most suitable parking space to the terminal. The server uses the parking information API to obtain the availability of parking spaces in real time and suggests the most suitable parking space. An example of a specific prompt is "Please search for available parking spaces around Tokyo Station in real time and suggest the most suitable parking space." Input: Request to the parking information API. Output: Availability of parking spaces and suggested parking information.

[0286] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0287] overview

[0288] The smart tour guide app, which incorporates the emotion engine of the present invention, is a system that improves the efficiency of travel planning and provides flexible responses during the trip. This system allows users to input their travel destinations and itineraries, and generates an optimal schedule using a generative AI model based on that information. It also obtains real-time weather and traffic information during the trip, and proposes and changes the schedule depending on the situation. Furthermore, in addition to the terminal's functions of managing the user's health and checking parking availability, it is equipped with an emotion engine that recognizes the user's emotions and makes appropriate proposals and changes based on the user's emotional state.

[0289] A flow where the user inputs the travel destination and dates and generates a schedule

[0290] 1. The user launches the Smart Tour Consulting app on their device and enters their travel destination and dates.

[0291] 2. The terminal sends the entered information to the server.

[0292] 3. The server calls the generative AI model based on the received travel destination and itinerary information, and collects information on tourist attractions, restaurants, events, etc. at the travel destination.

[0293] 4. Based on the information collected by the server, multiple optimal schedules are generated.

[0294] 5. The generated schedule is sent to the terminal and displayed to the user, who can then select the best one from the multiple schedules.

[0295] Get real-time information and schedule changes while traveling

[0296] 1. The device periodically collects weather and traffic information and sends it to the server.

[0297] 2. The server analyzes the real-time information and generates a new schedule if it affects the current schedule.

[0298] 3. If a new schedule is required, the server sends it to the terminal, which notifies the user, who then checks the new schedule and approves the changes.

[0299] User health management

[0300] 1. The device uses a camera to recognize the user's face and monitor their physical condition.

[0301] 2. If an abnormality or drowsiness is detected, the device will display a warning to the user and suggest rest or preventive measures.

[0302] Recognizing and responding to user emotions

[0303] 1. The emotion engine installed in the device analyzes the user's facial expressions, voice, etc. to recognize their emotional state.

[0304] 2. The device suggests an optimal travel schedule based on the user's emotional state.

[0305] 3. If a schedule change is required during the trip, the emotion engine generates a new schedule taking into account the emotional state.

[0306] 4. The device will suggest suitable places to rest and relax based on your emotional state.

[0307] Check parking availability

[0308] 1. When a user searches for a parking space, the device queries the server for nearby parking information based on the current location and destination.

[0309] 2. The server collects real-time information on available parking spaces in the surrounding area and selects the most suitable parking space.

[0310] 3. The selected parking lot information is sent to the terminal and suggested to the user, who then parks the car in the designated parking lot.

[0311] Specific examples

[0312] Suppose a user launches a travel app and enters "Tokyo" and "May 1st to 3rd." The device sends this information to a server, which uses a generative AI model to create multiple schedules. For example, a schedule might be generated in which the user visits tourist spot A on day 1, tourist spot B and restaurant C on day 2, and so on. This schedule is then sent to the device and displayed to the user.

[0313] If it starts to rain while the user is traveling to tourist spot A, the device automatically updates the weather information and reports it to the server. The server takes the rain into account and generates a new schedule that changes the itinerary to indoor sightseeing and sends it to the device. The user can then review and select the new schedule.

[0314] Furthermore, if a user begins to feel drowsy while driving, the device's camera will detect frequent blinking and display a warning, allowing the user to take a break at a nearby rest spot.

[0315] When a user searches for a parking space, the server collects information on available parking spaces in real time and selects the most suitable parking space. The device then suggests an appropriate parking space, and the user parks accordingly.

[0316] Furthermore, if the user feels stressed during their trip, the emotion engine will analyze the user's facial expressions and voice, and if it determines that relaxation is necessary, the device will suggest a spot where they can relax. For example, if the user feels tired, the device will suggest a nearby cafe.

[0317] In this way, the present invention allows users to enjoy traveling efficiently and comfortably, and also enables flexible responses according to the user's emotional state and physical condition, providing a safer and more satisfying travel experience.

[0318] The processing flow will be explained below.

[0319] A flow where the user inputs the travel destination and dates and generates a schedule

[0320] Step 1:

[0321] The user launches the Smart Tour Consulting app on their device.

[0322] Step 2:

[0323] The user inputs the travel destination and dates.

[0324] Step 3:

[0325] The terminal transmits the entered travel destination and itinerary information to the server.

[0326] Step 4:

[0327] The server receives the travel destination and itinerary information.

[0328] Step 5:

[0329] The server calls the generated AI model and collects information on tourist attractions, restaurants, events, etc. at the travel destination.

[0330] Step 6:

[0331] Based on the information collected by the server, multiple optimal schedules are generated.

[0332] Step 7:

[0333] The server transmits the generated schedule to the terminal.

[0334] Step 8:

[0335] The terminal displays the generated schedule to the user.

[0336] Step 9:

[0337] The user selects the best schedule from among multiple schedules.

[0338] Get real-time information and schedule changes while traveling

[0339] Step 1:

[0340] The device periodically collects weather and traffic information.

[0341] Step 2:

[0342] The device sends the collected weather and traffic information to the server.

[0343] Step 3:

[0344] The server receives and analyzes the real-time information.

[0345] Step 4:

[0346] The server evaluates whether the current schedule is affected.

[0347] Step 5:

[0348] The server generates new schedules as needed.

[0349] Step 6:

[0350] The server sends the new schedule to the terminal.

[0351] Step 7:

[0352] The terminal notifies the user of the new schedule.

[0353] Step 8:

[0354] The user reviews and approves the proposed schedule changes.

[0355] User health management and warning function

[0356] Step 1:

[0357] The device uses a camera to recognize the user's face and monitor their physical condition.

[0358] Step 2:

[0359] The device uses facial recognition technology to detect signs of drowsiness in the user (e.g., frequent blinking, rubbing of eyes, etc.).

[0360] Step 3:

[0361] If the device detects an abnormality, it displays a warning to the user.

[0362] Step 4:

[0363] The device will suggest appropriate rest areas and preventive measures to the user.

[0364] Recognizing and responding to user emotions

[0365] Step 1:

[0366] The emotion engine installed in the device analyzes the user's facial expressions and voice to recognize their emotional state.

[0367] Step 2:

[0368] The terminal proposes an optimal travel schedule based on the user's emotional state.

[0369] Step 3:

[0370] If a schedule change is required during the trip, the emotion engine will generate a new schedule taking into account the emotional state.

[0371] Step 4:

[0372] The device will suggest suitable places to rest and relax based on your emotional state.

[0373] Check parking availability

[0374] Step 1:

[0375] A user searches for parking near a destination.

[0376] Step 2:

[0377] The terminal queries the server for information about nearby parking lots based on the user's current location and destination.

[0378] Step 3:

[0379] The server collects real-time information on available parking spaces in the surrounding area.

[0380] Step 4:

[0381] The server selects the best parking space.

[0382] Step 5:

[0383] The server transmits the selected parking lot information to the terminal.

[0384] Step 6:

[0385] The terminal proposes optimal parking information to the user.

[0386] Through these steps, the present invention provides users with more efficient travel planning, flexible travel arrangements, health management, and parking management. Furthermore, by using an emotion engine, personalized responses based on the user's emotional state become possible, resulting in a more satisfying travel experience.

[0387] Example 2

[0388] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0389] Conventional travel planning applications have issues such as ineffective automatic schedule generation based on user input, difficulty in flexible schedule changes that take real-time information into account, a lack of suggestions that take into account the user's physical and emotional state, and an inability to effectively provide information on available parking spaces. These issues not only make the travel experience inefficient and inconvenient, but also lead to a lack of safety.

[0390] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for a user to input a travel destination and itinerary, a means for transmitting the input information to the server, a means for the server to generate multiple optimal travel schedules using a generative AI model based on the input information, and a means for transmitting the generated schedules to a terminal and displaying them. This allows the user to enjoy an efficient and comfortable travel experience.

[0391] Furthermore, this invention includes a means for the server to periodically acquire weather and traffic information, a means for evaluating the impact on the current schedule based on the acquired information, a means for generating a new schedule and proposing it to the terminal, a means for monitoring the user's face and physical condition using a camera attached to the terminal, a means for detecting abnormalities based on the monitoring results and displaying a warning, and a means for proposing rest areas and preventive measures, thereby enabling the user's physical condition management and quick schedule adjustments.

[0392] The device further includes a means for analyzing the user's emotional state using an emotion analysis engine built into the device, and a means for proposing optimal travel schedules and rest spots based on the analysis results, thereby enabling optimal suggestions to be made according to the user's emotional state.

[0393] In addition, the system includes a means for users to query the server for information to search for available parking spaces, a means for the server to collect information on available parking spaces in the vicinity in real time and select the most suitable parking space, and a means for transmitting and displaying the selected parking space information to the terminal, thereby enabling users to find parking spaces efficiently.

[0394] A "user" is a person who uses the system to create and execute a travel plan.

[0395] "Destination" refers to a geographic location or city that a user plans to visit.

[0396] "Itinerary" refers to the specific schedule of the travel period, including the start and end dates of the trip.

[0397] "Terminal" refers to an electronic device that a user uses as an interface, such as a smartphone, tablet, or PC.

[0398] "Server" refers to a central processing unit that processes data provided by users and generates travel schedules.

[0399] A "generative artificial intelligence model" refers to an algorithm or method that automatically generates an optimal travel schedule based on input data.

[0400] A "schedule" refers to a specific travel plan including the places to visit, the duration of stay, the means of transportation, etc.

[0401] "Weather Information" refers to data about current and forecast weather and climate.

[0402] "Traffic information" refers to data on current traffic conditions and road congestion.

[0403] "Facial recognition" refers to technology that uses a camera to identify a user's face and detect certain features of it.

[0404] "Physical condition" refers to data indicating the physical condition of the user, such as their health and fatigue level.

[0405] An "emotion analysis engine" refers to software or a system that analyzes a user's facial expressions, voice, etc. to recognize emotions.

[0406] A "rest spot" refers to a place suitable for a user to take a rest.

[0407] "Parking lot" means a designated area for parking vehicles.

[0408] "Availability" refers to the current status of available parking spaces.

[0409] This invention is a system that generates an optimal travel schedule based on user input of travel destinations and dates. The system includes a user terminal, a server, a generative AI model, a camera, and an emotion analysis engine.

[0410] The user operates a device to input the travel destination and itinerary. For example, the device can be a smartphone, tablet, or PC. The information entered by the user is sent from the device to the server as an HTTP POST request. This request includes the travel destination (e.g., "Tokyo") and the travel itinerary (e.g., "May 1st to 3rd").

[0411] The server calls a generative AI model based on the received information to generate an optimal travel schedule. For example, OpenAI's GPT-3 is used as the generative AI model. Here, the following prompt sentence is input to the AI ​​model:

[0412] "A user is traveling to Tokyo from May 1st to 3rd. Please create a three-day travel itinerary including recommended tourist spots and restaurants."

[0413] The generative AI model generates multiple schedules based on this, and the server sends these to the device.

[0414] The terminal presents the received schedule to the user, who can then select the most suitable schedule. The selected schedule is then used for management and display during the trip.

[0415] During a trip, the device periodically collects weather and traffic information and sends it to the server. The server analyzes this information and, if it affects the current schedule, generates a new schedule and sends it to the device. The user receives a notification, checks the new schedule, and approves or changes it as necessary.

[0416] The device also has a built-in camera that can be used to recognize the user's face and monitor their physical condition. If abnormalities or drowsiness (such as frequent blinking) are detected, the device will display a warning and suggest rest areas or preventive measures.

[0417] As an additional function, the device is equipped with an emotion analysis engine. This engine analyzes the user's facial expressions and voice to recognize their emotional state. Based on the emotional state, the device will suggest optimal travel schedules and rest spots. For example, if the user feels stressed, the device will suggest a place where they can relax (such as a cafe).

[0418] Furthermore, when a user searches for a parking space while traveling by car, the device inquires about available parking spaces with the server. The server collects information about nearby parking spaces in real time, selects the most suitable parking space, and sends the information to the device. The user can then park their car in the parking space provided.

[0419] In this way, the system of the present invention provides all kinds of support for users to enjoy an efficient and comfortable trip. By automatically generating schedules using generative AI models, responding in real time based on weather and traffic information, and monitoring the user's physical and emotional state and making optimal suggestions accordingly, users can have a safe and satisfying travel experience.

[0420] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0421] Step 1:

[0422] The user inputs the travel destination and dates.

[0423] Users input their travel destination and itinerary on devices such as smartphones, tablets, and PCs (input: travel destination and itinerary). This information becomes the basis for the user's travel plans. For example, they might input "Tokyo" and "May 1st to 3rd." The input information is processed internally on the device and temporarily saved (output: input information).

[0424] Step 2:

[0425] The terminal transmits the input information to the server.

[0426] The terminal sends the travel destination and itinerary information entered by the user to the server via an HTTP POST request (input: input information). The server receives the request and analyzes the contents (output: received data).

[0427] Step 3:

[0428] The server calls the generative AI model and generates a travel schedule.

[0429] The server calls a generative AI model (e.g., OpenAI's GPT-3) based on the received travel destinations and itinerary (input: received data). Here, the prompt is set as follows: "The user will travel to Tokyo from May 1st to 3rd. Please create a three-day travel schedule including recommended tourist spots and restaurants." The generative AI model outputs multiple travel schedules in response to the prompt (output: generated schedules).

[0430] Step 4:

[0431] The server transmits the generated schedule to the terminal.

[0432] The server sends the generated schedules to the terminal (input: generated schedules). The terminal receives them and saves them as schedule data (output: saved schedules).

[0433] Step 5:

[0434] The terminal displays the generated schedule to the user.

[0435] The terminal displays the received schedules on the screen to the user (input: saved schedules), and the user can view them and select the most suitable schedule (output: selected schedule).

[0436] Step 6:

[0437] The terminal periodically collects weather and traffic information and sends it to the server.

[0438] During the trip, the device periodically collects weather information (weather information service) and traffic information (map API) (input: external information). The collected information is sent to the server and analyzed there (output: analysis results).

[0439] Step 7:

[0440] The server will suggest schedule changes based on the information.

[0441] The server evaluates the impact of collected and analyzed weather and traffic information on the current schedule (input: analysis results). If necessary, it generates a new schedule and proposes it to the terminal (output: new schedule proposal).

[0442] Step 8:

[0443] The terminal notifies the user of the new schedule.

[0444] The terminal receives the new schedule sent from the server and notifies the user (input: new schedule proposal). The user can then review the new schedule and approve or change it (output: user decision).

[0445] Step 9:

[0446] The device uses a camera to monitor the user's physical condition.

[0447] The device's built-in camera recognizes the user's face and monitors their physical condition (e.g., frequent blinking) (input: facial recognition data). If an abnormality is detected, a warning is displayed and suggestions for rest areas or preventive measures are made (output: warning and suggestions).

[0448] Step 10:

[0449] The terminal uses an emotion analysis engine to analyze the user's emotional state.

[0450] The device's built-in emotion analysis engine analyzes the user's facial expressions and voice to recognize their emotional state (input: emotion analysis data). Based on the recognition results, it suggests optimal travel schedules and rest spots (output: emotion-based suggestions).

[0451] Step 11:

[0452] The server collects parking availability information in real time and sends it to the terminal.

[0453] When a user searches for a parking space, the device sends the current location and destination information to the server (input: location information). The server collects information on available parking spaces in real time, selects the most suitable parking space, and sends it to the device (output: parking suggestion).

[0454] Step 12:

[0455] The terminal presents the parking lot information to the user.

[0456] The terminal receives the parking information sent from the server and presents it to the user (input: parking suggestion). The user can then park accordingly (output: parking completed).

[0457] (Application example 2)

[0458] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0459] Conventional travel support systems have difficulty proposing optimal schedules that take into account the user's emotions and physical condition, and responding flexibly based on real-time information during the trip. In particular, there is a need for appropriate responses to sudden changes in physical condition and emotional fluctuations during travel. Furthermore, to improve the customer experience in stores, flexible suggestions based on the customer's emotional and physical conditions are necessary, but current systems are inadequate in this regard. Therefore, there is a need for a system that can provide users with a comfortable and safe travel experience while also improving customer satisfaction in stores.

[0460] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input a travel destination and itinerary; a means for transmitting the input information to the server; a means for the server to generate multiple optimal travel schedules using a generative AI model based on the input information; a means for transmitting the generated schedules to a terminal and displaying them; a means for analyzing the emotional state from the user's facial expressions and voice using an emotion recognition engine; and a means for adjusting the schedule and making appropriate suggestions based on the emotional state. This makes it possible to monitor the user's physical condition and emotional state during the trip in real time and always provide optimal schedules and suggestions.

[0461] A "travel destination" is a geographic location that a user wishes to visit.

[0462] "Itinerary" refers to a time plan such as a travel period or visiting schedule.

[0463] A "server" is a computer system that processes data and provides information over a network.

[0464] A "generative AI model" is an artificial intelligence algorithm or program that generates optimal schedules and proposals based on input information.

[0465] A "terminal" is a device used by a user, such as a smartphone or smart glasses.

[0466] An "emotion recognition engine" is software or hardware that analyzes a user's emotional state from their facial expressions and voice.

[0467] "Facial expressions" are used to read emotions from changes and movements of the user's face.

[0468] "Voice" refers to reading emotions from the tone and patterns of the user's voice.

[0469] "Physical condition" indicates the physical condition of the user, such as their health condition and fatigue level.

[0470] A "schedule" is a plan or timeline for a trip or activity.

[0471] "Suggestions" are information or advice that suggest optimal actions or options for the user.

[0472] "Traffic information" is data showing real-time conditions of traffic volume and means of transportation.

[0473] "Weather information" is data that indicates the weather conditions at your travel destination or current location.

[0474] "Facial recognition" is a technology that uses a camera to identify a user's face and analyze its features.

[0475] A "warning" is a notification to the user of a sign of danger or abnormality.

[0476] A "resting place" is a place where a user can rest and relax during their trip.

[0477] A "parking lot" is a space or facility for parking automobiles.

[0478] "Crowding situation" refers to the degree of crowding or congestion in a particular place or facility.

[0479] The system of the present invention begins when the user inputs their travel destination and itinerary. When the user inputs the destination and itinerary using a device such as a smartphone or smart glasses, the device sends this information to a server. The server uses a generative AI model based on the received information to generate multiple optimal travel itineraries. These schedules are sent to the device and displayed to the user. The user can then select the most appropriate one from the presented schedules.

[0480] The system periodically obtains weather and traffic information and evaluates its impact on the current schedule. Based on the results of this evaluation, it generates a new schedule and proposes it to the device. The device then notifies the user of the proposal, allowing them to review and select the new schedule.

[0481] The device also uses a camera to recognize the user's face and monitor their physical condition. If an abnormality is detected, the device will display a warning to the user and suggest appropriate rest areas and preventive measures. Furthermore, an emotion recognition engine can analyze the user's emotional state from their facial expressions and voice, adjusting their schedule accordingly and making appropriate suggestions.

[0482] The system also obtains real-time information on congestion and parking availability at places the user visits during their trip and makes appropriate suggestions based on that information. For example, if a store is crowded, it can suggest a different time or store. Parking availability is also checked in real time, and the system suggests the best parking spot for the user.

[0483] For example, a user might enter, "I'm looking for new sneakers and would like to know about the latest promotions." The AI ​​model will then provide optimal suggestions for the latest sneakers and promotions. It will also suggest alternative options if the store is crowded, and provide real-time parking information.

[0484] Examples of prompts include:

[0485] "A customer is looking for sneakers and wants to know about the latest promotions. Generate the best suggestions."

[0486] This system allows users to adjust their schedules optimally based on real-time information while traveling, and also improves the customer experience in stores, enabling flexible responses based on physical and emotional states, allowing for safe and comfortable travel and shopping experiences.

[0487] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0488] Step 1: The user enters the travel destination and dates.

[0489] The user launches an application that can be used on a smartphone or smart glasses and inputs their travel destination and itinerary. This information is collected and saved on the device. (Input: Travel destination, itinerary) (Output: Data on the device)

[0490] Step 2: The device sends the input information to the server.

[0491] The terminal sends the entered travel destination and itinerary information to the server. (Input: Data on the terminal) (Output: Data sent to the server)

[0492] Step 3: The server generates a schedule using the generative AI model.

[0493] The server uses the generative AI model to generate multiple optimal travel schedules based on the received travel destination and itinerary information. It also takes into account information collected from the internet about tourist attractions, facilities, events, and other information about the travel destination. (Input: Travel destination, itinerary) (Output: Optimal travel schedule)

[0494] Step 4: The server sends the generation schedule to the terminal.

[0495] The server sends the generated schedules to the terminal and displays them on the terminal. (Input: Optimal travel schedule) (Output: Data sent to the terminal)

[0496] Step 5: The terminal displays the schedule to the user.

[0497] The terminal displays the received schedule to the user, who then selects the optimal schedule from multiple options. (Input: Optimal travel schedule) (Output: Schedule displayed to user)

[0498] Step 6: The device periodically collects weather and traffic information.

[0499] The device periodically collects weather and traffic information and sends it to the server. (Input: Weather information, traffic information) (Output: Data sent to the server)

[0500] Step 7: The server adjusts the schedule based on the real-time information.

[0501] The server analyzes the weather and traffic information sent from the device, and if it affects the current schedule, it generates a new schedule and sends it to the device. (Input: Weather information, traffic information) (Output: Adjusted schedule)

[0502] Step 8: The terminal notifies the user of the adjusted schedule.

[0503] The terminal notifies the user of the new schedule, and the user confirms and approves the schedule changes. (Input: Adjusted schedule) (Output: Notification to user)

[0504] Step 9: The device uses the camera to perform facial recognition and health monitoring.

[0505] The device uses a camera to recognize the user's face and uses AI to monitor their physical condition. If an abnormality is detected, a warning is displayed. (Input: User's face image) (Output: Physical condition evaluation, warning display)

[0506] Step 10: The device uses an emotion recognition engine to analyze the user's facial expressions and voice.

[0507] The device analyzes the user's facial expressions and voice using an emotion recognition engine to infer their emotional state. It then adjusts the schedule based on their emotional state and makes appropriate suggestions. (Input: User's facial expression data, voice data) (Output: Emotional state assessment, adjusted schedule)

[0508] Step 11: The device collects congestion information and parking availability.

[0509] The device collects real-time information about congestion at the destination and parking availability, and sends it to the server to make optimal recommendations. (Input: congestion information, parking availability) (Output: data sent to the server)

[0510] Step 12: The server generates a proposal based on the new information.

[0511] The server analyzes the information sent in real time and generates new suggestions to help users enjoy their trip or shopping. (Input: congestion information, parking information) (Output: new suggestions)

[0512] Step 13: The terminal notifies the user of the new offer.

[0513] The device notifies the user of the new proposal. The user confirms the proposal and selects the next action. (Input: New proposal) (Output: Notification to user)

[0514] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0515] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0516] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0517] [Second embodiment]

[0518] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0519] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0520] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0521] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0522] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0523] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0524] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0525] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0526] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0527] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0528] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0529] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0530] overview

[0531] The smart tour guide app of this invention is a system that improves the efficiency of travel planning and provides flexible support during the trip. This system allows users to input their travel destinations and itineraries, and generates an optimal schedule using a generative AI model based on that information. It also obtains real-time weather and traffic information during the trip, and suggests or changes the schedule depending on the situation. Furthermore, the device also has the function of managing the user's health and checking the availability of parking spaces.

[0532] The user inputs the travel destination and dates

[0533] 1. The user launches the Smart Tour Guide app and enters their travel destination and dates.

[0534] 2. The terminal sends the entered information to the server.

[0535] The server generates the schedule

[0536] 1. The server calls the generative AI model based on the received travel destination and itinerary.

[0537] 2. The server collects information on tourist attractions, restaurants, events, etc. at the travel destination and generates several optimal schedules based on this information.

[0538] 3. The generated schedule is sent to the terminal and displayed to the user, who can then select the best one from the multiple schedules.

[0539] Get real-time information and schedule changes while traveling

[0540] 1. The device periodically collects weather and traffic information and sends it to the server.

[0541] 2. The server analyzes the real-time information and generates a new schedule if it affects the current schedule.

[0542] 3. If a new schedule is required, the server sends it to the terminal, which notifies the user, who then checks the new schedule and approves the changes.

[0543] User health management

[0544] 1. The device uses a camera to recognize the user's face and monitor their physical condition.

[0545] 2. If an abnormality or drowsiness is detected, the device will display a warning to the user and suggest rest or preventive measures.

[0546] Check parking availability

[0547] 1. When a user searches for a parking space, the device queries the server for nearby parking information based on the current location and destination.

[0548] 2. The server collects real-time information on available parking spaces in the surrounding area and selects the most suitable parking space.

[0549] 3. The selected parking lot information is sent to the terminal and suggested to the user, who then parks the car in the designated parking lot.

[0550] Specific examples

[0551] Suppose a user launches a travel app and enters "Tokyo" and "May 1st to 3rd." The device sends this information to a server, which uses a generative AI model to create multiple schedules. For example, a schedule might be generated in which the user visits tourist spot A on day 1, tourist spot B and restaurant C on day 2, and so on. This schedule is then sent to the device and displayed to the user.

[0552] If it starts raining while the user is traveling to tourist spot A, the device automatically updates the weather information and reports it to the server. The server generates a new schedule that changes outdoor activities to indoor sightseeing and sends it to the device. The user can then review and select the new schedule.

[0553] Furthermore, if a user begins to feel drowsy while driving, the device's camera will detect frequent blinking and display a warning, allowing the user to take a break at a nearby rest spot.

[0554] When searching for a parking space, the user searches for a parking space around Tokyo Station, and the server finds a suitable parking space and sends information about its availability to the terminal. Based on this information, the user can park in the most optimal location.

[0555] In this way, the present invention allows users to enjoy traveling efficiently and comfortably.

[0556] The processing flow will be explained below.

[0557] A flow where the user inputs the travel destination and dates and generates a schedule

[0558] Step 1:

[0559] The user launches the Smart Tour Consulting app on their device.

[0560] Step 2:

[0561] The user inputs the travel destination and dates.

[0562] Step 3:

[0563] The terminal transmits the travel destination and itinerary input by the user to the server.

[0564] Step 4:

[0565] The server receives the travel destination and itinerary information.

[0566] Step 5:

[0567] The server calls the generated AI model and collects information on tourist attractions, restaurants, events, etc. at the travel destination.

[0568] Step 6:

[0569] Based on the information collected by the server, multiple optimal schedules are generated.

[0570] Step 7:

[0571] The server transmits the generated schedule to the terminal.

[0572] Step 8:

[0573] The terminal displays the generated schedule to the user.

[0574] Step 9:

[0575] The user selects the best schedule from among multiple schedules.

[0576] Get real-time information and schedule changes while traveling

[0577] Step 1:

[0578] The device periodically collects weather and traffic information.

[0579] Step 2:

[0580] The device sends the collected weather and traffic information to the server.

[0581] Step 3:

[0582] The server receives and analyzes the real-time information.

[0583] Step 4:

[0584] The server evaluates whether the current schedule is affected.

[0585] Step 5:

[0586] The server generates new schedules as needed.

[0587] Step 6:

[0588] The server sends the new schedule to the terminal.

[0589] Step 7:

[0590] The terminal notifies the user of the new schedule.

[0591] Step 8:

[0592] The user reviews and approves the proposed schedule changes.

[0593] User health management and warning function

[0594] Step 1:

[0595] The device uses a camera to recognize the user's face and monitor their physical condition.

[0596] Step 2:

[0597] The device uses facial recognition technology to detect signs of drowsiness in the user (e.g., frequent blinking, rubbing of eyes, etc.).

[0598] Step 3:

[0599] If the device detects an abnormality, it displays a warning to the user.

[0600] Step 4:

[0601] The device will suggest appropriate rest areas and preventive measures to the user.

[0602] Check parking availability

[0603] Step 1:

[0604] A user searches for parking near a destination.

[0605] Step 2:

[0606] The terminal queries the server for information about nearby parking lots based on the user's current location and destination.

[0607] Step 3:

[0608] The server collects real-time information on available parking spaces in the surrounding area.

[0609] Step 4:

[0610] The server selects the best parking space.

[0611] Step 5:

[0612] The server transmits the selected parking lot information to the terminal.

[0613] Step 6:

[0614] The terminal proposes optimal parking information to the user.

[0615] Through the above steps, the present invention provides users with more efficient travel planning, flexible travel arrangements, health management, and parking lot management.

[0616] Example 1

[0617] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0618] In today's world, planning and executing a trip involves many variables, so a flexible and efficient travel planning system is necessary to ensure a comfortable and stress-free trip. Conventional travel planning systems have had difficulty adequately meeting diverse needs, such as optimizing schedules, responding to unexpected changes in circumstances, managing the user's health, and checking parking availability. The present invention aims to solve these problems and provide users with a fulfilling travel experience.

[0619] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0620] In this invention, the server includes a means for a user to input travel destinations and itineraries, a means for transmitting the input information to the server, a means for the server to generate multiple optimal travel schedules using a generative AI model based on the input information, and a means for transmitting the generated schedules to a terminal and displaying them, thereby enabling users to easily create optimal travel plans.

[0621] Furthermore, the server includes means for periodically acquiring weather and traffic information, means for evaluating the impact on the current schedule based on the acquired information, and means for generating a new schedule and proposing it to the terminal, thereby enabling the system to flexibly respond to unexpected changes in circumstances.

[0622] Furthermore, the terminal includes a means for recognizing the user's face and monitoring their physical condition using a camera, a means for detecting abnormalities based on the monitoring results and displaying a warning, and a means for suggesting rest areas and preventive measures, thereby protecting the user's safety and health.

[0623] Furthermore, the system includes a means for the terminal to search for available parking spaces and inquire of the server, a means for the server to select an appropriate parking space based on the current location and destination, and a means for transmitting information about the selected parking space to the terminal and suggesting it to the user, thereby efficiently solving parking problems.

[0624] In addition, the system includes a means for generating a prompt sentence for the server to call the generative AI model, a means for generating a schedule using the generative AI model based on the prompt sentence, and a means for presenting the schedule to the user as multiple options, thereby providing the user with a variety of options and realizing more satisfying travel plans.

[0625] "User" refers to a person who uses the system to provide input to plan a trip.

[0626] "Destination" refers to a geographic location that a user plans to visit.

[0627] "Schedule" refers to the start and end dates of the trip planned by the user, as well as the duration of the trip.

[0628] "Terminal" refers to the digital device that a user uses to enter their travel plans and receive system results.

[0629] "Server" refers to a remote computer system that receives user input, processes it, and sends the results to the terminal.

[0630] "Generative AI Model" refers to an artificial intelligence model used to generate travel itineraries based on input information.

[0631] "Schedule" refers to the plan of the itinerary and activities for each day at the travel destination.

[0632] "Weather information" refers to data about the weather around your travel destination.

[0633] "Traffic information" refers to data regarding traffic conditions around the travel destination.

[0634] "Facial recognition" refers to technology that uses a camera to identify a user's face.

[0635] "Physical condition" refers to information for monitoring the user's health condition and fatigue level.

[0636] "Parking Availability" refers to information about available parking spaces within a specified area.

[0637] "Prompt" refers to a textual instruction used to invoke a generative AI model.

[0638] overview

[0639] The smart tour guide system of the present invention is a system that improves the efficiency of travel planning and provides flexible support during the trip. The system allows users to input their travel destinations and itineraries, and generates an optimal schedule based on that information using a generative AI model on the server. It also obtains weather and traffic information during the trip in real time, and proposes or changes the schedule depending on the situation. Furthermore, the terminal also has the function of managing the user's health and checking the availability of parking spaces.

[0640] Hardware and software used

[0641] The main hardware and software used to implement this system are as follows:

[0642] User device: Mobile device such as smartphone or tablet (e.g. iOS, Android device)

[0643] Server: Cloud-based computing resources (e.g., Amazon Web Services, Google Cloud Platform)

[0644] Generative AI model: a generative AI algorithm (e.g., GPT-4)

[0645] External API: Weather API, Google Maps API, Yelp API, Parkopedia API

[0646] Facial recognition algorithm: TensorFlow Lite, Apple Core ML

[0647] Explanation of program processing

[0648] User inputs travel destination and dates

[0649] The user launches the smart tour guide app and enters their travel destination and itinerary in the form displayed on the UI. For example, they enter "Tokyo" and "May 1st to 3rd." The device then sends this information in JSON format to the server.

[0650] Server-based schedule generation

[0651] The server generates a prompt to call the generative AI model based on the received travel destination and itinerary information. For example, the prompt might look like this:

[0652] Travel Planner

[0653] Travel destination: Tokyo

[0654] Dates: May 1st to 3rd

[0655] The server inputs this prompt into a generative AI model and generates multiple optimal schedules. For example, the generated schedules are as follows:

[0656] Day 1: Visit Sensoji Temple in the morning, Ueno Zoo in the afternoon

[0657] Day 2: Visit Sensoji Temple in the morning and Ueno Zoo in the afternoon

[0658] The generated schedule is sent from the server in JSON format to the device and displayed on the device's UI. The user can then select the most suitable schedule from multiple options.

[0659] Get real-time information and schedule changes while traveling

[0660] The device periodically collects weather and traffic information through APIs and sends that information to the server. For example, weather information is obtained from the Weather API, and traffic information is obtained from the Google Maps API. The server analyzes the real-time information and generates a new schedule if it affects the current schedule. If a change is determined to be necessary, the new schedule is sent to the device and the user is notified.

[0661] User health management

[0662] The device monitors the user's face using a built-in camera. For example, it uses a camera sensor and a facial recognition algorithm (e.g., TensorFlow Lite). It estimates the user's physical condition from the user's complexion, blinking frequency, and facial expression, and displays an alert if an abnormality is detected. For example, it displays a message such as, "Are you feeling sleepy? There is a rest area nearby."

[0663] Check parking availability

[0664] The user uses the app's "Parking Lot Search" function to input their current location and destination information. The device sends this information to the server, which then uses a parking API (e.g., Parkopedia API) to obtain information about available parking lots in the area. Based on this information, the server selects the most suitable parking lot and sends it back to the device.

[0665] Specific examples

[0666] Suppose a user launches a travel app and enters "Tokyo" and "May 1st to 3rd." The device sends this information in JSON format to the server, which then uses a generative AI model to create multiple schedules. For example, a schedule might be generated in which the user visits tourist spot A on the first day, tourist spot B and restaurant C on the second day, and so on. This schedule is then sent to the device and displayed to the user.

[0667] If it starts raining while the user is traveling to tourist spot A, the device automatically updates the weather information and reports it to the server. The server generates a new schedule that changes outdoor activities to indoor sightseeing and sends it to the device. The user can then review and select the new schedule.

[0668] Furthermore, if the user begins to feel drowsy while driving, the device's camera will detect frequent blinking and display a warning. This warning will allow the user to take a rest at a nearby rest spot. When searching for parking, if the user searches for a parking lot near Tokyo Station, the server will find a suitable parking lot and send information about its availability to the device. The user can then park in the most optimal location based on this information.

[0669] In this way, the present invention allows users to enjoy traveling efficiently and comfortably.

[0670] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0671] Step 1:

[0672] The user enters the travel destination and itinerary. The user launches the smart tour guide app and enters the travel destination (e.g., Tokyo) and travel dates (e.g., May 1st to 3rd) in the form displayed in the UI. The input data is temporarily saved in the device's memory.

[0673] Input: Travel destination and itinerary information

[0674] Output: Travel destination and itinerary data in memory

[0675] Step 2:

[0676] The device sends the input information to the server, which converts the input data of the travel destination and itinerary into JSON format and sends it to the server using the HTTPS protocol, using Wi-Fi or mobile data connection.

[0677] Input: Travel destination and itinerary data in memory

[0678] Output: Data sent to the server in JSON format

[0679] Step 3:

[0680] The server generates the schedule. Based on the received travel destination and schedule information, the server generates a prompt to call the generative AI model. For example, the prompt might say, "The travel destination is Tokyo, and the dates are three days from May 1st to 3rd. Please generate a schedule including tourist spots, restaurants, and events." This prompt is input into the generative AI model to generate the schedule.

[0681] Input: Received travel destination and itinerary information

[0682] Output: Multiple schedules generated by the generative AI model

[0683] Step 4:

[0684] The server sends the generated schedule to the terminal. The generated schedules are converted into JSON format and sent to the terminal using the HTTPS protocol.

[0685] Input: Multiple generated schedules

[0686] Output: Schedule data sent to the device in JSON format

[0687] Step 5:

[0688] The device displays the schedule to the user. The received schedule is displayed on the UI, and the user can select the most appropriate one from multiple options. The schedule is displayed in a visually easy-to-understand list or calendar format.

[0689] Input: Received schedule data

[0690] Output: The schedule displayed on the user interface

[0691] Step 6:

[0692] Obtaining real-time information and changing schedules during travel. The device periodically obtains weather and traffic information via API (for example, using the Weather API or Google Maps API). The collected information is sent to the server, which analyzes the real-time information. If the current schedule is affected, a new schedule is generated and sent to the device. The user can review the new schedule and approve the changes.

[0693] Input: Weather and traffic information

[0694] Output: New schedule as needed

[0695] Step 7:

[0696] User health management. The device uses a built-in camera to recognize the user's face and monitor their physical condition. For example, it estimates their physical condition from factors such as facial color, blinking frequency, and facial expressions. If an abnormality is detected, it displays a warning and suggests rest areas and preventive measures.

[0697] Input: Face data captured from the camera

[0698] Output: Warning messages and suggestions for the user

[0699] Step 8:

[0700] Checking parking availability. The user uses the app's "parking search" function to input their current location and destination information. The device sends this information to the server, which then uses the parking API to obtain information on available parking in the surrounding area. The server then returns the optimal parking information to the device and makes a suggestion to the user.

[0701] Input: Current location and destination information

[0702] Output: Appropriate parking information

[0703] (Application example 1)

[0704] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0705] For food delivery services, improving delivery efficiency and managing the safety of delivery personnel are important. However, challenges include dynamically changing delivery schedules based on real-time weather and traffic information, managing the health of delivery personnel, and checking the availability of parking spaces around the destination. These factors can lead to delivery delays and a decline in service quality. Furthermore, if delivery personnel's fatigue and poor health are not adequately managed, the risk of accidents and delivery errors increases.

[0706] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0707] In this invention, the server includes a means for a user to input a delivery destination and delivery date and time, a means for transmitting the input information to the server, a means for the server to generate multiple optimal delivery schedules using a generative AI model based on the input information, and a means for transmitting the generated schedules to a terminal and displaying them, thereby enabling efficient and flexible generation of delivery schedules.

[0708] The server also includes a means for periodically acquiring weather and traffic information, a means for evaluating the impact of the acquired information on the current schedule, and a means for generating a new schedule and proposing it to the terminal, thereby enabling schedule adjustments in response to real-time changes in the situation.

[0709] The terminal also has a means for using a camera to recognize the face and monitor the physical condition of the delivery person, a means for detecting abnormalities and displaying a warning based on the monitoring results, and a means for suggesting rest areas and preventive measures, which will enable the health management and safety of delivery people to be ensured.

[0710] Furthermore, the server acquires the availability of parking spaces around the destination in real time, and includes a means for suggesting the most suitable parking space to the terminal based on the availability, thereby enabling the delivery person to find a parking space efficiently and quickly.

[0711] "User" refers to the person who uses or operates the delivery service.

[0712] "Delivery Destination" means the destination to which goods or items are to be delivered.

[0713] "Delivery Date and Time" means the date and time when the goods or items are to be delivered to their destination.

[0714] "Server" refers to a central computer system that processes data and collects and processes various types of information.

[0715] A "generative AI model" refers to an algorithm that uses artificial intelligence to suggest optimal schedules and solutions.

[0716] "Terminal" refers to a device or equipment used by a user, such as a smartphone or tablet.

[0717] A "schedule" is a timetable showing the order and plan of deliveries.

[0718] "Weather Information" means data regarding current and future weather conditions.

[0719] "Traffic information" refers to the latest data on road traffic, such as road congestion and traffic regulations.

[0720] "Facial recognition" refers to a technology that uses a camera to identify a person's face and then processes that information.

[0721] Monitoring "health status" refers to the process of monitoring the health status of delivery personnel in real time and detecting any abnormalities.

[0722] "Warning" refers to a notification that alerts the user when the system detects an abnormality.

[0723] "Rest Area" means a place where a delivery person temporarily stops to take rest.

[0724] "Preventive measures" refer to measures taken to prevent delivery personnel from becoming ill or having accidents.

[0725] "Parking Space" means a location available for a Delivery Person to temporarily park their vehicle.

[0726] "Availability" refers to information about whether a parking space is available.

[0727] "Real-time" means that the latest information is obtained and updated almost instantly.

[0728] The present invention relates to a system for improving delivery efficiency and managing the safety of delivery personnel in a food delivery service. Detailed embodiments of the present invention will be described below.

[0729] System Program Overview

[0730] This system includes a means for the user to input the delivery destination and delivery date and time, a means for transmitting the input information to a server, a means for generating an optimal delivery schedule using a generative AI model, and a means for transmitting and displaying the generated schedule to a terminal. The server also has a means for periodically obtaining weather and traffic information, evaluating the impact on the current schedule, generating a new schedule, and proposing it to the terminal. The terminal also has a means for recognizing the delivery person's face and monitoring their physical condition, detecting abnormalities, displaying a warning, and suggesting rest areas and preventive measures. It also includes a means for obtaining real-time information on available parking spaces around the destination and proposing the optimal parking space to the terminal.

[0731] Program processing and the hardware and software used

[0732] 1. User Input and Information Submission

[0733] Hardware: Smartphone or tablet

[0734] Software: User Interface Application

[0735] Processing: The user inputs the delivery address and delivery date and time. The input information is sent from the terminal to the server.

[0736] 2. Schedule generation using generative AI models

[0737] Hardware: Server

[0738] Software: Generative AI models (e.g., OpenAI GPT-3)

[0739] Processing: The server calls the generative AI model based on the received delivery destination and date and time information. The prompt "Generate the optimal delivery schedule for arriving at Tokyo Station at 12:00 on May 1, 2023" is entered, and the AI ​​model generates the optimal schedule. The generated schedule is sent to the device and displayed to the user.

[0740] 3. Real-time information acquisition and schedule updates

[0741] Hardware: Server

[0742] Software: Weather information API, traffic information API

[0743] Processing: The server periodically obtains weather and traffic information using an API. Based on the obtained information, it evaluates the impact on the current schedule and generates a new schedule. An example of a prompt sentence in this case is "Please update the delivery route taking into account the current weather (rain) and traffic conditions (congestion)." The new schedule is then resent to the device.

[0744] 4. Managing the health of delivery personnel

[0745] Hardware: Smartphone or tablet with a camera

[0746] Software: OpenCV (open source computer vision library)

[0747] Processing: The device uses a camera to recognize the delivery person's face and monitor their physical condition. If an abnormality is detected, it displays a warning and suggests resting places and preventive measures. For example, it displays a notification saying, "Frequent blinking has been detected. For safety reasons, we suggest you take a break at a nearby rest facility."

[0748] 5. Check availability of parking spaces

[0749] Hardware: Server

[0750] Software: Parking Information API

[0751] Processing: The server obtains the availability of parking spaces near the delivery destination in real time and suggests the most suitable parking location to the terminal. An example of a prompt sentence in this case is "Please search for available parking spaces around Tokyo Station in real time and suggest the most suitable parking location."

[0752] Specific examples

[0753] When a user uses their smartphone to enter "Tokyo Station" and "May 1, 2023, 12:00," the device sends this information to the server. The server uses a generative AI model to generate an optimal delivery schedule and sends this schedule to the device. If the weather worsens and traffic congestion occurs during the trip, the server obtains new real-time information, updates the schedule, and resends it to the device.

[0754] Furthermore, if a delivery person feels unwell during a delivery, the device's camera will use facial recognition to detect the abnormality and display a warning. This warning allows the delivery person to take a break safely. If the delivery person needs to find a parking space around Tokyo Station, the server will obtain the most suitable parking space in real time and send that information to the device, helping the delivery person park efficiently.

[0755] In this way, the present invention improves the delivery efficiency and safety of delivery personnel of food delivery services.

[0756] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0757] Step 1:

[0758] The user enters the delivery address and delivery date and time. The user uses a smartphone or tablet to enter the delivery address and delivery date and time into the application. This input information is sent from the device to the server. Input: Delivery address and delivery date and time. Output: Delivery information sent to the server.

[0759] Step 2:

[0760] The server uses the generative AI model to generate an optimal delivery schedule. Based on the delivery destination and date and time information received, the server sends a prompt to the generative AI model to generate an optimal schedule. An example of a specific prompt is, "Please generate an optimal delivery schedule that arrives at Tokyo Station at 12:00 on May 1, 2023." Input: Delivery information and prompt. Output: Optimal delivery schedule.

[0761] Step 3:

[0762] The generated schedule is sent to the terminal and displayed to the user. The server sends the generated schedule to the terminal, and the terminal displays it to the user. Input: Generated schedule. Output: Schedule displayed on the terminal.

[0763] Step 4:

[0764] The server periodically obtains weather and traffic information. The server obtains weather and traffic information in real time using the weather information API and traffic information API. Input: Requests to the weather information API and traffic information API. Output: Real-time weather and traffic information.

[0765] Step 5:

[0766] The impact on the current schedule is evaluated based on the acquired information. The server analyzes the acquired weather and traffic information and evaluates the impact on the current schedule. For example, if it starts to rain or traffic congestion occurs, it evaluates whether the current schedule will be delayed. Input: Real-time weather information and traffic information. Output: Result of the impact evaluation on the schedule.

[0767] Step 6:

[0768] If a new schedule needs to be generated and proposed to the device, the server again sends a prompt to the generation AI model to generate a new schedule. An example of a specific prompt is, "Please update the delivery route taking into account the current weather (rain) and traffic conditions (traffic jam)." The new schedule generated is sent to the device and proposed to the user. Input: Schedule impact assessment results and prompt. Output: New schedule.

[0769] Step 7:

[0770] The device uses a camera to recognize the delivery person's face and monitor their physical condition. The device uses a camera to recognize the delivery person's face and detects frequent blinking and signs of fatigue. Input: Camera image. Output: Results of facial recognition and physical condition monitoring.

[0771] Step 8:

[0772] If an abnormality is detected, the device will display a warning and suggest rest areas or preventive measures. For example, it may display a notification saying, "Frequent blinking has been detected. For safety reasons, we suggest you take a rest at a nearby rest area." Input: Monitoring results. Output: Warning display and suggested rest areas.

[0773] Step 9:

[0774] The server obtains the availability of parking spaces around the destination in real time and suggests the most suitable parking space to the terminal. The server uses the parking information API to obtain the availability of parking spaces in real time and suggests the most suitable parking space. An example of a specific prompt is "Please search for available parking spaces around Tokyo Station in real time and suggest the most suitable parking space." Input: Request to the parking information API. Output: Availability of parking spaces and suggested parking information.

[0775] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0776] overview

[0777] The smart tour guide app, which incorporates the emotion engine of the present invention, is a system that improves the efficiency of travel planning and provides flexible responses during the trip. This system allows users to input their travel destinations and itineraries, and generates an optimal schedule using a generative AI model based on that information. It also obtains real-time weather and traffic information during the trip, and proposes and changes the schedule depending on the situation. Furthermore, in addition to the terminal's functions of managing the user's health and checking parking availability, it is equipped with an emotion engine that recognizes the user's emotions and makes appropriate proposals and changes based on the user's emotional state.

[0778] A flow where the user inputs the travel destination and dates and generates a schedule

[0779] 1. The user launches the Smart Tour Consulting app on their device and enters their travel destination and dates.

[0780] 2. The terminal sends the entered information to the server.

[0781] 3. The server calls the generative AI model based on the received travel destination and itinerary information, and collects information on tourist attractions, restaurants, events, etc. at the travel destination.

[0782] 4. Based on the information collected by the server, multiple optimal schedules are generated.

[0783] 5. The generated schedule is sent to the terminal and displayed to the user, who can then select the best one from the multiple schedules.

[0784] Get real-time information and schedule changes while traveling

[0785] 1. The device periodically collects weather and traffic information and sends it to the server.

[0786] 2. The server analyzes the real-time information and generates a new schedule if it affects the current schedule.

[0787] 3. If a new schedule is required, the server sends it to the terminal, which notifies the user, who then checks the new schedule and approves the changes.

[0788] User health management

[0789] 1. The device uses a camera to recognize the user's face and monitor their physical condition.

[0790] 2. If an abnormality or drowsiness is detected, the device will display a warning to the user and suggest rest or preventive measures.

[0791] Recognizing and responding to user emotions

[0792] 1. The emotion engine installed in the device analyzes the user's facial expressions, voice, etc. to recognize their emotional state.

[0793] 2. The device suggests an optimal travel schedule based on the user's emotional state.

[0794] 3. If a schedule change is required during the trip, the emotion engine generates a new schedule taking into account the emotional state.

[0795] 4. The device will suggest suitable places to rest and relax based on your emotional state.

[0796] Check parking availability

[0797] 1. When a user searches for a parking space, the device queries the server for nearby parking information based on the current location and destination.

[0798] 2. The server collects real-time information on available parking spaces in the surrounding area and selects the most suitable parking space.

[0799] 3. The selected parking lot information is sent to the terminal and suggested to the user, who then parks the car in the designated parking lot.

[0800] Specific examples

[0801] Suppose a user launches a travel app and enters "Tokyo" and "May 1st to 3rd." The device sends this information to a server, which uses a generative AI model to create multiple schedules. For example, a schedule might be generated in which the user visits tourist spot A on day 1, tourist spot B and restaurant C on day 2, and so on. This schedule is then sent to the device and displayed to the user.

[0802] If it starts to rain while the user is traveling to tourist spot A, the device automatically updates the weather information and reports it to the server. The server takes the rain into account and generates a new schedule that changes the itinerary to indoor sightseeing and sends it to the device. The user can then review and select the new schedule.

[0803] Furthermore, if a user begins to feel drowsy while driving, the device's camera will detect frequent blinking and display a warning, allowing the user to take a break at a nearby rest spot.

[0804] When a user searches for a parking space, the server collects information on available parking spaces in real time and selects the most suitable parking space. The device then suggests an appropriate parking space, and the user parks accordingly.

[0805] Furthermore, if the user feels stressed during their trip, the emotion engine will analyze the user's facial expressions and voice, and if it determines that relaxation is necessary, the device will suggest a spot where they can relax. For example, if the user feels tired, the device will suggest a nearby cafe.

[0806] In this way, the present invention allows users to enjoy traveling efficiently and comfortably, and also enables flexible responses according to the user's emotional state and physical condition, providing a safer and more satisfying travel experience.

[0807] The processing flow will be explained below.

[0808] A flow where the user inputs the travel destination and dates and generates a schedule

[0809] Step 1:

[0810] The user launches the Smart Tour Consulting app on their device.

[0811] Step 2:

[0812] The user inputs the travel destination and dates.

[0813] Step 3:

[0814] The terminal transmits the entered travel destination and itinerary information to the server.

[0815] Step 4:

[0816] The server receives the travel destination and itinerary information.

[0817] Step 5:

[0818] The server calls the generated AI model and collects information on tourist attractions, restaurants, events, etc. at the travel destination.

[0819] Step 6:

[0820] Based on the information collected by the server, multiple optimal schedules are generated.

[0821] Step 7:

[0822] The server transmits the generated schedule to the terminal.

[0823] Step 8:

[0824] The terminal displays the generated schedule to the user.

[0825] Step 9:

[0826] The user selects the best schedule from among multiple schedules.

[0827] Get real-time information and schedule changes while traveling

[0828] Step 1:

[0829] The device periodically collects weather and traffic information.

[0830] Step 2:

[0831] The device sends the collected weather and traffic information to the server.

[0832] Step 3:

[0833] The server receives and analyzes the real-time information.

[0834] Step 4:

[0835] The server evaluates whether the current schedule is affected.

[0836] Step 5:

[0837] The server generates new schedules as needed.

[0838] Step 6:

[0839] The server sends the new schedule to the terminal.

[0840] Step 7:

[0841] The terminal notifies the user of the new schedule.

[0842] Step 8:

[0843] The user reviews and approves the proposed schedule changes.

[0844] User health management and warning function

[0845] Step 1:

[0846] The device uses a camera to recognize the user's face and monitor their physical condition.

[0847] Step 2:

[0848] The device uses facial recognition technology to detect signs of drowsiness in the user (e.g., frequent blinking, rubbing of eyes, etc.).

[0849] Step 3:

[0850] If the device detects an abnormality, it displays a warning to the user.

[0851] Step 4:

[0852] The device will suggest appropriate rest areas and preventive measures to the user.

[0853] Recognizing and responding to user emotions

[0854] Step 1:

[0855] The emotion engine installed in the device analyzes the user's facial expressions and voice to recognize their emotional state.

[0856] Step 2:

[0857] The terminal proposes an optimal travel schedule based on the user's emotional state.

[0858] Step 3:

[0859] If a schedule change is required during the trip, the emotion engine will generate a new schedule taking into account the emotional state.

[0860] Step 4:

[0861] The device will suggest suitable places to rest and relax based on your emotional state.

[0862] Check parking availability

[0863] Step 1:

[0864] A user searches for parking near a destination.

[0865] Step 2:

[0866] The terminal queries the server for information about nearby parking lots based on the user's current location and destination.

[0867] Step 3:

[0868] The server collects real-time information on available parking spaces in the surrounding area.

[0869] Step 4:

[0870] The server selects the best parking space.

[0871] Step 5:

[0872] The server transmits the selected parking lot information to the terminal.

[0873] Step 6:

[0874] The terminal proposes optimal parking information to the user.

[0875] Through these steps, the present invention provides users with more efficient travel planning, flexible travel arrangements, health management, and parking management. Furthermore, by using an emotion engine, personalized responses based on the user's emotional state become possible, resulting in a more satisfying travel experience.

[0876] Example 2

[0877] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0878] Conventional travel planning applications have issues such as ineffective automatic schedule generation based on user input, difficulty in flexible schedule changes that take real-time information into account, a lack of suggestions that take into account the user's physical and emotional state, and an inability to effectively provide information on available parking spaces. These issues not only make the travel experience inefficient and inconvenient, but also lead to a lack of safety.

[0879] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for a user to input a travel destination and itinerary, a means for transmitting the input information to the server, a means for the server to generate multiple optimal travel schedules using a generative AI model based on the input information, and a means for transmitting the generated schedules to a terminal and displaying them. This allows the user to enjoy an efficient and comfortable travel experience.

[0880] Furthermore, this invention includes a means for the server to periodically acquire weather and traffic information, a means for evaluating the impact on the current schedule based on the acquired information, a means for generating a new schedule and proposing it to the terminal, a means for monitoring the user's face and physical condition using a camera attached to the terminal, a means for detecting abnormalities based on the monitoring results and displaying a warning, and a means for proposing rest areas and preventive measures, thereby enabling the user's physical condition management and quick schedule adjustments.

[0881] The device further includes a means for analyzing the user's emotional state using an emotion analysis engine built into the device, and a means for proposing optimal travel schedules and rest spots based on the analysis results, thereby enabling optimal suggestions to be made according to the user's emotional state.

[0882] In addition, the system includes a means for users to query the server for information to search for available parking spaces, a means for the server to collect information on available parking spaces in the vicinity in real time and select the most suitable parking space, and a means for transmitting and displaying the selected parking space information to the terminal, thereby enabling users to find parking spaces efficiently.

[0883] A "user" is a person who uses the system to create and execute a travel plan.

[0884] "Destination" refers to a geographic location or city that a user plans to visit.

[0885] "Itinerary" refers to the specific schedule of the travel period, including the start and end dates of the trip.

[0886] "Terminal" refers to an electronic device that a user uses as an interface, such as a smartphone, tablet, or PC.

[0887] "Server" refers to a central processing unit that processes data provided by users and generates travel schedules.

[0888] A "generative artificial intelligence model" refers to an algorithm or method that automatically generates an optimal travel schedule based on input data.

[0889] A "schedule" refers to a specific travel plan including the places to visit, the duration of stay, the means of transportation, etc.

[0890] "Weather Information" refers to data about current and forecast weather and climate.

[0891] "Traffic information" refers to data on current traffic conditions and road congestion.

[0892] "Facial recognition" refers to technology that uses a camera to identify a user's face and detect certain features of it.

[0893] "Physical condition" refers to data indicating the physical condition of the user, such as their health and fatigue level.

[0894] An "emotion analysis engine" refers to software or a system that analyzes a user's facial expressions, voice, etc. to recognize emotions.

[0895] A "rest spot" refers to a place suitable for a user to take a rest.

[0896] "Parking lot" means a designated area for parking vehicles.

[0897] "Availability" refers to the current status of available parking spaces.

[0898] This invention is a system that generates an optimal travel schedule based on user input of travel destinations and dates. The system includes a user terminal, a server, a generative AI model, a camera, and an emotion analysis engine.

[0899] The user operates a device to input the travel destination and itinerary. For example, the device can be a smartphone, tablet, or PC. The information entered by the user is sent from the device to the server as an HTTP POST request. This request includes the travel destination (e.g., "Tokyo") and the travel itinerary (e.g., "May 1st to 3rd").

[0900] The server calls a generative AI model based on the received information to generate an optimal travel schedule. For example, OpenAI's GPT-3 is used as the generative AI model. Here, the following prompt sentence is input to the AI ​​model:

[0901] "A user is traveling to Tokyo from May 1st to 3rd. Please create a three-day travel itinerary including recommended tourist spots and restaurants."

[0902] The generative AI model generates multiple schedules based on this, and the server sends these to the device.

[0903] The terminal presents the received schedule to the user, who can then select the most suitable schedule. The selected schedule is then used for management and display during the trip.

[0904] During a trip, the device periodically collects weather and traffic information and sends it to the server. The server analyzes this information and, if it affects the current schedule, generates a new schedule and sends it to the device. The user receives a notification, checks the new schedule, and approves or changes it as necessary.

[0905] The device also has a built-in camera that can be used to recognize the user's face and monitor their physical condition. If abnormalities or drowsiness (such as frequent blinking) are detected, the device will display a warning and suggest rest areas or preventive measures.

[0906] As an additional function, the device is equipped with an emotion analysis engine. This engine analyzes the user's facial expressions and voice to recognize their emotional state. Based on the emotional state, the device will suggest optimal travel schedules and rest spots. For example, if the user feels stressed, the device will suggest a place where they can relax (such as a cafe).

[0907] Furthermore, when a user searches for a parking space while traveling by car, the device inquires about available parking spaces with the server. The server collects information about nearby parking spaces in real time, selects the most suitable parking space, and sends the information to the device. The user can then park their car in the parking space provided.

[0908] In this way, the system of the present invention provides all kinds of support for users to enjoy an efficient and comfortable trip. By automatically generating schedules using generative AI models, responding in real time based on weather and traffic information, and monitoring the user's physical and emotional state and making optimal suggestions accordingly, users can have a safe and satisfying travel experience.

[0909] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0910] Step 1:

[0911] The user inputs the travel destination and dates.

[0912] Users input their travel destination and itinerary on devices such as smartphones, tablets, and PCs (input: travel destination and itinerary). This information becomes the basis for the user's travel plans. For example, they might input "Tokyo" and "May 1st to 3rd." The input information is processed internally on the device and temporarily saved (output: input information).

[0913] Step 2:

[0914] The terminal transmits the input information to the server.

[0915] The terminal sends the travel destination and itinerary information entered by the user to the server via an HTTP POST request (input: input information). The server receives the request and analyzes the contents (output: received data).

[0916] Step 3:

[0917] The server calls the generative AI model and generates a travel schedule.

[0918] The server calls a generative AI model (e.g., OpenAI's GPT-3) based on the received travel destinations and itinerary (input: received data). Here, the prompt is set as follows: "The user will travel to Tokyo from May 1st to 3rd. Please create a three-day travel schedule including recommended tourist spots and restaurants." The generative AI model outputs multiple travel schedules in response to the prompt (output: generated schedules).

[0919] Step 4:

[0920] The server transmits the generated schedule to the terminal.

[0921] The server sends the generated schedules to the terminal (input: generated schedules). The terminal receives them and saves them as schedule data (output: saved schedules).

[0922] Step 5:

[0923] The terminal displays the generated schedule to the user.

[0924] The terminal displays the received schedules on the screen to the user (input: saved schedules), and the user can view them and select the most suitable schedule (output: selected schedule).

[0925] Step 6:

[0926] The terminal periodically collects weather and traffic information and sends it to the server.

[0927] During the trip, the device periodically collects weather information (weather information service) and traffic information (map API) (input: external information). The collected information is sent to the server and analyzed there (output: analysis results).

[0928] Step 7:

[0929] The server will suggest schedule changes based on the information.

[0930] The server evaluates the impact of collected and analyzed weather and traffic information on the current schedule (input: analysis results). If necessary, it generates a new schedule and proposes it to the terminal (output: new schedule proposal).

[0931] Step 8:

[0932] The terminal notifies the user of the new schedule.

[0933] The terminal receives the new schedule sent from the server and notifies the user (input: new schedule proposal). The user can then review the new schedule and approve or change it (output: user decision).

[0934] Step 9:

[0935] The device uses a camera to monitor the user's physical condition.

[0936] The device's built-in camera recognizes the user's face and monitors their physical condition (e.g., frequent blinking) (input: facial recognition data). If an abnormality is detected, a warning is displayed and suggestions for rest areas or preventive measures are made (output: warning and suggestions).

[0937] Step 10:

[0938] The terminal uses an emotion analysis engine to analyze the user's emotional state.

[0939] The device's built-in emotion analysis engine analyzes the user's facial expressions and voice to recognize their emotional state (input: emotion analysis data). Based on the recognition results, it suggests optimal travel schedules and rest spots (output: emotion-based suggestions).

[0940] Step 11:

[0941] The server collects parking availability information in real time and sends it to the terminal.

[0942] When a user searches for a parking space, the device sends the current location and destination information to the server (input: location information). The server collects information on available parking spaces in real time, selects the most suitable parking space, and sends it to the device (output: parking suggestion).

[0943] Step 12:

[0944] The terminal presents the parking lot information to the user.

[0945] The terminal receives the parking information sent from the server and presents it to the user (input: parking suggestion). The user can then park accordingly (output: parking completed).

[0946] (Application example 2)

[0947] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0948] Conventional travel support systems have difficulty proposing optimal schedules that take into account the user's emotions and physical condition, and responding flexibly based on real-time information during the trip. In particular, there is a need for appropriate responses to sudden changes in physical condition and emotional fluctuations during travel. Furthermore, to improve the customer experience in stores, flexible suggestions based on the customer's emotional and physical conditions are necessary, but current systems are inadequate in this regard. Therefore, there is a need for a system that can provide users with a comfortable and safe travel experience while also improving customer satisfaction in stores.

[0949] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input a travel destination and itinerary; a means for transmitting the input information to the server; a means for the server to generate multiple optimal travel schedules using a generative AI model based on the input information; a means for transmitting the generated schedules to a terminal and displaying them; a means for analyzing the emotional state from the user's facial expressions and voice using an emotion recognition engine; and a means for adjusting the schedule and making appropriate suggestions based on the emotional state. This makes it possible to monitor the user's physical condition and emotional state during the trip in real time and always provide optimal schedules and suggestions.

[0950] A "travel destination" is a geographic location that a user wishes to visit.

[0951] "Itinerary" refers to a time plan such as a travel period or visiting schedule.

[0952] A "server" is a computer system that processes data and provides information over a network.

[0953] A "generative AI model" is an artificial intelligence algorithm or program that generates optimal schedules and proposals based on input information.

[0954] A "terminal" is a device used by a user, such as a smartphone or smart glasses.

[0955] An "emotion recognition engine" is software or hardware that analyzes a user's emotional state from their facial expressions and voice.

[0956] "Facial expressions" are used to read emotions from changes and movements of the user's face.

[0957] "Voice" refers to reading emotions from the tone and patterns of the user's voice.

[0958] "Physical condition" indicates the physical condition of the user, such as their health condition and fatigue level.

[0959] A "schedule" is a plan or timeline for a trip or activity.

[0960] "Suggestions" are information or advice that suggest optimal actions or options for the user.

[0961] "Traffic information" is data showing real-time conditions of traffic volume and means of transportation.

[0962] "Weather information" is data that indicates the weather conditions at your travel destination or current location.

[0963] "Facial recognition" is a technology that uses a camera to identify a user's face and analyze its features.

[0964] A "warning" is a notification to the user of a sign of danger or abnormality.

[0965] A "resting place" is a place where a user can rest and relax during their trip.

[0966] A "parking lot" is a space or facility for parking automobiles.

[0967] "Crowding situation" refers to the degree of crowding or congestion in a particular place or facility.

[0968] The system of the present invention begins when the user inputs their travel destination and itinerary. When the user inputs the destination and itinerary using a device such as a smartphone or smart glasses, the device sends this information to a server. The server uses a generative AI model based on the received information to generate multiple optimal travel itineraries. These schedules are sent to the device and displayed to the user. The user can then select the most appropriate one from the presented schedules.

[0969] The system periodically obtains weather and traffic information and evaluates its impact on the current schedule. Based on the results of this evaluation, it generates a new schedule and proposes it to the device. The device then notifies the user of the proposal, allowing them to review and select the new schedule.

[0970] The device also uses a camera to recognize the user's face and monitor their physical condition. If an abnormality is detected, the device will display a warning to the user and suggest appropriate rest areas and preventive measures. Furthermore, an emotion recognition engine can analyze the user's emotional state from their facial expressions and voice, adjusting their schedule accordingly and making appropriate suggestions.

[0971] The system also obtains real-time information on congestion and parking availability at places the user visits during their trip and makes appropriate suggestions based on that information. For example, if a store is crowded, it can suggest a different time or store. Parking availability is also checked in real time, and the system suggests the best parking spot for the user.

[0972] For example, a user might enter, "I'm looking for new sneakers and would like to know about the latest promotions." The AI ​​model will then provide optimal suggestions for the latest sneakers and promotions. It will also suggest alternative options if the store is crowded, and provide real-time parking information.

[0973] Examples of prompts include:

[0974] "A customer is looking for sneakers and wants to know about the latest promotions. Generate the best suggestions."

[0975] This system allows users to adjust their schedules optimally based on real-time information while traveling, and also improves the customer experience in stores, enabling flexible responses based on physical and emotional states, allowing for safe and comfortable travel and shopping experiences.

[0976] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0977] Step 1: The user enters the travel destination and dates.

[0978] The user launches an application that can be used on a smartphone or smart glasses and inputs their travel destination and itinerary. This information is collected and saved on the device. (Input: Travel destination, itinerary) (Output: Data on the device)

[0979] Step 2: The device sends the input information to the server.

[0980] The terminal sends the entered travel destination and itinerary information to the server. (Input: Data on the terminal) (Output: Data sent to the server)

[0981] Step 3: The server generates a schedule using the generative AI model.

[0982] The server uses the generative AI model to generate multiple optimal travel schedules based on the received travel destination and itinerary information. It also takes into account information collected from the internet about tourist attractions, facilities, events, and other information about the travel destination. (Input: Travel destination, itinerary) (Output: Optimal travel schedule)

[0983] Step 4: The server sends the generation schedule to the terminal.

[0984] The server sends the generated schedules to the terminal and displays them on the terminal. (Input: Optimal travel schedule) (Output: Data sent to the terminal)

[0985] Step 5: The terminal displays the schedule to the user.

[0986] The terminal displays the received schedule to the user, who then selects the optimal schedule from multiple options. (Input: Optimal travel schedule) (Output: Schedule displayed to user)

[0987] Step 6: The device periodically collects weather and traffic information.

[0988] The device periodically collects weather and traffic information and sends it to the server. (Input: Weather information, traffic information) (Output: Data sent to the server)

[0989] Step 7: The server adjusts the schedule based on the real-time information.

[0990] The server analyzes the weather and traffic information sent from the device, and if it affects the current schedule, it generates a new schedule and sends it to the device. (Input: Weather information, traffic information) (Output: Adjusted schedule)

[0991] Step 8: The terminal notifies the user of the adjusted schedule.

[0992] The terminal notifies the user of the new schedule, and the user confirms and approves the schedule changes. (Input: Adjusted schedule) (Output: Notification to user)

[0993] Step 9: The device uses the camera to perform facial recognition and health monitoring.

[0994] The device uses a camera to recognize the user's face and uses AI to monitor their physical condition. If an abnormality is detected, a warning is displayed. (Input: User's face image) (Output: Physical condition evaluation, warning display)

[0995] Step 10: The device uses an emotion recognition engine to analyze the user's facial expressions and voice.

[0996] The device analyzes the user's facial expressions and voice using an emotion recognition engine to infer their emotional state. It then adjusts the schedule based on their emotional state and makes appropriate suggestions. (Input: User's facial expression data, voice data) (Output: Emotional state assessment, adjusted schedule)

[0997] Step 11: The device collects congestion information and parking availability.

[0998] The device collects real-time information about congestion at the destination and parking availability, and sends it to the server to make optimal recommendations. (Input: congestion information, parking availability) (Output: data sent to the server)

[0999] Step 12: The server generates a proposal based on the new information.

[1000] The server analyzes the information sent in real time and generates new suggestions to help users enjoy their trip or shopping. (Input: congestion information, parking information) (Output: new suggestions)

[1001] Step 13: The terminal notifies the user of the new offer.

[1002] The device notifies the user of the new proposal. The user confirms the proposal and selects the next action. (Input: New proposal) (Output: Notification to user)

[1003] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1004] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1005] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1006] [Third embodiment]

[1007] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1008] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1009] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1010] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1011] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1012] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1013] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1014] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1015] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1016] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1017] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1018] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1019] overview

[1020] The smart tour guide app of this invention is a system that improves the efficiency of travel planning and provides flexible support during the trip. This system allows users to input their travel destinations and itineraries, and generates an optimal schedule using a generative AI model based on that information. It also obtains real-time weather and traffic information during the trip, and suggests or changes the schedule depending on the situation. Furthermore, the device also has the function of managing the user's health and checking the availability of parking spaces.

[1021] The user inputs the travel destination and dates

[1022] 1. The user launches the Smart Tour Guide app and enters their travel destination and dates.

[1023] 2. The terminal sends the entered information to the server.

[1024] The server generates the schedule

[1025] 1. The server calls the generative AI model based on the received travel destination and itinerary.

[1026] 2. The server collects information on tourist attractions, restaurants, events, etc. at the travel destination and generates several optimal schedules based on this information.

[1027] 3. The generated schedule is sent to the terminal and displayed to the user, who can then select the best one from the multiple schedules.

[1028] Get real-time information and schedule changes while traveling

[1029] 1. The device periodically collects weather and traffic information and sends it to the server.

[1030] 2. The server analyzes the real-time information and generates a new schedule if it affects the current schedule.

[1031] 3. If a new schedule is required, the server sends it to the terminal, which notifies the user, who then checks the new schedule and approves the changes.

[1032] User health management

[1033] 1. The device uses a camera to recognize the user's face and monitor their physical condition.

[1034] 2. If an abnormality or drowsiness is detected, the device will display a warning to the user and suggest rest or preventive measures.

[1035] Check parking availability

[1036] 1. When a user searches for a parking space, the device queries the server for nearby parking information based on the current location and destination.

[1037] 2. The server collects real-time information on available parking spaces in the surrounding area and selects the most suitable parking space.

[1038] 3. The selected parking lot information is sent to the terminal and suggested to the user, who then parks the car in the designated parking lot.

[1039] Specific examples

[1040] Suppose a user launches a travel app and enters "Tokyo" and "May 1st to 3rd." The device sends this information to a server, which uses a generative AI model to create multiple schedules. For example, a schedule might be generated in which the user visits tourist spot A on day 1, tourist spot B and restaurant C on day 2, and so on. This schedule is then sent to the device and displayed to the user.

[1041] If it starts raining while the user is traveling to tourist spot A, the device automatically updates the weather information and reports it to the server. The server generates a new schedule that changes outdoor activities to indoor sightseeing and sends it to the device. The user can then review and select the new schedule.

[1042] Furthermore, if a user begins to feel drowsy while driving, the device's camera will detect frequent blinking and display a warning, allowing the user to take a break at a nearby rest spot.

[1043] When searching for a parking space, the user searches for a parking space around Tokyo Station, and the server finds a suitable parking space and sends information about its availability to the terminal. Based on this information, the user can park in the most optimal location.

[1044] In this way, the present invention allows users to enjoy traveling efficiently and comfortably.

[1045] The processing flow will be explained below.

[1046] A flow where the user inputs the travel destination and dates and generates a schedule

[1047] Step 1:

[1048] The user launches the Smart Tour Consulting app on their device.

[1049] Step 2:

[1050] The user inputs the travel destination and dates.

[1051] Step 3:

[1052] The terminal transmits the travel destination and itinerary input by the user to the server.

[1053] Step 4:

[1054] The server receives the travel destination and itinerary information.

[1055] Step 5:

[1056] The server calls the generated AI model and collects information on tourist attractions, restaurants, events, etc. at the travel destination.

[1057] Step 6:

[1058] Based on the information collected by the server, multiple optimal schedules are generated.

[1059] Step 7:

[1060] The server transmits the generated schedule to the terminal.

[1061] Step 8:

[1062] The terminal displays the generated schedule to the user.

[1063] Step 9:

[1064] The user selects the best schedule from among multiple schedules.

[1065] Get real-time information and schedule changes while traveling

[1066] Step 1:

[1067] The device periodically collects weather and traffic information.

[1068] Step 2:

[1069] The device sends the collected weather and traffic information to the server.

[1070] Step 3:

[1071] The server receives and analyzes the real-time information.

[1072] Step 4:

[1073] The server evaluates whether the current schedule is affected.

[1074] Step 5:

[1075] The server generates new schedules as needed.

[1076] Step 6:

[1077] The server sends the new schedule to the terminal.

[1078] Step 7:

[1079] The terminal notifies the user of the new schedule.

[1080] Step 8:

[1081] The user reviews and approves the proposed schedule changes.

[1082] User health management and warning function

[1083] Step 1:

[1084] The device uses a camera to recognize the user's face and monitor their physical condition.

[1085] Step 2:

[1086] The device uses facial recognition technology to detect signs of drowsiness in the user (e.g., frequent blinking, rubbing of eyes, etc.).

[1087] Step 3:

[1088] If the device detects an abnormality, it displays a warning to the user.

[1089] Step 4:

[1090] The device will suggest appropriate rest areas and preventive measures to the user.

[1091] Check parking availability

[1092] Step 1:

[1093] A user searches for parking near a destination.

[1094] Step 2:

[1095] The terminal queries the server for information about nearby parking lots based on the user's current location and destination.

[1096] Step 3:

[1097] The server collects real-time information on available parking spaces in the surrounding area.

[1098] Step 4:

[1099] The server selects the best parking space.

[1100] Step 5:

[1101] The server transmits the selected parking lot information to the terminal.

[1102] Step 6:

[1103] The terminal proposes optimal parking information to the user.

[1104] Through the above steps, the present invention provides users with more efficient travel planning, flexible travel arrangements, health management, and parking lot management.

[1105] Example 1

[1106] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1107] In today's world, planning and executing a trip involves many variables, so a flexible and efficient travel planning system is necessary to ensure a comfortable and stress-free trip. Conventional travel planning systems have had difficulty adequately meeting diverse needs, such as optimizing schedules, responding to unexpected changes in circumstances, managing the user's health, and checking parking availability. The present invention aims to solve these problems and provide users with a fulfilling travel experience.

[1108] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1109] In this invention, the server includes a means for a user to input travel destinations and itineraries, a means for transmitting the input information to the server, a means for the server to generate multiple optimal travel schedules using a generative AI model based on the input information, and a means for transmitting the generated schedules to a terminal and displaying them, thereby enabling users to easily create optimal travel plans.

[1110] Furthermore, the server includes means for periodically acquiring weather and traffic information, means for evaluating the impact on the current schedule based on the acquired information, and means for generating a new schedule and proposing it to the terminal, thereby enabling the system to flexibly respond to unexpected changes in circumstances.

[1111] Furthermore, the terminal includes a means for recognizing the user's face and monitoring their physical condition using a camera, a means for detecting abnormalities based on the monitoring results and displaying a warning, and a means for suggesting rest areas and preventive measures, thereby protecting the user's safety and health.

[1112] Furthermore, the system includes a means for the terminal to search for available parking spaces and inquire of the server, a means for the server to select an appropriate parking space based on the current location and destination, and a means for transmitting information about the selected parking space to the terminal and suggesting it to the user, thereby efficiently solving parking problems.

[1113] In addition, the system includes a means for generating a prompt sentence for the server to call the generative AI model, a means for generating a schedule using the generative AI model based on the prompt sentence, and a means for presenting the schedule to the user as multiple options, thereby providing the user with a variety of options and realizing more satisfying travel plans.

[1114] "User" refers to a person who uses the system to provide input to plan a trip.

[1115] "Destination" refers to a geographic location that a user plans to visit.

[1116] "Schedule" refers to the start and end dates of the trip planned by the user, as well as the duration of the trip.

[1117] "Terminal" refers to the digital device that a user uses to enter their travel plans and receive system results.

[1118] "Server" refers to a remote computer system that receives user input, processes it, and sends the results to the terminal.

[1119] "Generative AI Model" refers to an artificial intelligence model used to generate travel itineraries based on input information.

[1120] "Schedule" refers to the plan of the itinerary and activities for each day at the travel destination.

[1121] "Weather information" refers to data about the weather around your travel destination.

[1122] "Traffic information" refers to data regarding traffic conditions around the travel destination.

[1123] "Facial recognition" refers to technology that uses a camera to identify a user's face.

[1124] "Physical condition" refers to information for monitoring the user's health condition and fatigue level.

[1125] "Parking Availability" refers to information about available parking spaces within a specified area.

[1126] "Prompt" refers to a textual instruction used to invoke a generative AI model.

[1127] overview

[1128] The smart tour guide system of the present invention is a system that improves the efficiency of travel planning and provides flexible support during the trip. The system allows users to input their travel destinations and itineraries, and generates an optimal schedule based on that information using a generative AI model on the server. It also obtains weather and traffic information during the trip in real time, and proposes or changes the schedule depending on the situation. Furthermore, the terminal also has the function of managing the user's health and checking the availability of parking spaces.

[1129] Hardware and software used

[1130] The main hardware and software used to implement this system are as follows:

[1131] User device: Mobile device such as smartphone or tablet (e.g. iOS, Android device)

[1132] Server: Cloud-based computing resources (e.g., Amazon Web Services, Google Cloud Platform)

[1133] Generative AI model: a generative AI algorithm (e.g., GPT-4)

[1134] External API: Weather API, Google Maps API, Yelp API, Parkopedia API

[1135] Facial recognition algorithm: TensorFlow Lite, Apple Core ML

[1136] Explanation of program processing

[1137] User inputs travel destination and dates

[1138] The user launches the smart tour guide app and enters their travel destination and itinerary in the form displayed on the UI. For example, they enter "Tokyo" and "May 1st to 3rd." The device then sends this information in JSON format to the server.

[1139] Server-based schedule generation

[1140] The server generates a prompt to call the generative AI model based on the received travel destination and itinerary information. For example, the prompt might look like this:

[1141] Travel Planner

[1142] Travel destination: Tokyo

[1143] Dates: May 1st to 3rd

[1144] The server inputs this prompt into a generative AI model and generates multiple optimal schedules. For example, the generated schedules are as follows:

[1145] Day 1: Visit Sensoji Temple in the morning, Ueno Zoo in the afternoon

[1146] Day 2: Visit Sensoji Temple in the morning and Ueno Zoo in the afternoon

[1147] The generated schedule is sent from the server in JSON format to the device and displayed on the device's UI. The user can then select the most suitable schedule from multiple options.

[1148] Get real-time information and schedule changes while traveling

[1149] The device periodically collects weather and traffic information through APIs and sends that information to the server. For example, weather information is obtained from the Weather API, and traffic information is obtained from the Google Maps API. The server analyzes the real-time information and generates a new schedule if it affects the current schedule. If a change is determined to be necessary, the new schedule is sent to the device and the user is notified.

[1150] User health management

[1151] The device monitors the user's face using a built-in camera. For example, it uses a camera sensor and a facial recognition algorithm (e.g., TensorFlow Lite). It estimates the user's physical condition from the user's complexion, blinking frequency, and facial expression, and displays an alert if an abnormality is detected. For example, it displays a message such as, "Are you feeling sleepy? There is a rest area nearby."

[1152] Check parking availability

[1153] The user uses the app's "Parking Lot Search" function to input their current location and destination information. The device sends this information to the server, which then uses a parking API (e.g., Parkopedia API) to obtain information about available parking lots in the area. Based on this information, the server selects the most suitable parking lot and sends it back to the device.

[1154] Specific examples

[1155] Suppose a user launches a travel app and enters "Tokyo" and "May 1st to 3rd." The device sends this information in JSON format to the server, which then uses a generative AI model to create multiple schedules. For example, a schedule might be generated in which the user visits tourist spot A on the first day, tourist spot B and restaurant C on the second day, and so on. This schedule is then sent to the device and displayed to the user.

[1156] If it starts raining while the user is traveling to tourist spot A, the device automatically updates the weather information and reports it to the server. The server generates a new schedule that changes outdoor activities to indoor sightseeing and sends it to the device. The user can then review and select the new schedule.

[1157] Furthermore, if the user begins to feel drowsy while driving, the device's camera will detect frequent blinking and display a warning. This warning will allow the user to take a rest at a nearby rest spot. When searching for parking, if the user searches for a parking lot near Tokyo Station, the server will find a suitable parking lot and send information about its availability to the device. The user can then park in the most optimal location based on this information.

[1158] In this way, the present invention allows users to enjoy traveling efficiently and comfortably.

[1159] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1160] Step 1:

[1161] The user enters the travel destination and itinerary. The user launches the smart tour guide app and enters the travel destination (e.g., Tokyo) and travel dates (e.g., May 1st to 3rd) in the form displayed in the UI. The input data is temporarily saved in the device's memory.

[1162] Input: Travel destination and itinerary information

[1163] Output: Travel destination and itinerary data in memory

[1164] Step 2:

[1165] The device sends the input information to the server, which converts the input data of the travel destination and itinerary into JSON format and sends it to the server using the HTTPS protocol, using Wi-Fi or mobile data connection.

[1166] Input: Travel destination and itinerary data in memory

[1167] Output: Data sent to the server in JSON format

[1168] Step 3:

[1169] The server generates the schedule. Based on the received travel destination and schedule information, the server generates a prompt to call the generative AI model. For example, the prompt might say, "The travel destination is Tokyo, and the dates are three days from May 1st to 3rd. Please generate a schedule including tourist spots, restaurants, and events." This prompt is input into the generative AI model to generate the schedule.

[1170] Input: Received travel destination and itinerary information

[1171] Output: Multiple schedules generated by the generative AI model

[1172] Step 4:

[1173] The server sends the generated schedule to the terminal. The generated schedules are converted into JSON format and sent to the terminal using the HTTPS protocol.

[1174] Input: Multiple generated schedules

[1175] Output: Schedule data sent to the device in JSON format

[1176] Step 5:

[1177] The device displays the schedule to the user. The received schedule is displayed on the UI, and the user can select the most appropriate one from multiple options. The schedule is displayed in a visually easy-to-understand list or calendar format.

[1178] Input: Received schedule data

[1179] Output: The schedule displayed on the user interface

[1180] Step 6:

[1181] Obtaining real-time information and changing schedules during travel. The device periodically obtains weather and traffic information via API (for example, using the Weather API or Google Maps API). The collected information is sent to the server, which analyzes the real-time information. If the current schedule is affected, a new schedule is generated and sent to the device. The user can review the new schedule and approve the changes.

[1182] Input: Weather and traffic information

[1183] Output: New schedule as needed

[1184] Step 7:

[1185] User health management. The device uses a built-in camera to recognize the user's face and monitor their physical condition. For example, it estimates their physical condition from factors such as facial color, blinking frequency, and facial expressions. If an abnormality is detected, it displays a warning and suggests rest areas and preventive measures.

[1186] Input: Face data captured from the camera

[1187] Output: Warning messages and suggestions for the user

[1188] Step 8:

[1189] Checking parking availability. The user uses the app's "parking search" function to input their current location and destination information. The device sends this information to the server, which then uses the parking API to obtain information on available parking in the surrounding area. The server then returns the optimal parking information to the device and makes a suggestion to the user.

[1190] Input: Current location and destination information

[1191] Output: Appropriate parking information

[1192] (Application example 1)

[1193] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1194] For food delivery services, improving delivery efficiency and managing the safety of delivery personnel are important. However, challenges include dynamically changing delivery schedules based on real-time weather and traffic information, managing the health of delivery personnel, and checking the availability of parking spaces around the destination. These factors can lead to delivery delays and a decline in service quality. Furthermore, if delivery personnel's fatigue and poor health are not adequately managed, the risk of accidents and delivery errors increases.

[1195] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1196] In this invention, the server includes a means for a user to input a delivery destination and delivery date and time, a means for transmitting the input information to the server, a means for the server to generate multiple optimal delivery schedules using a generative AI model based on the input information, and a means for transmitting the generated schedules to a terminal and displaying them, thereby enabling efficient and flexible generation of delivery schedules.

[1197] The server also includes a means for periodically acquiring weather and traffic information, a means for evaluating the impact of the acquired information on the current schedule, and a means for generating a new schedule and proposing it to the terminal, thereby enabling schedule adjustments in response to real-time changes in the situation.

[1198] The terminal also has a means for using a camera to recognize the face and monitor the physical condition of the delivery person, a means for detecting abnormalities and displaying a warning based on the monitoring results, and a means for suggesting rest areas and preventive measures, which will enable the health management and safety of delivery people to be ensured.

[1199] Furthermore, the server acquires the availability of parking spaces around the destination in real time, and includes a means for suggesting the most suitable parking space to the terminal based on the availability, thereby enabling the delivery person to find a parking space efficiently and quickly.

[1200] "User" refers to the person who uses or operates the delivery service.

[1201] "Delivery Destination" means the destination to which goods or items are to be delivered.

[1202] "Delivery Date and Time" means the date and time when the goods or items are to be delivered to their destination.

[1203] "Server" refers to a central computer system that processes data and collects and processes various types of information.

[1204] A "generative AI model" refers to an algorithm that uses artificial intelligence to suggest optimal schedules and solutions.

[1205] "Terminal" refers to a device or equipment used by a user, such as a smartphone or tablet.

[1206] A "schedule" is a timetable showing the order and plan of deliveries.

[1207] "Weather Information" means data regarding current and future weather conditions.

[1208] "Traffic information" refers to the latest data on road traffic, such as road congestion and traffic regulations.

[1209] "Facial recognition" refers to a technology that uses a camera to identify a person's face and then processes that information.

[1210] Monitoring "health status" refers to the process of monitoring the health status of delivery personnel in real time and detecting any abnormalities.

[1211] "Warning" refers to a notification that alerts the user when the system detects an abnormality.

[1212] "Rest Area" means a place where a delivery person temporarily stops to take rest.

[1213] "Preventive measures" refer to measures taken to prevent delivery personnel from becoming ill or having accidents.

[1214] "Parking Space" means a location available for a Delivery Person to temporarily park their vehicle.

[1215] "Availability" refers to information about whether a parking space is available.

[1216] "Real-time" means that the latest information is obtained and updated almost instantly.

[1217] The present invention relates to a system for improving delivery efficiency and managing the safety of delivery personnel in a food delivery service. Detailed embodiments of the present invention will be described below.

[1218] System Program Overview

[1219] This system includes a means for the user to input the delivery destination and delivery date and time, a means for transmitting the input information to a server, a means for generating an optimal delivery schedule using a generative AI model, and a means for transmitting and displaying the generated schedule to a terminal. The server also has a means for periodically obtaining weather and traffic information, evaluating the impact on the current schedule, generating a new schedule, and proposing it to the terminal. The terminal also has a means for recognizing the delivery person's face and monitoring their physical condition, detecting abnormalities, displaying a warning, and suggesting rest areas and preventive measures. It also includes a means for obtaining real-time information on available parking spaces around the destination and proposing the optimal parking space to the terminal.

[1220] Program processing and the hardware and software used

[1221] 1. User Input and Information Submission

[1222] Hardware: Smartphone or tablet

[1223] Software: User Interface Application

[1224] Processing: The user inputs the delivery address and delivery date and time. The input information is sent from the terminal to the server.

[1225] 2. Schedule generation using generative AI models

[1226] Hardware: Server

[1227] Software: Generative AI models (e.g., OpenAI GPT-3)

[1228] Processing: The server calls the generative AI model based on the received delivery destination and date and time information. The prompt "Generate the optimal delivery schedule for arriving at Tokyo Station at 12:00 on May 1, 2023" is entered, and the AI ​​model generates the optimal schedule. The generated schedule is sent to the device and displayed to the user.

[1229] 3. Real-time information acquisition and schedule updates

[1230] Hardware: Server

[1231] Software: Weather information API, traffic information API

[1232] Processing: The server periodically obtains weather and traffic information using an API. Based on the obtained information, it evaluates the impact on the current schedule and generates a new schedule. An example of a prompt sentence in this case is "Please update the delivery route taking into account the current weather (rain) and traffic conditions (congestion)." The new schedule is then resent to the device.

[1233] 4. Managing the health of delivery personnel

[1234] Hardware: Smartphone or tablet with a camera

[1235] Software: OpenCV (open source computer vision library)

[1236] Processing: The device uses a camera to recognize the delivery person's face and monitor their physical condition. If an abnormality is detected, it displays a warning and suggests resting places and preventive measures. For example, it displays a notification saying, "Frequent blinking has been detected. For safety reasons, we suggest you take a break at a nearby rest facility."

[1237] 5. Check availability of parking spaces

[1238] Hardware: Server

[1239] Software: Parking Information API

[1240] Processing: The server obtains the availability of parking spaces near the delivery destination in real time and suggests the most suitable parking location to the terminal. An example of a prompt sentence in this case is "Please search for available parking spaces around Tokyo Station in real time and suggest the most suitable parking location."

[1241] Specific examples

[1242] When a user uses their smartphone to enter "Tokyo Station" and "May 1, 2023, 12:00," the device sends this information to the server. The server uses a generative AI model to generate an optimal delivery schedule and sends this schedule to the device. If the weather worsens and traffic congestion occurs during the trip, the server obtains new real-time information, updates the schedule, and resends it to the device.

[1243] Furthermore, if a delivery person feels unwell during a delivery, the device's camera will use facial recognition to detect the abnormality and display a warning. This warning allows the delivery person to take a break safely. If the delivery person needs to find a parking space around Tokyo Station, the server will obtain the most suitable parking space in real time and send that information to the device, helping the delivery person park efficiently.

[1244] In this way, the present invention improves the delivery efficiency and safety of delivery personnel of food delivery services.

[1245] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1246] Step 1:

[1247] The user enters the delivery address and delivery date and time. The user uses a smartphone or tablet to enter the delivery address and delivery date and time into the application. This input information is sent from the device to the server. Input: Delivery address and delivery date and time. Output: Delivery information sent to the server.

[1248] Step 2:

[1249] The server uses the generative AI model to generate an optimal delivery schedule. Based on the delivery destination and date and time information received, the server sends a prompt to the generative AI model to generate an optimal schedule. An example of a specific prompt is, "Please generate an optimal delivery schedule that arrives at Tokyo Station at 12:00 on May 1, 2023." Input: Delivery information and prompt. Output: Optimal delivery schedule.

[1250] Step 3:

[1251] The generated schedule is sent to the terminal and displayed to the user. The server sends the generated schedule to the terminal, and the terminal displays it to the user. Input: Generated schedule. Output: Schedule displayed on the terminal.

[1252] Step 4:

[1253] The server periodically obtains weather and traffic information. The server obtains weather and traffic information in real time using the weather information API and traffic information API. Input: Requests to the weather information API and traffic information API. Output: Real-time weather and traffic information.

[1254] Step 5:

[1255] The impact on the current schedule is evaluated based on the acquired information. The server analyzes the acquired weather and traffic information and evaluates the impact on the current schedule. For example, if it starts to rain or traffic congestion occurs, it evaluates whether the current schedule will be delayed. Input: Real-time weather information and traffic information. Output: Result of the impact evaluation on the schedule.

[1256] Step 6:

[1257] If a new schedule needs to be generated and proposed to the device, the server again sends a prompt to the generation AI model to generate a new schedule. An example of a specific prompt is, "Please update the delivery route taking into account the current weather (rain) and traffic conditions (traffic jam)." The new schedule generated is sent to the device and proposed to the user. Input: Schedule impact assessment results and prompt. Output: New schedule.

[1258] Step 7:

[1259] The device uses a camera to recognize the delivery person's face and monitor their physical condition. The device uses a camera to recognize the delivery person's face and detects frequent blinking and signs of fatigue. Input: Camera image. Output: Results of facial recognition and physical condition monitoring.

[1260] Step 8:

[1261] If an abnormality is detected, the device will display a warning and suggest rest areas or preventive measures. For example, it may display a notification saying, "Frequent blinking has been detected. For safety reasons, we suggest you take a rest at a nearby rest area." Input: Monitoring results. Output: Warning display and suggested rest areas.

[1262] Step 9:

[1263] The server obtains the availability of parking spaces around the destination in real time and suggests the most suitable parking space to the terminal. The server uses the parking information API to obtain the availability of parking spaces in real time and suggests the most suitable parking space. An example of a specific prompt is "Please search for available parking spaces around Tokyo Station in real time and suggest the most suitable parking space." Input: Request to the parking information API. Output: Availability of parking spaces and suggested parking information.

[1264] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1265] overview

[1266] The smart tour guide app, which incorporates the emotion engine of the present invention, is a system that improves the efficiency of travel planning and provides flexible responses during the trip. This system allows users to input their travel destinations and itineraries, and generates an optimal schedule using a generative AI model based on that information. It also obtains real-time weather and traffic information during the trip, and proposes and changes the schedule depending on the situation. Furthermore, in addition to the terminal's functions of managing the user's health and checking parking availability, it is equipped with an emotion engine that recognizes the user's emotions and makes appropriate proposals and changes based on the user's emotional state.

[1267] A flow where the user inputs the travel destination and dates and generates a schedule

[1268] 1. The user launches the Smart Tour Consulting app on their device and enters their travel destination and dates.

[1269] 2. The terminal sends the entered information to the server.

[1270] 3. The server calls the generative AI model based on the received travel destination and itinerary information, and collects information on tourist attractions, restaurants, events, etc. at the travel destination.

[1271] 4. Based on the information collected by the server, multiple optimal schedules are generated.

[1272] 5. The generated schedule is sent to the terminal and displayed to the user, who can then select the best one from the multiple schedules.

[1273] Get real-time information and schedule changes while traveling

[1274] 1. The device periodically collects weather and traffic information and sends it to the server.

[1275] 2. The server analyzes the real-time information and generates a new schedule if it affects the current schedule.

[1276] 3. If a new schedule is required, the server sends it to the terminal, which notifies the user, who then checks the new schedule and approves the changes.

[1277] User health management

[1278] 1. The device uses a camera to recognize the user's face and monitor their physical condition.

[1279] 2. If an abnormality or drowsiness is detected, the device will display a warning to the user and suggest rest or preventive measures.

[1280] Recognizing and responding to user emotions

[1281] 1. The emotion engine installed in the device analyzes the user's facial expressions, voice, etc. to recognize their emotional state.

[1282] 2. The device suggests an optimal travel schedule based on the user's emotional state.

[1283] 3. If a schedule change is required during the trip, the emotion engine generates a new schedule taking into account the emotional state.

[1284] 4. The device will suggest suitable places to rest and relax based on your emotional state.

[1285] Check parking availability

[1286] 1. When a user searches for a parking space, the device queries the server for nearby parking information based on the current location and destination.

[1287] 2. The server collects real-time information on available parking spaces in the surrounding area and selects the most suitable parking space.

[1288] 3. The selected parking lot information is sent to the terminal and suggested to the user, who then parks the car in the designated parking lot.

[1289] Specific examples

[1290] Suppose a user launches a travel app and enters "Tokyo" and "May 1st to 3rd." The device sends this information to a server, which uses a generative AI model to create multiple schedules. For example, a schedule might be generated in which the user visits tourist spot A on day 1, tourist spot B and restaurant C on day 2, and so on. This schedule is then sent to the device and displayed to the user.

[1291] If it starts to rain while the user is traveling to tourist spot A, the device automatically updates the weather information and reports it to the server. The server takes the rain into account and generates a new schedule that changes the itinerary to indoor sightseeing and sends it to the device. The user can then review and select the new schedule.

[1292] Furthermore, if a user begins to feel drowsy while driving, the device's camera will detect frequent blinking and display a warning, allowing the user to take a break at a nearby rest spot.

[1293] When a user searches for a parking space, the server collects information on available parking spaces in real time and selects the most suitable parking space. The device then suggests an appropriate parking space, and the user parks accordingly.

[1294] Furthermore, if the user feels stressed during their trip, the emotion engine will analyze the user's facial expressions and voice, and if it determines that relaxation is necessary, the device will suggest a spot where they can relax. For example, if the user feels tired, the device will suggest a nearby cafe.

[1295] In this way, the present invention allows users to enjoy traveling efficiently and comfortably, and also enables flexible responses according to the user's emotional state and physical condition, providing a safer and more satisfying travel experience.

[1296] The processing flow will be explained below.

[1297] A flow where the user inputs the travel destination and dates and generates a schedule

[1298] Step 1:

[1299] The user launches the Smart Tour Consulting app on their device.

[1300] Step 2:

[1301] The user inputs the travel destination and dates.

[1302] Step 3:

[1303] The terminal transmits the entered travel destination and itinerary information to the server.

[1304] Step 4:

[1305] The server receives the travel destination and itinerary information.

[1306] Step 5:

[1307] The server calls the generated AI model and collects information on tourist attractions, restaurants, events, etc. at the travel destination.

[1308] Step 6:

[1309] Based on the information collected by the server, multiple optimal schedules are generated.

[1310] Step 7:

[1311] The server transmits the generated schedule to the terminal.

[1312] Step 8:

[1313] The terminal displays the generated schedule to the user.

[1314] Step 9:

[1315] The user selects the best schedule from among multiple schedules.

[1316] Get real-time information and schedule changes while traveling

[1317] Step 1:

[1318] The device periodically collects weather and traffic information.

[1319] Step 2:

[1320] The device sends the collected weather and traffic information to the server.

[1321] Step 3:

[1322] The server receives and analyzes the real-time information.

[1323] Step 4:

[1324] The server evaluates whether the current schedule is affected.

[1325] Step 5:

[1326] The server generates new schedules as needed.

[1327] Step 6:

[1328] The server sends the new schedule to the terminal.

[1329] Step 7:

[1330] The terminal notifies the user of the new schedule.

[1331] Step 8:

[1332] The user reviews and approves the proposed schedule changes.

[1333] User health management and warning function

[1334] Step 1:

[1335] The device uses a camera to recognize the user's face and monitor their physical condition.

[1336] Step 2:

[1337] The device uses facial recognition technology to detect signs of drowsiness in the user (e.g., frequent blinking, rubbing of eyes, etc.).

[1338] Step 3:

[1339] If the device detects an abnormality, it displays a warning to the user.

[1340] Step 4:

[1341] The device will suggest appropriate rest areas and preventive measures to the user.

[1342] Recognizing and responding to user emotions

[1343] Step 1:

[1344] The emotion engine installed in the device analyzes the user's facial expressions and voice to recognize their emotional state.

[1345] Step 2:

[1346] The terminal proposes an optimal travel schedule based on the user's emotional state.

[1347] Step 3:

[1348] If a schedule change is required during the trip, the emotion engine will generate a new schedule taking into account the emotional state.

[1349] Step 4:

[1350] The device will suggest suitable places to rest and relax based on your emotional state.

[1351] Check parking availability

[1352] Step 1:

[1353] A user searches for parking near a destination.

[1354] Step 2:

[1355] The terminal queries the server for information about nearby parking lots based on the user's current location and destination.

[1356] Step 3:

[1357] The server collects real-time information on available parking spaces in the surrounding area.

[1358] Step 4:

[1359] The server selects the best parking space.

[1360] Step 5:

[1361] The server transmits the selected parking lot information to the terminal.

[1362] Step 6:

[1363] The terminal proposes optimal parking information to the user.

[1364] Through these steps, the present invention provides users with more efficient travel planning, flexible travel arrangements, health management, and parking management. Furthermore, by using an emotion engine, personalized responses based on the user's emotional state become possible, resulting in a more satisfying travel experience.

[1365] Example 2

[1366] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1367] Conventional travel planning applications have issues such as ineffective automatic schedule generation based on user input, difficulty in flexible schedule changes that take real-time information into account, a lack of suggestions that take into account the user's physical and emotional state, and an inability to effectively provide information on available parking spaces. These issues not only make the travel experience inefficient and inconvenient, but also lead to a lack of safety.

[1368] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for a user to input a travel destination and itinerary, a means for transmitting the input information to the server, a means for the server to generate multiple optimal travel schedules using a generative AI model based on the input information, and a means for transmitting the generated schedules to a terminal and displaying them. This allows the user to enjoy an efficient and comfortable travel experience.

[1369] Furthermore, this invention includes a means for the server to periodically acquire weather and traffic information, a means for evaluating the impact on the current schedule based on the acquired information, a means for generating a new schedule and proposing it to the terminal, a means for monitoring the user's face and physical condition using a camera attached to the terminal, a means for detecting abnormalities based on the monitoring results and displaying a warning, and a means for proposing rest areas and preventive measures, thereby enabling the user's physical condition management and quick schedule adjustments.

[1370] The device further includes a means for analyzing the user's emotional state using an emotion analysis engine built into the device, and a means for proposing optimal travel schedules and rest spots based on the analysis results, thereby enabling optimal suggestions to be made according to the user's emotional state.

[1371] In addition, the system includes a means for users to query the server for information to search for available parking spaces, a means for the server to collect information on available parking spaces in the vicinity in real time and select the most suitable parking space, and a means for transmitting and displaying the selected parking space information to the terminal, thereby enabling users to find parking spaces efficiently.

[1372] A "user" is a person who uses the system to create and execute a travel plan.

[1373] "Destination" refers to a geographic location or city that a user plans to visit.

[1374] "Itinerary" refers to the specific schedule of the travel period, including the start and end dates of the trip.

[1375] "Terminal" refers to an electronic device that a user uses as an interface, such as a smartphone, tablet, or PC.

[1376] "Server" refers to a central processing unit that processes data provided by users and generates travel schedules.

[1377] A "generative artificial intelligence model" refers to an algorithm or method that automatically generates an optimal travel schedule based on input data.

[1378] A "schedule" refers to a specific travel plan including the places to visit, the duration of stay, the means of transportation, etc.

[1379] "Weather Information" refers to data about current and forecast weather and climate.

[1380] "Traffic information" refers to data on current traffic conditions and road congestion.

[1381] "Facial recognition" refers to technology that uses a camera to identify a user's face and detect certain features of it.

[1382] "Physical condition" refers to data indicating the physical condition of the user, such as their health and fatigue level.

[1383] An "emotion analysis engine" refers to software or a system that analyzes a user's facial expressions, voice, etc. to recognize emotions.

[1384] A "rest spot" refers to a place suitable for a user to take a rest.

[1385] "Parking lot" means a designated area for parking vehicles.

[1386] "Availability" refers to the current status of available parking spaces.

[1387] This invention is a system that generates an optimal travel schedule based on user input of travel destinations and dates. The system includes a user terminal, a server, a generative AI model, a camera, and an emotion analysis engine.

[1388] The user operates a device to input the travel destination and itinerary. For example, the device can be a smartphone, tablet, or PC. The information entered by the user is sent from the device to the server as an HTTP POST request. This request includes the travel destination (e.g., "Tokyo") and the travel itinerary (e.g., "May 1st to 3rd").

[1389] The server calls a generative AI model based on the received information to generate an optimal travel schedule. For example, OpenAI's GPT-3 is used as the generative AI model. Here, the following prompt sentence is input to the AI ​​model:

[1390] "A user is traveling to Tokyo from May 1st to 3rd. Please create a three-day travel itinerary including recommended tourist spots and restaurants."

[1391] The generative AI model generates multiple schedules based on this, and the server sends these to the device.

[1392] The terminal presents the received schedule to the user, who can then select the most suitable schedule. The selected schedule is then used for management and display during the trip.

[1393] During a trip, the device periodically collects weather and traffic information and sends it to the server. The server analyzes this information and, if it affects the current schedule, generates a new schedule and sends it to the device. The user receives a notification, checks the new schedule, and approves or changes it as necessary.

[1394] The device also has a built-in camera that can be used to recognize the user's face and monitor their physical condition. If abnormalities or drowsiness (such as frequent blinking) are detected, the device will display a warning and suggest rest areas or preventive measures.

[1395] As an additional function, the device is equipped with an emotion analysis engine. This engine analyzes the user's facial expressions and voice to recognize their emotional state. Based on the emotional state, the device will suggest optimal travel schedules and rest spots. For example, if the user feels stressed, the device will suggest a place where they can relax (such as a cafe).

[1396] Furthermore, when a user searches for a parking space while traveling by car, the device inquires about available parking spaces with the server. The server collects information about nearby parking spaces in real time, selects the most suitable parking space, and sends the information to the device. The user can then park their car in the parking space provided.

[1397] In this way, the system of the present invention provides all kinds of support for users to enjoy an efficient and comfortable trip. By automatically generating schedules using generative AI models, responding in real time based on weather and traffic information, and monitoring the user's physical and emotional state and making optimal suggestions accordingly, users can have a safe and satisfying travel experience.

[1398] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1399] Step 1:

[1400] The user inputs the travel destination and dates.

[1401] Users input their travel destination and itinerary on devices such as smartphones, tablets, and PCs (input: travel destination and itinerary). This information becomes the basis for the user's travel plans. For example, they might input "Tokyo" and "May 1st to 3rd." The input information is processed internally on the device and temporarily saved (output: input information).

[1402] Step 2:

[1403] The terminal transmits the input information to the server.

[1404] The terminal sends the travel destination and itinerary information entered by the user to the server via an HTTP POST request (input: input information). The server receives the request and analyzes the contents (output: received data).

[1405] Step 3:

[1406] The server calls the generative AI model and generates a travel schedule.

[1407] The server calls a generative AI model (e.g., OpenAI's GPT-3) based on the received travel destinations and itinerary (input: received data). Here, the prompt is set as follows: "The user will travel to Tokyo from May 1st to 3rd. Please create a three-day travel schedule including recommended tourist spots and restaurants." The generative AI model outputs multiple travel schedules in response to the prompt (output: generated schedules).

[1408] Step 4:

[1409] The server transmits the generated schedule to the terminal.

[1410] The server sends the generated schedules to the terminal (input: generated schedules). The terminal receives them and saves them as schedule data (output: saved schedules).

[1411] Step 5:

[1412] The terminal displays the generated schedule to the user.

[1413] The terminal displays the received schedules on the screen to the user (input: saved schedules), and the user can view them and select the most suitable schedule (output: selected schedule).

[1414] Step 6:

[1415] The terminal periodically collects weather and traffic information and sends it to the server.

[1416] During the trip, the device periodically collects weather information (weather information service) and traffic information (map API) (input: external information). The collected information is sent to the server and analyzed there (output: analysis results).

[1417] Step 7:

[1418] The server will suggest schedule changes based on the information.

[1419] The server evaluates the impact of collected and analyzed weather and traffic information on the current schedule (input: analysis results). If necessary, it generates a new schedule and proposes it to the terminal (output: new schedule proposal).

[1420] Step 8:

[1421] The terminal notifies the user of the new schedule.

[1422] The terminal receives the new schedule sent from the server and notifies the user (input: new schedule proposal). The user can then review the new schedule and approve or change it (output: user decision).

[1423] Step 9:

[1424] The device uses a camera to monitor the user's physical condition.

[1425] The device's built-in camera recognizes the user's face and monitors their physical condition (e.g., frequent blinking) (input: facial recognition data). If an abnormality is detected, a warning is displayed and suggestions for rest areas or preventive measures are made (output: warning and suggestions).

[1426] Step 10:

[1427] The terminal uses an emotion analysis engine to analyze the user's emotional state.

[1428] The device's built-in emotion analysis engine analyzes the user's facial expressions and voice to recognize their emotional state (input: emotion analysis data). Based on the recognition results, it suggests optimal travel schedules and rest spots (output: emotion-based suggestions).

[1429] Step 11:

[1430] The server collects parking availability information in real time and sends it to the terminal.

[1431] When a user searches for a parking space, the device sends the current location and destination information to the server (input: location information). The server collects information on available parking spaces in real time, selects the most suitable parking space, and sends it to the device (output: parking suggestion).

[1432] Step 12:

[1433] The terminal presents the parking lot information to the user.

[1434] The terminal receives the parking information sent from the server and presents it to the user (input: parking suggestion). The user can then park accordingly (output: parking completed).

[1435] (Application example 2)

[1436] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1437] Conventional travel support systems have difficulty proposing optimal schedules that take into account the user's emotions and physical condition, and responding flexibly based on real-time information during the trip. In particular, there is a need for appropriate responses to sudden changes in physical condition and emotional fluctuations during travel. Furthermore, to improve the customer experience in stores, flexible suggestions based on the customer's emotional and physical conditions are necessary, but current systems are inadequate in this regard. Therefore, there is a need for a system that can provide users with a comfortable and safe travel experience while also improving customer satisfaction in stores.

[1438] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input a travel destination and itinerary; a means for transmitting the input information to the server; a means for the server to generate multiple optimal travel schedules using a generative AI model based on the input information; a means for transmitting the generated schedules to a terminal and displaying them; a means for analyzing the emotional state from the user's facial expressions and voice using an emotion recognition engine; and a means for adjusting the schedule and making appropriate suggestions based on the emotional state. This makes it possible to monitor the user's physical condition and emotional state during the trip in real time and always provide optimal schedules and suggestions.

[1439] A "travel destination" is a geographic location that a user wishes to visit.

[1440] "Itinerary" refers to a time plan such as a travel period or visiting schedule.

[1441] A "server" is a computer system that processes data and provides information over a network.

[1442] A "generative AI model" is an artificial intelligence algorithm or program that generates optimal schedules and proposals based on input information.

[1443] A "terminal" is a device used by a user, such as a smartphone or smart glasses.

[1444] An "emotion recognition engine" is software or hardware that analyzes a user's emotional state from their facial expressions and voice.

[1445] "Facial expressions" are used to read emotions from changes and movements of the user's face.

[1446] "Voice" refers to reading emotions from the tone and patterns of the user's voice.

[1447] "Physical condition" indicates the physical condition of the user, such as their health condition and fatigue level.

[1448] A "schedule" is a plan or timeline for a trip or activity.

[1449] "Suggestions" are information or advice that suggest optimal actions or options for the user.

[1450] "Traffic information" is data showing real-time conditions of traffic volume and means of transportation.

[1451] "Weather information" is data that indicates the weather conditions at your travel destination or current location.

[1452] "Facial recognition" is a technology that uses a camera to identify a user's face and analyze its features.

[1453] A "warning" is a notification to the user of a sign of danger or abnormality.

[1454] A "resting place" is a place where a user can rest and relax during their trip.

[1455] A "parking lot" is a space or facility for parking automobiles.

[1456] "Crowding situation" refers to the degree of crowding or congestion in a particular place or facility.

[1457] The system of the present invention begins when the user inputs their travel destination and itinerary. When the user inputs the destination and itinerary using a device such as a smartphone or smart glasses, the device sends this information to a server. The server uses a generative AI model based on the received information to generate multiple optimal travel itineraries. These schedules are sent to the device and displayed to the user. The user can then select the most appropriate one from the presented schedules.

[1458] The system periodically obtains weather and traffic information and evaluates its impact on the current schedule. Based on the results of this evaluation, it generates a new schedule and proposes it to the device. The device then notifies the user of the proposal, allowing them to review and select the new schedule.

[1459] The device also uses a camera to recognize the user's face and monitor their physical condition. If an abnormality is detected, the device will display a warning to the user and suggest appropriate rest areas and preventive measures. Furthermore, an emotion recognition engine can analyze the user's emotional state from their facial expressions and voice, adjusting their schedule accordingly and making appropriate suggestions.

[1460] The system also obtains real-time information on congestion and parking availability at places the user visits during their trip and makes appropriate suggestions based on that information. For example, if a store is crowded, it can suggest a different time or store. Parking availability is also checked in real time, and the system suggests the best parking spot for the user.

[1461] For example, a user might enter, "I'm looking for new sneakers and would like to know about the latest promotions." The AI ​​model will then provide optimal suggestions for the latest sneakers and promotions. It will also suggest alternative options if the store is crowded, and provide real-time parking information.

[1462] Examples of prompts include:

[1463] "A customer is looking for sneakers and wants to know about the latest promotions. Generate the best suggestions."

[1464] This system allows users to adjust their schedules optimally based on real-time information while traveling, and also improves the customer experience in stores, enabling flexible responses based on physical and emotional states, allowing for safe and comfortable travel and shopping experiences.

[1465] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1466] Step 1: The user enters the travel destination and dates.

[1467] The user launches an application that can be used on a smartphone or smart glasses and inputs their travel destination and itinerary. This information is collected and saved on the device. (Input: Travel destination, itinerary) (Output: Data on the device)

[1468] Step 2: The device sends the input information to the server.

[1469] The terminal sends the entered travel destination and itinerary information to the server. (Input: Data on the terminal) (Output: Data sent to the server)

[1470] Step 3: The server generates a schedule using the generative AI model.

[1471] The server uses the generative AI model to generate multiple optimal travel schedules based on the received travel destination and itinerary information. It also takes into account information collected from the internet about tourist attractions, facilities, events, and other information about the travel destination. (Input: Travel destination, itinerary) (Output: Optimal travel schedule)

[1472] Step 4: The server sends the generation schedule to the terminal.

[1473] The server sends the generated schedules to the terminal and displays them on the terminal. (Input: Optimal travel schedule) (Output: Data sent to the terminal)

[1474] Step 5: The terminal displays the schedule to the user.

[1475] The terminal displays the received schedule to the user, who then selects the optimal schedule from multiple options. (Input: Optimal travel schedule) (Output: Schedule displayed to user)

[1476] Step 6: The device periodically collects weather and traffic information.

[1477] The device periodically collects weather and traffic information and sends it to the server. (Input: Weather information, traffic information) (Output: Data sent to the server)

[1478] Step 7: The server adjusts the schedule based on the real-time information.

[1479] The server analyzes the weather and traffic information sent from the device, and if it affects the current schedule, it generates a new schedule and sends it to the device. (Input: Weather information, traffic information) (Output: Adjusted schedule)

[1480] Step 8: The terminal notifies the user of the adjusted schedule.

[1481] The terminal notifies the user of the new schedule, and the user confirms and approves the schedule changes. (Input: Adjusted schedule) (Output: Notification to user)

[1482] Step 9: The device uses the camera to perform facial recognition and health monitoring.

[1483] The device uses a camera to recognize the user's face and uses AI to monitor their physical condition. If an abnormality is detected, a warning is displayed. (Input: User's face image) (Output: Physical condition evaluation, warning display)

[1484] Step 10: The device uses an emotion recognition engine to analyze the user's facial expressions and voice.

[1485] The device analyzes the user's facial expressions and voice using an emotion recognition engine to infer their emotional state. It then adjusts the schedule based on their emotional state and makes appropriate suggestions. (Input: User's facial expression data, voice data) (Output: Emotional state assessment, adjusted schedule)

[1486] Step 11: The device collects congestion information and parking availability.

[1487] The device collects real-time information about congestion at the destination and parking availability, and sends it to the server to make optimal recommendations. (Input: congestion information, parking availability) (Output: data sent to the server)

[1488] Step 12: The server generates a proposal based on the new information.

[1489] The server analyzes the information sent in real time and generates new suggestions to help users enjoy their trip or shopping. (Input: congestion information, parking information) (Output: new suggestions)

[1490] Step 13: The terminal notifies the user of the new offer.

[1491] The device notifies the user of the new proposal. The user confirms the proposal and selects the next action. (Input: New proposal) (Output: Notification to user)

[1492] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1493] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1494] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1495] [Fourth embodiment]

[1496] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1497] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1498] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1499] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1500] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1501] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1502] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1503] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1504] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1505] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1506] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1507] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1508] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1509] overview

[1510] The smart tour guide app of this invention is a system that improves the efficiency of travel planning and provides flexible support during the trip. This system allows users to input their travel destinations and itineraries, and generates an optimal schedule using a generative AI model based on that information. It also obtains real-time weather and traffic information during the trip, and suggests or changes the schedule depending on the situation. Furthermore, the device also has the function of managing the user's health and checking the availability of parking spaces.

[1511] The user inputs the travel destination and dates

[1512] 1. The user launches the Smart Tour Guide app and enters their travel destination and dates.

[1513] 2. The terminal sends the entered information to the server.

[1514] The server generates the schedule

[1515] 1. The server calls the generative AI model based on the received travel destination and itinerary.

[1516] 2. The server collects information on tourist attractions, restaurants, events, etc. at the travel destination and generates several optimal schedules based on this information.

[1517] 3. The generated schedule is sent to the terminal and displayed to the user, who can then select the best one from the multiple schedules.

[1518] Get real-time information and schedule changes while traveling

[1519] 1. The device periodically collects weather and traffic information and sends it to the server.

[1520] 2. The server analyzes the real-time information and generates a new schedule if it affects the current schedule.

[1521] 3. If a new schedule is required, the server sends it to the terminal, which notifies the user, who then checks the new schedule and approves the changes.

[1522] User health management

[1523] 1. The device uses a camera to recognize the user's face and monitor their physical condition.

[1524] 2. If an abnormality or drowsiness is detected, the device will display a warning to the user and suggest rest or preventive measures.

[1525] Check parking availability

[1526] 1. When a user searches for a parking space, the device queries the server for nearby parking information based on the current location and destination.

[1527] 2. The server collects real-time information on available parking spaces in the surrounding area and selects the most suitable parking space.

[1528] 3. The selected parking lot information is sent to the terminal and suggested to the user, who then parks the car in the designated parking lot.

[1529] Specific examples

[1530] Suppose a user launches a travel app and enters "Tokyo" and "May 1st to 3rd." The device sends this information to a server, which uses a generative AI model to create multiple schedules. For example, a schedule might be generated in which the user visits tourist spot A on day 1, tourist spot B and restaurant C on day 2, and so on. This schedule is then sent to the device and displayed to the user.

[1531] If it starts raining while the user is traveling to tourist spot A, the device automatically updates the weather information and reports it to the server. The server generates a new schedule that changes outdoor activities to indoor sightseeing and sends it to the device. The user can then review and select the new schedule.

[1532] Furthermore, if a user begins to feel drowsy while driving, the device's camera will detect frequent blinking and display a warning, allowing the user to take a break at a nearby rest spot.

[1533] When searching for a parking space, the user searches for a parking space around Tokyo Station, and the server finds a suitable parking space and sends information about its availability to the terminal. Based on this information, the user can park in the most optimal location.

[1534] In this way, the present invention allows users to enjoy traveling efficiently and comfortably.

[1535] The processing flow will be explained below.

[1536] A flow where the user inputs the travel destination and dates and generates a schedule

[1537] Step 1:

[1538] The user launches the Smart Tour Consulting app on their device.

[1539] Step 2:

[1540] The user inputs the travel destination and dates.

[1541] Step 3:

[1542] The terminal transmits the travel destination and itinerary input by the user to the server.

[1543] Step 4:

[1544] The server receives the travel destination and itinerary information.

[1545] Step 5:

[1546] The server calls the generated AI model and collects information on tourist attractions, restaurants, events, etc. at the travel destination.

[1547] Step 6:

[1548] Based on the information collected by the server, multiple optimal schedules are generated.

[1549] Step 7:

[1550] The server transmits the generated schedule to the terminal.

[1551] Step 8:

[1552] The terminal displays the generated schedule to the user.

[1553] Step 9:

[1554] The user selects the best schedule from among multiple schedules.

[1555] Get real-time information and schedule changes while traveling

[1556] Step 1:

[1557] The device periodically collects weather and traffic information.

[1558] Step 2:

[1559] The device sends the collected weather and traffic information to the server.

[1560] Step 3:

[1561] The server receives and analyzes the real-time information.

[1562] Step 4:

[1563] The server evaluates whether the current schedule is affected.

[1564] Step 5:

[1565] The server generates new schedules as needed.

[1566] Step 6:

[1567] The server sends the new schedule to the terminal.

[1568] Step 7:

[1569] The terminal notifies the user of the new schedule.

[1570] Step 8:

[1571] The user reviews and approves the proposed schedule changes.

[1572] User health management and warning function

[1573] Step 1:

[1574] The device uses a camera to recognize the user's face and monitor their physical condition.

[1575] Step 2:

[1576] The device uses facial recognition technology to detect signs of drowsiness in the user (e.g., frequent blinking, rubbing of eyes, etc.).

[1577] Step 3:

[1578] If the device detects an abnormality, it displays a warning to the user.

[1579] Step 4:

[1580] The device will suggest appropriate rest areas and preventive measures to the user.

[1581] Check parking availability

[1582] Step 1:

[1583] A user searches for parking near a destination.

[1584] Step 2:

[1585] The terminal queries the server for information about nearby parking lots based on the user's current location and destination.

[1586] Step 3:

[1587] The server collects real-time information on available parking spaces in the surrounding area.

[1588] Step 4:

[1589] The server selects the best parking space.

[1590] Step 5:

[1591] The server transmits the selected parking lot information to the terminal.

[1592] Step 6:

[1593] The terminal proposes optimal parking information to the user.

[1594] Through the above steps, the present invention provides users with more efficient travel planning, flexible travel arrangements, health management, and parking lot management.

[1595] Example 1

[1596] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1597] In today's world, planning and executing a trip involves many variables, so a flexible and efficient travel planning system is necessary to ensure a comfortable and stress-free trip. Conventional travel planning systems have had difficulty adequately meeting diverse needs, such as optimizing schedules, responding to unexpected changes in circumstances, managing the user's health, and checking parking availability. The present invention aims to solve these problems and provide users with a fulfilling travel experience.

[1598] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1599] In this invention, the server includes a means for a user to input travel destinations and itineraries, a means for transmitting the input information to the server, a means for the server to generate multiple optimal travel schedules using a generative AI model based on the input information, and a means for transmitting the generated schedules to a terminal and displaying them, thereby enabling users to easily create optimal travel plans.

[1600] Furthermore, the server includes means for periodically acquiring weather and traffic information, means for evaluating the impact on the current schedule based on the acquired information, and means for generating a new schedule and proposing it to the terminal, thereby enabling the system to flexibly respond to unexpected changes in circumstances.

[1601] Furthermore, the terminal includes a means for recognizing the user's face and monitoring their physical condition using a camera, a means for detecting abnormalities based on the monitoring results and displaying a warning, and a means for suggesting rest areas and preventive measures, thereby protecting the user's safety and health.

[1602] Furthermore, the system includes a means for the terminal to search for available parking spaces and inquire of the server, a means for the server to select an appropriate parking space based on the current location and destination, and a means for transmitting information about the selected parking space to the terminal and suggesting it to the user, thereby efficiently solving parking problems.

[1603] In addition, the system includes a means for generating a prompt sentence for the server to call the generative AI model, a means for generating a schedule using the generative AI model based on the prompt sentence, and a means for presenting the schedule to the user as multiple options, thereby providing the user with a variety of options and realizing more satisfying travel plans.

[1604] "User" refers to a person who uses the system to provide input to plan a trip.

[1605] "Destination" refers to a geographic location that a user plans to visit.

[1606] "Schedule" refers to the start and end dates of the trip planned by the user, as well as the duration of the trip.

[1607] "Terminal" refers to the digital device that a user uses to enter their travel plans and receive system results.

[1608] "Server" refers to a remote computer system that receives user input, processes it, and sends the results to the terminal.

[1609] "Generative AI Model" refers to an artificial intelligence model used to generate travel itineraries based on input information.

[1610] "Schedule" refers to the plan of the itinerary and activities for each day at the travel destination.

[1611] "Weather information" refers to data about the weather around your travel destination.

[1612] "Traffic information" refers to data regarding traffic conditions around the travel destination.

[1613] "Facial recognition" refers to technology that uses a camera to identify a user's face.

[1614] "Physical condition" refers to information for monitoring the user's health condition and fatigue level.

[1615] "Parking Availability" refers to information about available parking spaces within a specified area.

[1616] "Prompt" refers to a textual instruction used to invoke a generative AI model.

[1617] overview

[1618] The smart tour guide system of the present invention is a system that improves the efficiency of travel planning and provides flexible support during the trip. The system allows users to input their travel destinations and itineraries, and generates an optimal schedule based on that information using a generative AI model on the server. It also obtains weather and traffic information during the trip in real time, and proposes or changes the schedule depending on the situation. Furthermore, the terminal also has the function of managing the user's health and checking the availability of parking spaces.

[1619] Hardware and software used

[1620] The main hardware and software used to implement this system are as follows:

[1621] User device: Mobile device such as smartphone or tablet (e.g. iOS, Android device)

[1622] Server: Cloud-based computing resources (e.g., Amazon Web Services, Google Cloud Platform)

[1623] Generative AI model: a generative AI algorithm (e.g., GPT-4)

[1624] External API: Weather API, Google Maps API, Yelp API, Parkopedia API

[1625] Facial recognition algorithm: TensorFlow Lite, Apple Core ML

[1626] Explanation of program processing

[1627] User inputs travel destination and dates

[1628] The user launches the smart tour guide app and enters their travel destination and itinerary in the form displayed on the UI. For example, they enter "Tokyo" and "May 1st to 3rd." The device then sends this information in JSON format to the server.

[1629] Server-based schedule generation

[1630] The server generates a prompt to call the generative AI model based on the received travel destination and itinerary information. For example, the prompt might look like this:

[1631] Travel Planner

[1632] Travel destination: Tokyo

[1633] Dates: May 1st to 3rd

[1634] The server inputs this prompt into a generative AI model and generates multiple optimal schedules. For example, the generated schedules are as follows:

[1635] Day 1: Visit Sensoji Temple in the morning, Ueno Zoo in the afternoon

[1636] Day 2: Visit Sensoji Temple in the morning and Ueno Zoo in the afternoon

[1637] The generated schedule is sent from the server in JSON format to the device and displayed on the device's UI. The user can then select the most suitable schedule from multiple options.

[1638] Get real-time information and schedule changes while traveling

[1639] The device periodically collects weather and traffic information through APIs and sends that information to the server. For example, weather information is obtained from the Weather API, and traffic information is obtained from the Google Maps API. The server analyzes the real-time information and generates a new schedule if it affects the current schedule. If a change is determined to be necessary, the new schedule is sent to the device and the user is notified.

[1640] User health management

[1641] The device monitors the user's face using a built-in camera. For example, it uses a camera sensor and a facial recognition algorithm (e.g., TensorFlow Lite). It estimates the user's physical condition from the user's complexion, blinking frequency, and facial expression, and displays an alert if an abnormality is detected. For example, it displays a message such as, "Are you feeling sleepy? There is a rest area nearby."

[1642] Check parking availability

[1643] The user uses the app's "Parking Lot Search" function to input their current location and destination information. The device sends this information to the server, which then uses a parking API (e.g., Parkopedia API) to obtain information about available parking lots in the area. Based on this information, the server selects the most suitable parking lot and sends it back to the device.

[1644] Specific examples

[1645] Suppose a user launches a travel app and enters "Tokyo" and "May 1st to 3rd." The device sends this information in JSON format to the server, which then uses a generative AI model to create multiple schedules. For example, a schedule might be generated in which the user visits tourist spot A on the first day, tourist spot B and restaurant C on the second day, and so on. This schedule is then sent to the device and displayed to the user.

[1646] If it starts raining while the user is traveling to tourist spot A, the device automatically updates the weather information and reports it to the server. The server generates a new schedule that changes outdoor activities to indoor sightseeing and sends it to the device. The user can then review and select the new schedule.

[1647] Furthermore, if the user begins to feel drowsy while driving, the device's camera will detect frequent blinking and display a warning. This warning will allow the user to take a rest at a nearby rest spot. When searching for parking, if the user searches for a parking lot near Tokyo Station, the server will find a suitable parking lot and send information about its availability to the device. The user can then park in the most optimal location based on this information.

[1648] In this way, the present invention allows users to enjoy traveling efficiently and comfortably.

[1649] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1650] Step 1:

[1651] The user enters the travel destination and itinerary. The user launches the smart tour guide app and enters the travel destination (e.g., Tokyo) and travel dates (e.g., May 1st to 3rd) in the form displayed in the UI. The input data is temporarily saved in the device's memory.

[1652] Input: Travel destination and itinerary information

[1653] Output: Travel destination and itinerary data in memory

[1654] Step 2:

[1655] The device sends the input information to the server, which converts the input data of the travel destination and itinerary into JSON format and sends it to the server using the HTTPS protocol, using Wi-Fi or mobile data connection.

[1656] Input: Travel destination and itinerary data in memory

[1657] Output: Data sent to the server in JSON format

[1658] Step 3:

[1659] The server generates the schedule. Based on the received travel destination and schedule information, the server generates a prompt to call the generative AI model. For example, the prompt might say, "The travel destination is Tokyo, and the dates are three days from May 1st to 3rd. Please generate a schedule including tourist spots, restaurants, and events." This prompt is input into the generative AI model to generate the schedule.

[1660] Input: Received travel destination and itinerary information

[1661] Output: Multiple schedules generated by the generative AI model

[1662] Step 4:

[1663] The server sends the generated schedule to the terminal. The generated schedules are converted into JSON format and sent to the terminal using the HTTPS protocol.

[1664] Input: Multiple generated schedules

[1665] Output: Schedule data sent to the device in JSON format

[1666] Step 5:

[1667] The device displays the schedule to the user. The received schedule is displayed on the UI, and the user can select the most appropriate one from multiple options. The schedule is displayed in a visually easy-to-understand list or calendar format.

[1668] Input: Received schedule data

[1669] Output: The schedule displayed on the user interface

[1670] Step 6:

[1671] Obtaining real-time information and changing schedules during travel. The device periodically obtains weather and traffic information via API (for example, using the Weather API or Google Maps API). The collected information is sent to the server, which analyzes the real-time information. If the current schedule is affected, a new schedule is generated and sent to the device. The user can review the new schedule and approve the changes.

[1672] Input: Weather and traffic information

[1673] Output: New schedule as needed

[1674] Step 7:

[1675] User health management. The device uses a built-in camera to recognize the user's face and monitor their physical condition. For example, it estimates their physical condition from factors such as facial color, blinking frequency, and facial expressions. If an abnormality is detected, it displays a warning and suggests rest areas and preventive measures.

[1676] Input: Face data captured from the camera

[1677] Output: Warning messages and suggestions for the user

[1678] Step 8:

[1679] Checking parking availability. The user uses the app's "parking search" function to input their current location and destination information. The device sends this information to the server, which then uses the parking API to obtain information on available parking in the surrounding area. The server then returns the optimal parking information to the device and makes a suggestion to the user.

[1680] Input: Current location and destination information

[1681] Output: Appropriate parking information

[1682] (Application example 1)

[1683] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1684] For food delivery services, improving delivery efficiency and managing the safety of delivery personnel are important. However, challenges include dynamically changing delivery schedules based on real-time weather and traffic information, managing the health of delivery personnel, and checking the availability of parking spaces around the destination. These factors can lead to delivery delays and a decline in service quality. Furthermore, if delivery personnel's fatigue and poor health are not adequately managed, the risk of accidents and delivery errors increases.

[1685] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1686] In this invention, the server includes a means for a user to input a delivery destination and delivery date and time, a means for transmitting the input information to the server, a means for the server to generate multiple optimal delivery schedules using a generative AI model based on the input information, and a means for transmitting the generated schedules to a terminal and displaying them, thereby enabling efficient and flexible generation of delivery schedules.

[1687] The server also includes a means for periodically acquiring weather and traffic information, a means for evaluating the impact of the acquired information on the current schedule, and a means for generating a new schedule and proposing it to the terminal, thereby enabling schedule adjustments in response to real-time changes in the situation.

[1688] The terminal also has a means for using a camera to recognize the face and monitor the physical condition of the delivery person, a means for detecting abnormalities and displaying a warning based on the monitoring results, and a means for suggesting rest areas and preventive measures, which will enable the health management and safety of delivery people to be ensured.

[1689] Furthermore, the server acquires the availability of parking spaces around the destination in real time, and includes a means for suggesting the most suitable parking space to the terminal based on the availability, thereby enabling the delivery person to find a parking space efficiently and quickly.

[1690] "User" refers to the person who uses or operates the delivery service.

[1691] "Delivery Destination" means the destination to which goods or items are to be delivered.

[1692] "Delivery Date and Time" means the date and time when the goods or items are to be delivered to their destination.

[1693] "Server" refers to a central computer system that processes data and collects and processes various types of information.

[1694] A "generative AI model" refers to an algorithm that uses artificial intelligence to suggest optimal schedules and solutions.

[1695] "Terminal" refers to a device or equipment used by a user, such as a smartphone or tablet.

[1696] A "schedule" is a timetable showing the order and plan of deliveries.

[1697] "Weather Information" means data regarding current and future weather conditions.

[1698] "Traffic information" refers to the latest data on road traffic, such as road congestion and traffic regulations.

[1699] "Facial recognition" refers to a technology that uses a camera to identify a person's face and then processes that information.

[1700] Monitoring "health status" refers to the process of monitoring the health status of delivery personnel in real time and detecting any abnormalities.

[1701] "Warning" refers to a notification that alerts the user when the system detects an abnormality.

[1702] "Rest Area" means a place where a delivery person temporarily stops to take rest.

[1703] "Preventive measures" refer to measures taken to prevent delivery personnel from becoming ill or having accidents.

[1704] "Parking Space" means a location available for a Delivery Person to temporarily park their vehicle.

[1705] "Availability" refers to information about whether a parking space is available.

[1706] "Real-time" means that the latest information is obtained and updated almost instantly.

[1707] The present invention relates to a system for improving delivery efficiency and managing the safety of delivery personnel in a food delivery service. Detailed embodiments of the present invention will be described below.

[1708] System Program Overview

[1709] This system includes a means for the user to input the delivery destination and delivery date and time, a means for transmitting the input information to a server, a means for generating an optimal delivery schedule using a generative AI model, and a means for transmitting and displaying the generated schedule to a terminal. The server also has a means for periodically obtaining weather and traffic information, evaluating the impact on the current schedule, generating a new schedule, and proposing it to the terminal. The terminal also has a means for recognizing the delivery person's face and monitoring their physical condition, detecting abnormalities, displaying a warning, and suggesting rest areas and preventive measures. It also includes a means for obtaining real-time information on available parking spaces around the destination and proposing the optimal parking space to the terminal.

[1710] Program processing and the hardware and software used

[1711] 1. User Input and Information Submission

[1712] Hardware: Smartphone or tablet

[1713] Software: User Interface Application

[1714] Processing: The user inputs the delivery address and delivery date and time. The input information is sent from the terminal to the server.

[1715] 2. Schedule generation using generative AI models

[1716] Hardware: Server

[1717] Software: Generative AI models (e.g., OpenAI GPT-3)

[1718] Processing: The server calls the generative AI model based on the received delivery destination and date and time information. The prompt "Generate the optimal delivery schedule for arriving at Tokyo Station at 12:00 on May 1, 2023" is entered, and the AI ​​model generates the optimal schedule. The generated schedule is sent to the device and displayed to the user.

[1719] 3. Real-time information acquisition and schedule updates

[1720] Hardware: Server

[1721] Software: Weather information API, traffic information API

[1722] Processing: The server periodically obtains weather and traffic information using an API. Based on the obtained information, it evaluates the impact on the current schedule and generates a new schedule. An example of a prompt sentence in this case is "Please update the delivery route taking into account the current weather (rain) and traffic conditions (congestion)." The new schedule is then resent to the device.

[1723] 4. Managing the health of delivery personnel

[1724] Hardware: Smartphone or tablet with a camera

[1725] Software: OpenCV (open source computer vision library)

[1726] Processing: The device uses a camera to recognize the delivery person's face and monitor their physical condition. If an abnormality is detected, it displays a warning and suggests resting places and preventive measures. For example, it displays a notification saying, "Frequent blinking has been detected. For safety reasons, we suggest you take a break at a nearby rest facility."

[1727] 5. Check availability of parking spaces

[1728] Hardware: Server

[1729] Software: Parking Information API

[1730] Processing: The server obtains the availability of parking spaces near the delivery destination in real time and suggests the most suitable parking location to the terminal. An example of a prompt sentence in this case is "Please search for available parking spaces around Tokyo Station in real time and suggest the most suitable parking location."

[1731] Specific examples

[1732] When a user uses their smartphone to enter "Tokyo Station" and "May 1, 2023, 12:00," the device sends this information to the server. The server uses a generative AI model to generate an optimal delivery schedule and sends this schedule to the device. If the weather worsens and traffic congestion occurs during the trip, the server obtains new real-time information, updates the schedule, and resends it to the device.

[1733] Furthermore, if a delivery person feels unwell during a delivery, the device's camera will use facial recognition to detect the abnormality and display a warning. This warning allows the delivery person to take a break safely. If the delivery person needs to find a parking space around Tokyo Station, the server will obtain the most suitable parking space in real time and send that information to the device, helping the delivery person park efficiently.

[1734] In this way, the present invention improves the delivery efficiency and safety of delivery personnel of food delivery services.

[1735] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1736] Step 1:

[1737] The user enters the delivery address and delivery date and time. The user uses a smartphone or tablet to enter the delivery address and delivery date and time into the application. This input information is sent from the device to the server. Input: Delivery address and delivery date and time. Output: Delivery information sent to the server.

[1738] Step 2:

[1739] The server uses the generative AI model to generate an optimal delivery schedule. Based on the delivery destination and date and time information received, the server sends a prompt to the generative AI model to generate an optimal schedule. An example of a specific prompt is, "Please generate an optimal delivery schedule that arrives at Tokyo Station at 12:00 on May 1, 2023." Input: Delivery information and prompt. Output: Optimal delivery schedule.

[1740] Step 3:

[1741] The generated schedule is sent to the terminal and displayed to the user. The server sends the generated schedule to the terminal, and the terminal displays it to the user. Input: Generated schedule. Output: Schedule displayed on the terminal.

[1742] Step 4:

[1743] The server periodically obtains weather and traffic information. The server obtains weather and traffic information in real time using the weather information API and traffic information API. Input: Requests to the weather information API and traffic information API. Output: Real-time weather and traffic information.

[1744] Step 5:

[1745] The impact on the current schedule is evaluated based on the acquired information. The server analyzes the acquired weather and traffic information and evaluates the impact on the current schedule. For example, if it starts to rain or traffic congestion occurs, it evaluates whether the current schedule will be delayed. Input: Real-time weather information and traffic information. Output: Result of the impact evaluation on the schedule.

[1746] Step 6:

[1747] If a new schedule needs to be generated and proposed to the device, the server again sends a prompt to the generation AI model to generate a new schedule. An example of a specific prompt is, "Please update the delivery route taking into account the current weather (rain) and traffic conditions (traffic jam)." The new schedule generated is sent to the device and proposed to the user. Input: Schedule impact assessment results and prompt. Output: New schedule.

[1748] Step 7:

[1749] The device uses a camera to recognize the delivery person's face and monitor their physical condition. The device uses a camera to recognize the delivery person's face and detects frequent blinking and signs of fatigue. Input: Camera image. Output: Results of facial recognition and physical condition monitoring.

[1750] Step 8:

[1751] If an abnormality is detected, the device will display a warning and suggest rest areas or preventive measures. For example, it may display a notification saying, "Frequent blinking has been detected. For safety reasons, we suggest you take a rest at a nearby rest area." Input: Monitoring results. Output: Warning display and suggested rest areas.

[1752] Step 9:

[1753] The server obtains the availability of parking spaces around the destination in real time and suggests the most suitable parking space to the terminal. The server uses the parking information API to obtain the availability of parking spaces in real time and suggests the most suitable parking space. An example of a specific prompt is "Please search for available parking spaces around Tokyo Station in real time and suggest the most suitable parking space." Input: Request to the parking information API. Output: Availability of parking spaces and suggested parking information.

[1754] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1755] overview

[1756] The smart tour guide app, which incorporates the emotion engine of the present invention, is a system that improves the efficiency of travel planning and provides flexible responses during the trip. This system allows users to input their travel destinations and itineraries, and generates an optimal schedule using a generative AI model based on that information. It also obtains real-time weather and traffic information during the trip, and proposes and changes the schedule depending on the situation. Furthermore, in addition to the terminal's functions of managing the user's health and checking parking availability, it is equipped with an emotion engine that recognizes the user's emotions and makes appropriate proposals and changes based on the user's emotional state.

[1757] A flow where the user inputs the travel destination and dates and generates a schedule

[1758] 1. The user launches the Smart Tour Consulting app on their device and enters their travel destination and dates.

[1759] 2. The terminal sends the entered information to the server.

[1760] 3. The server calls the generative AI model based on the received travel destination and itinerary information, and collects information on tourist attractions, restaurants, events, etc. at the travel destination.

[1761] 4. Based on the information collected by the server, multiple optimal schedules are generated.

[1762] 5. The generated schedule is sent to the terminal and displayed to the user, who can then select the best one from the multiple schedules.

[1763] Get real-time information and schedule changes while traveling

[1764] 1. The device periodically collects weather and traffic information and sends it to the server.

[1765] 2. The server analyzes the real-time information and generates a new schedule if it affects the current schedule.

[1766] 3. If a new schedule is required, the server sends it to the terminal, which notifies the user, who then checks the new schedule and approves the changes.

[1767] User health management

[1768] 1. The device uses a camera to recognize the user's face and monitor their physical condition.

[1769] 2. If an abnormality or drowsiness is detected, the device will display a warning to the user and suggest rest or preventive measures.

[1770] Recognizing and responding to user emotions

[1771] 1. The emotion engine installed in the device analyzes the user's facial expressions, voice, etc. to recognize their emotional state.

[1772] 2. The device suggests an optimal travel schedule based on the user's emotional state.

[1773] 3. If a schedule change is required during the trip, the emotion engine generates a new schedule taking into account the emotional state.

[1774] 4. The device will suggest suitable places to rest and relax based on your emotional state.

[1775] Check parking availability

[1776] 1. When a user searches for a parking space, the device queries the server for nearby parking information based on the current location and destination.

[1777] 2. The server collects real-time information on available parking spaces in the surrounding area and selects the most suitable parking space.

[1778] 3. The selected parking lot information is sent to the terminal and suggested to the user, who then parks the car in the designated parking lot.

[1779] Specific examples

[1780] Suppose a user launches a travel app and enters "Tokyo" and "May 1st to 3rd." The device sends this information to a server, which uses a generative AI model to create multiple schedules. For example, a schedule might be generated in which the user visits tourist spot A on day 1, tourist spot B and restaurant C on day 2, and so on. This schedule is then sent to the device and displayed to the user.

[1781] If it starts to rain while the user is traveling to tourist spot A, the device automatically updates the weather information and reports it to the server. The server takes the rain into account and generates a new schedule that changes the itinerary to indoor sightseeing and sends it to the device. The user can then review and select the new schedule.

[1782] Furthermore, if a user begins to feel drowsy while driving, the device's camera will detect frequent blinking and display a warning, allowing the user to take a break at a nearby rest spot.

[1783] When a user searches for a parking space, the server collects information on available parking spaces in real time and selects the most suitable parking space. The device then suggests an appropriate parking space, and the user parks accordingly.

[1784] Furthermore, if the user feels stressed during their trip, the emotion engine will analyze the user's facial expressions and voice, and if it determines that relaxation is necessary, the device will suggest a spot where they can relax. For example, if the user feels tired, the device will suggest a nearby cafe.

[1785] In this way, the present invention allows users to enjoy traveling efficiently and comfortably, and also enables flexible responses according to the user's emotional state and physical condition, providing a safer and more satisfying travel experience.

[1786] The processing flow will be explained below.

[1787] A flow where the user inputs the travel destination and dates and generates a schedule

[1788] Step 1:

[1789] The user launches the Smart Tour Consulting app on their device.

[1790] Step 2:

[1791] The user inputs the travel destination and dates.

[1792] Step 3:

[1793] The terminal transmits the entered travel destination and itinerary information to the server.

[1794] Step 4:

[1795] The server receives the travel destination and itinerary information.

[1796] Step 5:

[1797] The server calls the generated AI model and collects information on tourist attractions, restaurants, events, etc. at the travel destination.

[1798] Step 6:

[1799] Based on the information collected by the server, multiple optimal schedules are generated.

[1800] Step 7:

[1801] The server transmits the generated schedule to the terminal.

[1802] Step 8:

[1803] The terminal displays the generated schedule to the user.

[1804] Step 9:

[1805] The user selects the best schedule from among multiple schedules.

[1806] Get real-time information and schedule changes while traveling

[1807] Step 1:

[1808] The device periodically collects weather and traffic information.

[1809] Step 2:

[1810] The device sends the collected weather and traffic information to the server.

[1811] Step 3:

[1812] The server receives and analyzes the real-time information.

[1813] Step 4:

[1814] The server evaluates whether the current schedule is affected.

[1815] Step 5:

[1816] The server generates new schedules as needed.

[1817] Step 6:

[1818] The server sends the new schedule to the terminal.

[1819] Step 7:

[1820] The terminal notifies the user of the new schedule.

[1821] Step 8:

[1822] The user reviews and approves the proposed schedule changes.

[1823] User health management and warning function

[1824] Step 1:

[1825] The device uses a camera to recognize the user's face and monitor their physical condition.

[1826] Step 2:

[1827] The device uses facial recognition technology to detect signs of drowsiness in the user (e.g., frequent blinking, rubbing of eyes, etc.).

[1828] Step 3:

[1829] If the device detects an abnormality, it displays a warning to the user.

[1830] Step 4:

[1831] The device will suggest appropriate rest areas and preventive measures to the user.

[1832] Recognizing and responding to user emotions

[1833] Step 1:

[1834] The emotion engine installed in the device analyzes the user's facial expressions and voice to recognize their emotional state.

[1835] Step 2:

[1836] The terminal proposes an optimal travel schedule based on the user's emotional state.

[1837] Step 3:

[1838] If a schedule change is required during the trip, the emotion engine will generate a new schedule taking into account the emotional state.

[1839] Step 4:

[1840] The device will suggest suitable places to rest and relax based on your emotional state.

[1841] Check parking availability

[1842] Step 1:

[1843] A user searches for parking near a destination.

[1844] Step 2:

[1845] The terminal queries the server for information about nearby parking lots based on the user's current location and destination.

[1846] Step 3:

[1847] The server collects real-time information on available parking spaces in the surrounding area.

[1848] Step 4:

[1849] The server selects the best parking space.

[1850] Step 5:

[1851] The server transmits the selected parking lot information to the terminal.

[1852] Step 6:

[1853] The terminal proposes optimal parking information to the user.

[1854] Through these steps, the present invention provides users with more efficient travel planning, flexible travel arrangements, health management, and parking management. Furthermore, by using an emotion engine, personalized responses based on the user's emotional state become possible, resulting in a more satisfying travel experience.

[1855] Example 2

[1856] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1857] Conventional travel planning applications have issues such as ineffective automatic schedule generation based on user input, difficulty in flexible schedule changes that take real-time information into account, a lack of suggestions that take into account the user's physical and emotional state, and an inability to effectively provide information on available parking spaces. These issues not only make the travel experience inefficient and inconvenient, but also lead to a lack of safety.

[1858] The specification process by the specification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for a user to input a travel destination and itinerary, a means for transmitting the input information to the server, a means for the server to generate multiple optimal travel schedules using a generative AI model based on the input information, and a means for transmitting the generated schedules to a terminal and displaying them. This allows the user to enjoy an efficient and comfortable travel experience.

[1859] Furthermore, this invention includes a means for the server to periodically acquire weather and traffic information, a means for evaluating the impact on the current schedule based on the acquired information, a means for generating a new schedule and proposing it to the terminal, a means for monitoring the user's face and physical condition using a camera attached to the terminal, a means for detecting abnormalities based on the monitoring results and displaying a warning, and a means for proposing rest areas and preventive measures, thereby enabling the user's physical condition management and quick schedule adjustments.

[1860] The device further includes a means for analyzing the user's emotional state using an emotion analysis engine built into the device, and a means for proposing optimal travel schedules and rest spots based on the analysis results, thereby enabling optimal suggestions to be made according to the user's emotional state.

[1861] In addition, the system includes a means for users to query the server for information to search for available parking spaces, a means for the server to collect information on available parking spaces in the vicinity in real time and select the most suitable parking space, and a means for transmitting and displaying the selected parking space information to the terminal, thereby enabling users to find parking spaces efficiently.

[1862] A "user" is a person who uses the system to create and execute a travel plan.

[1863] "Destination" refers to a geographic location or city that a user plans to visit.

[1864] "Itinerary" refers to the specific schedule of the travel period, including the start and end dates of the trip.

[1865] "Terminal" refers to an electronic device that a user uses as an interface, such as a smartphone, tablet, or PC.

[1866] "Server" refers to a central processing unit that processes data provided by users and generates travel schedules.

[1867] A "generative artificial intelligence model" refers to an algorithm or method that automatically generates an optimal travel schedule based on input data.

[1868] A "schedule" refers to a specific travel plan including the places to visit, the duration of stay, the means of transportation, etc.

[1869] "Weather Information" refers to data about current and forecast weather and climate.

[1870] "Traffic information" refers to data on current traffic conditions and road congestion.

[1871] "Facial recognition" refers to technology that uses a camera to identify a user's face and detect certain features of it.

[1872] "Physical condition" refers to data indicating the physical condition of the user, such as their health and fatigue level.

[1873] An "emotion analysis engine" refers to software or a system that analyzes a user's facial expressions, voice, etc. to recognize emotions.

[1874] A "rest spot" refers to a place suitable for a user to take a rest.

[1875] "Parking lot" means a designated area for parking vehicles.

[1876] "Availability" refers to the current status of available parking spaces.

[1877] This invention is a system that generates an optimal travel schedule based on user input of travel destinations and dates. The system includes a user terminal, a server, a generative AI model, a camera, and an emotion analysis engine.

[1878] The user operates a device to input the travel destination and itinerary. For example, the device can be a smartphone, tablet, or PC. The information entered by the user is sent from the device to the server as an HTTP POST request. This request includes the travel destination (e.g., "Tokyo") and the travel itinerary (e.g., "May 1st to 3rd").

[1879] The server calls a generative AI model based on the received information to generate an optimal travel schedule. For example, OpenAI's GPT-3 is used as the generative AI model. Here, the following prompt sentence is input to the AI ​​model:

[1880] "A user is traveling to Tokyo from May 1st to 3rd. Please create a three-day travel itinerary including recommended tourist spots and restaurants."

[1881] The generative AI model generates multiple schedules based on this, and the server sends these to the device.

[1882] The terminal presents the received schedule to the user, who can then select the most suitable schedule. The selected schedule is then used for management and display during the trip.

[1883] During a trip, the device periodically collects weather and traffic information and sends it to the server. The server analyzes this information and, if it affects the current schedule, generates a new schedule and sends it to the device. The user receives a notification, checks the new schedule, and approves or changes it as necessary.

[1884] The device also has a built-in camera that can be used to recognize the user's face and monitor their physical condition. If abnormalities or drowsiness (such as frequent blinking) are detected, the device will display a warning and suggest rest areas or preventive measures.

[1885] As an additional function, the device is equipped with an emotion analysis engine. This engine analyzes the user's facial expressions and voice to recognize their emotional state. Based on the emotional state, the device will suggest optimal travel schedules and rest spots. For example, if the user feels stressed, the device will suggest a place where they can relax (such as a cafe).

[1886] Furthermore, when a user searches for a parking space while traveling by car, the device inquires about available parking spaces with the server. The server collects information about nearby parking spaces in real time, selects the most suitable parking space, and sends the information to the device. The user can then park their car in the parking space provided.

[1887] In this way, the system of the present invention provides all kinds of support for users to enjoy an efficient and comfortable trip. By automatically generating schedules using generative AI models, responding in real time based on weather and traffic information, and monitoring the user's physical and emotional state and making optimal suggestions accordingly, users can have a safe and satisfying travel experience.

[1888] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1889] Step 1:

[1890] The user inputs the travel destination and dates.

[1891] Users input their travel destination and itinerary on devices such as smartphones, tablets, and PCs (input: travel destination and itinerary). This information becomes the basis for the user's travel plans. For example, they might input "Tokyo" and "May 1st to 3rd." The input information is processed internally on the device and temporarily saved (output: input information).

[1892] Step 2:

[1893] The terminal transmits the input information to the server.

[1894] The terminal sends the travel destination and itinerary information entered by the user to the server via an HTTP POST request (input: input information). The server receives the request and analyzes the contents (output: received data).

[1895] Step 3:

[1896] The server calls the generative AI model and generates a travel schedule.

[1897] The server calls a generative AI model (e.g., OpenAI's GPT-3) based on the received travel destinations and itinerary (input: received data). Here, the prompt is set as follows: "The user will travel to Tokyo from May 1st to 3rd. Please create a three-day travel schedule including recommended tourist spots and restaurants." The generative AI model outputs multiple travel schedules in response to the prompt (output: generated schedules).

[1898] Step 4:

[1899] The server transmits the generated schedule to the terminal.

[1900] The server sends the generated schedules to the terminal (input: generated schedules). The terminal receives them and saves them as schedule data (output: saved schedules).

[1901] Step 5:

[1902] The terminal displays the generated schedule to the user.

[1903] The terminal displays the received schedules on the screen to the user (input: saved schedules), and the user can view them and select the most suitable schedule (output: selected schedule).

[1904] Step 6:

[1905] The terminal periodically collects weather and traffic information and sends it to the server.

[1906] During the trip, the device periodically collects weather information (weather information service) and traffic information (map API) (input: external information). The collected information is sent to the server and analyzed there (output: analysis results).

[1907] Step 7:

[1908] The server will suggest schedule changes based on the information.

[1909] The server evaluates the impact of collected and analyzed weather and traffic information on the current schedule (input: analysis results). If necessary, it generates a new schedule and proposes it to the terminal (output: new schedule proposal).

[1910] Step 8:

[1911] The terminal notifies the user of the new schedule.

[1912] The terminal receives the new schedule sent from the server and notifies the user (input: new schedule proposal). The user can then review the new schedule and approve or change it (output: user decision).

[1913] Step 9:

[1914] The device uses a camera to monitor the user's physical condition.

[1915] The device's built-in camera recognizes the user's face and monitors their physical condition (e.g., frequent blinking) (input: facial recognition data). If an abnormality is detected, a warning is displayed and suggestions for rest areas or preventive measures are made (output: warning and suggestions).

[1916] Step 10:

[1917] The terminal uses an emotion analysis engine to analyze the user's emotional state.

[1918] The device's built-in emotion analysis engine analyzes the user's facial expressions and voice to recognize their emotional state (input: emotion analysis data). Based on the recognition results, it suggests optimal travel schedules and rest spots (output: emotion-based suggestions).

[1919] Step 11:

[1920] The server collects parking availability information in real time and sends it to the terminal.

[1921] When a user searches for a parking space, the device sends the current location and destination information to the server (input: location information). The server collects information on available parking spaces in real time, selects the most suitable parking space, and sends it to the device (output: parking suggestion).

[1922] Step 12:

[1923] The terminal presents the parking lot information to the user.

[1924] The terminal receives the parking information sent from the server and presents it to the user (input: parking suggestion). The user can then park accordingly (output: parking completed).

[1925] (Application example 2)

[1926] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1927] Conventional travel support systems have difficulty proposing optimal schedules that take into account the user's emotions and physical condition, and responding flexibly based on real-time information during the trip. In particular, there is a need for appropriate responses to sudden changes in physical condition and emotional fluctuations during travel. Furthermore, to improve the customer experience in stores, flexible suggestions based on the customer's emotional and physical conditions are necessary, but current systems are inadequate in this regard. Therefore, there is a need for a system that can provide users with a comfortable and safe travel experience while also improving customer satisfaction in stores.

[1928] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for a user to input a travel destination and itinerary; a means for transmitting the input information to the server; a means for the server to generate multiple optimal travel schedules using a generative AI model based on the input information; a means for transmitting the generated schedules to a terminal and displaying them; a means for analyzing the emotional state from the user's facial expressions and voice using an emotion recognition engine; and a means for adjusting the schedule and making appropriate suggestions based on the emotional state. This makes it possible to monitor the user's physical condition and emotional state during the trip in real time and always provide optimal schedules and suggestions.

[1929] A "travel destination" is a geographic location that a user wishes to visit.

[1930] "Itinerary" refers to a time plan such as a travel period or visiting schedule.

[1931] A "server" is a computer system that processes data and provides information over a network.

[1932] A "generative AI model" is an artificial intelligence algorithm or program that generates optimal schedules and proposals based on input information.

[1933] A "terminal" is a device used by a user, such as a smartphone or smart glasses.

[1934] An "emotion recognition engine" is software or hardware that analyzes a user's emotional state from their facial expressions and voice.

[1935] "Facial expressions" are used to read emotions from changes and movements of the user's face.

[1936] "Voice" refers to reading emotions from the tone and patterns of the user's voice.

[1937] "Physical condition" indicates the physical condition of the user, such as their health condition and fatigue level.

[1938] A "schedule" is a plan or timeline for a trip or activity.

[1939] "Suggestions" are information or advice that suggest optimal actions or options for the user.

[1940] "Traffic information" is data showing real-time conditions of traffic volume and means of transportation.

[1941] "Weather information" is data that indicates the weather conditions at your travel destination or current location.

[1942] "Facial recognition" is a technology that uses a camera to identify a user's face and analyze its features.

[1943] A "warning" is a notification to the user of a sign of danger or abnormality.

[1944] A "resting place" is a place where a user can rest and relax during their trip.

[1945] A "parking lot" is a space or facility for parking automobiles.

[1946] "Crowding situation" refers to the degree of crowding or congestion in a particular place or facility.

[1947] The system of the present invention begins when the user inputs their travel destination and itinerary. When the user inputs the destination and itinerary using a device such as a smartphone or smart glasses, the device sends this information to a server. The server uses a generative AI model based on the received information to generate multiple optimal travel itineraries. These schedules are sent to the device and displayed to the user. The user can then select the most appropriate one from the presented schedules.

[1948] The system periodically obtains weather and traffic information and evaluates its impact on the current schedule. Based on the results of this evaluation, it generates a new schedule and proposes it to the device. The device then notifies the user of the proposal, allowing them to review and select the new schedule.

[1949] The device also uses a camera to recognize the user's face and monitor their physical condition. If an abnormality is detected, the device will display a warning to the user and suggest appropriate rest areas and preventive measures. Furthermore, an emotion recognition engine can analyze the user's emotional state from their facial expressions and voice, adjusting their schedule accordingly and making appropriate suggestions.

[1950] The system also obtains real-time information on congestion and parking availability at places the user visits during their trip and makes appropriate suggestions based on that information. For example, if a store is crowded, it can suggest a different time or store. Parking availability is also checked in real time, and the system suggests the best parking spot for the user.

[1951] For example, a user might enter, "I'm looking for new sneakers and would like to know about the latest promotions." The AI ​​model will then provide optimal suggestions for the latest sneakers and promotions. It will also suggest alternative options if the store is crowded, and provide real-time parking information.

[1952] Examples of prompts include:

[1953] "A customer is looking for sneakers and wants to know about the latest promotions. Generate the best suggestions."

[1954] This system allows users to adjust their schedules optimally based on real-time information while traveling, and also improves the customer experience in stores, enabling flexible responses based on physical and emotional states, allowing for safe and comfortable travel and shopping experiences.

[1955] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1956] Step 1: The user enters the travel destination and dates.

[1957] The user launches an application that can be used on a smartphone or smart glasses and inputs their travel destination and itinerary. This in...

Claims

1. a means for a user to input a travel destination and itinerary; means for transmitting the input information to a server; A means for a server to generate a plurality of optimal travel schedules using a generation AI model based on the input information; means for transmitting the generated schedule to a terminal and displaying it; A system including:

2. means for the server to periodically acquire weather information and traffic information; means for assessing the impact on a current schedule based on the obtained information; and means for generating a new schedule and proposing it to the terminal. The system of claim 1 .

3. A means for the terminal to recognize the user's face and monitor their physical condition using a camera; a means for detecting an abnormality and displaying a warning based on the monitoring result; Further measures include suggesting rest areas and preventative measures; The system of claim 1 .

4. A means for the terminal to inquire of the server about available parking spaces nearby based on the user's current location and destination; The server collects information on parking availability in real time and selects the most suitable parking space. The parking lot information selection unit may further include a means for transmitting the parking lot information to a terminal and proposing the information. The system of claim 1 .

Images