System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting and transmitting driver profile information to a cloud server; means for inputting and transmitting destination information to a cloud server; means for collecting real-time traffic information and calculating an optimal route; means for providing a real-time optimal route to a user; and means for generating and providing tips and useful information based on user interests.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Long drives can often be boring. This is especially true when conventional car navigation systems do not provide real-time traffic information, increasing driver stress. Furthermore, the information provided is generally general and does not address the individual interests of each user, failing to capture the driver's attention. Therefore, there is a need for systems that provide information tailored to each user's individual interests while still allowing for safe and comfortable driving. [Means for solving the problem]

[0005] The present invention provides a system including a means for inputting driver profile information and transmitting it to a cloud server, a means for inputting destination information and transmitting it to a cloud server, a means for collecting real-time traffic information and calculating an optimal route, a means for providing the optimal route to the user in real time, and a means for generating and providing trivia and useful information based on the user's interests. This allows the driver to always select the optimal route based on the latest traffic information, and furthermore, by receiving information related to the user's interests while driving, it is expected that long drives will become more enjoyable.

[0006] "Driver profile information" is personal data entered by the driver, such as name, age, and areas of interest.

[0007] A "cloud server" is a remote server that stores, manages, and processes data over the Internet.

[0008] "Destination information" refers to the location information of the destination specified by the driver.

[0009] "Real-time traffic information" is the latest data on current traffic conditions, including information on congestion, accidents, construction, etc.

[0010] An "optimal route" is the result of calculating the shortest and most efficient route to a specified destination.

[0011] "User" refers to the driver who uses the system.

[0012] The "means for providing an optimal route" refers to a device or program that includes a function for displaying or providing audio guidance to the driver about the calculated optimal route.

[0013] "Trivia and useful information based on the user's interests" refers to knowledge and information related to the driver's individual interests and concerns.

[0014] A "generative artificial intelligence model" refers to an artificial intelligence technology that has the ability to generate new information based on input data. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention describes a car navigation system equipped with a generative AI model to improve the driver's driving experience. The system consists of a cloud server, an in-vehicle terminal, and a driver.

[0037] Basic system configuration

[0038] 1. User Initial Setup:

[0039] The user starts up the in-car device and enters profile information such as name, age, and areas of interest. For example, this information might include "Name: Tarou, Age: 35, Areas of interest: History and nature."

[0040] The device sends this profile information to a cloud server, which stores the information in a database.

[0041] 2. Destination setting and route calculation:

[0042] The user inputs destination information into the in-vehicle terminal. For example, the user inputs "Tokyo Tower."

[0043] The device sends this destination information to a cloud server, which then calculates the optimal route based on real-time traffic information.

[0044] The calculated route information is sent to the in-vehicle terminal and displayed to the user, and voice guidance is also provided.

[0045] 3. Providing interesting information:

[0046] The server generates interesting trivia and useful information at appropriate times based on the user's profile information. For example, if a user is interested in "history," it generates information such as "Tokyo Tower was completed in 1958."

[0047] The generated information is transmitted to an in-vehicle terminal and provided to the user by voice.

[0048] Specific examples

[0049] Initial Settings:

[0050] The user follows the instructions on the device screen and enters "Name: Tarou, Age: 35, Interests: History and Nature." The device sends this information to the server, which then stores the received profile in a database.

[0051] Enter destination and get directions:

[0052] The user inputs "Tokyo Tower" as the destination for a trip. The device sends the location information of "Tokyo Tower" to the server, which then collects real-time traffic information and calculates the optimal route. The calculated route is sent to the device and displayed to the user in real time.

[0053] Providing information while driving:

[0054] While driving, the server uses a generative AI model to generate historical trivia based on the user's interests, and sends information such as "Tokyo Tower was completed in 1958" to the device. The device then relays the received information to the user via voice.

[0055] In this way, users can safely reach their destination while receiving real-time information relevant to their interests while driving. The system also provides optimal routes, helping users avoid traffic congestion and inconvenience. This makes long drives less boring and more enjoyable.

[0056] The processing flow will be explained below.

[0057] Step 1:

[0058] The user starts up the in-car terminal and enters profile information (such as name, age, areas of interest, etc.).

[0059] Step 2:

[0060] The device sends the entered profile information to the cloud server. For example, it sends "Name: Tarou, Age: 35, Areas of Interest: History and Nature."

[0061] Step 3:

[0062] The server stores the received profile information in a database.

[0063] Step 4:

[0064] The user inputs destination information (for example, "Tokyo Tower") into the in-vehicle terminal.

[0065] Step 5:

[0066] The terminal transmits the destination information to the cloud server.

[0067] Step 6:

[0068] The server collects real-time traffic information and calculates the optimal route, including data on congestion, accidents, and construction.

[0069] Step 7:

[0070] The server transmits the calculated optimum route information to the terminal.

[0071] Step 8:

[0072] The terminal displays the received optimum route information to the user and starts voice guidance.

[0073] Step 9:

[0074] While driving, the server uses an AI model to generate trivia and useful information based on the user's profile information. For example, based on the "History" category, it generates information such as "Tokyo Tower was completed in 1958."

[0075] Step 10:

[0076] The server transmits the generated information to the terminal.

[0077] Step 11:

[0078] The device provides the received information to the user via voice, allowing the user to receive information relevant to their interests in real time while driving safely.

[0079] Example 1

[0080] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0081] Conventional car navigation systems only provide support to help drivers reach their destination, making it difficult to improve the quality of the driving experience itself. Furthermore, they lacked features such as route recalculation based on real-time traffic information and information provision based on the driver's interests, making long drives boring and monotonous. This resulted in a decline in the quality and efficiency of information provision while driving, and insufficient safety.

[0082] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0083] In this invention, the server includes means for inputting driver profile information and transmitting it to the cloud server, means for inputting destination information and transmitting it to the cloud server, means for collecting real-time traffic information and calculating an optimal route, means for transmitting the calculated route information to an in-vehicle terminal and displaying and providing voice guidance to the user, means for using a generative artificial intelligence model to generate trivia and useful information based on the driver's profile information and current location information, and means for transmitting the generated information to the in-vehicle terminal and providing it to the driver by voice. This makes it possible to provide individual information based on the driver's profile and guidance on an optimal route that reflects real-time traffic information, thereby improving the quality of the driving experience and ensuring safety while driving.

[0084] "Driver profile information" refers to information about an individual, such as the driver's name, age, and areas of interest.

[0085] A "cloud server" refers to a server that provides data and services via the Internet.

[0086] "Destination information" refers to information about a destination that the driver inputs into the in-vehicle terminal.

[0087] "Real-time traffic information" is data showing current traffic conditions, including information on congestion and traffic restrictions.

[0088] "Optimal route" refers to the most efficient and safe route to your destination, calculated based on real-time traffic information.

[0089] "In-vehicle terminal" refers to an electronic terminal equipped with navigation and information provision functions that is installed in a vehicle.

[0090] "Voice guidance" refers to route guidance and information provided by voice generated by an in-vehicle terminal.

[0091] A "generative artificial intelligence model" refers to an artificial intelligence program that uses techniques such as natural language processing to generate new information.

[0092] A "prompt sentence" refers to an input sentence that causes a generative artificial intelligence model to generate information.

[0093] "Trivia and Helpful Information" refers to short pieces of information and helpful data provided based on a driver's interests and profile.

[0094] The present invention relates to a car navigation system equipped with a generative AI model for improving the driver's driving experience. The system consists of a cloud server, an in-vehicle terminal, and a driver.

[0095] Basic system configuration

[0096] 1. Initial Setup

[0097] The user starts up the in-car device and enters profile information such as name, age, and areas of interest. Input is done using the in-car device's touchscreen or voice recognition function. For example, the user might enter information such as "Name: Tarou, Age: 35, Interests: History and Nature." The device then sends this profile information to a cloud server via an internet connection (using 4G or 5G networks). The server then stores the received profile information in an SQL database.

[0098] 2. Setting a destination and calculating a route

[0099] The user inputs destination information into the in-car device. For example, they input "Tokyo Tower" using the touchscreen. The device then sends this destination information to a cloud server. The server uses the Google Maps API, HERE API, etc. to collect traffic information in real time and calculates the optimal route by sending a request such as "current_location: 'x,y', destination: 'Tokyo Tower'" to the API. The calculated route information is then sent from the server to the in-car device and displayed and provided to the user via display and voice guidance.

[0100] 3. Providing interesting information

[0101] The server references the saved user profile information and confirms, for example, that the user is interested in "history." Next, it generates and sends a prompt to the generative AI model (e.g., OpenAI GPT-3). An example prompt might be, "The user is interested in history and is currently driving toward Tokyo Tower. Please generate some historical trivia related to Tokyo Tower." Based on this prompt, the generative AI model generates information such as, "Tokyo Tower was completed in 1958." This information is sent from the server to the in-vehicle device and provided to the user audibly using a speech synthesis function (using Amazon Polly or Google Text-to-Speech).

[0102] Specific examples

[0103] 1. Initial Setup

[0104] The user follows the on-board device's screen and enters "Name: Tarou, Age: 35, Interests: History and Nature." The device then sends this information to the cloud server, which stores it in a database.

[0105] 2. Enter your destination and get route guidance

[0106] The user inputs "Tokyo Tower" as their destination. The device sends this information to a cloud server, which then collects real-time traffic information and calculates the optimal route. The calculated route is sent to the device and displayed and guided to the user in real time via display and voice guidance.

[0107] 3. Providing information while driving

[0108] While driving, the server uses a generative AI model to generate historical trivia, such as "Tokyo Tower was completed in 1958." This information is sent to the device and provided to the user via voice.

[0109] In this way, drivers can safely reach their destination while receiving interesting information in real time. Furthermore, by providing optimal routes, it is possible to avoid traffic congestion and inconvenience. This makes long driving trips less boring and more enjoyable.

[0110] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0111] Step 1:

[0112] The user starts up the in-car device and enters profile information such as name, age, and areas of interest. Input is done using the in-car device's touchscreen or voice recognition function. For example, the input information might be "Name: Tarou, Age: 35, Interests: History and Nature." This information is aggregated as input information. The device then sends this information to a cloud server via an internet connection (using 4G or 5G networks). This information is the output information.

[0113] Step 2:

[0114] The server stores the received profile information in a database. Specifically, it uses an SQL database to store it as a record such as "username: 'Tarou', age: 35, interests: 'history, nature'". The input information is the profile information sent from the device, which is processed and stored in the database. Updates to this database become the output information.

[0115] Step 3:

[0116] The user inputs destination information into the in-car terminal. For example, they input "Tokyo Tower" using the touch screen. This is the input of destination information. The terminal then sends the destination information to the cloud server. The destination information becomes output data.

[0117] Step 4:

[0118] The server collects real-time traffic information using Google Maps API, HERE API, etc. It receives traffic condition data by sending a request to the API such as "current_location: 'x,y', destination: 'Tokyo Tower'". This is the input information. The optimal route is calculated based on the input data. The Dijkstra algorithm or A algorithm is used for the calculation. The calculation result is generated as output data.

[0119] Step 5:

[0120] The server sends the calculated route information to the in-vehicle terminal. The sent data includes multiple waypoints (passing points). This is the output route information. The terminal displays the received route information on the user's display and begins guiding the user using voice guidance. Detailed navigation information such as "Turn right at the next intersection" is displayed on the display. This is the final output information.

[0121] Step 6:

[0122] The server references the saved user profile information and confirms that the user is interested in "history." This is the input information. Next, it generates and sends a prompt to the generative AI model. An example of a prompt is, "The user is interested in history and is currently driving toward Tokyo Tower. Please generate some historical trivia related to Tokyo Tower." The prompt becomes the input to the generative AI model.

[0123] Step 7:

[0124] The server sends a prompt to a generative AI model (e.g., OpenAI GPT-3) and generates appropriate information. The generative AI model generates information such as "Tokyo Tower was completed in 1958." This is the output data.

[0125] Step 8:

[0126] The server sends the generated information to the in-vehicle terminal. This information may also be sent in audio format. This is the output information. The terminal then conveys the received information to the user by voice. Specifically, it uses a function that generates speech from text (for example, Amazon Polly or Google Text-to-Speech). This is the final output information.

[0127] Through these steps, drivers can reach their destination safely while receiving interesting information in real time, and avoid traffic congestion and inconvenience by being offered optimal routes.

[0128] (Application example 1)

[0129] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0130] Conventional driving experience enhancement systems have limited means of providing drivers with information of interest, resulting in a lack of entertainment while driving. Furthermore, route guidance to a destination often lacks real-time traffic information updates, making it difficult to respond to traffic congestion and route changes. The present invention aims to solve these problems and improve the travel experience for passengers in autonomous vehicles.

[0131] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0132] In this invention, the server includes means for inputting driver profile information and transmitting it to the cloud server, means for inputting destination information and transmitting it to the cloud server, means for collecting real-time traffic information and calculating the optimal route, means for providing the optimal route to the user in real time, means for generating and providing trivia and useful information based on the user's interests, and means that is installed in the autonomous vehicle and provides information by voice in real time to the user while traveling. This makes it possible to provide information based on the user's profile and provide optimal route guidance, thereby enriching the travel experience.

[0133] "Driver profile information" refers to personal information such as the user's name, age, and areas of interest, and is information that is sent to the cloud server.

[0134] A "cloud server" is a remote server that provides data storage, processing, and access over the Internet.

[0135] "Destination information" refers to geographical information about a place the user wants to visit, and is information that is sent to the cloud server.

[0136] "Real-time traffic information" is data showing current traffic conditions, including traffic volume, speed limits, accident information, and the like.

[0137] The "optimal route" is the most efficient and safe route to your destination, calculated based on real-time traffic information.

[0138] "Means for providing the user with the optimal route in real time" refers to means for displaying the calculated optimal route to the user and providing voice guidance.

[0139] "Trivia and useful information based on your interests" refers to information about history, nature, and other topics that are likely to be of interest to you based on your profile information.

[0140] A "generative artificial intelligence model" is an artificial intelligence system that has the ability to generate new information based on massive amounts of data.

[0141] An "autonomous vehicle" is a vehicle equipped with a system that automates human driving behavior and is capable of running without the intervention of a driver.

[0142] "Means for providing information by voice" refers to a method of converting generated information into voice and conveying it to the user.

[0143] The present invention relates to a system that uses a generative AI model to provide users with interesting information based on driver profile information and destination information, thereby improving the travel experience in an autonomous vehicle. Specific embodiments are described below.

[0144] System configuration

[0145] 1. Cloud Server

[0146] Receives and processes profile information, destination information, and real-time traffic information.

[0147] It is equipped with a generative AI model that generates user-based trivia and useful information.

[0148] 2. In-vehicle terminal

[0149] Provide a means for users to enter profile and destination information.

[0150] It receives data from the cloud server and provides users with the best route and information.

[0151] A voice output means is used to provide information to the user audibly.

[0152] 3. Self-driving vehicles

[0153] The vehicle itself is equipped with a system that automates driving, so the user does not need to be involved in driving.

[0154] Enriching users' travel experience by providing information while driving.

[0155] How it works

[0156] Entering user profile information

[0157] The user starts up the in-car device and enters profile information such as name, age, areas of interest, etc. This information is sent to the cloud server and stored in the server's database.

[0158] Enter your destination and calculate your route

[0159] When a user inputs destination information into the in-vehicle terminal, the terminal sends this information to a cloud server. The cloud server collects traffic information in real time, calculates the optimal route, and sends it to the in-vehicle terminal. The in-vehicle terminal then displays the calculated route information to the user and provides voice guidance.

[0160] Providing interesting information

[0161] The cloud server uses a generative AI model to generate trivia and useful information related to the user's interests based on the user's profile information. The generated information is sent to the in-vehicle device in real time and provided to the user via a voice synthesis system. Specifically, a prompt sentence is entered and passed to the generative AI model, and the generated text is then played back by the voice synthesis system.

[0162] Hardware and software used

[0163] Cloud server: Uses AWS or Google Cloud Platform.

[0164] In-vehicle terminal: A device such as a tablet or smartphone.

[0165] Generative AI models: Generative artificial intelligence models such as OpenAI's GPT and Google's BERT.

[0166] Speech synthesis system: Uses Google Text-to-Speech, Amazon Polly, etc.

[0167] Specific examples

[0168] For example, if a user enters information such as "Name: Takashi, Age: 30, Interests: History and Culture" and sets "Museum" as the destination, the cloud server will generate information such as "The museum houses ancient Roman artifacts." This information is sent back to the in-vehicle terminal and provided to the user aloud via a speech synthesis system.

[0169] Example prompt sentence:

[0170] "Takashi-san, the museum houses ancient Roman artifacts."

[0171] In this way, users can receive information of interest in real time while on the move, enriching their in-car experience.

[0172] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0173] Step 1:

[0174] The user enters profile information on the in-car terminal.

[0175] Specifically, users enter information such as their name, age, and areas of interest into the device screen, and this profile information is then sent to the cloud server.

[0176] Input: User's first and last name, age, and interests

[0177] Output: Send profile information to cloud server

[0178] Step 2:

[0179] The cloud server stores the received profile information in a database.

[0180] Specifically, the server analyzes the received data, converts it into an appropriate format, and stores it in a database. This information is then used by a generative AI model to generate information based on the user's interests.

[0181] Input: User profile information

[0182] Output: Profile information stored in a database

[0183] Step 3:

[0184] The user inputs destination information into the in-car terminal.

[0185] Specifically, users input their "destination" on their device, which is then sent to a cloud server, which then uses it to calculate the optimal route.

[0186] Input: Destination information

[0187] Output: Send destination information to the cloud server

[0188] Step 4:

[0189] A cloud server collects real-time traffic information and calculates the optimal route.

[0190] Specifically, the server collects real-time information on current traffic conditions, road closures, traffic congestion, etc., and calculates the optimal route to the destination based on this information. This calculation is performed using an algorithm, and the calculation results are sent to the in-vehicle terminal.

[0191] Input: Destination information, real-time traffic information

[0192] Output: Calculated optimal route information

[0193] Step 5:

[0194] The in-car device displays the optimal route to the user and begins voice guidance.

[0195] Specifically, route information sent from a cloud server is displayed on the terminal, and a voice synthesis system is used to provide real-time voice guidance to the user.

[0196] Input: Optimal route information

[0197] Output: Route display, voice guidance

[0198] Step 6:

[0199] The cloud server generates interesting information based on the user's profile information using a generative AI model.

[0200] Specifically, a prompt sentence is created based on the user's interest information (e.g., history, nature), and passed to a generative AI model to generate useful information and trivia.

[0201] Input: User profile information

[0202] Output: interesting information generated

[0203] Step 7:

[0204] The generated information is sent from the cloud server to the in-vehicle terminal, and the information is provided to the user using a voice synthesis system.

[0205] Specifically, the generated information is sent to the terminal, converted into voice by a voice synthesis system, and provided to the user.

[0206] Input: Generated interesting information

[0207] Output: Providing information as synthesized speech

[0208] Specific examples

[0209] For example, if a user enters the information "Name: Takashi, Age: 30, Interests: History and Culture" and sets "Museum" as the destination, the prompt might look like this:

[0210] "Takashi-san, the museum houses ancient Roman artifacts."

[0211] This information is generated on a cloud server, sent to the in-vehicle terminal, and provided to the user audibly through a voice synthesis system.

[0212] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0213] This paper describes a system that combines a car navigation system that uses a generative AI model to improve the driver's driving experience with an emotion engine to provide information tailored to individual user needs. This system consists of a cloud server, an in-vehicle terminal, a driver, and an emotion engine.

[0214] Basic system configuration

[0215] 1. User Initial Setup:

[0216] The user starts up the in-car device and enters their profile information (such as name, age, and areas of interest). For example, this information might include "Name: Tarou, Age: 35, Areas of Interest: History and Nature."

[0217] The device sends this profile information to a cloud server, which stores the information in a database.

[0218] 2. Destination setting and route calculation:

[0219] The user inputs destination information into the in-vehicle terminal. For example, the user inputs "Tokyo Tower."

[0220] The device sends this destination information to a cloud server, which then calculates the optimal route based on real-time traffic information.

[0221] The calculated route information is sent to the in-vehicle terminal and displayed to the user, and voice guidance is also provided.

[0222] 3. Emotion Recognition with Emotion Engine:

[0223] The emotion engine analyzes the driver's voice and facial expressions, for example, by monitoring voice tone and facial expressions in real time to understand the driver's emotional state.

[0224] It analyzes whether the user is relaxed or stressed and gives real-time feedback while driving.

[0225] 4. Providing interesting information:

[0226] The server uses a generative AI model to generate interesting trivia and useful information at the appropriate time based on the user's profile information and the recognition results of the emotion engine. For example, if the user is interested in "history" and is relaxing, the server will generate information such as "Tokyo Tower was completed in 1958." On the other hand, if the user is feeling stressed, the server will provide relaxing music or topics of interest.

[0227] The generated information is transmitted to an in-vehicle terminal and provided to the user by voice.

[0228] Specific processing of the program

[0229] Emotion Engine in action:

[0230] The device captures the user's voice and facial expressions through a camera and microphone, and the emotion engine analyzes them in real time. The analysis results are then sent to the server.

[0231] For example, if the user is smiling while driving, it is determined that the user is in a relaxed state, whereas if the user is speaking in a loud voice, it is determined that the user is feeling stressed.

[0232] Information generation and provision:

[0233] The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine.

[0234] When the user is in a relaxed state, interesting trivia is provided aloud, such as "Tokyo Tower is 333 meters tall."

[0235] For users who are under stress, the system plays relaxing music and provides encouraging words about driving.

[0236] In this way, while driving, users can always select the optimal route based on the latest traffic information, and receive information and music tailored to their interests and emotional state, making long drives stress-free and more fulfilling. Real-time feedback from the emotion engine also improves driver safety. For example, if a stressful state is detected, a message recommending safe driving can be displayed or music that promotes relaxation can be played. Providing both physical and psychological comfort simultaneously can significantly improve the driver's overall driving experience.

[0237] The processing flow will be explained below.

[0238] Step 1:

[0239] The user starts up the in-car device and enters profile information (such as name, age, and areas of interest). For example, the user enters "Name: Tarou, Age: 35, Areas of interest: History and nature."

[0240] Step 2:

[0241] The device sends the entered profile information to a cloud server, which stores the received information in a database.

[0242] Step 3:

[0243] The user inputs destination information into the in-vehicle terminal. For example, the user inputs "Tokyo Tower."

[0244] Step 4:

[0245] The device sends this destination information to a cloud server, which collects real-time traffic information and calculates the optimal route.

[0246] Step 5:

[0247] The server sends the calculated optimal route information to the terminal, which displays the received optimal route information to the user and starts voice guidance.

[0248] Step 6:

[0249] The device captures the user's voice and facial expressions through a camera and microphone, and the emotion engine analyzes them in real time. The analysis results are then sent to a cloud server.

[0250] Step 7:

[0251] The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine. If the user is in a relaxed state, it generates trivia such as "Tokyo Tower was completed in 1958." If the user is in a stressed state, it generates relaxing music or an encouraging message.

[0252] Step 8:

[0253] The server sends the generated information to the terminal, and the terminal provides the received information to the user by voice.

[0254] Step 9:

[0255] The device then plays back the received information through a speaker and provides it to the user. For example, it might say, "Tokyo Tower was completed in 1958."

[0256] Step 10:

[0257] The server monitors traffic conditions in real time and recalculates routes as needed. If a new route is required, it notifies the device and updates the optimal route information.

[0258] Step 11:

[0259] The device displays the updated optimal route information to the user and updates the voice guidance. The device continues to provide voice guidance to the user about the recalculated route.

[0260] In this way, the system improves the user's driving experience by providing personalized information based on the user's profile and emotional state, and by providing optimal route guidance that reflects traffic conditions in real time.

[0261] Example 2

[0262] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0263] Conventional car navigation systems do not adequately provide information that corresponds to the driver's emotional state or individual interests, making it difficult to relieve stress associated with long drives and resulting in a less than fulfilling driving experience.In addition, route calculations based solely on real-time traffic information make it difficult to provide an optimal driving experience that corresponds to the driver's emotions and interests.

[0264] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for inputting driver profile information and transmitting it to a cloud server, a means for inputting destination information and transmitting it to the cloud server, a means for collecting real-time traffic information and calculating an optimal route, a means for capturing the driver's voice and facial expression and analyzing the emotional state, a means for transmitting the analysis results to the cloud server, a means for the cloud server to generate appropriate information using a generative artificial intelligence model based on the driver's profile information and emotional state, and a means for providing the generated information to the driver by voice. This makes it possible to provide information according to the driver's emotional state and interests, thereby reducing stress and enhancing the driving experience.

[0265] "Driver profile information" is basic information about each individual driver, such as the driver's name, age, and areas of interest.

[0266] A "cloud server" is a remote server that operates on the Internet and stores and processes driver profile information and destination information.

[0267] "Destination information" refers to information such as the address and place name of the destination that the driver inputs into the in-vehicle terminal.

[0268] "Real-time traffic information" is data that provides current road conditions, traffic volume, accident information, etc. in real time.

[0269] "Optimal route" refers to the most efficient and safe route for a driver based on real-time traffic information.

[0270] "Driver's voice and facial expression" refers to the voice and facial expression of the driver, and is data for analyzing the emotional state through these.

[0271] "Emotional state" refers to the driver's current psychological state (relaxed, stressed, etc.).

[0272] A "generative artificial intelligence model" is an AI technology that generates appropriate information based on input data.

[0273] "Relevant information" is interesting tidbits or helpful information generated based on the driver's profile information and emotional state.

[0274] An "in-vehicle terminal" is a device installed in a vehicle that allows the driver to input profile information and destination information, and capture voice and facial expressions.

[0275] "Means for providing by voice" refers to technology or devices for conveying the generated information to the driver by voice.

[0276] The present invention relates to a car navigation system that improves the driver's driving experience by providing information tailored to the driver's emotional state and individual interests. This system is composed of a cloud server, an in-vehicle terminal, a driver, and an emotion engine.

[0277] Basic system configuration

[0278] User Initial Settings

[0279] The user starts up the in-car terminal and uses the touchscreen to enter profile information (such as name, age, and areas of interest). For example, this information might include "Name: Tarou, Age: 35, Areas of Interest: History and Nature." The terminal then sends this profile information to a cloud server, which then stores the information in a database. The hardware used is an in-car terminal with a touchscreen, and the software is a cloud database management system.

[0280] Destination setting and route calculation

[0281] The user inputs destination information into the in-vehicle terminal. For example, they input "Tokyo Tower." The terminal sends this destination information to a cloud server. The server calculates the optimal route based on real-time traffic information. The calculated route information is sent to the in-vehicle terminal and displayed to the user, while voice guidance is also provided. The hardware used here is an in-vehicle terminal with GPS functionality, and the software is a real-time traffic information API and a route calculation algorithm.

[0282] emotion recognition

[0283] The emotion engine analyzes the driver's voice and facial expressions. Using the device's camera and microphone, it monitors the driver's voice tone and facial expressions in real time to understand the driver's emotional state. The analysis results are sent to a cloud server. The hardware used here is an in-car device equipped with a camera and microphone, and the software is an emotion analysis algorithm. For example, if the user is smiling while driving, it is determined that the user is relaxed. If the user's voice is raised, it can be determined that the user is stressed.

[0284] Information generation and provision

[0285] The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine. For users who are in a relaxed state, interesting trivia is provided by voice. For example, information such as "Tokyo Tower is 333 meters tall" is generated. On the other hand, for users who are in a stressed state, relaxing music is played or encouraging words about driving are provided. As a concrete example, the prompt text for the generative AI model is as follows:

[0286] Prompt: "Generate some trivia about Tokyo Tower for a relaxed user who is interested in history."

[0287] AI model output: "Tokyo Tower was completed in 1958. It is 333 meters tall."

[0288] The generated information is sent to the in-vehicle terminal and provided to the driver using the terminal's voice synthesis function. The software used here is a generative artificial intelligence model and a voice synthesis system.

[0289] In this way, while driving, users can always select the optimal route based on the latest traffic information, and receive information and music tailored to their interests and emotional state, making long drives stress-free and more fulfilling. Real-time feedback from the emotion engine also improves driver safety. For example, if a stressful state is detected, a message recommending safe driving can be displayed or music that promotes relaxation can be played. Providing both physical and psychological comfort simultaneously can significantly improve the driver's overall driving experience.

[0290] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0291] System program processing flow

[0292] Step 1: User Initial Setup

[0293] Input: User profile information (e.g., name, age, interests)

[0294] Output: Profile information sent to cloud server and stored in database

[0295] 1. The user starts up the in-car device and operates the touch screen to enter profile information.

[0296] Specific operation: The user enters "Name: Tarou, Age: 35, Areas of interest: History and nature" into the input form on the device.

[0297] 2. The device sends the entered profile information to the cloud server.

[0298] Specific operation: When the device presses the send button, it sends the profile information to the cloud server via the Internet connection.

[0299] 3. The server stores the received profile information in a database.

[0300] What happens: The server's database management system automatically creates a record for the new user and records their profile information.

[0301] Step 2: Set destination and calculate route

[0302] Input: Destination information (e.g. Tokyo Tower)

[0303] Output: Optimal route information is displayed on the in-car device and voice guidance begins.

[0304] 1. The user inputs destination information into the in-vehicle terminal.

[0305] Specific operation: The user uses the touch screen and the keyboard to input "Tokyo Tower."

[0306] 2. The device sends the entered destination information to the cloud server.

[0307] Specific operation: The terminal transmits destination information to the server via the Internet in real time.

[0308] 3. The server calculates the optimal route based on real-time traffic information.

[0309] Specific operation: The server obtains data from the traffic information API and uses an algorithm to calculate the optimal route.

[0310] 4. The server sends the calculated route information to the in-vehicle terminal, which displays and provides voice guidance to the user.

[0311] Specific operation: The server sends route information to the terminal, the terminal displays the route map on the screen, and the voice guidance system provides instructions by voice.

[0312] Step 3: Emotion Recognition

[0313] Input: Driver's voice and facial expression capture data

[0314] Output: The analyzed emotional state (e.g., relaxed, stressed) is sent to the cloud server.

[0315] 1. The device captures the driver's voice and facial expressions through a camera and microphone.

[0316] Specific operation: The device's built-in camera and microphone automatically record the user's face and voice.

[0317] 2. The emotion engine analyzes the captured data in real time.

[0318] How it works: The built-in software uses visual recognition and voice analysis technology to analyze the user's emotions (relaxed, stressed, etc.).

[0319] 3. The analysis results are sent to the cloud server.

[0320] Specific operation: The device sends analysis data to the server in real time.

[0321] Step 4: Information generation and provision

[0322] Input: Profile information, emotional state

[0323] Output: The generated appropriate information (e.g., trivia or relaxing music) is provided to the user via the in-car terminal.

[0324] 1. The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine.

[0325] Specific operation: The server inputs the prompt "Profile information: History lover, Emotional state: Relaxed" to the generated AI model.

[0326] Example prompt sentence:

[0327] Prompt: "Generate some trivia about Tokyo Tower for a relaxed user who is interested in history."

[0328] AI model output: "Tokyo Tower was completed in 1958. It is 333 meters tall."

[0329] 2. The server sends the generated information to the in-vehicle terminal, which then provides it to the user via voice.

[0330] Specific operation: The server sends the generated information to the terminal, and the terminal's voice synthesis function conveys the information to the user.

[0331] Through the above processing steps, the system can provide optimized information according to the driver's emotional state and interests, improving the driving experience.

[0332] (Application example 2)

[0333] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0334] Conventional car navigation systems only provide optimal routes using the driver's profile and real-time traffic information, but lack information tailored to the driver's emotional state and interests. This can make long drives stressful and reduce safety. Furthermore, if the information provided during driving does not match the driver's interests, satisfaction can decrease.

[0335] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting driver profile information and transmitting it to a cloud server, means for inputting destination information and transmitting it to the cloud server, means for collecting real-time traffic information and calculating an optimal route, means for providing the optimal route to the user in real time, means for analyzing the user's emotional state, and means for generating and providing information based on the user's interests and emotional state. This makes it possible to provide appropriate information according to the driver's emotional state, thereby reducing stress while driving, improving safety, and improving the driving experience.

[0336] "Driver profile information" refers to personal information such as the driver's name, age, and areas of interest.

[0337] A "cloud server" refers to a server that provides resources and services over the Internet.

[0338] "Destination information" refers to information such as the name and address of the destination set by the driver.

[0339] "Real-time traffic information" refers to data collected in real time about current traffic conditions.

[0340] An "optimal route" refers to the most efficient route for a driver's travel.

[0341] "User" refers to a driver who uses the system.

[0342] "Emotional state" refers to the driver's current emotional and psychological state.

[0343] A "generative artificial intelligence model" refers to an artificial intelligence model that automatically generates appropriate information based on input data.

[0344] This invention is a car navigation system that provides customized information based on the driver's profile information and real-time emotional state. The system consists of a cloud server, an in-vehicle terminal, a driver, and an emotion engine.

[0345] First, the user starts up the in-car device and enters their profile information (such as name, age, and areas of interest). For example, a user might enter information such as "Name: ABC, Age: 35, Areas of Interest: History and Nature." This information is sent from the device to the cloud server and stored in a database.

[0346] Next, the user inputs destination information into the in-vehicle terminal. For example, they input "major tourist destinations." This destination information is also sent to the cloud server, which calculates the optimal route based on real-time traffic information. The calculated route information is sent back to the in-vehicle terminal and displayed to the user. Voice guidance is also provided.

[0347] While driving, the emotion engine analyzes the driver's voice and facial expressions in real time. Specifically, the vehicle's camera and microphone capture the driver's video and audio, which the emotion engine analyzes. For example, it determines whether the driver is relaxed or stressed. The results of this analysis are sent to a cloud server.

[0348] The cloud server uses a generative AI model based on the driver's profile information and the analysis results of the emotion engine to generate appropriate information. For example, if the driver is relaxed and interested in "history and nature," the generative AI model will generate information such as "Major tourist destinations were completed in 1958." On the other hand, if the driver is feeling stressed, the model will provide relaxing music and encouraging words.

[0349] The generated information is sent to an in-vehicle terminal and provided to the driver via voice, allowing the driver to receive customized information at the right time, making the drive more comfortable.

[0350] As a concrete example, we will show an example of a prompt sentence to be input to the generative AI model.

[0351] Example prompt sentence:

[0352] User profile:

[0353] Name: ABC, Age: 35, Areas of Interest: History and Nature

[0354] Driver's state: Relaxed

[0355] prompt:

[0356] Based on the user profile and driver status, generate interesting information about their history to help them relax.

[0357] This system aims to reduce driver stress and improve safety by dynamically customizing information provided while driving based on the driver's emotional state and interests, and also to improve the driver's driving experience.

[0358] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0359] Step 1:

[0360] The user inputs profile information from the in-vehicle terminal, and the terminal transmits the information to the cloud server.

[0361] Specifically, when a user enters data such as their name, age, and areas of interest and presses the "Submit" button, the device sends the input data to the cloud server as a POST request, and the input data is stored directly in the cloud database.

[0362] Input: User profile information (name, age, interests)

[0363] Output: Profile information stored on the cloud server

[0364] Step 2:

[0365] The user inputs destination information from the in-vehicle terminal, and the terminal transmits the information to the cloud server.

[0366] Specifically, when a user inputs a destination (e.g., a major tourist spot) and presses the "Send" button, the device sends the input data to the cloud server as a POST request. The cloud server receives this data and calculates the optimal route based on real-time traffic information.

[0367] Input: User's destination information

[0368] Output: Optimal route calculation by cloud server

[0369] Step 3:

[0370] The cloud server sends the calculated optimal route information to the in-vehicle terminal, which then displays it to the user.

[0371] Specifically, the cloud server returns the calculation results to the in-vehicle terminal as a response, and the terminal displays the information on the screen and starts voice guidance.

[0372] Input: Optimal route information calculated by the cloud server

[0373] Output: Optimal route information and voice guidance displayed on the in-car device

[0374] Step 4:

[0375] The device captures the driver's voice and facial expressions through a camera and microphone, and the emotion engine analyzes them in real time.

[0376] Specifically, the in-car camera captures the driver's facial expressions and the microphone captures their voice, and the data is sent to the emotion engine, which analyzes it to determine the driver's emotional state and sends the results to a cloud server.

[0377] Input: Driver's facial image and voice data

[0378] Output: Analysis result of emotional state by emotion engine

[0379] Step 5:

[0380] The cloud server uses a generative AI model to generate appropriate information based on the driver's profile information and the analysis results of the emotion engine.

[0381] Specifically, the cloud server sends data along with the prompt text to the generative AI model and receives the generated information, which includes content that corresponds to the driver's interests and emotional state.

[0382] Input: Driver profile information, emotional state

[0383] Output: Customized information from a generative AI model

[0384] Step 6:

[0385] The cloud server transmits the generated customization information to the in-vehicle terminal, which then provides it to the user via voice.

[0386] Specifically, the cloud server sends the generated information back to the in-vehicle terminal as a response, and the terminal then conveys the information to the user via voice. For example, it might provide interesting trivia to a relaxed driver, or relaxing music or words of encouragement to a stressed driver.

[0387] Input: Customization information generated by the generative AI model

[0388] Output: Audio information presented to the user

[0389] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0390] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0391] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0392] [Second embodiment]

[0393] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0394] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0395] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0396] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0397] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0398] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0399] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0400] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0401] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0402] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0403] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0404] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0405] This invention describes a car navigation system equipped with a generative AI model to improve the driver's driving experience. The system consists of a cloud server, an in-vehicle terminal, and a driver.

[0406] Basic system configuration

[0407] 1. User Initial Setup:

[0408] The user starts up the in-car device and enters profile information such as name, age, and areas of interest. For example, this information might include "Name: Tarou, Age: 35, Areas of interest: History and nature."

[0409] The device sends this profile information to a cloud server, which stores the information in a database.

[0410] 2. Destination setting and route calculation:

[0411] The user inputs destination information into the in-vehicle terminal. For example, the user inputs "Tokyo Tower."

[0412] The device sends this destination information to a cloud server, which then calculates the optimal route based on real-time traffic information.

[0413] The calculated route information is sent to the in-vehicle terminal and displayed to the user, and voice guidance is also provided.

[0414] 3. Providing interesting information:

[0415] The server generates interesting trivia and useful information at appropriate times based on the user's profile information. For example, if a user is interested in "history," it generates information such as "Tokyo Tower was completed in 1958."

[0416] The generated information is transmitted to an in-vehicle terminal and provided to the user by voice.

[0417] Specific examples

[0418] Initial Settings:

[0419] The user follows the instructions on the device screen and enters "Name: Tarou, Age: 35, Interests: History and Nature." The device sends this information to the server, which then stores the received profile in a database.

[0420] Enter destination and get directions:

[0421] The user inputs "Tokyo Tower" as the destination for a trip. The device sends the location information of "Tokyo Tower" to the server, which then collects real-time traffic information and calculates the optimal route. The calculated route is sent to the device and displayed to the user in real time.

[0422] Providing information while driving:

[0423] While driving, the server uses a generative AI model to generate historical trivia based on the user's interests, and sends information such as "Tokyo Tower was completed in 1958" to the device. The device then relays the received information to the user via voice.

[0424] In this way, users can safely reach their destination while receiving real-time information relevant to their interests while driving. The system also provides optimal routes, helping users avoid traffic congestion and inconvenience. This makes long drives less boring and more enjoyable.

[0425] The processing flow will be explained below.

[0426] Step 1:

[0427] The user starts up the in-car terminal and enters profile information (such as name, age, areas of interest, etc.).

[0428] Step 2:

[0429] The device sends the entered profile information to the cloud server. For example, it sends "Name: Tarou, Age: 35, Areas of Interest: History and Nature."

[0430] Step 3:

[0431] The server stores the received profile information in a database.

[0432] Step 4:

[0433] The user inputs destination information (for example, "Tokyo Tower") into the in-vehicle terminal.

[0434] Step 5:

[0435] The terminal transmits the destination information to the cloud server.

[0436] Step 6:

[0437] The server collects real-time traffic information and calculates the optimal route, including data on congestion, accidents, and construction.

[0438] Step 7:

[0439] The server transmits the calculated optimum route information to the terminal.

[0440] Step 8:

[0441] The terminal displays the received optimum route information to the user and starts voice guidance.

[0442] Step 9:

[0443] While driving, the server uses an AI model to generate trivia and useful information based on the user's profile information. For example, based on the "History" category, it generates information such as "Tokyo Tower was completed in 1958."

[0444] Step 10:

[0445] The server transmits the generated information to the terminal.

[0446] Step 11:

[0447] The device provides the received information to the user via voice, allowing the user to receive information relevant to their interests in real time while driving safely.

[0448] Example 1

[0449] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0450] Conventional car navigation systems only provide support to help drivers reach their destination, making it difficult to improve the quality of the driving experience itself. Furthermore, they lacked features such as route recalculation based on real-time traffic information and information provision based on the driver's interests, making long drives boring and monotonous. This resulted in a decline in the quality and efficiency of information provision while driving, and insufficient safety.

[0451] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0452] In this invention, the server includes means for inputting driver profile information and transmitting it to the cloud server, means for inputting destination information and transmitting it to the cloud server, means for collecting real-time traffic information and calculating an optimal route, means for transmitting the calculated route information to an in-vehicle terminal and displaying and providing voice guidance to the user, means for using a generative artificial intelligence model to generate trivia and useful information based on the driver's profile information and current location information, and means for transmitting the generated information to the in-vehicle terminal and providing it to the driver by voice. This makes it possible to provide individual information based on the driver's profile and guidance on an optimal route that reflects real-time traffic information, thereby improving the quality of the driving experience and ensuring safety while driving.

[0453] "Driver profile information" refers to information about an individual, such as the driver's name, age, and areas of interest.

[0454] A "cloud server" refers to a server that provides data and services via the Internet.

[0455] "Destination information" refers to information about a destination that the driver inputs into the in-vehicle terminal.

[0456] "Real-time traffic information" is data showing current traffic conditions, including information on congestion and traffic restrictions.

[0457] "Optimal route" refers to the most efficient and safe route to your destination, calculated based on real-time traffic information.

[0458] "In-vehicle terminal" refers to an electronic terminal equipped with navigation and information provision functions that is installed in a vehicle.

[0459] "Voice guidance" refers to route guidance and information provided by voice generated by an in-vehicle terminal.

[0460] A "generative artificial intelligence model" refers to an artificial intelligence program that uses techniques such as natural language processing to generate new information.

[0461] A "prompt sentence" refers to an input sentence that causes a generative artificial intelligence model to generate information.

[0462] "Trivia and Helpful Information" refers to short pieces of information and helpful data provided based on a driver's interests and profile.

[0463] The present invention relates to a car navigation system equipped with a generative AI model for improving the driver's driving experience. The system consists of a cloud server, an in-vehicle terminal, and a driver.

[0464] Basic system configuration

[0465] 1. Initial Setup

[0466] The user starts up the in-car device and enters profile information such as name, age, and areas of interest. Input is done using the in-car device's touchscreen or voice recognition function. For example, the user might enter information such as "Name: Tarou, Age: 35, Interests: History and Nature." The device then sends this profile information to a cloud server via an internet connection (using 4G or 5G networks). The server then stores the received profile information in an SQL database.

[0467] 2. Setting a destination and calculating a route

[0468] The user inputs destination information into the in-car device. For example, they input "Tokyo Tower" using the touchscreen. The device then sends this destination information to a cloud server. The server uses the Google Maps API, HERE API, etc. to collect traffic information in real time and calculates the optimal route by sending a request such as "current_location: 'x,y', destination: 'Tokyo Tower'" to the API. The calculated route information is then sent from the server to the in-car device and displayed and provided to the user via display and voice guidance.

[0469] 3. Providing interesting information

[0470] The server references the saved user profile information and confirms, for example, that the user is interested in "history." Next, it generates and sends a prompt to the generative AI model (e.g., OpenAI GPT-3). An example prompt might be, "The user is interested in history and is currently driving toward Tokyo Tower. Please generate some historical trivia related to Tokyo Tower." Based on this prompt, the generative AI model generates information such as, "Tokyo Tower was completed in 1958." This information is sent from the server to the in-vehicle device and provided to the user audibly using a speech synthesis function (using Amazon Polly or Google Text-to-Speech).

[0471] Specific examples

[0472] 1. Initial Setup

[0473] The user follows the on-board device's screen and enters "Name: Tarou, Age: 35, Interests: History and Nature." The device then sends this information to the cloud server, which stores it in a database.

[0474] 2. Enter your destination and get route guidance

[0475] The user inputs "Tokyo Tower" as their destination. The device sends this information to a cloud server, which then collects real-time traffic information and calculates the optimal route. The calculated route is sent to the device and displayed and guided to the user in real time via display and voice guidance.

[0476] 3. Providing information while driving

[0477] While driving, the server uses a generative AI model to generate historical trivia, such as "Tokyo Tower was completed in 1958." This information is sent to the device and provided to the user via voice.

[0478] In this way, drivers can safely reach their destination while receiving interesting information in real time. Furthermore, by providing optimal routes, it is possible to avoid traffic congestion and inconvenience. This makes long driving trips less boring and more enjoyable.

[0479] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0480] Step 1:

[0481] The user starts up the in-car device and enters profile information such as name, age, and areas of interest. Input is done using the in-car device's touchscreen or voice recognition function. For example, the input information might be "Name: Tarou, Age: 35, Interests: History and Nature." This information is aggregated as input information. The device then sends this information to a cloud server via an internet connection (using 4G or 5G networks). This information is the output information.

[0482] Step 2:

[0483] The server stores the received profile information in a database. Specifically, it uses an SQL database to store it as a record such as "username: 'Tarou', age: 35, interests: 'history, nature'". The input information is the profile information sent from the device, which is processed and stored in the database. Updates to this database become the output information.

[0484] Step 3:

[0485] The user inputs destination information into the in-car terminal. For example, they input "Tokyo Tower" using the touch screen. This is the input of destination information. The terminal then sends the destination information to the cloud server. The destination information becomes output data.

[0486] Step 4:

[0487] The server collects real-time traffic information using Google Maps API, HERE API, etc. It receives traffic condition data by sending a request to the API such as "current_location: 'x,y', destination: 'Tokyo Tower'". This is the input information. The optimal route is calculated based on the input data. The Dijkstra algorithm or A algorithm is used for the calculation. The calculation result is generated as output data.

[0488] Step 5:

[0489] The server sends the calculated route information to the in-vehicle terminal. The sent data includes multiple waypoints (passing points). This is the output route information. The terminal displays the received route information on the user's display and begins guiding the user using voice guidance. Detailed navigation information such as "Turn right at the next intersection" is displayed on the display. This is the final output information.

[0490] Step 6:

[0491] The server references the saved user profile information and confirms that the user is interested in "history." This is the input information. Next, it generates and sends a prompt to the generative AI model. An example of a prompt is, "The user is interested in history and is currently driving toward Tokyo Tower. Please generate some historical trivia related to Tokyo Tower." The prompt becomes the input to the generative AI model.

[0492] Step 7:

[0493] The server sends a prompt to a generative AI model (e.g., OpenAI GPT-3) and generates appropriate information. The generative AI model generates information such as "Tokyo Tower was completed in 1958." This is the output data.

[0494] Step 8:

[0495] The server sends the generated information to the in-vehicle terminal. This information may also be sent in audio format. This is the output information. The terminal then conveys the received information to the user by voice. Specifically, it uses a function that generates speech from text (for example, Amazon Polly or Google Text-to-Speech). This is the final output information.

[0496] Through these steps, drivers can reach their destination safely while receiving interesting information in real time, and avoid traffic congestion and inconvenience by being offered optimal routes.

[0497] (Application example 1)

[0498] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0499] Conventional driving experience enhancement systems have limited means of providing drivers with information of interest, resulting in a lack of entertainment while driving. Furthermore, route guidance to a destination often lacks real-time traffic information updates, making it difficult to respond to traffic congestion and route changes. The present invention aims to solve these problems and improve the travel experience for passengers in autonomous vehicles.

[0500] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0501] In this invention, the server includes means for inputting driver profile information and transmitting it to the cloud server, means for inputting destination information and transmitting it to the cloud server, means for collecting real-time traffic information and calculating the optimal route, means for providing the optimal route to the user in real time, means for generating and providing trivia and useful information based on the user's interests, and means that is installed in the autonomous vehicle and provides information by voice in real time to the user while traveling. This makes it possible to provide information based on the user's profile and provide optimal route guidance, thereby enriching the travel experience.

[0502] "Driver profile information" refers to personal information such as the user's name, age, and areas of interest, and is information that is sent to the cloud server.

[0503] A "cloud server" is a remote server that provides data storage, processing, and access over the Internet.

[0504] "Destination information" refers to geographical information about a place the user wants to visit, and is information that is sent to the cloud server.

[0505] "Real-time traffic information" is data showing current traffic conditions, including traffic volume, speed limits, accident information, and the like.

[0506] The "optimal route" is the most efficient and safe route to your destination, calculated based on real-time traffic information.

[0507] "Means for providing the user with the optimal route in real time" refers to means for displaying the calculated optimal route to the user and providing voice guidance.

[0508] "Trivia and useful information based on your interests" refers to information about history, nature, and other topics that are likely to be of interest to you based on your profile information.

[0509] A "generative artificial intelligence model" is an artificial intelligence system that has the ability to generate new information based on massive amounts of data.

[0510] An "autonomous vehicle" is a vehicle equipped with a system that automates human driving behavior and is capable of running without the intervention of a driver.

[0511] "Means for providing information by voice" refers to a method of converting generated information into voice and conveying it to the user.

[0512] The present invention relates to a system that uses a generative AI model to provide users with interesting information based on driver profile information and destination information, thereby improving the travel experience in an autonomous vehicle. Specific embodiments are described below.

[0513] System configuration

[0514] 1. Cloud Server

[0515] Receives and processes profile information, destination information, and real-time traffic information.

[0516] It is equipped with a generative AI model that generates user-based trivia and useful information.

[0517] 2. In-vehicle terminal

[0518] Provide a means for users to enter profile and destination information.

[0519] It receives data from the cloud server and provides users with the best route and information.

[0520] A voice output means is used to provide information to the user audibly.

[0521] 3. Self-driving vehicles

[0522] The vehicle itself is equipped with a system that automates driving, so the user does not need to be involved in driving.

[0523] Enriching users' travel experience by providing information while driving.

[0524] How it works

[0525] Entering user profile information

[0526] The user starts up the in-car device and enters profile information such as name, age, areas of interest, etc. This information is sent to the cloud server and stored in the server's database.

[0527] Enter your destination and calculate your route

[0528] When a user inputs destination information into the in-vehicle terminal, the terminal sends this information to a cloud server. The cloud server collects traffic information in real time, calculates the optimal route, and sends it to the in-vehicle terminal. The in-vehicle terminal then displays the calculated route information to the user and provides voice guidance.

[0529] Providing interesting information

[0530] The cloud server uses a generative AI model to generate trivia and useful information related to the user's interests based on the user's profile information. The generated information is sent to the in-vehicle device in real time and provided to the user via a voice synthesis system. Specifically, a prompt sentence is entered and passed to the generative AI model, and the generated text is then played back by the voice synthesis system.

[0531] Hardware and software used

[0532] Cloud server: Uses AWS or Google Cloud Platform.

[0533] In-vehicle terminal: A device such as a tablet or smartphone.

[0534] Generative AI models: Generative artificial intelligence models such as OpenAI's GPT and Google's BERT.

[0535] Speech synthesis system: Uses Google Text-to-Speech, Amazon Polly, etc.

[0536] Specific examples

[0537] For example, if a user enters information such as "Name: Takashi, Age: 30, Interests: History and Culture" and sets "Museum" as the destination, the cloud server will generate information such as "The museum houses ancient Roman artifacts." This information is sent back to the in-vehicle terminal and provided to the user aloud via a speech synthesis system.

[0538] Example prompt sentence:

[0539] "Takashi-san, the museum houses ancient Roman artifacts."

[0540] In this way, users can receive information of interest in real time while on the move, enriching their in-car experience.

[0541] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0542] Step 1:

[0543] The user enters profile information on the in-car terminal.

[0544] Specifically, users enter information such as their name, age, and areas of interest into the device screen, and this profile information is then sent to the cloud server.

[0545] Input: User's first and last name, age, and interests

[0546] Output: Send profile information to cloud server

[0547] Step 2:

[0548] The cloud server stores the received profile information in a database.

[0549] Specifically, the server analyzes the received data, converts it into an appropriate format, and stores it in a database. This information is then used by a generative AI model to generate information based on the user's interests.

[0550] Input: User profile information

[0551] Output: Profile information stored in a database

[0552] Step 3:

[0553] The user inputs destination information into the in-car terminal.

[0554] Specifically, users input their "destination" on their device, which is then sent to a cloud server, which then uses it to calculate the optimal route.

[0555] Input: Destination information

[0556] Output: Send destination information to the cloud server

[0557] Step 4:

[0558] A cloud server collects real-time traffic information and calculates the optimal route.

[0559] Specifically, the server collects real-time information on current traffic conditions, road closures, traffic congestion, etc., and calculates the optimal route to the destination based on this information. This calculation is performed using an algorithm, and the calculation results are sent to the in-vehicle terminal.

[0560] Input: Destination information, real-time traffic information

[0561] Output: Calculated optimal route information

[0562] Step 5:

[0563] The in-car device displays the optimal route to the user and begins voice guidance.

[0564] Specifically, route information sent from a cloud server is displayed on the terminal, and a voice synthesis system is used to provide real-time voice guidance to the user.

[0565] Input: Optimal route information

[0566] Output: Route display, voice guidance

[0567] Step 6:

[0568] The cloud server generates interesting information based on the user's profile information using a generative AI model.

[0569] Specifically, a prompt sentence is created based on the user's interest information (e.g., history, nature), and passed to a generative AI model to generate useful information and trivia.

[0570] Input: User profile information

[0571] Output: interesting information generated

[0572] Step 7:

[0573] The generated information is sent from the cloud server to the in-vehicle terminal, and the information is provided to the user using a voice synthesis system.

[0574] Specifically, the generated information is sent to the terminal, converted into voice by a voice synthesis system, and provided to the user.

[0575] Input: Generated interesting information

[0576] Output: Providing information as synthesized speech

[0577] Specific examples

[0578] For example, if a user enters the information "Name: Takashi, Age: 30, Interests: History and Culture" and sets "Museum" as the destination, the prompt might look like this:

[0579] "Takashi-san, the museum houses ancient Roman artifacts."

[0580] This information is generated on a cloud server, sent to the in-vehicle terminal, and provided to the user audibly through a voice synthesis system.

[0581] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0582] This paper describes a system that combines a car navigation system that uses a generative AI model to improve the driver's driving experience with an emotion engine to provide information tailored to individual user needs. This system consists of a cloud server, an in-vehicle terminal, a driver, and an emotion engine.

[0583] Basic system configuration

[0584] 1. User Initial Setup:

[0585] The user starts up the in-car device and enters their profile information (such as name, age, and areas of interest). For example, this information might include "Name: Tarou, Age: 35, Areas of Interest: History and Nature."

[0586] The device sends this profile information to a cloud server, which stores the information in a database.

[0587] 2. Destination setting and route calculation:

[0588] The user inputs destination information into the in-vehicle terminal. For example, the user inputs "Tokyo Tower."

[0589] The device sends this destination information to a cloud server, which then calculates the optimal route based on real-time traffic information.

[0590] The calculated route information is sent to the in-vehicle terminal and displayed to the user, and voice guidance is also provided.

[0591] 3. Emotion Recognition with Emotion Engine:

[0592] The emotion engine analyzes the driver's voice and facial expressions, for example, by monitoring voice tone and facial expressions in real time to understand the driver's emotional state.

[0593] It analyzes whether the user is relaxed or stressed and gives real-time feedback while driving.

[0594] 4. Providing interesting information:

[0595] The server uses a generative AI model to generate interesting trivia and useful information at the appropriate time based on the user's profile information and the recognition results of the emotion engine. For example, if the user is interested in "history" and is relaxing, the server will generate information such as "Tokyo Tower was completed in 1958." On the other hand, if the user is feeling stressed, the server will provide relaxing music or topics of interest.

[0596] The generated information is transmitted to an in-vehicle terminal and provided to the user by voice.

[0597] Specific processing of the program

[0598] Emotion Engine in action:

[0599] The device captures the user's voice and facial expressions through a camera and microphone, and the emotion engine analyzes them in real time. The analysis results are then sent to the server.

[0600] For example, if the user is smiling while driving, it is determined that the user is in a relaxed state, whereas if the user is speaking in a loud voice, it is determined that the user is feeling stressed.

[0601] Information generation and provision:

[0602] The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine.

[0603] When the user is in a relaxed state, interesting trivia is provided aloud, such as "Tokyo Tower is 333 meters tall."

[0604] For users who are under stress, the system plays relaxing music and provides encouraging words about driving.

[0605] In this way, while driving, users can always select the optimal route based on the latest traffic information, and receive information and music tailored to their interests and emotional state, making long drives stress-free and more fulfilling. Real-time feedback from the emotion engine also improves driver safety. For example, if a stressful state is detected, a message recommending safe driving can be displayed or music that promotes relaxation can be played. Providing both physical and psychological comfort simultaneously can significantly improve the driver's overall driving experience.

[0606] The processing flow will be explained below.

[0607] Step 1:

[0608] The user starts up the in-car device and enters profile information (such as name, age, and areas of interest). For example, the user enters "Name: Tarou, Age: 35, Areas of interest: History and nature."

[0609] Step 2:

[0610] The device sends the entered profile information to a cloud server, which stores the received information in a database.

[0611] Step 3:

[0612] The user inputs destination information into the in-vehicle terminal. For example, the user inputs "Tokyo Tower."

[0613] Step 4:

[0614] The device sends this destination information to a cloud server, which collects real-time traffic information and calculates the optimal route.

[0615] Step 5:

[0616] The server sends the calculated optimal route information to the terminal, which displays the received optimal route information to the user and starts voice guidance.

[0617] Step 6:

[0618] The device captures the user's voice and facial expressions through a camera and microphone, and the emotion engine analyzes them in real time. The analysis results are then sent to a cloud server.

[0619] Step 7:

[0620] The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine. If the user is in a relaxed state, it generates trivia such as "Tokyo Tower was completed in 1958." If the user is in a stressed state, it generates relaxing music or an encouraging message.

[0621] Step 8:

[0622] The server sends the generated information to the terminal, and the terminal provides the received information to the user by voice.

[0623] Step 9:

[0624] The device then plays back the received information through a speaker and provides it to the user. For example, it might say, "Tokyo Tower was completed in 1958."

[0625] Step 10:

[0626] The server monitors traffic conditions in real time and recalculates routes as needed. If a new route is required, it notifies the device and updates the optimal route information.

[0627] Step 11:

[0628] The device displays the updated optimal route information to the user and updates the voice guidance. The device continues to provide voice guidance to the user about the recalculated route.

[0629] In this way, the system improves the user's driving experience by providing personalized information based on the user's profile and emotional state, and by providing optimal route guidance that reflects traffic conditions in real time.

[0630] Example 2

[0631] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0632] Conventional car navigation systems do not adequately provide information that corresponds to the driver's emotional state or individual interests, making it difficult to relieve stress associated with long drives and resulting in a less than fulfilling driving experience.In addition, route calculations based solely on real-time traffic information make it difficult to provide an optimal driving experience that corresponds to the driver's emotions and interests.

[0633] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for inputting driver profile information and transmitting it to a cloud server, a means for inputting destination information and transmitting it to the cloud server, a means for collecting real-time traffic information and calculating an optimal route, a means for capturing the driver's voice and facial expression and analyzing the emotional state, a means for transmitting the analysis results to the cloud server, a means for the cloud server to generate appropriate information using a generative artificial intelligence model based on the driver's profile information and emotional state, and a means for providing the generated information to the driver by voice. This makes it possible to provide information according to the driver's emotional state and interests, thereby reducing stress and enhancing the driving experience.

[0634] "Driver profile information" is basic information about each individual driver, such as the driver's name, age, and areas of interest.

[0635] A "cloud server" is a remote server that operates on the Internet and stores and processes driver profile information and destination information.

[0636] "Destination information" refers to information such as the address and place name of the destination that the driver inputs into the in-vehicle terminal.

[0637] "Real-time traffic information" is data that provides current road conditions, traffic volume, accident information, etc. in real time.

[0638] "Optimal route" refers to the most efficient and safe route for a driver based on real-time traffic information.

[0639] "Driver's voice and facial expression" refers to the voice and facial expression of the driver, and is data for analyzing the emotional state through these.

[0640] "Emotional state" refers to the driver's current psychological state (relaxed, stressed, etc.).

[0641] A "generative artificial intelligence model" is an AI technology that generates appropriate information based on input data.

[0642] "Relevant information" is interesting tidbits or helpful information generated based on the driver's profile information and emotional state.

[0643] An "in-vehicle terminal" is a device installed in a vehicle that allows the driver to input profile information and destination information, and capture voice and facial expressions.

[0644] "Means for providing by voice" refers to technology or devices for conveying the generated information to the driver by voice.

[0645] The present invention relates to a car navigation system that improves the driver's driving experience by providing information tailored to the driver's emotional state and individual interests. This system is composed of a cloud server, an in-vehicle terminal, a driver, and an emotion engine.

[0646] Basic system configuration

[0647] User Initial Settings

[0648] The user starts up the in-car terminal and uses the touchscreen to enter profile information (such as name, age, and areas of interest). For example, this information might include "Name: Tarou, Age: 35, Areas of Interest: History and Nature." The terminal then sends this profile information to a cloud server, which then stores the information in a database. The hardware used is an in-car terminal with a touchscreen, and the software is a cloud database management system.

[0649] Destination setting and route calculation

[0650] The user inputs destination information into the in-vehicle terminal. For example, they input "Tokyo Tower." The terminal sends this destination information to a cloud server. The server calculates the optimal route based on real-time traffic information. The calculated route information is sent to the in-vehicle terminal and displayed to the user, while voice guidance is also provided. The hardware used here is an in-vehicle terminal with GPS functionality, and the software is a real-time traffic information API and a route calculation algorithm.

[0651] emotion recognition

[0652] The emotion engine analyzes the driver's voice and facial expressions. Using the device's camera and microphone, it monitors the driver's voice tone and facial expressions in real time to understand the driver's emotional state. The analysis results are sent to a cloud server. The hardware used here is an in-car device equipped with a camera and microphone, and the software is an emotion analysis algorithm. For example, if the user is smiling while driving, it is determined that the user is relaxed. If the user's voice is raised, it can be determined that the user is stressed.

[0653] Information generation and provision

[0654] The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine. For users who are in a relaxed state, interesting trivia is provided by voice. For example, information such as "Tokyo Tower is 333 meters tall" is generated. On the other hand, for users who are in a stressed state, relaxing music is played or encouraging words about driving are provided. As a concrete example, the prompt text for the generative AI model is as follows:

[0655] Prompt: "Generate some trivia about Tokyo Tower for a relaxed user who is interested in history."

[0656] AI model output: "Tokyo Tower was completed in 1958. It is 333 meters tall."

[0657] The generated information is sent to the in-vehicle terminal and provided to the driver using the terminal's voice synthesis function. The software used here is a generative artificial intelligence model and a voice synthesis system.

[0658] In this way, while driving, users can always select the optimal route based on the latest traffic information, and receive information and music tailored to their interests and emotional state, making long drives stress-free and more fulfilling. Real-time feedback from the emotion engine also improves driver safety. For example, if a stressful state is detected, a message recommending safe driving can be displayed or music that promotes relaxation can be played. Providing both physical and psychological comfort simultaneously can significantly improve the driver's overall driving experience.

[0659] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0660] System program processing flow

[0661] Step 1: User Initial Setup

[0662] Input: User profile information (e.g., name, age, interests)

[0663] Output: Profile information sent to cloud server and stored in database

[0664] 1. The user starts up the in-car device and operates the touch screen to enter profile information.

[0665] Specific operation: The user enters "Name: Tarou, Age: 35, Areas of interest: History and nature" into the input form on the device.

[0666] 2. The device sends the entered profile information to the cloud server.

[0667] Specific operation: When the device presses the send button, it sends the profile information to the cloud server via the Internet connection.

[0668] 3. The server stores the received profile information in a database.

[0669] What happens: The server's database management system automatically creates a record for the new user and records their profile information.

[0670] Step 2: Set destination and calculate route

[0671] Input: Destination information (e.g. Tokyo Tower)

[0672] Output: Optimal route information is displayed on the in-car device and voice guidance begins.

[0673] 1. The user inputs destination information into the in-vehicle terminal.

[0674] Specific operation: The user uses the touch screen and the keyboard to input "Tokyo Tower."

[0675] 2. The device sends the entered destination information to the cloud server.

[0676] Specific operation: The terminal transmits destination information to the server via the Internet in real time.

[0677] 3. The server calculates the optimal route based on real-time traffic information.

[0678] Specific operation: The server obtains data from the traffic information API and uses an algorithm to calculate the optimal route.

[0679] 4. The server sends the calculated route information to the in-vehicle terminal, which displays and provides voice guidance to the user.

[0680] Specific operation: The server sends route information to the terminal, the terminal displays the route map on the screen, and the voice guidance system provides instructions by voice.

[0681] Step 3: Emotion Recognition

[0682] Input: Driver's voice and facial expression capture data

[0683] Output: The analyzed emotional state (e.g., relaxed, stressed) is sent to the cloud server.

[0684] 1. The device captures the driver's voice and facial expressions through a camera and microphone.

[0685] Specific operation: The device's built-in camera and microphone automatically record the user's face and voice.

[0686] 2. The emotion engine analyzes the captured data in real time.

[0687] How it works: The built-in software uses visual recognition and voice analysis technology to analyze the user's emotions (relaxed, stressed, etc.).

[0688] 3. The analysis results are sent to the cloud server.

[0689] Specific operation: The device sends analysis data to the server in real time.

[0690] Step 4: Information generation and provision

[0691] Input: Profile information, emotional state

[0692] Output: The generated appropriate information (e.g., trivia or relaxing music) is provided to the user via the in-car terminal.

[0693] 1. The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine.

[0694] Specific operation: The server inputs the prompt "Profile information: History lover, Emotional state: Relaxed" to the generated AI model.

[0695] Example prompt sentence:

[0696] Prompt: "Generate some trivia about Tokyo Tower for a relaxed user who is interested in history."

[0697] AI model output: "Tokyo Tower was completed in 1958. It is 333 meters tall."

[0698] 2. The server sends the generated information to the in-vehicle terminal, which then provides it to the user via voice.

[0699] Specific operation: The server sends the generated information to the terminal, and the terminal's voice synthesis function conveys the information to the user.

[0700] Through the above processing steps, the system can provide optimized information according to the driver's emotional state and interests, improving the driving experience.

[0701] (Application example 2)

[0702] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0703] Conventional car navigation systems only provide optimal routes using the driver's profile and real-time traffic information, but lack information tailored to the driver's emotional state and interests. This can make long drives stressful and reduce safety. Furthermore, if the information provided during driving does not match the driver's interests, satisfaction can decrease.

[0704] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting driver profile information and transmitting it to a cloud server, means for inputting destination information and transmitting it to the cloud server, means for collecting real-time traffic information and calculating an optimal route, means for providing the optimal route to the user in real time, means for analyzing the user's emotional state, and means for generating and providing information based on the user's interests and emotional state. This makes it possible to provide appropriate information according to the driver's emotional state, thereby reducing stress while driving, improving safety, and improving the driving experience.

[0705] "Driver profile information" refers to personal information such as the driver's name, age, and areas of interest.

[0706] A "cloud server" refers to a server that provides resources and services over the Internet.

[0707] "Destination information" refers to information such as the name and address of the destination set by the driver.

[0708] "Real-time traffic information" refers to data collected in real time about current traffic conditions.

[0709] An "optimal route" refers to the most efficient route for a driver's travel.

[0710] "User" refers to a driver who uses the system.

[0711] "Emotional state" refers to the driver's current emotional and psychological state.

[0712] A "generative artificial intelligence model" refers to an artificial intelligence model that automatically generates appropriate information based on input data.

[0713] This invention is a car navigation system that provides customized information based on the driver's profile information and real-time emotional state. The system consists of a cloud server, an in-vehicle terminal, a driver, and an emotion engine.

[0714] First, the user starts up the in-car device and enters their profile information (such as name, age, and areas of interest). For example, a user might enter information such as "Name: ABC, Age: 35, Areas of Interest: History and Nature." This information is sent from the device to the cloud server and stored in a database.

[0715] Next, the user inputs destination information into the in-vehicle terminal. For example, they input "major tourist destinations." This destination information is also sent to the cloud server, which calculates the optimal route based on real-time traffic information. The calculated route information is sent back to the in-vehicle terminal and displayed to the user. Voice guidance is also provided.

[0716] While driving, the emotion engine analyzes the driver's voice and facial expressions in real time. Specifically, the vehicle's camera and microphone capture the driver's video and audio, which the emotion engine analyzes. For example, it determines whether the driver is relaxed or stressed. The results of this analysis are sent to a cloud server.

[0717] The cloud server uses a generative AI model based on the driver's profile information and the analysis results of the emotion engine to generate appropriate information. For example, if the driver is relaxed and interested in "history and nature," the generative AI model will generate information such as "Major tourist destinations were completed in 1958." On the other hand, if the driver is feeling stressed, the model will provide relaxing music and encouraging words.

[0718] The generated information is sent to an in-vehicle terminal and provided to the driver via voice, allowing the driver to receive customized information at the right time, making the drive more comfortable.

[0719] As a concrete example, we will show an example of a prompt sentence to be input to the generative AI model.

[0720] Example prompt sentence:

[0721] User profile:

[0722] Name: ABC, Age: 35, Areas of Interest: History and Nature

[0723] Driver's state: Relaxed

[0724] prompt:

[0725] Based on the user profile and driver status, generate interesting information about their history to help them relax.

[0726] This system aims to reduce driver stress and improve safety by dynamically customizing information provided while driving based on the driver's emotional state and interests, and also to improve the driver's driving experience.

[0727] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0728] Step 1:

[0729] The user inputs profile information from the in-vehicle terminal, and the terminal transmits the information to the cloud server.

[0730] Specifically, when a user enters data such as their name, age, and areas of interest and presses the "Submit" button, the device sends the input data to the cloud server as a POST request, and the input data is stored directly in the cloud database.

[0731] Input: User profile information (name, age, interests)

[0732] Output: Profile information stored on the cloud server

[0733] Step 2:

[0734] The user inputs destination information from the in-vehicle terminal, and the terminal transmits the information to the cloud server.

[0735] Specifically, when a user inputs a destination (e.g., a major tourist spot) and presses the "Send" button, the device sends the input data to the cloud server as a POST request. The cloud server receives this data and calculates the optimal route based on real-time traffic information.

[0736] Input: User's destination information

[0737] Output: Optimal route calculation by cloud server

[0738] Step 3:

[0739] The cloud server sends the calculated optimal route information to the in-vehicle terminal, which then displays it to the user.

[0740] Specifically, the cloud server returns the calculation results to the in-vehicle terminal as a response, and the terminal displays the information on the screen and starts voice guidance.

[0741] Input: Optimal route information calculated by the cloud server

[0742] Output: Optimal route information and voice guidance displayed on the in-car device

[0743] Step 4:

[0744] The device captures the driver's voice and facial expressions through a camera and microphone, and the emotion engine analyzes them in real time.

[0745] Specifically, the in-car camera captures the driver's facial expressions and the microphone captures their voice, and the data is sent to the emotion engine, which analyzes it to determine the driver's emotional state and sends the results to a cloud server.

[0746] Input: Driver's facial image and voice data

[0747] Output: Analysis result of emotional state by emotion engine

[0748] Step 5:

[0749] The cloud server uses a generative AI model to generate appropriate information based on the driver's profile information and the analysis results of the emotion engine.

[0750] Specifically, the cloud server sends data along with the prompt text to the generative AI model and receives the generated information, which includes content that corresponds to the driver's interests and emotional state.

[0751] Input: Driver profile information, emotional state

[0752] Output: Customized information from a generative AI model

[0753] Step 6:

[0754] The cloud server transmits the generated customization information to the in-vehicle terminal, which then provides it to the user via voice.

[0755] Specifically, the cloud server sends the generated information back to the in-vehicle terminal as a response, and the terminal then conveys the information to the user via voice. For example, it might provide interesting trivia to a relaxed driver, or relaxing music or words of encouragement to a stressed driver.

[0756] Input: Customization information generated by the generative AI model

[0757] Output: Audio information presented to the user

[0758] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0759] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0760] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0761] [Third embodiment]

[0762] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0763] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0764] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0765] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0766] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0767] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0768] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0769] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0770] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0771] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0772] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0773] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0774] This invention describes a car navigation system equipped with a generative AI model to improve the driver's driving experience. The system consists of a cloud server, an in-vehicle terminal, and a driver.

[0775] Basic system configuration

[0776] 1. User Initial Setup:

[0777] The user starts up the in-car device and enters profile information such as name, age, and areas of interest. For example, this information might include "Name: Tarou, Age: 35, Areas of interest: History and nature."

[0778] The device sends this profile information to a cloud server, which stores the information in a database.

[0779] 2. Destination setting and route calculation:

[0780] The user inputs destination information into the in-vehicle terminal. For example, the user inputs "Tokyo Tower."

[0781] The device sends this destination information to a cloud server, which then calculates the optimal route based on real-time traffic information.

[0782] The calculated route information is sent to the in-vehicle terminal and displayed to the user, and voice guidance is also provided.

[0783] 3. Providing interesting information:

[0784] The server generates interesting trivia and useful information at appropriate times based on the user's profile information. For example, if a user is interested in "history," it generates information such as "Tokyo Tower was completed in 1958."

[0785] The generated information is transmitted to an in-vehicle terminal and provided to the user by voice.

[0786] Specific examples

[0787] Initial Settings:

[0788] The user follows the instructions on the device screen and enters "Name: Tarou, Age: 35, Interests: History and Nature." The device sends this information to the server, which then stores the received profile in a database.

[0789] Enter destination and get directions:

[0790] The user inputs "Tokyo Tower" as the destination for a trip. The device sends the location information of "Tokyo Tower" to the server, which then collects real-time traffic information and calculates the optimal route. The calculated route is sent to the device and displayed to the user in real time.

[0791] Providing information while driving:

[0792] While driving, the server uses a generative AI model to generate historical trivia based on the user's interests, and sends information such as "Tokyo Tower was completed in 1958" to the device. The device then relays the received information to the user via voice.

[0793] In this way, users can safely reach their destination while receiving real-time information relevant to their interests while driving. The system also provides optimal routes, helping users avoid traffic congestion and inconvenience. This makes long drives less boring and more enjoyable.

[0794] The processing flow will be explained below.

[0795] Step 1:

[0796] The user starts up the in-car terminal and enters profile information (such as name, age, areas of interest, etc.).

[0797] Step 2:

[0798] The device sends the entered profile information to the cloud server. For example, it sends "Name: Tarou, Age: 35, Areas of Interest: History and Nature."

[0799] Step 3:

[0800] The server stores the received profile information in a database.

[0801] Step 4:

[0802] The user inputs destination information (for example, "Tokyo Tower") into the in-vehicle terminal.

[0803] Step 5:

[0804] The terminal transmits the destination information to the cloud server.

[0805] Step 6:

[0806] The server collects real-time traffic information and calculates the optimal route, including data on congestion, accidents, and construction.

[0807] Step 7:

[0808] The server transmits the calculated optimum route information to the terminal.

[0809] Step 8:

[0810] The terminal displays the received optimum route information to the user and starts voice guidance.

[0811] Step 9:

[0812] While driving, the server uses an AI model to generate trivia and useful information based on the user's profile information. For example, based on the "History" category, it generates information such as "Tokyo Tower was completed in 1958."

[0813] Step 10:

[0814] The server transmits the generated information to the terminal.

[0815] Step 11:

[0816] The device provides the received information to the user via voice, allowing the user to receive information relevant to their interests in real time while driving safely.

[0817] Example 1

[0818] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0819] Conventional car navigation systems only provide support to help drivers reach their destination, making it difficult to improve the quality of the driving experience itself. Furthermore, they lacked features such as route recalculation based on real-time traffic information and information provision based on the driver's interests, making long drives boring and monotonous. This resulted in a decline in the quality and efficiency of information provision while driving, and insufficient safety.

[0820] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0821] In this invention, the server includes means for inputting driver profile information and transmitting it to the cloud server, means for inputting destination information and transmitting it to the cloud server, means for collecting real-time traffic information and calculating an optimal route, means for transmitting the calculated route information to an in-vehicle terminal and displaying and providing voice guidance to the user, means for using a generative artificial intelligence model to generate trivia and useful information based on the driver's profile information and current location information, and means for transmitting the generated information to the in-vehicle terminal and providing it to the driver by voice. This makes it possible to provide individual information based on the driver's profile and guidance on an optimal route that reflects real-time traffic information, thereby improving the quality of the driving experience and ensuring safety while driving.

[0822] "Driver profile information" refers to information about an individual, such as the driver's name, age, and areas of interest.

[0823] A "cloud server" refers to a server that provides data and services via the Internet.

[0824] "Destination information" refers to information about a destination that the driver inputs into the in-vehicle terminal.

[0825] "Real-time traffic information" is data showing current traffic conditions, including information on congestion and traffic restrictions.

[0826] "Optimal route" refers to the most efficient and safe route to your destination, calculated based on real-time traffic information.

[0827] "In-vehicle terminal" refers to an electronic terminal equipped with navigation and information provision functions that is installed in a vehicle.

[0828] "Voice guidance" refers to route guidance and information provided by voice generated by an in-vehicle terminal.

[0829] A "generative artificial intelligence model" refers to an artificial intelligence program that uses techniques such as natural language processing to generate new information.

[0830] A "prompt sentence" refers to an input sentence that causes a generative artificial intelligence model to generate information.

[0831] "Trivia and Helpful Information" refers to short pieces of information and helpful data provided based on a driver's interests and profile.

[0832] The present invention relates to a car navigation system equipped with a generative AI model for improving the driver's driving experience. The system consists of a cloud server, an in-vehicle terminal, and a driver.

[0833] Basic system configuration

[0834] 1. Initial Setup

[0835] The user starts up the in-car device and enters profile information such as name, age, and areas of interest. Input is done using the in-car device's touchscreen or voice recognition function. For example, the user might enter information such as "Name: Tarou, Age: 35, Interests: History and Nature." The device then sends this profile information to a cloud server via an internet connection (using 4G or 5G networks). The server then stores the received profile information in an SQL database.

[0836] 2. Setting a destination and calculating a route

[0837] The user inputs destination information into the in-car device. For example, they input "Tokyo Tower" using the touchscreen. The device then sends this destination information to a cloud server. The server uses the Google Maps API, HERE API, etc. to collect traffic information in real time and calculates the optimal route by sending a request such as "current_location: 'x,y', destination: 'Tokyo Tower'" to the API. The calculated route information is then sent from the server to the in-car device and displayed and provided to the user via display and voice guidance.

[0838] 3. Providing interesting information

[0839] The server references the saved user profile information and confirms, for example, that the user is interested in "history." Next, it generates and sends a prompt to the generative AI model (e.g., OpenAI GPT-3). An example prompt might be, "The user is interested in history and is currently driving toward Tokyo Tower. Please generate some historical trivia related to Tokyo Tower." Based on this prompt, the generative AI model generates information such as, "Tokyo Tower was completed in 1958." This information is sent from the server to the in-vehicle device and provided to the user audibly using a speech synthesis function (using Amazon Polly or Google Text-to-Speech).

[0840] Specific examples

[0841] 1. Initial Setup

[0842] The user follows the on-board device's screen and enters "Name: Tarou, Age: 35, Interests: History and Nature." The device then sends this information to the cloud server, which stores it in a database.

[0843] 2. Enter your destination and get route guidance

[0844] The user inputs "Tokyo Tower" as their destination. The device sends this information to a cloud server, which then collects real-time traffic information and calculates the optimal route. The calculated route is sent to the device and displayed and guided to the user in real time via display and voice guidance.

[0845] 3. Providing information while driving

[0846] While driving, the server uses a generative AI model to generate historical trivia, such as "Tokyo Tower was completed in 1958." This information is sent to the device and provided to the user via voice.

[0847] In this way, drivers can safely reach their destination while receiving interesting information in real time. Furthermore, by providing optimal routes, it is possible to avoid traffic congestion and inconvenience. This makes long driving trips less boring and more enjoyable.

[0848] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0849] Step 1:

[0850] The user starts up the in-car device and enters profile information such as name, age, and areas of interest. Input is done using the in-car device's touchscreen or voice recognition function. For example, the input information might be "Name: Tarou, Age: 35, Interests: History and Nature." This information is aggregated as input information. The device then sends this information to a cloud server via an internet connection (using 4G or 5G networks). This information is the output information.

[0851] Step 2:

[0852] The server stores the received profile information in a database. Specifically, it uses an SQL database to store it as a record such as "username: 'Tarou', age: 35, interests: 'history, nature'". The input information is the profile information sent from the device, which is processed and stored in the database. Updates to this database become the output information.

[0853] Step 3:

[0854] The user inputs destination information into the in-car terminal. For example, they input "Tokyo Tower" using the touch screen. This is the input of destination information. The terminal then sends the destination information to the cloud server. The destination information becomes output data.

[0855] Step 4:

[0856] The server collects real-time traffic information using Google Maps API, HERE API, etc. It receives traffic condition data by sending a request to the API such as "current_location: 'x,y', destination: 'Tokyo Tower'". This is the input information. The optimal route is calculated based on the input data. The Dijkstra algorithm or A algorithm is used for the calculation. The calculation result is generated as output data.

[0857] Step 5:

[0858] The server sends the calculated route information to the in-vehicle terminal. The sent data includes multiple waypoints (passing points). This is the output route information. The terminal displays the received route information on the user's display and begins guiding the user using voice guidance. Detailed navigation information such as "Turn right at the next intersection" is displayed on the display. This is the final output information.

[0859] Step 6:

[0860] The server references the saved user profile information and confirms that the user is interested in "history." This is the input information. Next, it generates and sends a prompt to the generative AI model. An example of a prompt is, "The user is interested in history and is currently driving toward Tokyo Tower. Please generate some historical trivia related to Tokyo Tower." The prompt becomes the input to the generative AI model.

[0861] Step 7:

[0862] The server sends a prompt to a generative AI model (e.g., OpenAI GPT-3) and generates appropriate information. The generative AI model generates information such as "Tokyo Tower was completed in 1958." This is the output data.

[0863] Step 8:

[0864] The server sends the generated information to the in-vehicle terminal. This information may also be sent in audio format. This is the output information. The terminal then conveys the received information to the user by voice. Specifically, it uses a function that generates speech from text (for example, Amazon Polly or Google Text-to-Speech). This is the final output information.

[0865] Through these steps, drivers can reach their destination safely while receiving interesting information in real time, and avoid traffic congestion and inconvenience by being offered optimal routes.

[0866] (Application example 1)

[0867] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0868] Conventional driving experience enhancement systems have limited means of providing drivers with information of interest, resulting in a lack of entertainment while driving. Furthermore, route guidance to a destination often lacks real-time traffic information updates, making it difficult to respond to traffic congestion and route changes. The present invention aims to solve these problems and improve the travel experience for passengers in autonomous vehicles.

[0869] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0870] In this invention, the server includes means for inputting driver profile information and transmitting it to the cloud server, means for inputting destination information and transmitting it to the cloud server, means for collecting real-time traffic information and calculating the optimal route, means for providing the optimal route to the user in real time, means for generating and providing trivia and useful information based on the user's interests, and means that is installed in the autonomous vehicle and provides information by voice in real time to the user while traveling. This makes it possible to provide information based on the user's profile and provide optimal route guidance, thereby enriching the travel experience.

[0871] "Driver profile information" refers to personal information such as the user's name, age, and areas of interest, and is information that is sent to the cloud server.

[0872] A "cloud server" is a remote server that provides data storage, processing, and access over the Internet.

[0873] "Destination information" refers to geographical information about a place the user wants to visit, and is information that is sent to the cloud server.

[0874] "Real-time traffic information" is data showing current traffic conditions, including traffic volume, speed limits, accident information, and the like.

[0875] The "optimal route" is the most efficient and safe route to your destination, calculated based on real-time traffic information.

[0876] "Means for providing the user with the optimal route in real time" refers to means for displaying the calculated optimal route to the user and providing voice guidance.

[0877] "Trivia and useful information based on your interests" refers to information about history, nature, and other topics that are likely to be of interest to you based on your profile information.

[0878] A "generative artificial intelligence model" is an artificial intelligence system that has the ability to generate new information based on massive amounts of data.

[0879] An "autonomous vehicle" is a vehicle equipped with a system that automates human driving behavior and is capable of running without the intervention of a driver.

[0880] "Means for providing information by voice" refers to a method of converting generated information into voice and conveying it to the user.

[0881] The present invention relates to a system that uses a generative AI model to provide users with interesting information based on driver profile information and destination information, thereby improving the travel experience in an autonomous vehicle. Specific embodiments are described below.

[0882] System configuration

[0883] 1. Cloud Server

[0884] Receives and processes profile information, destination information, and real-time traffic information.

[0885] It is equipped with a generative AI model that generates user-based trivia and useful information.

[0886] 2. In-vehicle terminal

[0887] Provide a means for users to enter profile and destination information.

[0888] It receives data from the cloud server and provides users with the best route and information.

[0889] A voice output means is used to provide information to the user audibly.

[0890] 3. Self-driving vehicles

[0891] The vehicle itself is equipped with a system that automates driving, so the user does not need to be involved in driving.

[0892] Enriching users' travel experience by providing information while driving.

[0893] How it works

[0894] Entering user profile information

[0895] The user starts up the in-car device and enters profile information such as name, age, areas of interest, etc. This information is sent to the cloud server and stored in the server's database.

[0896] Enter your destination and calculate your route

[0897] When a user inputs destination information into the in-vehicle terminal, the terminal sends this information to a cloud server. The cloud server collects traffic information in real time, calculates the optimal route, and sends it to the in-vehicle terminal. The in-vehicle terminal then displays the calculated route information to the user and provides voice guidance.

[0898] Providing interesting information

[0899] The cloud server uses a generative AI model to generate trivia and useful information related to the user's interests based on the user's profile information. The generated information is sent to the in-vehicle device in real time and provided to the user via a voice synthesis system. Specifically, a prompt sentence is entered and passed to the generative AI model, and the generated text is then played back by the voice synthesis system.

[0900] Hardware and software used

[0901] Cloud server: Uses AWS or Google Cloud Platform.

[0902] In-vehicle terminal: A device such as a tablet or smartphone.

[0903] Generative AI models: Generative artificial intelligence models such as OpenAI's GPT and Google's BERT.

[0904] Speech synthesis system: Uses Google Text-to-Speech, Amazon Polly, etc.

[0905] Specific examples

[0906] For example, if a user enters information such as "Name: Takashi, Age: 30, Interests: History and Culture" and sets "Museum" as the destination, the cloud server will generate information such as "The museum houses ancient Roman artifacts." This information is sent back to the in-vehicle terminal and provided to the user aloud via a speech synthesis system.

[0907] Example prompt sentence:

[0908] "Takashi-san, the museum houses ancient Roman artifacts."

[0909] In this way, users can receive information of interest in real time while on the move, enriching their in-car experience.

[0910] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0911] Step 1:

[0912] The user enters profile information on the in-car terminal.

[0913] Specifically, users enter information such as their name, age, and areas of interest into the device screen, and this profile information is then sent to the cloud server.

[0914] Input: User's first and last name, age, and interests

[0915] Output: Send profile information to cloud server

[0916] Step 2:

[0917] The cloud server stores the received profile information in a database.

[0918] Specifically, the server analyzes the received data, converts it into an appropriate format, and stores it in a database. This information is then used by a generative AI model to generate information based on the user's interests.

[0919] Input: User profile information

[0920] Output: Profile information stored in a database

[0921] Step 3:

[0922] The user inputs destination information into the in-car terminal.

[0923] Specifically, users input their "destination" on their device, which is then sent to a cloud server, which then uses it to calculate the optimal route.

[0924] Input: Destination information

[0925] Output: Send destination information to the cloud server

[0926] Step 4:

[0927] A cloud server collects real-time traffic information and calculates the optimal route.

[0928] Specifically, the server collects real-time information on current traffic conditions, road closures, traffic congestion, etc., and calculates the optimal route to the destination based on this information. This calculation is performed using an algorithm, and the calculation results are sent to the in-vehicle terminal.

[0929] Input: Destination information, real-time traffic information

[0930] Output: Calculated optimal route information

[0931] Step 5:

[0932] The in-car device displays the optimal route to the user and begins voice guidance.

[0933] Specifically, route information sent from a cloud server is displayed on the terminal, and a voice synthesis system is used to provide real-time voice guidance to the user.

[0934] Input: Optimal route information

[0935] Output: Route display, voice guidance

[0936] Step 6:

[0937] The cloud server generates interesting information based on the user's profile information using a generative AI model.

[0938] Specifically, a prompt sentence is created based on the user's interest information (e.g., history, nature), and passed to a generative AI model to generate useful information and trivia.

[0939] Input: User profile information

[0940] Output: interesting information generated

[0941] Step 7:

[0942] The generated information is sent from the cloud server to the in-vehicle terminal, and the information is provided to the user using a voice synthesis system.

[0943] Specifically, the generated information is sent to the terminal, converted into voice by a voice synthesis system, and provided to the user.

[0944] Input: Generated interesting information

[0945] Output: Providing information as synthesized speech

[0946] Specific examples

[0947] For example, if a user enters the information "Name: Takashi, Age: 30, Interests: History and Culture" and sets "Museum" as the destination, the prompt might look like this:

[0948] "Takashi-san, the museum houses ancient Roman artifacts."

[0949] This information is generated on a cloud server, sent to the in-vehicle terminal, and provided to the user audibly through a voice synthesis system.

[0950] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0951] This paper describes a system that combines a car navigation system that uses a generative AI model to improve the driver's driving experience with an emotion engine to provide information tailored to individual user needs. This system consists of a cloud server, an in-vehicle terminal, a driver, and an emotion engine.

[0952] Basic system configuration

[0953] 1. User Initial Setup:

[0954] The user starts up the in-car device and enters their profile information (such as name, age, and areas of interest). For example, this information might include "Name: Tarou, Age: 35, Areas of Interest: History and Nature."

[0955] The device sends this profile information to a cloud server, which stores the information in a database.

[0956] 2. Destination setting and route calculation:

[0957] The user inputs destination information into the in-vehicle terminal. For example, the user inputs "Tokyo Tower."

[0958] The device sends this destination information to a cloud server, which then calculates the optimal route based on real-time traffic information.

[0959] The calculated route information is sent to the in-vehicle terminal and displayed to the user, and voice guidance is also provided.

[0960] 3. Emotion Recognition with Emotion Engine:

[0961] The emotion engine analyzes the driver's voice and facial expressions, for example, by monitoring voice tone and facial expressions in real time to understand the driver's emotional state.

[0962] It analyzes whether the user is relaxed or stressed and gives real-time feedback while driving.

[0963] 4. Providing interesting information:

[0964] The server uses a generative AI model to generate interesting trivia and useful information at the appropriate time based on the user's profile information and the recognition results of the emotion engine. For example, if the user is interested in "history" and is relaxing, the server will generate information such as "Tokyo Tower was completed in 1958." On the other hand, if the user is feeling stressed, the server will provide relaxing music or topics of interest.

[0965] The generated information is transmitted to an in-vehicle terminal and provided to the user by voice.

[0966] Specific processing of the program

[0967] Emotion Engine in action:

[0968] The device captures the user's voice and facial expressions through a camera and microphone, and the emotion engine analyzes them in real time. The analysis results are then sent to the server.

[0969] For example, if the user is smiling while driving, it is determined that the user is in a relaxed state, whereas if the user is speaking in a loud voice, it is determined that the user is feeling stressed.

[0970] Information generation and provision:

[0971] The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine.

[0972] When the user is in a relaxed state, interesting trivia is provided aloud, such as "Tokyo Tower is 333 meters tall."

[0973] For users who are under stress, the system plays relaxing music and provides encouraging words about driving.

[0974] In this way, while driving, users can always select the optimal route based on the latest traffic information, and receive information and music tailored to their interests and emotional state, making long drives stress-free and more fulfilling. Real-time feedback from the emotion engine also improves driver safety. For example, if a stressful state is detected, a message recommending safe driving can be displayed or music that promotes relaxation can be played. Providing both physical and psychological comfort simultaneously can significantly improve the driver's overall driving experience.

[0975] The processing flow will be explained below.

[0976] Step 1:

[0977] The user starts up the in-car device and enters profile information (such as name, age, and areas of interest). For example, the user enters "Name: Tarou, Age: 35, Areas of interest: History and nature."

[0978] Step 2:

[0979] The device sends the entered profile information to a cloud server, which stores the received information in a database.

[0980] Step 3:

[0981] The user inputs destination information into the in-vehicle terminal. For example, the user inputs "Tokyo Tower."

[0982] Step 4:

[0983] The device sends this destination information to a cloud server, which collects real-time traffic information and calculates the optimal route.

[0984] Step 5:

[0985] The server sends the calculated optimal route information to the terminal, which displays the received optimal route information to the user and starts voice guidance.

[0986] Step 6:

[0987] The device captures the user's voice and facial expressions through a camera and microphone, and the emotion engine analyzes them in real time. The analysis results are then sent to a cloud server.

[0988] Step 7:

[0989] The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine. If the user is in a relaxed state, it generates trivia such as "Tokyo Tower was completed in 1958." If the user is in a stressed state, it generates relaxing music or an encouraging message.

[0990] Step 8:

[0991] The server sends the generated information to the terminal, and the terminal provides the received information to the user by voice.

[0992] Step 9:

[0993] The device then plays back the received information through a speaker and provides it to the user. For example, it might say, "Tokyo Tower was completed in 1958."

[0994] Step 10:

[0995] The server monitors traffic conditions in real time and recalculates routes as needed. If a new route is required, it notifies the device and updates the optimal route information.

[0996] Step 11:

[0997] The device displays the updated optimal route information to the user and updates the voice guidance. The device continues to provide voice guidance to the user about the recalculated route.

[0998] In this way, the system improves the user's driving experience by providing personalized information based on the user's profile and emotional state, and by providing optimal route guidance that reflects traffic conditions in real time.

[0999] Example 2

[1000] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1001] Conventional car navigation systems do not adequately provide information that corresponds to the driver's emotional state or individual interests, making it difficult to relieve stress associated with long drives and resulting in a less than fulfilling driving experience.In addition, route calculations based solely on real-time traffic information make it difficult to provide an optimal driving experience that corresponds to the driver's emotions and interests.

[1002] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for inputting driver profile information and transmitting it to a cloud server, a means for inputting destination information and transmitting it to the cloud server, a means for collecting real-time traffic information and calculating an optimal route, a means for capturing the driver's voice and facial expression and analyzing the emotional state, a means for transmitting the analysis results to the cloud server, a means for the cloud server to generate appropriate information using a generative artificial intelligence model based on the driver's profile information and emotional state, and a means for providing the generated information to the driver by voice. This makes it possible to provide information according to the driver's emotional state and interests, thereby reducing stress and enhancing the driving experience.

[1003] "Driver profile information" is basic information about each individual driver, such as the driver's name, age, and areas of interest.

[1004] A "cloud server" is a remote server that operates on the Internet and stores and processes driver profile information and destination information.

[1005] "Destination information" refers to information such as the address and place name of the destination that the driver inputs into the in-vehicle terminal.

[1006] "Real-time traffic information" is data that provides current road conditions, traffic volume, accident information, etc. in real time.

[1007] "Optimal route" refers to the most efficient and safe route for a driver based on real-time traffic information.

[1008] "Driver's voice and facial expression" refers to the voice and facial expression of the driver, and is data for analyzing the emotional state through these.

[1009] "Emotional state" refers to the driver's current psychological state (relaxed, stressed, etc.).

[1010] A "generative artificial intelligence model" is an AI technology that generates appropriate information based on input data.

[1011] "Relevant information" is interesting tidbits or helpful information generated based on the driver's profile information and emotional state.

[1012] An "in-vehicle terminal" is a device installed in a vehicle that allows the driver to input profile information and destination information, and capture voice and facial expressions.

[1013] "Means for providing by voice" refers to technology or devices for conveying the generated information to the driver by voice.

[1014] The present invention relates to a car navigation system that improves the driver's driving experience by providing information tailored to the driver's emotional state and individual interests. This system is composed of a cloud server, an in-vehicle terminal, a driver, and an emotion engine.

[1015] Basic system configuration

[1016] User Initial Settings

[1017] The user starts up the in-car terminal and uses the touchscreen to enter profile information (such as name, age, and areas of interest). For example, this information might include "Name: Tarou, Age: 35, Areas of Interest: History and Nature." The terminal then sends this profile information to a cloud server, which then stores the information in a database. The hardware used is an in-car terminal with a touchscreen, and the software is a cloud database management system.

[1018] Destination setting and route calculation

[1019] The user inputs destination information into the in-vehicle terminal. For example, they input "Tokyo Tower." The terminal sends this destination information to a cloud server. The server calculates the optimal route based on real-time traffic information. The calculated route information is sent to the in-vehicle terminal and displayed to the user, while voice guidance is also provided. The hardware used here is an in-vehicle terminal with GPS functionality, and the software is a real-time traffic information API and a route calculation algorithm.

[1020] emotion recognition

[1021] The emotion engine analyzes the driver's voice and facial expressions. Using the device's camera and microphone, it monitors the driver's voice tone and facial expressions in real time to understand the driver's emotional state. The analysis results are sent to a cloud server. The hardware used here is an in-car device equipped with a camera and microphone, and the software is an emotion analysis algorithm. For example, if the user is smiling while driving, it is determined that the user is relaxed. If the user's voice is raised, it can be determined that the user is stressed.

[1022] Information generation and provision

[1023] The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine. For users who are in a relaxed state, interesting trivia is provided by voice. For example, information such as "Tokyo Tower is 333 meters tall" is generated. On the other hand, for users who are in a stressed state, relaxing music is played or encouraging words about driving are provided. As a concrete example, the prompt text for the generative AI model is as follows:

[1024] Prompt: "Generate some trivia about Tokyo Tower for a relaxed user who is interested in history."

[1025] AI model output: "Tokyo Tower was completed in 1958. It is 333 meters tall."

[1026] The generated information is sent to the in-vehicle terminal and provided to the driver using the terminal's voice synthesis function. The software used here is a generative artificial intelligence model and a voice synthesis system.

[1027] In this way, while driving, users can always select the optimal route based on the latest traffic information, and receive information and music tailored to their interests and emotional state, making long drives stress-free and more fulfilling. Real-time feedback from the emotion engine also improves driver safety. For example, if a stressful state is detected, a message recommending safe driving can be displayed or music that promotes relaxation can be played. Providing both physical and psychological comfort simultaneously can significantly improve the driver's overall driving experience.

[1028] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1029] System program processing flow

[1030] Step 1: User Initial Setup

[1031] Input: User profile information (e.g., name, age, interests)

[1032] Output: Profile information sent to cloud server and stored in database

[1033] 1. The user starts up the in-car device and operates the touch screen to enter profile information.

[1034] Specific operation: The user enters "Name: Tarou, Age: 35, Areas of interest: History and nature" into the input form on the device.

[1035] 2. The device sends the entered profile information to the cloud server.

[1036] Specific operation: When the device presses the send button, it sends the profile information to the cloud server via the Internet connection.

[1037] 3. The server stores the received profile information in a database.

[1038] What happens: The server's database management system automatically creates a record for the new user and records their profile information.

[1039] Step 2: Set destination and calculate route

[1040] Input: Destination information (e.g. Tokyo Tower)

[1041] Output: Optimal route information is displayed on the in-car device and voice guidance begins.

[1042] 1. The user inputs destination information into the in-vehicle terminal.

[1043] Specific operation: The user uses the touch screen and the keyboard to input "Tokyo Tower."

[1044] 2. The device sends the entered destination information to the cloud server.

[1045] Specific operation: The terminal transmits destination information to the server via the Internet in real time.

[1046] 3. The server calculates the optimal route based on real-time traffic information.

[1047] Specific operation: The server obtains data from the traffic information API and uses an algorithm to calculate the optimal route.

[1048] 4. The server sends the calculated route information to the in-vehicle terminal, which displays and provides voice guidance to the user.

[1049] Specific operation: The server sends route information to the terminal, the terminal displays the route map on the screen, and the voice guidance system provides instructions by voice.

[1050] Step 3: Emotion Recognition

[1051] Input: Driver's voice and facial expression capture data

[1052] Output: The analyzed emotional state (e.g., relaxed, stressed) is sent to the cloud server.

[1053] 1. The device captures the driver's voice and facial expressions through a camera and microphone.

[1054] Specific operation: The device's built-in camera and microphone automatically record the user's face and voice.

[1055] 2. The emotion engine analyzes the captured data in real time.

[1056] How it works: The built-in software uses visual recognition and voice analysis technology to analyze the user's emotions (relaxed, stressed, etc.).

[1057] 3. The analysis results are sent to the cloud server.

[1058] Specific operation: The device sends analysis data to the server in real time.

[1059] Step 4: Information generation and provision

[1060] Input: Profile information, emotional state

[1061] Output: The generated appropriate information (e.g., trivia or relaxing music) is provided to the user via the in-car terminal.

[1062] 1. The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine.

[1063] Specific operation: The server inputs the prompt "Profile information: History lover, Emotional state: Relaxed" to the generated AI model.

[1064] Example prompt sentence:

[1065] Prompt: "Generate some trivia about Tokyo Tower for a relaxed user who is interested in history."

[1066] AI model output: "Tokyo Tower was completed in 1958. It is 333 meters tall."

[1067] 2. The server sends the generated information to the in-vehicle terminal, which then provides it to the user via voice.

[1068] Specific operation: The server sends the generated information to the terminal, and the terminal's voice synthesis function conveys the information to the user.

[1069] Through the above processing steps, the system can provide optimized information according to the driver's emotional state and interests, improving the driving experience.

[1070] (Application example 2)

[1071] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1072] Conventional car navigation systems only provide optimal routes using the driver's profile and real-time traffic information, but lack information tailored to the driver's emotional state and interests. This can make long drives stressful and reduce safety. Furthermore, if the information provided during driving does not match the driver's interests, satisfaction can decrease.

[1073] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting driver profile information and transmitting it to a cloud server, means for inputting destination information and transmitting it to the cloud server, means for collecting real-time traffic information and calculating an optimal route, means for providing the optimal route to the user in real time, means for analyzing the user's emotional state, and means for generating and providing information based on the user's interests and emotional state. This makes it possible to provide appropriate information according to the driver's emotional state, thereby reducing stress while driving, improving safety, and improving the driving experience.

[1074] "Driver profile information" refers to personal information such as the driver's name, age, and areas of interest.

[1075] A "cloud server" refers to a server that provides resources and services over the Internet.

[1076] "Destination information" refers to information such as the name and address of the destination set by the driver.

[1077] "Real-time traffic information" refers to data collected in real time about current traffic conditions.

[1078] An "optimal route" refers to the most efficient route for a driver's travel.

[1079] "User" refers to a driver who uses the system.

[1080] "Emotional state" refers to the driver's current emotional and psychological state.

[1081] A "generative artificial intelligence model" refers to an artificial intelligence model that automatically generates appropriate information based on input data.

[1082] This invention is a car navigation system that provides customized information based on the driver's profile information and real-time emotional state. The system consists of a cloud server, an in-vehicle terminal, a driver, and an emotion engine.

[1083] First, the user starts up the in-car device and enters their profile information (such as name, age, and areas of interest). For example, a user might enter information such as "Name: ABC, Age: 35, Areas of Interest: History and Nature." This information is sent from the device to the cloud server and stored in a database.

[1084] Next, the user inputs destination information into the in-vehicle terminal. For example, they input "major tourist destinations." This destination information is also sent to the cloud server, which calculates the optimal route based on real-time traffic information. The calculated route information is sent back to the in-vehicle terminal and displayed to the user. Voice guidance is also provided.

[1085] While driving, the emotion engine analyzes the driver's voice and facial expressions in real time. Specifically, the vehicle's camera and microphone capture the driver's video and audio, which the emotion engine analyzes. For example, it determines whether the driver is relaxed or stressed. The results of this analysis are sent to a cloud server.

[1086] The cloud server uses a generative AI model based on the driver's profile information and the analysis results of the emotion engine to generate appropriate information. For example, if the driver is relaxed and interested in "history and nature," the generative AI model will generate information such as "Major tourist destinations were completed in 1958." On the other hand, if the driver is feeling stressed, the model will provide relaxing music and encouraging words.

[1087] The generated information is sent to an in-vehicle terminal and provided to the driver via voice, allowing the driver to receive customized information at the right time, making the drive more comfortable.

[1088] As a concrete example, we will show an example of a prompt sentence to be input to the generative AI model.

[1089] Example prompt sentence:

[1090] User profile:

[1091] Name: ABC, Age: 35, Areas of Interest: History and Nature

[1092] Driver's state: Relaxed

[1093] prompt:

[1094] Based on the user profile and driver status, generate interesting information about their history to help them relax.

[1095] This system aims to reduce driver stress and improve safety by dynamically customizing information provided while driving based on the driver's emotional state and interests, and also to improve the driver's driving experience.

[1096] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1097] Step 1:

[1098] The user inputs profile information from the in-vehicle terminal, and the terminal transmits the information to the cloud server.

[1099] Specifically, when a user enters data such as their name, age, and areas of interest and presses the "Submit" button, the device sends the input data to the cloud server as a POST request, and the input data is stored directly in the cloud database.

[1100] Input: User profile information (name, age, interests)

[1101] Output: Profile information stored on the cloud server

[1102] Step 2:

[1103] The user inputs destination information from the in-vehicle terminal, and the terminal transmits the information to the cloud server.

[1104] Specifically, when a user inputs a destination (e.g., a major tourist spot) and presses the "Send" button, the device sends the input data to the cloud server as a POST request. The cloud server receives this data and calculates the optimal route based on real-time traffic information.

[1105] Input: User's destination information

[1106] Output: Optimal route calculation by cloud server

[1107] Step 3:

[1108] The cloud server sends the calculated optimal route information to the in-vehicle terminal, which then displays it to the user.

[1109] Specifically, the cloud server returns the calculation results to the in-vehicle terminal as a response, and the terminal displays the information on the screen and starts voice guidance.

[1110] Input: Optimal route information calculated by the cloud server

[1111] Output: Optimal route information and voice guidance displayed on the in-car device

[1112] Step 4:

[1113] The device captures the driver's voice and facial expressions through a camera and microphone, and the emotion engine analyzes them in real time.

[1114] Specifically, the in-car camera captures the driver's facial expressions and the microphone captures their voice, and the data is sent to the emotion engine, which analyzes it to determine the driver's emotional state and sends the results to a cloud server.

[1115] Input: Driver's facial image and voice data

[1116] Output: Analysis result of emotional state by emotion engine

[1117] Step 5:

[1118] The cloud server uses a generative AI model to generate appropriate information based on the driver's profile information and the analysis results of the emotion engine.

[1119] Specifically, the cloud server sends data along with the prompt text to the generative AI model and receives the generated information, which includes content that corresponds to the driver's interests and emotional state.

[1120] Input: Driver profile information, emotional state

[1121] Output: Customized information from a generative AI model

[1122] Step 6:

[1123] The cloud server transmits the generated customization information to the in-vehicle terminal, which then provides it to the user via voice.

[1124] Specifically, the cloud server sends the generated information back to the in-vehicle terminal as a response, and the terminal then conveys the information to the user via voice. For example, it might provide interesting trivia to a relaxed driver, or relaxing music or words of encouragement to a stressed driver.

[1125] Input: Customization information generated by the generative AI model

[1126] Output: Audio information presented to the user

[1127] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1128] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1129] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1130] [Fourth embodiment]

[1131] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1132] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1133] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1134] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1135] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1136] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1137] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1138] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1139] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1140] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1141] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1142] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1143] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1144] This invention describes a car navigation system equipped with a generative AI model to improve the driver's driving experience. The system consists of a cloud server, an in-vehicle terminal, and a driver.

[1145] Basic system configuration

[1146] 1. User Initial Setup:

[1147] The user starts up the in-car device and enters profile information such as name, age, and areas of interest. For example, this information might include "Name: Tarou, Age: 35, Areas of interest: History and nature."

[1148] The device sends this profile information to a cloud server, which stores the information in a database.

[1149] 2. Destination setting and route calculation:

[1150] The user inputs destination information into the in-vehicle terminal. For example, the user inputs "Tokyo Tower."

[1151] The device sends this destination information to a cloud server, which then calculates the optimal route based on real-time traffic information.

[1152] The calculated route information is sent to the in-vehicle terminal and displayed to the user, and voice guidance is also provided.

[1153] 3. Providing interesting information:

[1154] The server generates interesting trivia and useful information at appropriate times based on the user's profile information. For example, if a user is interested in "history," it generates information such as "Tokyo Tower was completed in 1958."

[1155] The generated information is transmitted to an in-vehicle terminal and provided to the user by voice.

[1156] Specific examples

[1157] Initial Settings:

[1158] The user follows the instructions on the device screen and enters "Name: Tarou, Age: 35, Interests: History and Nature." The device sends this information to the server, which then stores the received profile in a database.

[1159] Enter destination and get directions:

[1160] The user inputs "Tokyo Tower" as the destination for a trip. The device sends the location information of "Tokyo Tower" to the server, which then collects real-time traffic information and calculates the optimal route. The calculated route is sent to the device and displayed to the user in real time.

[1161] Providing information while driving:

[1162] While driving, the server uses a generative AI model to generate historical trivia based on the user's interests, and sends information such as "Tokyo Tower was completed in 1958" to the device. The device then relays the received information to the user via voice.

[1163] In this way, users can safely reach their destination while receiving real-time information relevant to their interests while driving. The system also provides optimal routes, helping users avoid traffic congestion and inconvenience. This makes long drives less boring and more enjoyable.

[1164] The processing flow will be explained below.

[1165] Step 1:

[1166] The user starts up the in-car terminal and enters profile information (such as name, age, areas of interest, etc.).

[1167] Step 2:

[1168] The device sends the entered profile information to the cloud server. For example, it sends "Name: Tarou, Age: 35, Areas of Interest: History and Nature."

[1169] Step 3:

[1170] The server stores the received profile information in a database.

[1171] Step 4:

[1172] The user inputs destination information (for example, "Tokyo Tower") into the in-vehicle terminal.

[1173] Step 5:

[1174] The terminal transmits the destination information to the cloud server.

[1175] Step 6:

[1176] The server collects real-time traffic information and calculates the optimal route, including data on congestion, accidents, and construction.

[1177] Step 7:

[1178] The server transmits the calculated optimum route information to the terminal.

[1179] Step 8:

[1180] The terminal displays the received optimum route information to the user and starts voice guidance.

[1181] Step 9:

[1182] While driving, the server uses an AI model to generate trivia and useful information based on the user's profile information. For example, based on the "History" category, it generates information such as "Tokyo Tower was completed in 1958."

[1183] Step 10:

[1184] The server transmits the generated information to the terminal.

[1185] Step 11:

[1186] The device provides the received information to the user via voice, allowing the user to receive information relevant to their interests in real time while driving safely.

[1187] Example 1

[1188] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1189] Conventional car navigation systems only provide support to help drivers reach their destination, making it difficult to improve the quality of the driving experience itself. Furthermore, they lacked features such as route recalculation based on real-time traffic information and information provision based on the driver's interests, making long drives boring and monotonous. This resulted in a decline in the quality and efficiency of information provision while driving, and insufficient safety.

[1190] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1191] In this invention, the server includes means for inputting driver profile information and transmitting it to the cloud server, means for inputting destination information and transmitting it to the cloud server, means for collecting real-time traffic information and calculating an optimal route, means for transmitting the calculated route information to an in-vehicle terminal and displaying and providing voice guidance to the user, means for using a generative artificial intelligence model to generate trivia and useful information based on the driver's profile information and current location information, and means for transmitting the generated information to the in-vehicle terminal and providing it to the driver by voice. This makes it possible to provide individual information based on the driver's profile and guidance on an optimal route that reflects real-time traffic information, thereby improving the quality of the driving experience and ensuring safety while driving.

[1192] "Driver profile information" refers to information about an individual, such as the driver's name, age, and areas of interest.

[1193] A "cloud server" refers to a server that provides data and services via the Internet.

[1194] "Destination information" refers to information about a destination that the driver inputs into the in-vehicle terminal.

[1195] "Real-time traffic information" is data showing current traffic conditions, including information on congestion and traffic restrictions.

[1196] "Optimal route" refers to the most efficient and safe route to your destination, calculated based on real-time traffic information.

[1197] "In-vehicle terminal" refers to an electronic terminal equipped with navigation and information provision functions that is installed in a vehicle.

[1198] "Voice guidance" refers to route guidance and information provided by voice generated by an in-vehicle terminal.

[1199] A "generative artificial intelligence model" refers to an artificial intelligence program that uses techniques such as natural language processing to generate new information.

[1200] A "prompt sentence" refers to an input sentence that causes a generative artificial intelligence model to generate information.

[1201] "Trivia and Helpful Information" refers to short pieces of information and helpful data provided based on a driver's interests and profile.

[1202] The present invention relates to a car navigation system equipped with a generative AI model for improving the driver's driving experience. The system consists of a cloud server, an in-vehicle terminal, and a driver.

[1203] Basic system configuration

[1204] 1. Initial Setup

[1205] The user starts up the in-car device and enters profile information such as name, age, and areas of interest. Input is done using the in-car device's touchscreen or voice recognition function. For example, the user might enter information such as "Name: Tarou, Age: 35, Interests: History and Nature." The device then sends this profile information to a cloud server via an internet connection (using 4G or 5G networks). The server then stores the received profile information in an SQL database.

[1206] 2. Setting a destination and calculating a route

[1207] The user inputs destination information into the in-car device. For example, they input "Tokyo Tower" using the touchscreen. The device then sends this destination information to a cloud server. The server uses the Google Maps API, HERE API, etc. to collect traffic information in real time and calculates the optimal route by sending a request such as "current_location: 'x,y', destination: 'Tokyo Tower'" to the API. The calculated route information is then sent from the server to the in-car device and displayed and provided to the user via display and voice guidance.

[1208] 3. Providing interesting information

[1209] The server references the saved user profile information and confirms, for example, that the user is interested in "history." Next, it generates and sends a prompt to the generative AI model (e.g., OpenAI GPT-3). An example prompt might be, "The user is interested in history and is currently driving toward Tokyo Tower. Please generate some historical trivia related to Tokyo Tower." Based on this prompt, the generative AI model generates information such as, "Tokyo Tower was completed in 1958." This information is sent from the server to the in-vehicle device and provided to the user audibly using a speech synthesis function (using Amazon Polly or Google Text-to-Speech).

[1210] Specific examples

[1211] 1. Initial Setup

[1212] The user follows the on-board device's screen and enters "Name: Tarou, Age: 35, Interests: History and Nature." The device then sends this information to the cloud server, which stores it in a database.

[1213] 2. Enter your destination and get route guidance

[1214] The user inputs "Tokyo Tower" as their destination. The device sends this information to a cloud server, which then collects real-time traffic information and calculates the optimal route. The calculated route is sent to the device and displayed and guided to the user in real time via display and voice guidance.

[1215] 3. Providing information while driving

[1216] While driving, the server uses a generative AI model to generate historical trivia, such as "Tokyo Tower was completed in 1958." This information is sent to the device and provided to the user via voice.

[1217] In this way, drivers can safely reach their destination while receiving interesting information in real time. Furthermore, by providing optimal routes, it is possible to avoid traffic congestion and inconvenience. This makes long driving trips less boring and more enjoyable.

[1218] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1219] Step 1:

[1220] The user starts up the in-car device and enters profile information such as name, age, and areas of interest. Input is done using the in-car device's touchscreen or voice recognition function. For example, the input information might be "Name: Tarou, Age: 35, Interests: History and Nature." This information is aggregated as input information. The device then sends this information to a cloud server via an internet connection (using 4G or 5G networks). This information is the output information.

[1221] Step 2:

[1222] The server stores the received profile information in a database. Specifically, it uses an SQL database to store it as a record such as "username: 'Tarou', age: 35, interests: 'history, nature'". The input information is the profile information sent from the device, which is processed and stored in the database. Updates to this database become the output information.

[1223] Step 3:

[1224] The user inputs destination information into the in-car terminal. For example, they input "Tokyo Tower" using the touch screen. This is the input of destination information. The terminal then sends the destination information to the cloud server. The destination information becomes output data.

[1225] Step 4:

[1226] The server collects real-time traffic information using Google Maps API, HERE API, etc. It receives traffic condition data by sending a request to the API such as "current_location: 'x,y', destination: 'Tokyo Tower'". This is the input information. The optimal route is calculated based on the input data. The Dijkstra algorithm or A algorithm is used for the calculation. The calculation result is generated as output data.

[1227] Step 5:

[1228] The server sends the calculated route information to the in-vehicle terminal. The sent data includes multiple waypoints (passing points). This is the output route information. The terminal displays the received route information on the user's display and begins guiding the user using voice guidance. Detailed navigation information such as "Turn right at the next intersection" is displayed on the display. This is the final output information.

[1229] Step 6:

[1230] The server references the saved user profile information and confirms that the user is interested in "history." This is the input information. Next, it generates and sends a prompt to the generative AI model. An example of a prompt is, "The user is interested in history and is currently driving toward Tokyo Tower. Please generate some historical trivia related to Tokyo Tower." The prompt becomes the input to the generative AI model.

[1231] Step 7:

[1232] The server sends a prompt to a generative AI model (e.g., OpenAI GPT-3) and generates appropriate information. The generative AI model generates information such as "Tokyo Tower was completed in 1958." This is the output data.

[1233] Step 8:

[1234] The server sends the generated information to the in-vehicle terminal. This information may also be sent in audio format. This is the output information. The terminal then conveys the received information to the user by voice. Specifically, it uses a function that generates speech from text (for example, Amazon Polly or Google Text-to-Speech). This is the final output information.

[1235] Through these steps, drivers can reach their destination safely while receiving interesting information in real time, and avoid traffic congestion and inconvenience by being offered optimal routes.

[1236] (Application example 1)

[1237] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1238] Conventional driving experience enhancement systems have limited means of providing drivers with information of interest, resulting in a lack of entertainment while driving. Furthermore, route guidance to a destination often lacks real-time traffic information updates, making it difficult to respond to traffic congestion and route changes. The present invention aims to solve these problems and improve the travel experience for passengers in autonomous vehicles.

[1239] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1240] In this invention, the server includes means for inputting driver profile information and transmitting it to the cloud server, means for inputting destination information and transmitting it to the cloud server, means for collecting real-time traffic information and calculating the optimal route, means for providing the optimal route to the user in real time, means for generating and providing trivia and useful information based on the user's interests, and means that is installed in the autonomous vehicle and provides information by voice in real time to the user while traveling. This makes it possible to provide information based on the user's profile and provide optimal route guidance, thereby enriching the travel experience.

[1241] "Driver profile information" refers to personal information such as the user's name, age, and areas of interest, and is information that is sent to the cloud server.

[1242] A "cloud server" is a remote server that provides data storage, processing, and access over the Internet.

[1243] "Destination information" refers to geographical information about a place the user wants to visit, and is information that is sent to the cloud server.

[1244] "Real-time traffic information" is data showing current traffic conditions, including traffic volume, speed limits, accident information, and the like.

[1245] The "optimal route" is the most efficient and safe route to your destination, calculated based on real-time traffic information.

[1246] "Means for providing the user with the optimal route in real time" refers to means for displaying the calculated optimal route to the user and providing voice guidance.

[1247] "Trivia and useful information based on your interests" refers to information about history, nature, and other topics that are likely to be of interest to you based on your profile information.

[1248] A "generative artificial intelligence model" is an artificial intelligence system that has the ability to generate new information based on massive amounts of data.

[1249] An "autonomous vehicle" is a vehicle equipped with a system that automates human driving behavior and is capable of running without the intervention of a driver.

[1250] "Means for providing information by voice" refers to a method of converting generated information into voice and conveying it to the user.

[1251] The present invention relates to a system that uses a generative AI model to provide users with interesting information based on driver profile information and destination information, thereby improving the travel experience in an autonomous vehicle. Specific embodiments are described below.

[1252] System configuration

[1253] 1. Cloud Server

[1254] Receives and processes profile information, destination information, and real-time traffic information.

[1255] It is equipped with a generative AI model that generates user-based trivia and useful information.

[1256] 2. In-vehicle terminal

[1257] Provide a means for users to enter profile and destination information.

[1258] It receives data from the cloud server and provides users with the best route and information.

[1259] A voice output means is used to provide information to the user audibly.

[1260] 3. Self-driving vehicles

[1261] The vehicle itself is equipped with a system that automates driving, so the user does not need to be involved in driving.

[1262] Enriching users' travel experience by providing information while driving.

[1263] How it works

[1264] Entering user profile information

[1265] The user starts up the in-car device and enters profile information such as name, age, areas of interest, etc. This information is sent to the cloud server and stored in the server's database.

[1266] Enter your destination and calculate your route

[1267] When a user inputs destination information into the in-vehicle terminal, the terminal sends this information to a cloud server. The cloud server collects traffic information in real time, calculates the optimal route, and sends it to the in-vehicle terminal. The in-vehicle terminal then displays the calculated route information to the user and provides voice guidance.

[1268] Providing interesting information

[1269] The cloud server uses a generative AI model to generate trivia and useful information related to the user's interests based on the user's profile information. The generated information is sent to the in-vehicle device in real time and provided to the user via a voice synthesis system. Specifically, a prompt sentence is entered and passed to the generative AI model, and the generated text is then played back by the voice synthesis system.

[1270] Hardware and software used

[1271] Cloud server: Uses AWS or Google Cloud Platform.

[1272] In-vehicle terminal: A device such as a tablet or smartphone.

[1273] Generative AI models: Generative artificial intelligence models such as OpenAI's GPT and Google's BERT.

[1274] Speech synthesis system: Uses Google Text-to-Speech, Amazon Polly, etc.

[1275] Specific examples

[1276] For example, if a user enters information such as "Name: Takashi, Age: 30, Interests: History and Culture" and sets "Museum" as the destination, the cloud server will generate information such as "The museum houses ancient Roman artifacts." This information is sent back to the in-vehicle terminal and provided to the user aloud via a speech synthesis system.

[1277] Example prompt sentence:

[1278] "Takashi-san, the museum houses ancient Roman artifacts."

[1279] In this way, users can receive information of interest in real time while on the move, enriching their in-car experience.

[1280] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1281] Step 1:

[1282] The user enters profile information on the in-car terminal.

[1283] Specifically, users enter information such as their name, age, and areas of interest into the device screen, and this profile information is then sent to the cloud server.

[1284] Input: User's first and last name, age, and interests

[1285] Output: Send profile information to cloud server

[1286] Step 2:

[1287] The cloud server stores the received profile information in a database.

[1288] Specifically, the server analyzes the received data, converts it into an appropriate format, and stores it in a database. This information is then used by a generative AI model to generate information based on the user's interests.

[1289] Input: User profile information

[1290] Output: Profile information stored in a database

[1291] Step 3:

[1292] The user inputs destination information into the in-car terminal.

[1293] Specifically, users input their "destination" on their device, which is then sent to a cloud server, which then uses it to calculate the optimal route.

[1294] Input: Destination information

[1295] Output: Send destination information to the cloud server

[1296] Step 4:

[1297] A cloud server collects real-time traffic information and calculates the optimal route.

[1298] Specifically, the server collects real-time information on current traffic conditions, road closures, traffic congestion, etc., and calculates the optimal route to the destination based on this information. This calculation is performed using an algorithm, and the calculation results are sent to the in-vehicle terminal.

[1299] Input: Destination information, real-time traffic information

[1300] Output: Calculated optimal route information

[1301] Step 5:

[1302] The in-car device displays the optimal route to the user and begins voice guidance.

[1303] Specifically, route information sent from a cloud server is displayed on the terminal, and a voice synthesis system is used to provide real-time voice guidance to the user.

[1304] Input: Optimal route information

[1305] Output: Route display, voice guidance

[1306] Step 6:

[1307] The cloud server generates interesting information based on the user's profile information using a generative AI model.

[1308] Specifically, a prompt sentence is created based on the user's interest information (e.g., history, nature), and passed to a generative AI model to generate useful information and trivia.

[1309] Input: User profile information

[1310] Output: interesting information generated

[1311] Step 7:

[1312] The generated information is sent from the cloud server to the in-vehicle terminal, and the information is provided to the user using a voice synthesis system.

[1313] Specifically, the generated information is sent to the terminal, converted into voice by a voice synthesis system, and provided to the user.

[1314] Input: Generated interesting information

[1315] Output: Providing information as synthesized speech

[1316] Specific examples

[1317] For example, if a user enters the information "Name: Takashi, Age: 30, Interests: History and Culture" and sets "Museum" as the destination, the prompt might look like this:

[1318] "Takashi-san, the museum houses ancient Roman artifacts."

[1319] This information is generated on a cloud server, sent to the in-vehicle terminal, and provided to the user audibly through a voice synthesis system.

[1320] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1321] This paper describes a system that combines a car navigation system that uses a generative AI model to improve the driver's driving experience with an emotion engine to provide information tailored to individual user needs. This system consists of a cloud server, an in-vehicle terminal, a driver, and an emotion engine.

[1322] Basic system configuration

[1323] 1. User Initial Setup:

[1324] The user starts up the in-car device and enters their profile information (such as name, age, and areas of interest). For example, this information might include "Name: Tarou, Age: 35, Areas of Interest: History and Nature."

[1325] The device sends this profile information to a cloud server, which stores the information in a database.

[1326] 2. Destination setting and route calculation:

[1327] The user inputs destination information into the in-vehicle terminal. For example, the user inputs "Tokyo Tower."

[1328] The device sends this destination information to a cloud server, which then calculates the optimal route based on real-time traffic information.

[1329] The calculated route information is sent to the in-vehicle terminal and displayed to the user, and voice guidance is also provided.

[1330] 3. Emotion Recognition with Emotion Engine:

[1331] The emotion engine analyzes the driver's voice and facial expressions, for example, by monitoring voice tone and facial expressions in real time to understand the driver's emotional state.

[1332] It analyzes whether the user is relaxed or stressed and gives real-time feedback while driving.

[1333] 4. Providing interesting information:

[1334] The server uses a generative AI model to generate interesting trivia and useful information at the appropriate time based on the user's profile information and the recognition results of the emotion engine. For example, if the user is interested in "history" and is relaxing, the server will generate information such as "Tokyo Tower was completed in 1958." On the other hand, if the user is feeling stressed, the server will provide relaxing music or topics of interest.

[1335] The generated information is transmitted to an in-vehicle terminal and provided to the user by voice.

[1336] Specific processing of the program

[1337] Emotion Engine in action:

[1338] The device captures the user's voice and facial expressions through a camera and microphone, and the emotion engine analyzes them in real time. The analysis results are then sent to the server.

[1339] For example, if the user is smiling while driving, it is determined that the user is in a relaxed state, whereas if the user is speaking in a loud voice, it is determined that the user is feeling stressed.

[1340] Information generation and provision:

[1341] The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine.

[1342] When the user is in a relaxed state, interesting trivia is provided aloud, such as "Tokyo Tower is 333 meters tall."

[1343] For users who are under stress, the system plays relaxing music and provides encouraging words about driving.

[1344] In this way, while driving, users can always select the optimal route based on the latest traffic information, and receive information and music tailored to their interests and emotional state, making long drives stress-free and more fulfilling. Real-time feedback from the emotion engine also improves driver safety. For example, if a stressful state is detected, a message recommending safe driving can be displayed or music that promotes relaxation can be played. Providing both physical and psychological comfort simultaneously can significantly improve the driver's overall driving experience.

[1345] The processing flow will be explained below.

[1346] Step 1:

[1347] The user starts up the in-car device and enters profile information (such as name, age, and areas of interest). For example, the user enters "Name: Tarou, Age: 35, Areas of interest: History and nature."

[1348] Step 2:

[1349] The device sends the entered profile information to a cloud server, which stores the received information in a database.

[1350] Step 3:

[1351] The user inputs destination information into the in-vehicle terminal. For example, the user inputs "Tokyo Tower."

[1352] Step 4:

[1353] The device sends this destination information to a cloud server, which collects real-time traffic information and calculates the optimal route.

[1354] Step 5:

[1355] The server sends the calculated optimal route information to the terminal, which displays the received optimal route information to the user and starts voice guidance.

[1356] Step 6:

[1357] The device captures the user's voice and facial expressions through a camera and microphone, and the emotion engine analyzes them in real time. The analysis results are then sent to a cloud server.

[1358] Step 7:

[1359] The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine. If the user is in a relaxed state, it generates trivia such as "Tokyo Tower was completed in 1958." If the user is in a stressed state, it generates relaxing music or an encouraging message.

[1360] Step 8:

[1361] The server sends the generated information to the terminal, and the terminal provides the received information to the user by voice.

[1362] Step 9:

[1363] The device then plays back the received information through a speaker and provides it to the user. For example, it might say, "Tokyo Tower was completed in 1958."

[1364] Step 10:

[1365] The server monitors traffic conditions in real time and recalculates routes as needed. If a new route is required, it notifies the device and updates the optimal route information.

[1366] Step 11:

[1367] The device displays the updated optimal route information to the user and updates the voice guidance. The device continues to provide voice guidance to the user about the recalculated route.

[1368] In this way, the system improves the user's driving experience by providing personalized information based on the user's profile and emotional state, and by providing optimal route guidance that reflects traffic conditions in real time.

[1369] Example 2

[1370] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1371] Conventional car navigation systems do not adequately provide information that corresponds to the driver's emotional state or individual interests, making it difficult to relieve stress associated with long drives and resulting in a less than fulfilling driving experience.In addition, route calculations based solely on real-time traffic information make it difficult to provide an optimal driving experience that corresponds to the driver's emotions and interests.

[1372] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for inputting driver profile information and transmitting it to a cloud server, a means for inputting destination information and transmitting it to the cloud server, a means for collecting real-time traffic information and calculating an optimal route, a means for capturing the driver's voice and facial expression and analyzing the emotional state, a means for transmitting the analysis results to the cloud server, a means for the cloud server to generate appropriate information using a generative artificial intelligence model based on the driver's profile information and emotional state, and a means for providing the generated information to the driver by voice. This makes it possible to provide information according to the driver's emotional state and interests, thereby reducing stress and enhancing the driving experience.

[1373] "Driver profile information" is basic information about each individual driver, such as the driver's name, age, and areas of interest.

[1374] A "cloud server" is a remote server that operates on the Internet and stores and processes driver profile information and destination information.

[1375] "Destination information" refers to information such as the address and place name of the destination that the driver inputs into the in-vehicle terminal.

[1376] "Real-time traffic information" is data that provides current road conditions, traffic volume, accident information, etc. in real time.

[1377] "Optimal route" refers to the most efficient and safe route for a driver based on real-time traffic information.

[1378] "Driver's voice and facial expression" refers to the voice and facial expression of the driver, and is data for analyzing the emotional state through these.

[1379] "Emotional state" refers to the driver's current psychological state (relaxed, stressed, etc.).

[1380] A "generative artificial intelligence model" is an AI technology that generates appropriate information based on input data.

[1381] "Relevant information" is interesting tidbits or helpful information generated based on the driver's profile information and emotional state.

[1382] An "in-vehicle terminal" is a device installed in a vehicle that allows the driver to input profile information and destination information, and capture voice and facial expressions.

[1383] "Means for providing by voice" refers to technology or devices for conveying the generated information to the driver by voice.

[1384] The present invention relates to a car navigation system that improves the driver's driving experience by providing information tailored to the driver's emotional state and individual interests. This system is composed of a cloud server, an in-vehicle terminal, a driver, and an emotion engine.

[1385] Basic system configuration

[1386] User Initial Settings

[1387] The user starts up the in-car terminal and uses the touchscreen to enter profile information (such as name, age, and areas of interest). For example, this information might include "Name: Tarou, Age: 35, Areas of Interest: History and Nature." The terminal then sends this profile information to a cloud server, which then stores the information in a database. The hardware used is an in-car terminal with a touchscreen, and the software is a cloud database management system.

[1388] Destination setting and route calculation

[1389] The user inputs destination information into the in-vehicle terminal. For example, they input "Tokyo Tower." The terminal sends this destination information to a cloud server. The server calculates the optimal route based on real-time traffic information. The calculated route information is sent to the in-vehicle terminal and displayed to the user, while voice guidance is also provided. The hardware used here is an in-vehicle terminal with GPS functionality, and the software is a real-time traffic information API and a route calculation algorithm.

[1390] emotion recognition

[1391] The emotion engine analyzes the driver's voice and facial expressions. Using the device's camera and microphone, it monitors the driver's voice tone and facial expressions in real time to understand the driver's emotional state. The analysis results are sent to a cloud server. The hardware used here is an in-car device equipped with a camera and microphone, and the software is an emotion analysis algorithm. For example, if the user is smiling while driving, it is determined that the user is relaxed. If the user's voice is raised, it can be determined that the user is stressed.

[1392] Information generation and provision

[1393] The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine. For users who are in a relaxed state, interesting trivia is provided by voice. For example, information such as "Tokyo Tower is 333 meters tall" is generated. On the other hand, for users who are in a stressed state, relaxing music is played or encouraging words about driving are provided. As a concrete example, the prompt text for the generative AI model is as follows:

[1394] Prompt: "Generate some trivia about Tokyo Tower for a relaxed user who is interested in history."

[1395] AI model output: "Tokyo Tower was completed in 1958. It is 333 meters tall."

[1396] The generated information is sent to the in-vehicle terminal and provided to the driver using the terminal's voice synthesis function. The software used here is a generative artificial intelligence model and a voice synthesis system.

[1397] In this way, while driving, users can always select the optimal route based on the latest traffic information, and receive information and music tailored to their interests and emotional state, making long drives stress-free and more fulfilling. Real-time feedback from the emotion engine also improves driver safety. For example, if a stressful state is detected, a message recommending safe driving can be displayed or music that promotes relaxation can be played. Providing both physical and psychological comfort simultaneously can significantly improve the driver's overall driving experience.

[1398] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1399] System program processing flow

[1400] Step 1: User Initial Setup

[1401] Input: User profile information (e.g., name, age, interests)

[1402] Output: Profile information sent to cloud server and stored in database

[1403] 1. The user starts up the in-car device and operates the touch screen to enter profile information.

[1404] Specific operation: The user enters "Name: Tarou, Age: 35, Areas of interest: History and nature" into the input form on the device.

[1405] 2. The device sends the entered profile information to the cloud server.

[1406] Specific operation: When the device presses the send button, it sends the profile information to the cloud server via the Internet connection.

[1407] 3. The server stores the received profile information in a database.

[1408] What happens: The server's database management system automatically creates a record for the new user and records their profile information.

[1409] Step 2: Set destination and calculate route

[1410] Input: Destination information (e.g. Tokyo Tower)

[1411] Output: Optimal route information is displayed on the in-car device and voice guidance begins.

[1412] 1. The user inputs destination information into the in-vehicle terminal.

[1413] Specific operation: The user uses the touch screen and the keyboard to input "Tokyo Tower."

[1414] 2. The device sends the entered destination information to the cloud server.

[1415] Specific operation: The terminal transmits destination information to the server via the Internet in real time.

[1416] 3. The server calculates the optimal route based on real-time traffic information.

[1417] Specific operation: The server obtains data from the traffic information API and uses an algorithm to calculate the optimal route.

[1418] 4. The server sends the calculated route information to the in-vehicle terminal, which displays and provides voice guidance to the user.

[1419] Specific operation: The server sends route information to the terminal, the terminal displays the route map on the screen, and the voice guidance system provides instructions by voice.

[1420] Step 3: Emotion Recognition

[1421] Input: Driver's voice and facial expression capture data

[1422] Output: The analyzed emotional state (e.g., relaxed, stressed) is sent to the cloud server.

[1423] 1. The device captures the driver's voice and facial expressions through a camera and microphone.

[1424] Specific operation: The device's built-in camera and microphone automatically record the user's face and voice.

[1425] 2. The emotion engine analyzes the captured data in real time.

[1426] How it works: The built-in software uses visual recognition and voice analysis technology to analyze the user's emotions (relaxed, stressed, etc.).

[1427] 3. The analysis results are sent to the cloud server.

[1428] Specific operation: The device sends analysis data to the server in real time.

[1429] Step 4: Information generation and provision

[1430] Input: Profile information, emotional state

[1431] Output: The generated appropriate information (e.g., trivia or relaxing music) is provided to the user via the in-car terminal.

[1432] 1. The server uses a generative AI model to generate appropriate information based on the user's profile information and the analysis results of the emotion engine.

[1433] Specific operation: The server inputs the prompt "Profile information: History lover, Emotional state: Relaxed" to the generated AI model.

[1434] Example prompt sentence:

[1435] Prompt: "Generate some trivia about Tokyo Tower for a relaxed user who is interested in history."

[1436] AI model output: "Tokyo Tower was completed in 1958. It is 333 meters tall."

[1437] 2. The server sends the generated information to the in-vehicle terminal, which then provides it to the user via voice.

[1438] Specific operation: The server sends the generated information to the terminal, and the terminal's voice synthesis function conveys the information to the user.

[1439] Through the above processing steps, the system can provide optimized information according to the driver's emotional state and interests, improving the driving experience.

[1440] (Application example 2)

[1441] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1442] Conventional car navigation systems only provide optimal routes using the driver's profile and real-time traffic information, but lack information tailored to the driver's emotional state and interests. This can make long drives stressful and reduce safety. Furthermore, if the information provided during driving does not match the driver's interests, satisfaction can decrease.

[1443] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for inputting driver profile information and transmitting it to a cloud server, means for inputting destination information and transmitting it to the cloud server, means for collecting real-time traffic information and calculating an optimal route, means for providing the optimal route to the user in real time, means for analyzing the user's emotional state, and means for generating and providing information based on the user's interests and emotional state. This makes it possible to provide appropriate information according to the driver's emotional state, thereby reducing stress while driving, improving safety, and improving the driving experience.

[1444] "Driver profile information" refers to personal information such as the driver's name, age, and areas of interest.

[1445] A "cloud server" refers to a server that provides resources and services over the Internet.

[1446] "Destination information" refers to information such as the name and address of the destination set by the driver.

[1447] "Real-time traffic information" refers to data collected in real time about current traffic conditions.

[1448] An "optimal route" refers to the most efficient route for a driver's travel.

[1449] "User" refers to a driver who uses the system.

[1450] "Emotional state" refers to the driver's current emotional and psychological state.

[1451] A "generative artificial intelligence model" refers to an artificial intelligence model that automatically generates appropriate information based on input data.

[1452] This invention is a car navigation system that provides customized information based on the driver's profile information and real-time emotional state. The system consists of a cloud server, an in-vehicle terminal, a driver, and an emotion engine.

[1453] First, the user starts up the in-car device and enters their profile information (such as name, age, and areas of interest). For example, a user might enter information such as "Name: ABC, Age: 35, Areas of Interest: History and Nature." This information is sent from the device to the cloud server and stored in a database.

[1454] Next, the user inputs destination information into the in-vehicle terminal. For example, they input "major tourist destinations." This destination information is also sent to the cloud server, which calculates the optimal route based on real-time traffic information. The calculated route information is sent back to the in-vehicle terminal and displayed to the user. Voice guidance is also provided.

[1455] While driving, the emotion engine analyzes the driver's voice and facial expressions in real time. Specifically, the vehicle's camera and microphone capture the driver's video and audio, which the emotion engine analyzes. For example, it determines whether the driver is relaxed or stressed. The results of this analysis are sent to a cloud server.

[1456] The cloud server uses a generative AI model based on the driver's profile information and the analysis results of the emotion engine to generate appropriate information. For example, if the driver is relaxed and interested in "history and nature," the generative AI model will generate information such as "Major tourist destinations were completed in 1958." On the other hand, if the driver is feeling stressed, the model will provide relaxing music and encouraging words.

[1457] The generated information is sent to an in-vehicle terminal and provided to the driver via voice, allowing the driver to receive customized information at the right time, making the drive more comfortable.

[1458] As a concrete example, we will show an example of a prompt sentence to be input to the generative AI model.

[1459] Example prompt sentence:

[1460] User profile:

[1461] Name: ABC, Age: 35, Areas of Interest: History and Nature

[1462] Driver's state: Relaxed

[1463] prompt:

[1464] Based on the user profile and driver status, generate interesting information about their history to help them relax.

[1465] This system aims to reduce driver stress and improve safety by dynamically customizing information provided while driving based on the driver's emotional state and interests, and also to improve the driver's driving experience.

[1466] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1467] Step 1:

[1468] The user inputs profile information from the in-vehicle terminal, and the terminal transmits the information to the cloud server.

[1469] Specifically, when a user enters data such as their name, age, and areas of interest and presses the "Submit" button, the device sends the input data to the cloud server as a POST request, and the input data is stored directly in the cloud database.

[1470] Input: User profile information (name, age, interests)

[1471] Output: Profile information stored on the cloud server

[1472] Step 2:

[1473] The user inputs destination information from the in-vehicle terminal, and the terminal transmits the information to the cloud server.

[1474] Specifically, when a user inputs a destination (e.g., a major tourist spot) and presses the "Send" button, the device sends the input data to the cloud server as a POST request. The cloud server receives this data and calculates the optimal route based on real-time traffic information.

[1475] Input: User's destination information

[1476] Output: Optimal route calculation by cloud server

[1477] Step 3:

[1478] The cloud server sends the calculated optimal route information to the in-vehicle terminal, which then displays it to the user.

[1479] Specifically, the cloud server returns the calculation results to the in-vehicle terminal as a response, and the terminal displays the information on the screen and starts voice guidance.

[1480] Input: Optimal route information calculated by the cloud server

[1481] Output: Optimal route information and voice guidance displayed on the in-car device

[1482] Step 4:

[1483] The device captures the driver's voice and facial expressions through a camera and microphone, and the emotion engine analyzes them in real time.

[1484] Specifically, the in-car camera captures the driver's facial expressions and the microphone captures their voice, and the data is sent to the emotion engine, which analyzes it to determine the driver's emotional state and sends the results to a cloud server.

[1485] Input: Driver's facial image and voice data

[1486] Output: Analysis result of emotional state by emotion engine

[1487] Step 5:

[1488] The cloud server uses a generative AI model to generate appropriate information based on the driver's profile information and the analysis results of the emotion engine.

[1489] Specifically, the cloud server sends data along with the prompt text to the generative AI model and receives the generated information, which includes content that corresponds to the driver's interests and emotional state.

[1490] Input: Driver profile information, emotional state

[1491] Output: Customized information from a generative AI model

[1492] Step 6:

[1493] The cloud server transmits the generated customization information to the in-vehicle terminal, which then provides it to the user via voice.

[1494] Specifically, the cloud server sends the generated information back to the in-vehicle terminal as a response, and the terminal then conveys the information to the user via voice. For example, it might provide interesting trivia to a relaxed driver, or relaxing music or words of encouragement to a stressed driver.

[1495] Input: Customization information generated by the generative AI model

[1496] Output: Audio information presented to the user

[1497] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1498] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1499] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1500] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1501] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1502] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1503] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1504] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1505] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1506] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1507] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1508] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1509] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1510] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1511] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1512] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1513] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1514] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1515] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1516] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1517] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1518] The following is further disclosed regarding the above embodiment.

[1519] (Claim 1)

[1520] A means for inputting driver profile information and transmitting it to a cloud server;

[1521] A means for inputting destination information and transmitting it to a cloud server;

[1522] A means of collecting real-time traffic information and calculating optimal routes;

[1523] A means of providing users with the best route in real time,

[1524] A means for generating and providing user-interest trivia and useful information;

[1525] A system including:

[1526] (Claim 2)

[1527] 2. The system of claim 1, wherein the means for calculating the optimal route monitors traffic conditions in real time and recalculates the route as needed.

[1528] (Claim 3)

[1529] 2. The system of claim 1, wherein the means for generating trivia and useful information based on the user's interests uses a generative artificial intelligence model.

[1530] "Example 1"

[1531] (Claim 1)

[1532] A means for inputting driver profile information and transmitting it to a cloud server;

[1533] A means for inputting destination information and transmitting it to a cloud server;

[1534] A means of collecting real-time traffic information and calculating optimal routes;

[1535] means for transmitting the calculated route information to an in-vehicle terminal and displaying and providing voice guidance to a user;

[1536] a means for using a generative artificial intelligence model to generate trivia and useful information based on the driver's profile information and current location information;

[1537] means for transmitting the generated information to an in-vehicle terminal and providing the information to the driver by voice;

[1538] A system including:

[1539] (Claim 2)

[1540] 2. The system of claim 1, wherein the means for calculating the optimal route monitors traffic conditions in real time and recalculates the route as needed.

[1541] (Claim 3)

[1542] 10. The system of claim 1, further comprising means for generating and transmitting prompt sentences to the generative artificial intelligence model based on the driver's profile information.

[1543] "Application Example 1"

[1544] (Claim 1)

[1545] A means for inputting driver profile information and transmitting it to a cloud server;

[1546] A means for inputting destination information and transmitting it to a cloud server;

[1547] A means of collecting real-time traffic information and calculating optimal routes;

[1548] A means of providing users with the best route in real time,

[1549] A means for generating and providing user-interest trivia and useful information;

[1550] A means to be installed in an autonomous vehicle to provide real-time audio information to the user while they are traveling; and

[1551] A system including:

[1552] (Claim 2)

[1553] 2. The system of claim 1, wherein the means for calculating the optimal route monitors traffic conditions in real time and recalculates the route as needed.

[1554] (Claim 3)

[1555] 2. The system of claim 1, wherein the means for generating trivia and useful information based on the user's interests uses a generative artificial intelligence model.

[1556] "Example 2: Combining Emotion Engines"

[1557] (Claim 1)

[1558] A means for inputting driver profile information and transmitting it to a cloud server;

[1559] A means for inputting destination information and transmitting it to a cloud server;

[1560] A means of collecting real-time traffic information and calculating optimal routes;

[1561] means for capturing the driver's voice and facial expressions and analyzing the driver's emotional state;

[1562] means for transmitting the analysis results to a cloud server;

[1563] A cloud server generates appropriate information using a generative artificial intelligence model based on the driver's profile information and emotional state;

[1564] means for providing the generated information to the driver by voice;

[1565] A system including:

[1566] (Claim 2)

[1567] 2. The system of claim 1, wherein the means for calculating the optimal route monitors traffic conditions in real time and recalculates the route as needed.

[1568] (Claim 3)

[1569] 10. The system of claim 1, wherein the cloud server includes means for using a generative artificial intelligence model to generate appropriate information based on the profile information and emotional state.

[1570] "Application example 2 when combining emotion engines"

[1571] (Claim 1)

[1572] A means for inputting driver profile information and transmitting it to a cloud server;

[1573] A means for inputting destination information and transmitting it to a cloud server;

[1574] A means of collecting real-time traffic information and calculating optimal routes;

[1575] A means of providing users with the best route in real time,

[1576] a means for analyzing the emotional state of a user;

[1577] means for generating and providing information based on the user's interests and emotional state;

[1578] A system including:

[1579] (Claim 2)

[1580] 2. The system of claim 1, wherein the means for calculating the optimal route monitors traffic conditions in real time and recalculates the route as needed.

[1581] (Claim 3)

[1582] 10. The system of claim 1, wherein the means for generating information based on the user's interests and emotional state uses a generative artificial intelligence model. [Explanation of symbols]

[1583] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for inputting driver profile information and transmitting it to a cloud server; A means for inputting destination information and transmitting it to a cloud server; A means of collecting real-time traffic information and calculating optimal routes; A means of providing users with the best route in real time, A means for generating and providing user-interest trivia and useful information; A system including:

2. 2. The system of claim 1, wherein the means for calculating the optimal route monitors traffic conditions in real time and recalculates the route as needed.

3. 2. The system according to claim 1, wherein the means for generating trivia and useful information based on the user's interests uses a generative artificial intelligence model.

Images