System

Abstract

A system is provided.SOLUTION: The system includes means for providing an interactive interface through which a user can intuitively indicate a place to go or a point to stop by, means for obtaining traffic information from an external information source that is updated in real time and suggesting an optimal route to avoid congestion, means for obtaining and providing region-specific ratings and business information from an external information service, means for personalizing a route based on past behavior data and preferences of a user, means for providing entertainment functions, and means for supporting safe driving based on vehicle data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Today's drivers want to reach their destinations efficiently, enjoy entertainment while on the go, and be supported in safe driving. However, conventional navigation systems do not adequately meet these needs. Specifically, they struggle to utilize real-time traffic information to avoid traffic jams, and their functionality for providing up-to-date local ratings and business information is limited. Furthermore, the lack of personalized information based on user preferences and past behavior leads to low driver satisfaction. Furthermore, the overall driving experience is hindered by a lack of entertainment features and support for safe driving while driving. Therefore, there is a need for a navigation system that can solve these issues and provide a more comfortable and safe driving experience. [Means for solving the problem]

[0005] The present invention provides a system with an interactive interface that allows users to intuitively indicate where they want to go and where they want to stop. This system obtains traffic information from external sources that is updated in real time and suggests optimal routes that avoid congestion. It also obtains local ratings and business information from external information services and provides them to users. It also personalizes routes based on the user's past behavioral data and preferences to meet the driver's individual needs. It also provides entertainment functions to reduce boredom while driving. It also has a function to support safe driving based on vehicle data to ensure driver safety. This system uses generative AI technology to generate original character chatbots that can interact with users. It also provides a means for users to share and sell the chatbot characters they create. As described above, the present invention solves many of the challenges faced by modern drivers and provides a comfortable and safe driving experience.

[0006] An "interactive interface" is an interface for communication between a user and a system, allowing the user to specify places they want to go or points they want to stop at in natural language.

[0007] "External Sources" refers to data providers or services external to the System that provide real-time updated traffic information, location-specific ratings, and business information.

[0008] "Traffic information" refers to real-time information about road conditions such as congestion, accidents, and road closures.

[0009] "Route optimization" is a calculation method that allows users to reach their destination via the optimal route based on the traffic information obtained.

[0010] "Ratings and business information" refers to information such as user ratings, opening hours, and closing days for restaurants and tourist attractions specific to the region.

[0011] "Personalization" is the process of individually customizing the information and suggestions provided to users based on their past behavioral data and preferences.

[0012] "Entertainment features" refers to entertainment features such as music playback, quizzes, and chats to reduce boredom while driving.

[0013] "Safe Driving Support" is a function that provides messages and alerts to encourage drivers to drive safely based on vehicle data.

[0014] "Generative AI technology" is a means of generating new data and content (e.g., chatbot characters) using artificial intelligence technology.

[0015] A "chatbot" is an artificially intelligent character created using generative AI technology to interact with users.

[0016] "Sharing" is the act of sharing the generated chatbot character with other users or devices.

[0017] "Sales" refers to the act of providing the generated chatbot character to other users through online marketplaces and the like in exchange for payment. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention proposes a car navigation system with an interactive interface that allows users to intuitively specify where they want to go and where they want to stop. This not only realizes the comfortable and efficient driving that modern drivers desire, but also provides entertainment while traveling and support for safe driving. Specific embodiments of the present invention are described below.

[0040] 1. User Interface Design

[0041] Users can input instructions through a conversational interface using voice or text. For example, they can say, "I want to go to a nice cafe nearby." The device then analyzes this with a natural language processing engine and extracts keywords to understand the user's intent.

[0042] 2. Obtaining real-time traffic information

[0043] The server periodically retrieves the latest traffic information from external sources (e.g., traffic information API). This information includes congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time.

[0044] 3. Obtaining ratings and business information for restaurants and tourist spots

[0045] The device retrieves ratings and business information for restaurants and tourist spots in the specified area from external information services (e.g., rating site APIs). Based on this data, it creates a list of places that best meet the user's requirements.

[0046] 4. Personalized Route Suggestions

[0047] The server manages the user's past behavioral data and preferences as profile information, and based on this, the device suggests optimized routes to the user, for example, prioritizing cafes that have been highly rated in the past.

[0048] 5. Providing entertainment features

[0049] Users can make requests to their device, such as "play the next song," to enjoy entertainment such as music playback, quizzes, and chat functions. The device then plays music by calling an external music service (e.g., music distribution API). It also generates quizzes from a quiz database, providing entertainment while driving.

[0050] 6. Safe driving support

[0051] The device monitors vehicle data (e.g., speed, distance between vehicles) in real time and displays alerts to encourage safe driving. For example, if the distance between vehicles is too small, a message such as "There is insufficient distance between vehicles ahead. Please be careful" will be displayed.

[0052] 7. Creating a Character Chatbot

[0053] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile. This character is personalized based on the user's preferences and interests. The device then uses the generated character to engage in natural conversations with the user.

[0054] 8. Sharing and selling characters

[0055] Users can obtain a dedicated sharing link from the server to share the chatbot characters they have created with other users. They can also sell chatbot characters through the marketplace. The server provides this functionality through the network.

[0056] In this way, the present invention provides users with intuitive operation, real-time information and personalized suggestions, resulting in a comfortable and safe driving experience.

[0057] The processing flow will be explained below.

[0058] Step 1:

[0059] The user speaks to the device, saying, "I want to go to a nice cafe nearby." The device receives this command and uses a speech recognition engine to convert the voice data into text data.

[0060] Step 2:

[0061] The device passes the converted text data to a natural language processing engine, which extracts keywords such as "cafe," "nearby," and "delicious."

[0062] Step 3:

[0063] The server sends a request to an external information source (e.g., a traffic information API) to obtain real-time updated traffic information, and stores the obtained traffic information in a database.

[0064] Step 4:

[0065] The device obtains a list of local cafes, along with their ratings and business information, from an external information service (e.g., a rating site API). Based on this information, the device creates a list of the cafes with the highest ratings.

[0066] Step 5:

[0067] The device calls the Google Maps API to calculate the optimal route based on the traffic and cafe information obtained in steps 3 and 4. If there is traffic congestion, it suggests a detour route.

[0068] Step 6:

[0069] The server stores the user's past behavioral data and preferences as a profile, and the device accesses this profile information to prioritize suggestions for cafes that the user prefers.

[0070] Step 7:

[0071] Users can request "play my favorite music." The device sends the request to the music distribution API and plays music that matches the user's preferences. It also offers entertainment features such as quizzes.

[0072] Step 8:

[0073] The device monitors vehicle sensor information in real time and provides alerts if speed, distance, or improper driving is detected.

[0074] Step 9:

[0075] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile, and the device uses this character to interact with the user.

[0076] Step 10:

[0077] Users can share their chatbot characters with friends by obtaining a sharing link from the server. Characters can also be sold on the marketplace. The server manages transactions and generates sharing links.

[0078] Example 1

[0079] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0080] Conventional car navigation systems have issues with not being able to intuitively indicate where users want to go or stop, and it is difficult to efficiently provide real-time updated traffic information, local ratings, and business information. Furthermore, they lack the means to provide entertainment and safe driving support, failing to fully realize a comfortable and safe driving experience for users. Furthermore, there is a lack of means to provide personalized route suggestions based on user preferences and natural dialogue.

[0081] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0082] In this invention, the server includes means for providing an interactive interface that allows a user to intuitively indicate places they want to go and points they want to stop at, means for acquiring traffic information from external information sources that is updated in real time and proposing optimal routes that avoid congestion, means for acquiring and providing area-specific ratings and business information from external information services, means for personalizing routes based on the user's past behavioral data and preferences, means for providing entertainment functions, means for supporting safe driving based on vehicle data, means for analyzing voice input from the user and performing natural language processing, means for storing information acquired from external databases in a local database and updating it as needed, and means for generating a customized character chatbot using generative artificial intelligence technology to enable natural dialogue with the user.This allows the user to intuitively specify their destination, receive real-time information and personalized suggestions, and provide enhanced entertainment and support for safe driving, resulting in a comfortable and safe driving experience.

[0083] An "interactive interface" is a user interface that allows users to intuitively specify places they want to go or points they want to stop at using voice or text.

[0084] An "external information source" is an external data providing system or service that provides traffic information, rating information, business information, and the like.

[0085] "Traffic information" refers to real-time information about road conditions such as congestion, accidents, and road closures.

[0086] "Reviews and business information" refers to detailed information about reviews and business operations of restaurants, tourist spots, etc.

[0087] "Personalization" refers to individualization based on a user's past behavioral data and preferences.

[0088] "Entertainment functions" are functions that provide entertainment to users, such as playing music and asking quizzes.

[0089] "Safe Driving Support" is a support function that warns of dangers while driving based on vehicle data and encourages safe driving.

[0090] "Natural language processing" is a technology that analyzes voice and text input from users and understands their intent.

[0091] A "local database" is a database for storing information obtained from outside and updating it as necessary.

[0092] "Generative AI technology" is an AI technology for generating customized character chatbots based on user profile information.

[0093] A "character chatbot" is a virtual character generated using generative artificial intelligence technology to enable natural conversations with users.

[0094] The present invention provides a car navigation system with an interactive interface that allows users to intuitively specify destinations and stops. This system supports users in comfortable and efficient driving, and also provides entertainment while traveling and assists in safe driving.

[0095] First, the user inputs instructions via voice or text through a conversational interface. This interface converts the voice into text using a speech recognition engine (e.g., Google Cloud Speech-to-Text). Next, a natural language processing engine (e.g., Google Cloud Natural Language API) is used to analyze the user's instructions and extract keywords. For example, if the user says, "I want to go to a delicious cafe nearby," the system extracts the keywords "nearby" and "delicious cafe."

[0096] The server periodically retrieves the latest traffic information from external sources (e.g., HERE API) and stores information on congestion, accidents, road closures, etc. in a database (e.g., MySQL). This information is updated in real time and pushed to the device as needed.

[0097] The device also obtains ratings and business information for local restaurants and tourist spots from external information services (e.g., Yelp API) and stores this information in a local database. Based on this stored information, it lists the best places for the user and displays them on the screen. Furthermore, the server manages the user's past behavioral data as profile information, and the device calculates and suggests the best route based on this information. For example, it can prioritize routes to cafes that have been highly rated in the past.

[0098] As an entertainment feature, the user can make requests to the device, such as "Play the next song." The device will then call a music distribution service (e.g., Spotify API) and play the specified song. The device can also select questions from a quiz database and pose them to the user, providing entertainment while driving.

[0099] As a safe driving function, the device collects and analyzes data such as speed and following distance from various sensors in the vehicle in real time, and displays an alert to the user if danger is predicted. For example, it displays a warning message such as "There is insufficient distance between vehicles ahead. Please be careful."

[0100] Furthermore, the server uses generative AI technology (e.g., OpenAI GPT-4) to generate a customized character chatbot based on the user's profile. Using this chatbot, the device can have natural conversations with the user. Users can obtain a dedicated sharing link to share the chatbot character they created with other users, or sell the chatbot character through a marketplace.

[0101] Examples of prompts include:

[0102] "Tell me a recommended cafe nearby."

[0103] "Turn right at the next traffic light"

[0104] "Tell me the latest traffic information"

[0105] "Show me suggestions for cafes I've been to in the past."

[0106] "Play some Mozart"

[0107] "Show an alert if the following distance is too close"

[0108] "Generate a chatbot character based on user preferences"

[0109] "Get a link to share your generated character with other users."

[0110] In this way, the present invention allows users to intuitively operate the device and enjoy real-time information and personalized suggestions, resulting in a comfortable and safe driving experience.

[0111] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0112] Step 1:

[0113] The user inputs instructions through a conversational interface using voice or text. For example, the user might input, "I want to go to a nearby delicious cafe." The input voice data is sent to a speech recognition engine (e.g., Google Cloud Speech-to-Text) and converted into text data.

[0114] Input: User instructions via voice or text

[0115] Data processing: Speech to text conversion using a voice recognition engine

[0116] Output: Textual user instructions

[0117] Step 2:

[0118] The device sends the text data converted by the speech recognition engine to a natural language processing engine (e.g., Google Cloud Natural Language API) for analysis. Keywords are extracted to understand the user's intent. For example, keywords such as "nearby" and "delicious cafe" are extracted.

[0119] Input: Textual user instructions

[0120] Data Calculation: Keyword Extraction Using Natural Language Processing Engine

[0121] Output: Extracted keywords

[0122] Step 3:

[0123] The server periodically sends requests to external sources (e.g., HERE API) to obtain the latest traffic information. The obtained data is stored in a database (e.g., MySQL) and updated in real time. Push notifications are sent to the device when necessary.

[0124] Input: Request to external source

[0125] Data processing: Acquisition of traffic information and storage in a database

[0126] Output: Saved real-time traffic information data

[0127] Step 4:

[0128] The device sends a request to an external information service (e.g., Yelp API) to obtain ratings and business information for restaurants and tourist attractions in the area specified by the user. This information is stored in a local database.

[0129] Input: A request to an external information service

[0130] Data processing: Acquisition of evaluation information and storage in a local database

[0131] Output: Saved evaluation and sales information data

[0132] Step 5:

[0133] The device selects and lists the best places for the user based on the stored rating and business information data and real-time traffic information data. It then displays the list on the screen. For example, it presents the user with a list of "delicious cafes nearby."

[0134] Input: Rating and business information data, traffic information data

[0135] Data calculation: listing and optimized location suggestions

[0136] Output: A list of the best places listed by the user

[0137] Step 6:

[0138] The server manages the user's past behavior data (e.g., places rated in the past) as profile information. The device references this profile information and calculates and proposes the optimal route based on the user's past behavior. For example, it may prioritize routes to cafes that have been highly rated in the past.

[0139] Input: User's past behavior data

[0140] Data calculation: Referencing profile information and calculating routes

[0141] Output: Optimal route suggested to the user

[0142] Step 7:

[0143] The user inputs an entertainment request, such as "Play the next song," into the device. The device then calls a music distribution service (e.g., Spotify API) and plays the specified song. It also selects questions from a quiz database and presents them to the user in the form of a quiz.

[0144] Input: User entertainment request

[0145] Data calculation: Calling music distribution services and quiz questions

[0146] Output: Music playback and quiz

[0147] Step 8:

[0148] The device acquires and analyzes the vehicle's sensor data (e.g., speed and following distance) in real time. If there is a problem with safe driving, an alert will be displayed to the user. For example, if there is insufficient distance between the vehicle ahead, the device will display the message "Insufficient distance between the vehicle ahead. Please be careful."

[0149] Input: Vehicle sensor data

[0150] Data Calculation: Real-time analysis and safe driving alerts

[0151] Output: Alert the user

[0152] Step 9:

[0153] The server uses generative AI technology (e.g., OpenAI GPT-4) to generate a customized character chatbot based on the user's profile information. The generated chatbot is sent to the device and engages in natural conversations with the user.

[0154] Input: User profile information

[0155] Data Computation: Generating Character Chatbots with Generative AI Technology

[0156] Output: A chatbot that enables natural dialogue with users

[0157] Step 10:

[0158] Users can share their chatbot characters with other users by obtaining a dedicated sharing link from the server. Chatbot characters can also be sold through the marketplace.

[0159] Input: Generated chatbot character

[0160] Data processing: generating sharing links and selling on the marketplace

[0161] Output: Shared link and sales method

[0162] (Application example 1)

[0163] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0164] Today's drivers demand intuitive and efficient navigation systems. Especially important are interfaces that can be easily operated via voice, and the ability to obtain real-time traffic information and suggest optimal routes. However, few systems can simultaneously provide entertainment and safe driving support in addition to these functions. They also demand personalized suggestions based on individual user preferences and behavioral history. Furthermore, they also want chatbots that use generative AI technology to make the interface more user-friendly.

[0165] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0166] In this invention, the server includes: means for providing an interactive interface that allows a user to intuitively indicate places the user wants to go and points the user wants to stop at; means for acquiring traffic information from external information sources that is updated in real time and proposing optimal routes that avoid congestion; means for acquiring and providing area-specific ratings and business information from external information services; means for personalizing routes based on the user's past behavioral data and preferences; means for providing entertainment functions; means for supporting safe driving based on vehicle data; means for analyzing the user's voice commands using voice recognition technology and understanding the user's intentions using a natural language processing engine; means for displaying information in real time on the smart glasses; and means for generating an original character chatbot using generative artificial intelligence technology and having the chatbot interact with the user through the smart glasses. This allows the driver to intuitively operate the system and enjoy a comfortable and safe driving experience that combines real-time information with personalized suggestions.

[0167] - "User" means a person who uses a navigation system to reach a destination.

[0168] An "interactive interface" is a system that accepts and responds to instructions from the user through voice or text.

[0169] "External information sources" refer to external services or APIs that provide data such as traffic information and store information.

[0170] "Traffic information" refers to data on traffic conditions, including information on congestion, accidents, road closures, etc.

[0171] "Ratings and business information" is data showing user ratings and business status of stores and tourist attractions in the area.

[0172] "Personalization" refers to individual optimization based on a user's past behavioral data and preferences.

[0173] "Entertainment features" are features that allow users to enjoy themselves, such as music playback, quizzes, and chat.

[0174] "Vehicle data" is information about the vehicle's operation, such as speed and following distance.

[0175] "Voice recognition technology" is a technology that analyzes the user's voice and converts it into text data.

[0176] A "natural language processing engine" is a technology that understands the user's intent from text data and generates an appropriate response.

[0177] "Smart glasses" are glasses-type devices that have a built-in display and provide visual information when worn by the user.

[0178] "Generative artificial intelligence technology" is a technology that uses AI to generate new characters and content.

[0179] An "original character chatbot" is a conversational character created based on a user's profile using generative AI technology.

[0180] The present invention provides an interactive navigation system that allows users to intuitively specify where they want to go, provides real-time traffic information, and realizes a safe and comfortable driving experience. Specific embodiments for implementing this system are described below.

[0181] 1. User Interface Design

[0182] Users input commands by voice through a conversational interface, such as "I want to go to a nice cafe nearby." This voice command is converted into text data by the device's voice recognition technology (e.g., speech_recognition library), and then analyzed by a natural language processing engine (e.g., IntentRecognizer) to understand the user's intent.

[0183] 2. Obtaining real-time traffic information

[0184] The server periodically retrieves the latest traffic information from external sources (e.g., traffic information APIs), including traffic congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time. The device displays this information on the smart glasses display and suggests optimal routes for the user.

[0185] 3. Obtaining evaluations and sales information

[0186] The device obtains ratings and business information for restaurants and tourist spots in the area from external information services (e.g., rating site APIs), and based on this, it creates a list of places that best suit the user's needs.

[0187] 4. Personalized Route Suggestions

[0188] The server manages the user's past behavioral data and preferences as profile information. Based on this, the device suggests an optimized route to the user. By prioritizing cafes that have been highly rated in the past, the device can provide the user with a more satisfying experience.

[0189] 5. Providing entertainment features

[0190] Users can make requests to their device, such as "play the next song," to enjoy music playback, quizzes, and chat functions. The device then calls an external music service (e.g., music distribution API) to play music. It also generates quizzes from a quiz database, providing entertainment while driving.

[0191] 6. Safe driving support

[0192] The device monitors vehicle data (e.g., speed, distance between vehicles) in real time and displays alerts to encourage safe driving. For example, if the distance between vehicles is too small, a message such as "There is insufficient distance between vehicles ahead. Please be careful" will be displayed.

[0193] 7. Character Chatbot Generation and Interaction

[0194] The server uses generative AI technology to generate an original character chatbot based on the user's profile, which then interacts with the user through smart glasses, enabling natural communication.

[0195] 8. Sharing and selling characters

[0196] Users can share their chatbot characters with other users by obtaining a dedicated sharing link from the server, and can also sell their chatbot characters through the marketplace.

[0197] Examples and prompts

[0198] As a concrete example, the system functions can be realized by using the following prompt sentences:

[0199] Example prompt:

[0200] User: "I want to go to a recommended cafe nearby."

[0201] AI system: "Activate the speech recognition engine to understand the user's intent."

[0202] AI system: "Get information about nearby cafes from an external API."

[0203] AI system: "Recommended cafes will be displayed on the screen."

[0204] User: "Okay, let's go to the second cafe."

[0205] AI system: "Route guidance in progress. Turn right."

[0206] User: "Play some music."

[0207] AI system: "Playing your favorite song"

[0208] This allows users to enjoy a comfortable driving experience that is intuitive to operate and combines real-time information with personalized suggestions.

[0209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0210] Step 1:

[0211] The user inputs instructions by voice, such as "I want to go to a nice cafe nearby." This voice input is captured through the device's microphone.

[0212] Step 2:

[0213] The device uses speech recognition technology (e.g., the speech_recognition library) to convert speech to text data. The input of this step is speech data, and the output is text data.

[0214] Step 3:

[0215] The device uses a natural language processing engine (e.g., IntentRecognizer) to analyze the user's intent from the text data. The input is text data, and the output is data indicating the user's intent (e.g., the intent to "find a cafe").

[0216] Step 4:

[0217] The device sends the user's intention to the server and queries the relevant local cafe information. The input is the user's intention data, and the output is the cafe information obtained through the external API.

[0218] Step 5:

[0219] The server obtains ratings and business information for restaurants and tourist attractions from external information sources (e.g., rating site APIs). This includes region-specific rating data and business information. The input is data indicating the region, and the output is the ratings and business information.

[0220] Step 6:

[0221] The server filters relevant information based on the user's past behavioral data and preferences to generate a personalized cafe list. The input is the user's past behavioral data and externally acquired ratings and sales information, and the output is a personalized list of cafes.

[0222] Step 7:

[0223] The terminal displays the cafe list obtained from the server on the display of the smart glasses. The input is the cafe list received from the server, and the output is the information displayed on the display of the smart glasses.

[0224] Step 8:

[0225] The user selects a cafe and then gives a voice command, such as "Let's go to the second cafe," which is also captured through the smart glasses.

[0226] Step 9:

[0227] The device again uses voice recognition technology and a natural language processing engine to analyze the user's new command: the input is voice data, and the output is the intent data: "Go to the second cafe."

[0228] Step 10:

[0229] The device retrieves optimal route information from the server and displays route guidance on the smart glasses. The input is the user's intention, current location, and route information received from the server, and the output is the display of route guidance.

[0230] Step 11:

[0231] The user types in a voice command requesting "play music," which is also captured through the smart glasses.

[0232] Step 12:

[0233] The device uses voice recognition technology and a natural language processing engine to analyze the music request and then calls an external music service (e.g., a music distribution API) to play the music. The input is audio data, and the output is the music being played.

[0234] Step 13:

[0235] The terminal monitors vehicle data (e.g., speed, distance between vehicles) in real time and displays safe driving alerts on the smart glasses. The input is vehicle data, and the output is the display of safe driving alerts.

[0236] Step 14:

[0237] The server uses generative AI technology to generate an original character chatbot based on the user's profile, and the device displays this character on the smart glasses and allows the user to interact with it. The input is the user profile data, and the output is the interaction with the generated character.

[0238] Step 15:

[0239] To share the chatbot character that a user has created with other users, the user obtains a dedicated sharing link from the server. The input is the generated character data, and the output is the sharing link.

[0240] This allows users to enjoy a comfortable driving experience that combines real-time information with personalized suggestions, with more intuitive operation than usual.

[0241] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0242] The present invention provides a car navigation system that has an interactive interface that allows users to intuitively indicate places they want to go or points they want to stop at, and that also combines an emotion engine that recognizes the user's emotions. This system allows the user to have a more personalized experience and realize a more comfortable driving experience. Specific embodiments of the present invention are described below.

[0243] 1. User Interface Design

[0244] Users can input instructions through a conversational interface using voice or text. For example, they can say, "I want to go to a nice cafe nearby." The device then uses a speech recognition engine to convert the voice data into text.

[0245] 2. Introducing the Emotion Engine

[0246] The device implements an emotion engine that analyzes emotions from the user's voice and text input. The emotion engine recognizes the user's emotional state (e.g., joy, anger, sadness, or happiness) from the user's tone of voice and text expressions, and saves the analysis results.

[0247] 3. Obtaining real-time traffic information

[0248] The server periodically retrieves the latest traffic information from external sources (e.g., traffic information API). This information includes congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time.

[0249] 4. Obtaining ratings and business information for restaurants and tourist spots

[0250] The device retrieves a list of cafes in the specified area, along with their ratings and business information, from an external information service (e.g., a rating site API). Based on this information, the device creates a list of the cafes with the highest ratings.

[0251] 5. Personalized Route Suggestions

[0252] The server manages the user's past behavioral data and preferences as profile information. The device accesses this profile information and suggests optimized routes to the user. For example, it may prioritize cafes that users have given high ratings to in the past or places based on their current emotional state.

[0253] 6. Provision of entertainment features

[0254] Users can enjoy entertainment such as music playback, quizzes, and chat functions. The device receives requests such as "play the next song" and calls an external music service (e.g., music distribution API) to play music. In addition, an emotion engine analyzes the user's emotions and adjusts appropriate entertainment content.

[0255] 7. Safe driving support

[0256] The device monitors vehicle sensor information in real time and provides alerts when speed, distance, or improper driving is detected. In addition, an emotion engine adjusts the content and timing of alerts based on the user's emotional state.

[0257] 8. Creating a Character Chatbot

[0258] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile. The device then uses this character to engage in natural conversations with the user. Using generative AI technology, the character's expressions also change dynamically to reflect the user's emotional state.

[0259] 9. Sharing and selling characters

[0260] Users can obtain a dedicated sharing link from the server to share the chatbot characters they have created with other users. They can also sell chatbot characters through the marketplace. The server provides this functionality through the network.

[0261] Specific examples

[0262] As a concrete example, consider the case where a user verbally commands, "I want to go to a nearby, relaxing cafe." The device converts this command into text using a speech recognition engine and analyzes it using a natural language processing engine. Next, the emotion engine analyzes the user's tone of voice and recognizes that the user is a little tired. The device obtains cafe information from an external information service, and the server uses past behavioral data and emotion information to create a list of highly rated cafes that are relaxing. The device then suggests the user the optimal route to that cafe. While driving, the device plays relaxing music that matches the user's preferences. The device also obtains safety data from vehicle sensors and works with the emotion engine to provide the user with safety alerts at the appropriate time.

[0263] In this way, the present invention makes it possible to realize a system that allows the user to enjoy a more comfortable and safe driving experience.

[0264] The processing flow will be explained below.

[0265] Step 1:

[0266] The user speaks to the device, saying, "I want to go to a quiet cafe nearby." The device receives this instruction and uses a speech recognition engine to convert the voice data into text data.

[0267] Step 2:

[0268] The device passes the converted text data to a natural language processing engine, which extracts keywords such as "cafe," "nearby," and "calm."

[0269] Step 3:

[0270] The device uses an emotion engine to analyze the user's emotional state from their voice and determines that they are "tired." The emotion engine then stores this information in their profile.

[0271] Step 4:

[0272] The server sends a request to an external information source (e.g., a traffic information API) to obtain the latest traffic information, and stores the obtained traffic information in a database.

[0273] Step 5:

[0274] The terminal obtains a list of cafes in a specified area, along with their ratings and business information, from an external information service (e.g., a rating site API).

[0275] Step 6:

[0276] The device will use the cafe information obtained in step 5 to create a list of cafes that are highly rated and have a relaxing atmosphere.

[0277] Step 7:

[0278] The device uses the Google Maps API to calculate the optimal route based on the traffic information obtained in step 4 and the cafe information listed in step 6.

[0279] Step 8:

[0280] Based on the user's past behavioral data and the emotional profile obtained in step 3, the server selects the cafe that best suits the user's preferences and current emotional state, and then personalizes and suggests a route to the cafe.

[0281] Step 9:

[0282] The user checks the suggested cafe and route on the device and confirms, "I want to go there."

[0283] Step 10:

[0284] The user requests, "Play some relaxing music." The device sends a request to the music distribution API and plays relaxing music.

[0285] Step 11:

[0286] While driving, the device monitors vehicle sensor information to check speed and following distance, and provides alerts if improper driving is detected.

[0287] Step 12:

[0288] The device uses an emotion engine to monitor the user's emotional state and adjust the entertainment content and timing of safe driving alerts as needed.

[0289] Step 13:

[0290] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile, and the device uses this character to interact with the user.

[0291] Step 14:

[0292] Users can share their chatbot characters with friends by obtaining a sharing link from the server. Characters can also be sold on the marketplace. The server manages transactions and generates sharing links.

[0293] In this way, users can operate intuitively and enjoy real-time information and personalized suggestions based on their emotional state, ensuring a comfortable and safe driving experience.

[0294] Example 2

[0295] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0296] Conventional car navigation systems simply provide routes to a destination and lack the ability to provide a personalized experience based on the user's emotional state and individual preferences. Furthermore, entertainment and safe driving support functions are limited, creating a need for an integrated solution to improve the overall driving experience. Furthermore, providing interactive character chatbots using generative AI technology and creating new user experiences based on them are also challenges.

[0297] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0298] In this invention, the server includes means for providing an interactive interface that allows the user to intuitively indicate places the user wants to go and points the user wants to stop at, means for acquiring traffic information from external information sources that is updated in real time and proposing optimal routes that avoid congestion, means for acquiring and providing area-specific ratings and business information from external information services, means for personalizing routes based on the user's past behavioral data and preferences, means for analyzing the user's emotional state from voice and text input, means for adjusting and providing entertainment functions according to the emotional state, means for supporting safe driving based on vehicle data and adjusting the content and timing of alerts based on the emotional state, and means for generating an original character chatbot based on the user's profile and dynamically changing and interacting with the chatbot. This enables optimal route suggestions, entertainment, and safe driving support based on the user's emotions and preferences, providing a more personalized and comfortable driving experience.

[0299] An "interactive interface" is an interface that allows a user to intuitively input instructions by voice or text, and the system returns an appropriate response to that input.

[0300] "External information sources" refer to information services and APIs that provide traffic information, traffic accident information, traffic congestion information, etc. from outside the system.

[0301] "Traffic Information" is real-time information about current road conditions, traffic congestion, traffic accidents, road closures, etc.

[0302] "Region-specific ratings" refers to user ratings and word-of-mouth information about places and facilities in a specific region.

[0303] "Business information" refers to information such as opening hours, regular holidays, and special opening days of restaurants, tourist attractions, etc.

[0304] "External information services" refer to services and APIs that provide evaluations and sales information.

[0305] "Personalizing your route" means suggesting the best route based on the user's past behavioral data and individual preferences.

[0306] "Emotional state" refers to the emotions such as joy, anger, sadness, and happiness that the user is currently feeling, and is analyzed from voice and text input.

[0307] "Entertainment features" are features that users can enjoy while driving, such as music playback, quizzes, and chat functions.

[0308] "Safe Driving Support" is a function that uses information from the vehicle's sensors to detect driving speed, vehicle distance, and inappropriate driving, and provides alerts to the user.

[0309] An "alert" is a notification that the system uses to warn or alert the user to encourage safe driving.

[0310] "Generative AI technology" is a technology that uses artificial intelligence to generate new content and characters.

[0311] "Profile Information" refers to information about a user, such as the user's past behavioral data, preferences, and emotional state.

[0312] A "character chatbot" is a virtual character created using generative AI technology that can engage in natural conversations with users.

[0313] "Dynamic change" means that the chatbot's behavior and responses change in real time depending on the user's emotional state and situation.

[0314] "Marketplace" means an online platform for sharing or selling user-generated chatbot characters.

[0315] This invention is a car navigation system that features an interactive interface that allows users to intuitively indicate where they want to go and where they want to stop, and provides real-time traffic information, emotion analysis, personalized route suggestions, entertainment functions, safe driving support, and a character chatbot that utilizes generative AI technology.

[0316] User Interface Design

[0317] Users input instructions to the system through voice or text. For example, if a user says, "I want to go to a nice cafe nearby," the device uses a speech recognition engine (e.g., Google Speech-to-Text) to convert the voice into text data. Then, a natural language processing engine (e.g., OpenAI GPT) analyzes this text data.

[0318] Introducing the Emotion Engine

[0319] The device implements an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the user's emotional state from voice and text. This engine analyzes the user's emotional state (e.g., joy, anger, sadness, or happiness) from the user's tone of voice and text expressions and saves the results.

[0320] Obtaining real-time traffic information

[0321] The server periodically retrieves the latest traffic information from external sources (e.g., Traffic API), including information on congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time.

[0322] Obtaining ratings and business information for restaurants and tourist spots

[0323] The device uses external information services (e.g., Yelp API) that provide location-specific ratings and business information to obtain a list of cafes and their ratings and business information, allowing it to present the location that best suits the user's request.

[0324] Personalized route suggestions

[0325] The server manages the user's past behavioral data and preferences as profile information. The device then uses this profile information to suggest the most suitable route for the user. Specifically, it prioritizes cafes that users have given high ratings to in the past and places that correspond to their current emotional state.

[0326] Providing entertainment features

[0327] Users can enjoy entertainment such as music playback, quizzes, and chat functions. The device receives requests such as "play the next song" and calls an external music service (e.g., Spotify API) to play music. In addition, an emotion engine analyzes the user's emotions and adjusts appropriate entertainment content.

[0328] Safe driving support

[0329] The device monitors vehicle sensor information in real time and provides alerts when speed, distance, or improper driving is detected. The emotion engine adjusts the content and timing of alerts based on the user's emotional state.

[0330] Character chatbot generation

[0331] The server uses generative AI technology (e.g., OpenAI GPT) to generate a chatbot with an original character based on the user's profile. The device then uses this character to engage in natural conversations with the user. Generative AI technology dynamically changes the character's expressions according to the user's emotional state.

[0332] Sharing and selling characters

[0333] Users can obtain a dedicated sharing link from the server to share their chatbot characters with other users, and can also sell chatbot characters through the marketplace.

[0334] Specific examples

[0335] Consider a scenario where a user verbally commands, "I want to go to a nearby relaxing cafe." The device converts this command into text using a speech recognition engine (e.g., Google Speech-to-Text) and analyzes it using a natural language processing engine (e.g., OpenAI GPT). Next, an emotion engine (e.g., IBM Watson Tone Analyzer) analyzes the user's tone of voice and recognizes that the user is slightly tired. The device obtains cafe information from an external information service (e.g., Yelp API), and the server uses past behavioral data and emotion information to create a list of highly rated cafes that are relaxing. The device then suggests the user the optimal route to the cafe. While driving, the device plays relaxing music using an external music service (e.g., Spotify API). The device also obtains safety data from vehicle sensors and, in cooperation with the emotion engine, provides the user with safety alerts at the appropriate time.

[0336] Prompt Sentence Examples

[0337] "If a user says, 'I want to go to a quiet cafe nearby,' what kind of processing and data is required?"

[0338] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0339] Step 1:

[0340] The user launches the device's conversational interface. The user inputs a command by voice or text, such as "I want to go to a nearby delicious cafe." In this case, the input is voice data, which is passed to the device. Specifically, the user performs input operations using a microphone or keyboard.

[0341] Step 2:

[0342] The device uses a speech recognition engine (e.g., a speech recognition engine API) to convert the user's voice data into text data. The output is text data, which is used as input data for the next processing step. The specific operation involves the process in which the speech recognition engine analyzes the voice waveform and converts it into corresponding text.

[0343] Step 3:

[0344] The device passes the converted text data to a natural language processing engine (e.g., natural language processing API) and analyzes the user's instructions. The input is text data, and the output is the intent data resulting from the analysis. For example, the intent extracted is "I want to go to a nearby delicious cafe." Specifically, the natural language processing engine understands the context of the text and concludes on the user's request.

[0345] Step 4:

[0346] The device uses an emotion engine (e.g., emotion analysis API) to analyze the user's emotional state from text data and voice tone. The input is text data and voice tone data, and the output is emotional state data. For example, it may recognize that the user is slightly tired. Specifically, the emotion engine analyzes the intonation of the voice and the tone of the text to determine the emotional state.

[0347] Step 5:

[0348] The server periodically obtains the latest traffic information from an external information source (e.g., a traffic information API). The input is an API request, and the output is traffic information data. Specifically, the server sends an API request and receives real-time data such as congestion and accident information.

[0349] Step 6:

[0350] The device obtains cafe ratings and business information from an external information service (e.g., a rating site API). The input is an API request, and the output is a list of cafes and their rating data. Specific operations include sending an API request, parsing the results, and generating a list of cafes.

[0351] Step 7:

[0352] The device accesses the user's profile information (e.g., past behavioral data, preferences) stored on the server and proposes the optimal route. The input is the profile data, and the output is optimal route guidance data. Specifically, the server evaluates candidate locations based on the past behavioral data and the user's preferences, and generates the optimal route.

[0353] Step 8:

[0354] When a user requests "play the next song," the device calls an external music service (e.g., a music distribution API) to play music. The input is the user's voice request, and the output is playback instruction data. Specifically, the device sends an API request to the music distribution service and streams the music data.

[0355] Step 9:

[0356] The device monitors vehicle sensor information in real time and provides alerts to support safe driving. The input is sensor information and the output is alert data. Specific operations include analyzing sensor information to evaluate speed and vehicle distance and detect inappropriate driving.

[0357] Step 10:

[0358] The server uses generative AI technology (e.g., generative AI model) to generate an original character chatbot based on the user's profile. The input is profile information, and the output is chatbot data. Specifically, the server uses generative AI technology to simulate the character's dialogue.

[0359] Step 11:

[0360] The server generates a dedicated sharing link so that users can share their chatbot characters with other users. The input is the chatbot data, and the output is the sharing link. Specifically, the server registers the chatbot data on the sharing platform and generates the link.

[0361] (Application example 2)

[0362] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0363] Conventional car navigation systems only provide simple route guidance without considering the user's emotions or entertainment needs. This makes it difficult to provide a comfortable and personalized driving experience for users. Furthermore, their safe driving support features are limited, and they do not provide appropriate alerts or entertainment tailored to the user's emotional state. This makes it difficult to reduce user stress and discomfort, and furthermore, ensures safe driving. The present invention aims to solve these problems of the conventional technology and provide users with a personalized, comfortable driving experience and safe driving support.

[0364] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for analyzing the user's emotional state and proposing entertainment content and routes based on the emotion, a means for monitoring the vehicle's sensor information in real time and adjusting the content and timing of alerts based on the user's emotional state, and a means for acquiring and proposing area-specific reviews and business information from external information services. This enables personalized entertainment and route suggestions that take the user's emotional state into consideration. Furthermore, safety alerts can be provided at appropriate times according to the user's emotional state, resulting in a comfortable and safe driving experience.

[0365] An "interactive interface" is an input method that allows users to intuitively specify their destination or points of interest using voice or text.

[0366] "External information sources updated in real time" refers to information provision services that provide traffic information, rating information, etc. with the latest data over time.

[0367] "Means of obtaining traffic information and proposing the optimal route to avoid congestion" refers to a function that imports traffic data in real time from external traffic information services and, based on that data, presents users with the optimal route to avoid congestion.

[0368] "Means for obtaining and providing region-specific ratings and business information from external information services" refers to a function that obtains ratings and business information on restaurants, tourist attractions, etc. related to a specified region from external information services and provides it to users.

[0369] "Means to personalize routes based on the user's past behavioral data and preferences" is a function that suggests individualized routes based on information such as the user's past travel history, favorite categories, and preferred locations.

[0370] "Means for providing entertainment functions" refers to functions that provide users with entertainment content such as music, quizzes, and chat.

[0371] "Means to support safe driving based on vehicle data" refers to a function that provides users with alerts and notifications to encourage safe driving based on data obtained from sensors, cameras, etc. installed in the vehicle.

[0372] "Means for analyzing the user's emotional state and suggesting entertainment content and routes based on that emotion" refers to a function that analyzes the user's emotional state (e.g., happy, tired) from voice and text data, and personalizes entertainment content and routes based on the analysis results.

[0373] "Means for monitoring vehicle sensor information in real time and adjusting the content and timing of alerts based on the user's emotional state" refers to a function that monitors data obtained in real time from various sensors installed in the vehicle and optimizes the content and timing of alerts according to the user's emotional state.

[0374] The system of the present invention includes an interactive interface that allows users to intuitively indicate where they want to go and where they want to stop, routes that are suggested based on traffic information obtained from external sources that is updated in real time, personalized entertainment content and route suggestions based on the user's emotional state, and functions that support safe driving.

[0375] 1. User Interface Design

[0376] A conversational interface uses a speech recognition engine (e.g., Google Speech Recognition API) to convert the user's speech into text. When a user speaks a command such as "I want to go to a nice cafe nearby," the speech data is converted into text by the recognition engine.

[0377] 2. Introducing the Emotion Engine

[0378] Using an emotion engine (e.g., EmotionRecognizer), the emotional state of the user is analyzed from voice and text, allowing accurate understanding of the user's emotional state, such as whether they are tired or happy.

[0379] 3. Obtaining real-time traffic information

[0380] The server periodically obtains the latest traffic information using a traffic information API (e.g., a general traffic information API), thereby obtaining real-time information on traffic congestion and accidents and storing it in a database.

[0381] 4. Obtaining ratings and business information for restaurants and tourist spots

[0382] The device retrieves a list of cafes in the specified area, along with their ratings and business information, from an external information service (e.g., a rating site API). Based on this information, the device creates a list of the cafes with the highest ratings.

[0383] 5. Personalized Route Suggestions

[0384] The server manages the user's past behavioral data and preferences (favorite restaurant trends and records of places visited) as profile information. The device then suggests the optimal route based on this profile information and the results of the emotion engine's analysis.

[0385] 6. Provision of entertainment features

[0386] The device provides features such as music playback (using external music service APIs), quizzes, and chat functions. The emotion engine analyzes the user's emotions and dynamically adjusts the entertainment content to suit their needs.

[0387] 7. Safe driving support

[0388] The device monitors vehicle sensor information in real time to support safe driving, and improves safety by adjusting the content and timing of safety alerts based on the user's emotional state.

[0389] 8. Creating a Character Chatbot

[0390] The server uses generative AI technology (e.g., generative AI model) to generate a chatbot with an original character based on the user's profile. The device then uses this character to engage in natural conversations with the user.

[0391] 9. Sharing and selling characters

[0392] Users can obtain a dedicated sharing link from the server to share their chatbot characters with other users, and can also sell their characters through the marketplace.

[0393] Specific examples

[0394] For example, if a user says, "I want to go to a nearby relaxing cafe," the device will convert this into text using a speech recognition engine and analyze the user's emotional state using an emotion engine. If the device detects that the user is tired, it will retrieve cafe information from a rating site API and create a list of highly rated cafes that are relaxing and calming based on past behavioral data and emotional information. It will then suggest the optimal route to the cafe to the user and play relaxing music. It will also retrieve safety data from vehicle sensors and link with the emotion engine to provide safety alerts at the appropriate time.

[0395] Prompt Sentence Examples

[0396] "Based on the invention below, you will design an emotional car navigation app for self-driving cars. The app should include a function that allows the user to specify a destination by voice, and the emotion engine analyzes the user's emotions, suggests the optimal route, and provides entertainment."

[0397] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0398] Step 1:

[0399] The user inputs instructions by voice. The user may say, "I want to go to a nice cafe nearby." This voice input becomes the input to the system.

[0400] Step 2:

[0401] The device uses a speech recognition engine (e.g., Google Speech Recognition API) to convert the voice data into text data. The speech recognition engine analyzes the voice data and outputs it as text data.

[0402] Step 3:

[0403] The device uses an emotion engine (e.g., EmotionRecognizer) to analyze the user's emotional state from text and voice data, and outputs the emotional state (e.g., tired, happy) obtained through this analysis.

[0404] Step 4:

[0405] The server uses a traffic information API (e.g., a general traffic information API) to obtain the latest traffic information in real time. The obtained traffic information is stored in a database and becomes input data for the system.

[0406] Step 5:

[0407] The device uses an external information service (e.g., a rating site API) to obtain a list of cafes in the specified area, along with their ratings and business information. This data is input, and a list of highly rated cafes is output.

[0408] Step 6:

[0409] The server retrieves the user's past behavioral data and preferences (e.g., a list of cafes visited and their ratings) from the database as profile information. The retrieved profile information becomes input data for the system.

[0410] Step 7:

[0411] The device then proposes the optimal route based on the acquired profile information, the results of the emotion engine analysis, and traffic and cafe information. The device analyzes this data and outputs a personalized route for the user.

[0412] Step 8:

[0413] The device provides entertainment functions. Based on the analysis results of the emotion engine, it dynamically adjusts and provides music, quizzes, and chat functions that suit the user. This function outputs entertainment content optimized for the user's emotional state.

[0414] Step 9:

[0415] The device monitors the vehicle's sensor information in real time and adjusts the content and timing of safety alerts according to the user's emotional state. Alerts are output at appropriate times based on the sensor information and the user's emotional state.

[0416] Step 10:

[0417] The server uses generative AI technology (e.g., generative AI model) to generate a chatbot with an original character based on the user's profile. The generated character is used as an output to the system to interact with the user.

[0418] Step 11:

[0419] Users can share the generated character chatbots with other users or sell them on the marketplace. The server generates a sharing link and provides a sales function in cooperation with the marketplace. This function is provided as an output to the user.

[0420] Adding specific examples

[0421] For example, if a user says, "I want to go to a nearby relaxing cafe," the system converts the speech to text and then uses its emotion engine to interpret it as "tired." It then retrieves cafe information from a review site API and, taking into account the user's past review data, creates a list of the most suitable cafes and suggests them to the user along with a route. It also plays relaxing music and adjusts safety alerts while driving based on the user's emotional state.

[0422] Prompt Sentence Examples

[0423] "Based on the invention below, you will design an emotional car navigation app for self-driving cars. The app should include a function that allows the user to specify a destination by voice, and the emotion engine analyzes the user's emotions, suggests the optimal route, and provides entertainment."

[0424] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0425] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0426] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0427] [Second embodiment]

[0428] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0429] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0430] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0431] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0432] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0433] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0434] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0435] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0436] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0437] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0438] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0439] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0440] The present invention proposes a car navigation system with an interactive interface that allows users to intuitively specify where they want to go and where they want to stop. This not only realizes the comfortable and efficient driving that modern drivers desire, but also provides entertainment while traveling and support for safe driving. Specific embodiments of the present invention are described below.

[0441] 1. User Interface Design

[0442] Users can input instructions through a conversational interface using voice or text. For example, they can say, "I want to go to a nice cafe nearby." The device then analyzes this with a natural language processing engine and extracts keywords to understand the user's intent.

[0443] 2. Obtaining real-time traffic information

[0444] The server periodically retrieves the latest traffic information from external sources (e.g., traffic information API). This information includes congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time.

[0445] 3. Obtaining ratings and business information for restaurants and tourist spots

[0446] The device retrieves ratings and business information for restaurants and tourist spots in the specified area from external information services (e.g., rating site APIs). Based on this data, it creates a list of places that best meet the user's requirements.

[0447] 4. Personalized Route Suggestions

[0448] The server manages the user's past behavioral data and preferences as profile information, and based on this, the device suggests optimized routes to the user, for example, prioritizing cafes that have been highly rated in the past.

[0449] 5. Providing entertainment features

[0450] Users can make requests to their device, such as "play the next song," to enjoy entertainment such as music playback, quizzes, and chat functions. The device then plays music by calling an external music service (e.g., music distribution API). It also generates quizzes from a quiz database, providing entertainment while driving.

[0451] 6. Safe driving support

[0452] The device monitors vehicle data (e.g., speed, distance between vehicles) in real time and displays alerts to encourage safe driving. For example, if the distance between vehicles is too small, a message such as "There is insufficient distance between vehicles ahead. Please be careful" will be displayed.

[0453] 7. Creating a Character Chatbot

[0454] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile. This character is personalized based on the user's preferences and interests. The device then uses the generated character to engage in natural conversations with the user.

[0455] 8. Sharing and selling characters

[0456] Users can obtain a dedicated sharing link from the server to share the chatbot characters they have created with other users. They can also sell chatbot characters through the marketplace. The server provides this functionality through the network.

[0457] In this way, the present invention provides users with intuitive operation, real-time information and personalized suggestions, resulting in a comfortable and safe driving experience.

[0458] The processing flow will be explained below.

[0459] Step 1:

[0460] The user speaks to the device, saying, "I want to go to a nice cafe nearby." The device receives this command and uses a speech recognition engine to convert the voice data into text data.

[0461] Step 2:

[0462] The device passes the converted text data to a natural language processing engine, which extracts keywords such as "cafe," "nearby," and "delicious."

[0463] Step 3:

[0464] The server sends a request to an external information source (e.g., a traffic information API) to obtain real-time updated traffic information, and stores the obtained traffic information in a database.

[0465] Step 4:

[0466] The device obtains a list of local cafes, along with their ratings and business information, from an external information service (e.g., a rating site API). Based on this information, the device creates a list of the cafes with the highest ratings.

[0467] Step 5:

[0468] The device calls the Google Maps API to calculate the optimal route based on the traffic and cafe information obtained in steps 3 and 4. If there is traffic congestion, it suggests a detour route.

[0469] Step 6:

[0470] The server stores the user's past behavioral data and preferences as a profile, and the device accesses this profile information to prioritize suggestions for cafes that the user prefers.

[0471] Step 7:

[0472] Users can request "play my favorite music." The device sends the request to the music distribution API and plays music that matches the user's preferences. It also offers entertainment features such as quizzes.

[0473] Step 8:

[0474] The device monitors vehicle sensor information in real time and provides alerts if speed, distance, or improper driving is detected.

[0475] Step 9:

[0476] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile, and the device uses this character to interact with the user.

[0477] Step 10:

[0478] Users can share their chatbot characters with friends by obtaining a sharing link from the server. Characters can also be sold on the marketplace. The server manages transactions and generates sharing links.

[0479] Example 1

[0480] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0481] Conventional car navigation systems have issues with not being able to intuitively indicate where users want to go or stop, and it is difficult to efficiently provide real-time updated traffic information, local ratings, and business information. Furthermore, they lack the means to provide entertainment and safe driving support, failing to fully realize a comfortable and safe driving experience for users. Furthermore, there is a lack of means to provide personalized route suggestions based on user preferences and natural dialogue.

[0482] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0483] In this invention, the server includes means for providing an interactive interface that allows a user to intuitively indicate places they want to go and points they want to stop at, means for acquiring traffic information from external information sources that is updated in real time and proposing optimal routes that avoid congestion, means for acquiring and providing area-specific ratings and business information from external information services, means for personalizing routes based on the user's past behavioral data and preferences, means for providing entertainment functions, means for supporting safe driving based on vehicle data, means for analyzing voice input from the user and performing natural language processing, means for storing information acquired from external databases in a local database and updating it as needed, and means for generating a customized character chatbot using generative artificial intelligence technology to enable natural dialogue with the user.This allows the user to intuitively specify their destination, receive real-time information and personalized suggestions, and provide enhanced entertainment and support for safe driving, resulting in a comfortable and safe driving experience.

[0484] An "interactive interface" is a user interface that allows users to intuitively specify places they want to go or points they want to stop at using voice or text.

[0485] An "external information source" is an external data providing system or service that provides traffic information, rating information, business information, and the like.

[0486] "Traffic information" refers to real-time information about road conditions such as congestion, accidents, and road closures.

[0487] "Reviews and business information" refers to detailed information about reviews and business operations of restaurants, tourist spots, etc.

[0488] "Personalization" refers to individualization based on a user's past behavioral data and preferences.

[0489] "Entertainment functions" are functions that provide entertainment to users, such as playing music and asking quizzes.

[0490] "Safe Driving Support" is a support function that warns of dangers while driving based on vehicle data and encourages safe driving.

[0491] "Natural language processing" is a technology that analyzes voice and text input from users and understands their intent.

[0492] A "local database" is a database for storing information obtained from outside and updating it as necessary.

[0493] "Generative AI technology" is an AI technology for generating customized character chatbots based on user profile information.

[0494] A "character chatbot" is a virtual character generated using generative artificial intelligence technology to enable natural conversations with users.

[0495] The present invention provides a car navigation system with an interactive interface that allows users to intuitively specify destinations and stops. This system supports users in comfortable and efficient driving, and also provides entertainment while traveling and assists in safe driving.

[0496] First, the user inputs instructions via voice or text through a conversational interface. This interface converts the voice into text using a speech recognition engine (e.g., Google Cloud Speech-to-Text). Next, a natural language processing engine (e.g., Google Cloud Natural Language API) is used to analyze the user's instructions and extract keywords. For example, if the user says, "I want to go to a delicious cafe nearby," the system extracts the keywords "nearby" and "delicious cafe."

[0497] The server periodically retrieves the latest traffic information from external sources (e.g., HERE API) and stores information on congestion, accidents, road closures, etc. in a database (e.g., MySQL). This information is updated in real time and pushed to the device as needed.

[0498] The device also obtains ratings and business information for local restaurants and tourist spots from external information services (e.g., Yelp API) and stores this information in a local database. Based on this stored information, it lists the best places for the user and displays them on the screen. Furthermore, the server manages the user's past behavioral data as profile information, and the device calculates and suggests the best route based on this information. For example, it can prioritize routes to cafes that have been highly rated in the past.

[0499] As an entertainment feature, the user can make requests to the device, such as "Play the next song." The device will then call a music distribution service (e.g., Spotify API) and play the specified song. The device can also select questions from a quiz database and pose them to the user, providing entertainment while driving.

[0500] As a safe driving function, the device collects and analyzes data such as speed and following distance from various sensors in the vehicle in real time, and displays an alert to the user if danger is predicted. For example, it displays a warning message such as "There is insufficient distance between vehicles ahead. Please be careful."

[0501] Furthermore, the server uses generative AI technology (e.g., OpenAI GPT-4) to generate a customized character chatbot based on the user's profile. Using this chatbot, the device can have natural conversations with the user. Users can obtain a dedicated sharing link to share the chatbot character they created with other users, or sell the chatbot character through a marketplace.

[0502] Examples of prompts include:

[0503] "Tell me a recommended cafe nearby."

[0504] "Turn right at the next traffic light"

[0505] "Tell me the latest traffic information"

[0506] "Show me suggestions for cafes I've been to in the past."

[0507] "Play some Mozart"

[0508] "Show an alert if the following distance is too close"

[0509] "Generate a chatbot character based on user preferences"

[0510] "Get a link to share your generated character with other users."

[0511] In this way, the present invention allows users to intuitively operate the device and enjoy real-time information and personalized suggestions, resulting in a comfortable and safe driving experience.

[0512] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0513] Step 1:

[0514] The user inputs instructions through a conversational interface using voice or text. For example, the user might input, "I want to go to a nearby delicious cafe." The input voice data is sent to a speech recognition engine (e.g., Google Cloud Speech-to-Text) and converted into text data.

[0515] Input: User instructions via voice or text

[0516] Data processing: Speech to text conversion using a voice recognition engine

[0517] Output: Textual user instructions

[0518] Step 2:

[0519] The device sends the text data converted by the speech recognition engine to a natural language processing engine (e.g., Google Cloud Natural Language API) for analysis. Keywords are extracted to understand the user's intent. For example, keywords such as "nearby" and "delicious cafe" are extracted.

[0520] Input: Textual user instructions

[0521] Data Calculation: Keyword Extraction Using Natural Language Processing Engine

[0522] Output: Extracted keywords

[0523] Step 3:

[0524] The server periodically sends requests to external sources (e.g., HERE API) to obtain the latest traffic information. The obtained data is stored in a database (e.g., MySQL) and updated in real time. Push notifications are sent to the device when necessary.

[0525] Input: Request to external source

[0526] Data processing: Acquisition of traffic information and storage in a database

[0527] Output: Saved real-time traffic information data

[0528] Step 4:

[0529] The device sends a request to an external information service (e.g., Yelp API) to obtain ratings and business information for restaurants and tourist attractions in the area specified by the user. This information is stored in a local database.

[0530] Input: A request to an external information service

[0531] Data processing: Acquisition of evaluation information and storage in a local database

[0532] Output: Saved evaluation and sales information data

[0533] Step 5:

[0534] The device selects and lists the best places for the user based on the stored rating and business information data and real-time traffic information data. It then displays the list on the screen. For example, it presents the user with a list of "delicious cafes nearby."

[0535] Input: Rating and business information data, traffic information data

[0536] Data calculation: listing and optimized location suggestions

[0537] Output: A list of the best places listed by the user

[0538] Step 6:

[0539] The server manages the user's past behavior data (e.g., places rated in the past) as profile information. The device references this profile information and calculates and proposes the optimal route based on the user's past behavior. For example, it may prioritize routes to cafes that have been highly rated in the past.

[0540] Input: User's past behavior data

[0541] Data calculation: Referencing profile information and calculating routes

[0542] Output: Optimal route suggested to the user

[0543] Step 7:

[0544] The user inputs an entertainment request, such as "Play the next song," into the device. The device then calls a music distribution service (e.g., Spotify API) and plays the specified song. It also selects questions from a quiz database and presents them to the user in the form of a quiz.

[0545] Input: User entertainment request

[0546] Data calculation: Calling music distribution services and quiz questions

[0547] Output: Music playback and quiz

[0548] Step 8:

[0549] The device acquires and analyzes the vehicle's sensor data (e.g., speed and following distance) in real time. If there is a problem with safe driving, an alert will be displayed to the user. For example, if there is insufficient distance between the vehicle ahead, the device will display the message "Insufficient distance between the vehicle ahead. Please be careful."

[0550] Input: Vehicle sensor data

[0551] Data Calculation: Real-time analysis and safe driving alerts

[0552] Output: Alert the user

[0553] Step 9:

[0554] The server uses generative AI technology (e.g., OpenAI GPT-4) to generate a customized character chatbot based on the user's profile information. The generated chatbot is sent to the device and engages in natural conversations with the user.

[0555] Input: User profile information

[0556] Data Computation: Generating Character Chatbots with Generative AI Technology

[0557] Output: A chatbot that enables natural dialogue with users

[0558] Step 10:

[0559] Users can share their chatbot characters with other users by obtaining a dedicated sharing link from the server. Chatbot characters can also be sold through the marketplace.

[0560] Input: Generated chatbot character

[0561] Data processing: generating sharing links and selling on the marketplace

[0562] Output: Shared link and sales method

[0563] (Application example 1)

[0564] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0565] Today's drivers demand intuitive and efficient navigation systems. Especially important are interfaces that can be easily operated via voice, and the ability to obtain real-time traffic information and suggest optimal routes. However, few systems can simultaneously provide entertainment and safe driving support in addition to these functions. They also demand personalized suggestions based on individual user preferences and behavioral history. Furthermore, they also want chatbots that use generative AI technology to make the interface more user-friendly.

[0566] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0567] In this invention, the server includes: means for providing an interactive interface that allows a user to intuitively indicate places the user wants to go and points the user wants to stop at; means for acquiring traffic information from external information sources that is updated in real time and proposing optimal routes that avoid congestion; means for acquiring and providing area-specific ratings and business information from external information services; means for personalizing routes based on the user's past behavioral data and preferences; means for providing entertainment functions; means for supporting safe driving based on vehicle data; means for analyzing the user's voice commands using voice recognition technology and understanding the user's intentions using a natural language processing engine; means for displaying information in real time on the smart glasses; and means for generating an original character chatbot using generative artificial intelligence technology and having the chatbot interact with the user through the smart glasses. This allows the driver to intuitively operate the system and enjoy a comfortable and safe driving experience that combines real-time information with personalized suggestions.

[0568] - "User" means a person who uses a navigation system to reach a destination.

[0569] An "interactive interface" is a system that accepts and responds to instructions from the user through voice or text.

[0570] "External information sources" refer to external services or APIs that provide data such as traffic information and store information.

[0571] "Traffic information" refers to data on traffic conditions, including information on congestion, accidents, road closures, etc.

[0572] "Ratings and business information" is data showing user ratings and business status of stores and tourist attractions in the area.

[0573] "Personalization" refers to individual optimization based on a user's past behavioral data and preferences.

[0574] "Entertainment features" are features that allow users to enjoy themselves, such as music playback, quizzes, and chat.

[0575] "Vehicle data" is information about the vehicle's operation, such as speed and following distance.

[0576] "Voice recognition technology" is a technology that analyzes the user's voice and converts it into text data.

[0577] A "natural language processing engine" is a technology that understands the user's intent from text data and generates an appropriate response.

[0578] "Smart glasses" are glasses-type devices that have a built-in display and provide visual information when worn by the user.

[0579] "Generative artificial intelligence technology" is a technology that uses AI to generate new characters and content.

[0580] An "original character chatbot" is a conversational character created based on a user's profile using generative AI technology.

[0581] The present invention provides an interactive navigation system that allows users to intuitively specify where they want to go, provides real-time traffic information, and realizes a safe and comfortable driving experience. Specific embodiments for implementing this system are described below.

[0582] 1. User Interface Design

[0583] Users input commands by voice through a conversational interface, such as "I want to go to a nice cafe nearby." This voice command is converted into text data by the device's voice recognition technology (e.g., speech_recognition library), and then analyzed by a natural language processing engine (e.g., IntentRecognizer) to understand the user's intent.

[0584] 2. Obtaining real-time traffic information

[0585] The server periodically retrieves the latest traffic information from external sources (e.g., traffic information APIs), including traffic congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time. The device displays this information on the smart glasses display and suggests optimal routes for the user.

[0586] 3. Obtaining evaluations and sales information

[0587] The device obtains ratings and business information for restaurants and tourist spots in the area from external information services (e.g., rating site APIs), and based on this, it creates a list of places that best suit the user's needs.

[0588] 4. Personalized Route Suggestions

[0589] The server manages the user's past behavioral data and preferences as profile information. Based on this, the device suggests an optimized route to the user. By prioritizing cafes that have been highly rated in the past, the device can provide the user with a more satisfying experience.

[0590] 5. Providing entertainment features

[0591] Users can make requests to their device, such as "play the next song," to enjoy music playback, quizzes, and chat functions. The device then calls an external music service (e.g., music distribution API) to play music. It also generates quizzes from a quiz database, providing entertainment while driving.

[0592] 6. Safe driving support

[0593] The device monitors vehicle data (e.g., speed, distance between vehicles) in real time and displays alerts to encourage safe driving. For example, if the distance between vehicles is too small, a message such as "There is insufficient distance between vehicles ahead. Please be careful" will be displayed.

[0594] 7. Character Chatbot Generation and Interaction

[0595] The server uses generative AI technology to generate an original character chatbot based on the user's profile, which then interacts with the user through smart glasses, enabling natural communication.

[0596] 8. Sharing and selling characters

[0597] Users can share their chatbot characters with other users by obtaining a dedicated sharing link from the server, and can also sell their chatbot characters through the marketplace.

[0598] Examples and prompts

[0599] As a concrete example, the system functions can be realized by using the following prompt sentences:

[0600] Example prompt:

[0601] User: "I want to go to a recommended cafe nearby."

[0602] AI system: "Activate the speech recognition engine to understand the user's intent."

[0603] AI system: "Get information about nearby cafes from an external API."

[0604] AI system: "Recommended cafes will be displayed on the screen."

[0605] User: "Okay, let's go to the second cafe."

[0606] AI system: "Route guidance in progress. Turn right."

[0607] User: "Play some music."

[0608] AI system: "Playing your favorite song"

[0609] This allows users to enjoy a comfortable driving experience that is intuitive to operate and combines real-time information with personalized suggestions.

[0610] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0611] Step 1:

[0612] The user inputs instructions by voice, such as "I want to go to a nice cafe nearby." This voice input is captured through the device's microphone.

[0613] Step 2:

[0614] The device uses speech recognition technology (e.g., the speech_recognition library) to convert speech to text data. The input of this step is speech data, and the output is text data.

[0615] Step 3:

[0616] The device uses a natural language processing engine (e.g., IntentRecognizer) to analyze the user's intent from the text data. The input is text data, and the output is data indicating the user's intent (e.g., the intent to "find a cafe").

[0617] Step 4:

[0618] The device sends the user's intention to the server and queries the relevant local cafe information. The input is the user's intention data, and the output is the cafe information obtained through the external API.

[0619] Step 5:

[0620] The server obtains ratings and business information for restaurants and tourist attractions from external information sources (e.g., rating site APIs). This includes region-specific rating data and business information. The input is data indicating the region, and the output is the ratings and business information.

[0621] Step 6:

[0622] The server filters relevant information based on the user's past behavioral data and preferences to generate a personalized cafe list. The input is the user's past behavioral data and externally acquired ratings and sales information, and the output is a personalized list of cafes.

[0623] Step 7:

[0624] The terminal displays the cafe list obtained from the server on the display of the smart glasses. The input is the cafe list received from the server, and the output is the information displayed on the display of the smart glasses.

[0625] Step 8:

[0626] The user selects a cafe and then gives a voice command, such as "Let's go to the second cafe," which is also captured through the smart glasses.

[0627] Step 9:

[0628] The device again uses voice recognition technology and a natural language processing engine to analyze the user's new command: the input is voice data, and the output is the intent data: "Go to the second cafe."

[0629] Step 10:

[0630] The device retrieves optimal route information from the server and displays route guidance on the smart glasses. The input is the user's intention, current location, and route information received from the server, and the output is the display of route guidance.

[0631] Step 11:

[0632] The user types in a voice command requesting "play music," which is also captured through the smart glasses.

[0633] Step 12:

[0634] The device uses voice recognition technology and a natural language processing engine to analyze the music request and then calls an external music service (e.g., a music distribution API) to play the music. The input is audio data, and the output is the music being played.

[0635] Step 13:

[0636] The terminal monitors vehicle data (e.g., speed, distance between vehicles) in real time and displays safe driving alerts on the smart glasses. The input is vehicle data, and the output is the display of safe driving alerts.

[0637] Step 14:

[0638] The server uses generative AI technology to generate an original character chatbot based on the user's profile, and the device displays this character on the smart glasses and allows the user to interact with it. The input is the user profile data, and the output is the interaction with the generated character.

[0639] Step 15:

[0640] To share the chatbot character that a user has created with other users, the user obtains a dedicated sharing link from the server. The input is the generated character data, and the output is the sharing link.

[0641] This allows users to enjoy a comfortable driving experience that combines real-time information with personalized suggestions, with more intuitive operation than usual.

[0642] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0643] The present invention provides a car navigation system that has an interactive interface that allows users to intuitively indicate places they want to go or points they want to stop at, and that also combines an emotion engine that recognizes the user's emotions. This system allows the user to have a more personalized experience and realize a more comfortable driving experience. Specific embodiments of the present invention are described below.

[0644] 1. User Interface Design

[0645] Users can input instructions through a conversational interface using voice or text. For example, they can say, "I want to go to a nice cafe nearby." The device then uses a speech recognition engine to convert the voice data into text.

[0646] 2. Introducing the Emotion Engine

[0647] The device implements an emotion engine that analyzes emotions from the user's voice and text input. The emotion engine recognizes the user's emotional state (e.g., joy, anger, sadness, or happiness) from the user's tone of voice and text expressions, and saves the analysis results.

[0648] 3. Obtaining real-time traffic information

[0649] The server periodically retrieves the latest traffic information from external sources (e.g., traffic information API). This information includes congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time.

[0650] 4. Obtaining ratings and business information for restaurants and tourist spots

[0651] The device retrieves a list of cafes in the specified area, along with their ratings and business information, from an external information service (e.g., a rating site API). Based on this information, the device creates a list of the cafes with the highest ratings.

[0652] 5. Personalized Route Suggestions

[0653] The server manages the user's past behavioral data and preferences as profile information. The device accesses this profile information and suggests optimized routes to the user. For example, it may prioritize cafes that users have given high ratings to in the past or places based on their current emotional state.

[0654] 6. Provision of entertainment features

[0655] Users can enjoy entertainment such as music playback, quizzes, and chat functions. The device receives requests such as "play the next song" and calls an external music service (e.g., music distribution API) to play music. In addition, an emotion engine analyzes the user's emotions and adjusts appropriate entertainment content.

[0656] 7. Safe driving support

[0657] The device monitors vehicle sensor information in real time and provides alerts when speed, distance, or improper driving is detected. In addition, an emotion engine adjusts the content and timing of alerts based on the user's emotional state.

[0658] 8. Creating a Character Chatbot

[0659] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile. The device then uses this character to engage in natural conversations with the user. Using generative AI technology, the character's expressions also change dynamically to reflect the user's emotional state.

[0660] 9. Sharing and selling characters

[0661] Users can obtain a dedicated sharing link from the server to share the chatbot characters they have created with other users. They can also sell chatbot characters through the marketplace. The server provides this functionality through the network.

[0662] Specific examples

[0663] As a concrete example, consider the case where a user verbally commands, "I want to go to a nearby, relaxing cafe." The device converts this command into text using a speech recognition engine and analyzes it using a natural language processing engine. Next, the emotion engine analyzes the user's tone of voice and recognizes that the user is a little tired. The device obtains cafe information from an external information service, and the server uses past behavioral data and emotion information to create a list of highly rated cafes that are relaxing. The device then suggests the user the optimal route to that cafe. While driving, the device plays relaxing music that matches the user's preferences. The device also obtains safety data from vehicle sensors and works with the emotion engine to provide the user with safety alerts at the appropriate time.

[0664] In this way, the present invention makes it possible to realize a system that allows the user to enjoy a more comfortable and safe driving experience.

[0665] The processing flow will be explained below.

[0666] Step 1:

[0667] The user speaks to the device, saying, "I want to go to a quiet cafe nearby." The device receives this instruction and uses a speech recognition engine to convert the voice data into text data.

[0668] Step 2:

[0669] The device passes the converted text data to a natural language processing engine, which extracts keywords such as "cafe," "nearby," and "calm."

[0670] Step 3:

[0671] The device uses an emotion engine to analyze the user's emotional state from their voice and determines that they are "tired." The emotion engine then stores this information in their profile.

[0672] Step 4:

[0673] The server sends a request to an external information source (e.g., a traffic information API) to obtain the latest traffic information, and stores the obtained traffic information in a database.

[0674] Step 5:

[0675] The terminal obtains a list of cafes in a specified area, along with their ratings and business information, from an external information service (e.g., a rating site API).

[0676] Step 6:

[0677] The device will use the cafe information obtained in step 5 to create a list of cafes that are highly rated and have a relaxing atmosphere.

[0678] Step 7:

[0679] The device uses the Google Maps API to calculate the optimal route based on the traffic information obtained in step 4 and the cafe information listed in step 6.

[0680] Step 8:

[0681] Based on the user's past behavioral data and the emotional profile obtained in step 3, the server selects the cafe that best suits the user's preferences and current emotional state, and then personalizes and suggests a route to the cafe.

[0682] Step 9:

[0683] The user checks the suggested cafe and route on the device and confirms, "I want to go there."

[0684] Step 10:

[0685] The user requests, "Play some relaxing music." The device sends a request to the music distribution API and plays relaxing music.

[0686] Step 11:

[0687] While driving, the device monitors vehicle sensor information to check speed and following distance, and provides alerts if improper driving is detected.

[0688] Step 12:

[0689] The device uses an emotion engine to monitor the user's emotional state and adjust the entertainment content and timing of safe driving alerts as needed.

[0690] Step 13:

[0691] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile, and the device uses this character to interact with the user.

[0692] Step 14:

[0693] Users can share their chatbot characters with friends by obtaining a sharing link from the server. Characters can also be sold on the marketplace. The server manages transactions and generates sharing links.

[0694] In this way, users can operate intuitively and enjoy real-time information and personalized suggestions based on their emotional state, ensuring a comfortable and safe driving experience.

[0695] Example 2

[0696] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0697] Conventional car navigation systems simply provide routes to a destination and lack the ability to provide a personalized experience based on the user's emotional state and individual preferences. Furthermore, entertainment and safe driving support functions are limited, creating a need for an integrated solution to improve the overall driving experience. Furthermore, providing interactive character chatbots using generative AI technology and creating new user experiences based on them are also challenges.

[0698] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0699] In this invention, the server includes means for providing an interactive interface that allows the user to intuitively indicate places the user wants to go and points the user wants to stop at, means for acquiring traffic information from external information sources that is updated in real time and proposing optimal routes that avoid congestion, means for acquiring and providing area-specific ratings and business information from external information services, means for personalizing routes based on the user's past behavioral data and preferences, means for analyzing the user's emotional state from voice and text input, means for adjusting and providing entertainment functions according to the emotional state, means for supporting safe driving based on vehicle data and adjusting the content and timing of alerts based on the emotional state, and means for generating an original character chatbot based on the user's profile and dynamically changing and interacting with the chatbot. This enables optimal route suggestions, entertainment, and safe driving support based on the user's emotions and preferences, providing a more personalized and comfortable driving experience.

[0700] An "interactive interface" is an interface that allows a user to intuitively input instructions by voice or text, and the system returns an appropriate response to that input.

[0701] "External information sources" refer to information services and APIs that provide traffic information, traffic accident information, traffic congestion information, etc. from outside the system.

[0702] "Traffic Information" is real-time information about current road conditions, traffic congestion, traffic accidents, road closures, etc.

[0703] "Region-specific ratings" refers to user ratings and word-of-mouth information about places and facilities in a specific region.

[0704] "Business information" refers to information such as opening hours, regular holidays, and special opening days of restaurants, tourist attractions, etc.

[0705] "External information services" refer to services and APIs that provide evaluations and sales information.

[0706] "Personalizing your route" means suggesting the best route based on the user's past behavioral data and individual preferences.

[0707] "Emotional state" refers to the emotions such as joy, anger, sadness, and happiness that the user is currently feeling, and is analyzed from voice and text input.

[0708] "Entertainment features" are features that users can enjoy while driving, such as music playback, quizzes, and chat functions.

[0709] "Safe Driving Support" is a function that uses information from the vehicle's sensors to detect driving speed, vehicle distance, and inappropriate driving, and provides alerts to the user.

[0710] An "alert" is a notification that the system uses to warn or alert the user to encourage safe driving.

[0711] "Generative AI technology" is a technology that uses artificial intelligence to generate new content and characters.

[0712] "Profile Information" refers to information about a user, such as the user's past behavioral data, preferences, and emotional state.

[0713] A "character chatbot" is a virtual character created using generative AI technology that can engage in natural conversations with users.

[0714] "Dynamic change" means that the chatbot's behavior and responses change in real time depending on the user's emotional state and situation.

[0715] "Marketplace" means an online platform for sharing or selling user-generated chatbot characters.

[0716] This invention is a car navigation system that features an interactive interface that allows users to intuitively indicate where they want to go and where they want to stop, and provides real-time traffic information, emotion analysis, personalized route suggestions, entertainment functions, safe driving support, and a character chatbot that utilizes generative AI technology.

[0717] User Interface Design

[0718] Users input instructions to the system through voice or text. For example, if a user says, "I want to go to a nice cafe nearby," the device uses a speech recognition engine (e.g., Google Speech-to-Text) to convert the voice into text data. Then, a natural language processing engine (e.g., OpenAI GPT) analyzes this text data.

[0719] Introducing the Emotion Engine

[0720] The device implements an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the user's emotional state from voice and text. This engine analyzes the user's emotional state (e.g., joy, anger, sadness, or happiness) from the user's tone of voice and text expressions and saves the results.

[0721] Obtaining real-time traffic information

[0722] The server periodically retrieves the latest traffic information from external sources (e.g., Traffic API), including information on congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time.

[0723] Obtaining ratings and business information for restaurants and tourist spots

[0724] The device uses external information services (e.g., Yelp API) that provide location-specific ratings and business information to obtain a list of cafes and their ratings and business information, allowing it to present the location that best suits the user's request.

[0725] Personalized route suggestions

[0726] The server manages the user's past behavioral data and preferences as profile information. The device then uses this profile information to suggest the most suitable route for the user. Specifically, it prioritizes cafes that users have given high ratings to in the past and places that correspond to their current emotional state.

[0727] Providing entertainment features

[0728] Users can enjoy entertainment such as music playback, quizzes, and chat functions. The device receives requests such as "play the next song" and calls an external music service (e.g., Spotify API) to play music. In addition, an emotion engine analyzes the user's emotions and adjusts appropriate entertainment content.

[0729] Safe driving support

[0730] The device monitors vehicle sensor information in real time and provides alerts when speed, distance, or improper driving is detected. The emotion engine adjusts the content and timing of alerts based on the user's emotional state.

[0731] Character chatbot generation

[0732] The server uses generative AI technology (e.g., OpenAI GPT) to generate a chatbot with an original character based on the user's profile. The device then uses this character to engage in natural conversations with the user. Generative AI technology dynamically changes the character's expressions according to the user's emotional state.

[0733] Sharing and selling characters

[0734] Users can obtain a dedicated sharing link from the server to share their chatbot characters with other users, and can also sell chatbot characters through the marketplace.

[0735] Specific examples

[0736] Consider a scenario where a user verbally commands, "I want to go to a nearby relaxing cafe." The device converts this command into text using a speech recognition engine (e.g., Google Speech-to-Text) and analyzes it using a natural language processing engine (e.g., OpenAI GPT). Next, an emotion engine (e.g., IBM Watson Tone Analyzer) analyzes the user's tone of voice and recognizes that the user is slightly tired. The device obtains cafe information from an external information service (e.g., Yelp API), and the server uses past behavioral data and emotion information to create a list of highly rated cafes that are relaxing. The device then suggests the user the optimal route to the cafe. While driving, the device plays relaxing music using an external music service (e.g., Spotify API). The device also obtains safety data from vehicle sensors and, in cooperation with the emotion engine, provides the user with safety alerts at the appropriate time.

[0737] Prompt Sentence Examples

[0738] "If a user says, 'I want to go to a quiet cafe nearby,' what kind of processing and data is required?"

[0739] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0740] Step 1:

[0741] The user launches the device's conversational interface. The user inputs a command by voice or text, such as "I want to go to a nearby delicious cafe." In this case, the input is voice data, which is passed to the device. Specifically, the user performs input operations using a microphone or keyboard.

[0742] Step 2:

[0743] The device uses a speech recognition engine (e.g., a speech recognition engine API) to convert the user's voice data into text data. The output is text data, which is used as input data for the next processing step. The specific operation involves the process in which the speech recognition engine analyzes the voice waveform and converts it into corresponding text.

[0744] Step 3:

[0745] The device passes the converted text data to a natural language processing engine (e.g., natural language processing API) and analyzes the user's instructions. The input is text data, and the output is the intent data resulting from the analysis. For example, the intent extracted is "I want to go to a nearby delicious cafe." Specifically, the natural language processing engine understands the context of the text and concludes on the user's request.

[0746] Step 4:

[0747] The device uses an emotion engine (e.g., emotion analysis API) to analyze the user's emotional state from text data and voice tone. The input is text data and voice tone data, and the output is emotional state data. For example, it may recognize that the user is slightly tired. Specifically, the emotion engine analyzes the intonation of the voice and the tone of the text to determine the emotional state.

[0748] Step 5:

[0749] The server periodically obtains the latest traffic information from an external information source (e.g., a traffic information API). The input is an API request, and the output is traffic information data. Specifically, the server sends an API request and receives real-time data such as congestion and accident information.

[0750] Step 6:

[0751] The device obtains cafe ratings and business information from an external information service (e.g., a rating site API). The input is an API request, and the output is a list of cafes and their rating data. Specific operations include sending an API request, parsing the results, and generating a list of cafes.

[0752] Step 7:

[0753] The device accesses the user's profile information (e.g., past behavioral data, preferences) stored on the server and proposes the optimal route. The input is the profile data, and the output is optimal route guidance data. Specifically, the server evaluates candidate locations based on the past behavioral data and the user's preferences, and generates the optimal route.

[0754] Step 8:

[0755] When a user requests "play the next song," the device calls an external music service (e.g., a music distribution API) to play music. The input is the user's voice request, and the output is playback instruction data. Specifically, the device sends an API request to the music distribution service and streams the music data.

[0756] Step 9:

[0757] The device monitors vehicle sensor information in real time and provides alerts to support safe driving. The input is sensor information and the output is alert data. Specific operations include analyzing sensor information to evaluate speed and vehicle distance and detect inappropriate driving.

[0758] Step 10:

[0759] The server uses generative AI technology (e.g., generative AI model) to generate an original character chatbot based on the user's profile. The input is profile information, and the output is chatbot data. Specifically, the server uses generative AI technology to simulate the character's dialogue.

[0760] Step 11:

[0761] The server generates a dedicated sharing link so that users can share their chatbot characters with other users. The input is the chatbot data, and the output is the sharing link. Specifically, the server registers the chatbot data on the sharing platform and generates the link.

[0762] (Application example 2)

[0763] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0764] Conventional car navigation systems only provide simple route guidance without considering the user's emotions or entertainment needs. This makes it difficult to provide a comfortable and personalized driving experience for users. Furthermore, their safe driving support features are limited, and they do not provide appropriate alerts or entertainment tailored to the user's emotional state. This makes it difficult to reduce user stress and discomfort, and furthermore, ensures safe driving. The present invention aims to solve these problems of the conventional technology and provide users with a personalized, comfortable driving experience and safe driving support.

[0765] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for analyzing the user's emotional state and proposing entertainment content and routes based on the emotion, a means for monitoring the vehicle's sensor information in real time and adjusting the content and timing of alerts based on the user's emotional state, and a means for acquiring and proposing area-specific reviews and business information from external information services. This enables personalized entertainment and route suggestions that take the user's emotional state into consideration. Furthermore, safety alerts can be provided at appropriate times according to the user's emotional state, resulting in a comfortable and safe driving experience.

[0766] An "interactive interface" is an input method that allows users to intuitively specify their destination or points of interest using voice or text.

[0767] "External information sources updated in real time" refers to information provision services that provide traffic information, rating information, etc. with the latest data over time.

[0768] "Means of obtaining traffic information and proposing the optimal route to avoid congestion" refers to a function that imports traffic data in real time from external traffic information services and, based on that data, presents users with the optimal route to avoid congestion.

[0769] "Means for obtaining and providing region-specific ratings and business information from external information services" refers to a function that obtains ratings and business information on restaurants, tourist attractions, etc. related to a specified region from external information services and provides it to users.

[0770] "Means to personalize routes based on the user's past behavioral data and preferences" is a function that suggests individualized routes based on information such as the user's past travel history, favorite categories, and preferred locations.

[0771] "Means for providing entertainment functions" refers to functions that provide users with entertainment content such as music, quizzes, and chat.

[0772] "Means to support safe driving based on vehicle data" refers to a function that provides users with alerts and notifications to encourage safe driving based on data obtained from sensors, cameras, etc. installed in the vehicle.

[0773] "Means for analyzing the user's emotional state and suggesting entertainment content and routes based on that emotion" refers to a function that analyzes the user's emotional state (e.g., happy, tired) from voice and text data, and personalizes entertainment content and routes based on the analysis results.

[0774] "Means for monitoring vehicle sensor information in real time and adjusting the content and timing of alerts based on the user's emotional state" refers to a function that monitors data obtained in real time from various sensors installed in the vehicle and optimizes the content and timing of alerts according to the user's emotional state.

[0775] The system of the present invention includes an interactive interface that allows users to intuitively indicate where they want to go and where they want to stop, routes that are suggested based on traffic information obtained from external sources that is updated in real time, personalized entertainment content and route suggestions based on the user's emotional state, and functions that support safe driving.

[0776] 1. User Interface Design

[0777] A conversational interface uses a speech recognition engine (e.g., Google Speech Recognition API) to convert the user's speech into text. When a user speaks a command such as "I want to go to a nice cafe nearby," the speech data is converted into text by the recognition engine.

[0778] 2. Introducing the Emotion Engine

[0779] Using an emotion engine (e.g., EmotionRecognizer), the emotional state of the user is analyzed from voice and text, allowing accurate understanding of the user's emotional state, such as whether they are tired or happy.

[0780] 3. Obtaining real-time traffic information

[0781] The server periodically obtains the latest traffic information using a traffic information API (e.g., a general traffic information API), thereby obtaining real-time information on traffic congestion and accidents and storing it in a database.

[0782] 4. Obtaining ratings and business information for restaurants and tourist spots

[0783] The device retrieves a list of cafes in the specified area, along with their ratings and business information, from an external information service (e.g., a rating site API). Based on this information, the device creates a list of the cafes with the highest ratings.

[0784] 5. Personalized Route Suggestions

[0785] The server manages the user's past behavioral data and preferences (favorite restaurant trends and records of places visited) as profile information. The device then suggests the optimal route based on this profile information and the results of the emotion engine's analysis.

[0786] 6. Provision of entertainment features

[0787] The device provides features such as music playback (using external music service APIs), quizzes, and chat functions. The emotion engine analyzes the user's emotions and dynamically adjusts the entertainment content to suit their needs.

[0788] 7. Safe driving support

[0789] The device monitors vehicle sensor information in real time to support safe driving, and improves safety by adjusting the content and timing of safety alerts based on the user's emotional state.

[0790] 8. Creating a Character Chatbot

[0791] The server uses generative AI technology (e.g., generative AI model) to generate a chatbot with an original character based on the user's profile. The device then uses this character to engage in natural conversations with the user.

[0792] 9. Sharing and selling characters

[0793] Users can obtain a dedicated sharing link from the server to share their chatbot characters with other users, and can also sell their characters through the marketplace.

[0794] Specific examples

[0795] For example, if a user says, "I want to go to a nearby relaxing cafe," the device will convert this into text using a speech recognition engine and analyze the user's emotional state using an emotion engine. If the device detects that the user is tired, it will retrieve cafe information from a rating site API and create a list of highly rated cafes that are relaxing and calming based on past behavioral data and emotional information. It will then suggest the optimal route to the cafe to the user and play relaxing music. It will also retrieve safety data from vehicle sensors and link with the emotion engine to provide safety alerts at the appropriate time.

[0796] Prompt Sentence Examples

[0797] "Based on the invention below, you will design an emotional car navigation app for self-driving cars. The app should include a function that allows the user to specify a destination by voice, and the emotion engine analyzes the user's emotions, suggests the optimal route, and provides entertainment."

[0798] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0799] Step 1:

[0800] The user inputs instructions by voice. The user may say, "I want to go to a nice cafe nearby." This voice input becomes the input to the system.

[0801] Step 2:

[0802] The device uses a speech recognition engine (e.g., Google Speech Recognition API) to convert the voice data into text data. The speech recognition engine analyzes the voice data and outputs it as text data.

[0803] Step 3:

[0804] The device uses an emotion engine (e.g., EmotionRecognizer) to analyze the user's emotional state from text and voice data, and outputs the emotional state (e.g., tired, happy) obtained through this analysis.

[0805] Step 4:

[0806] The server uses a traffic information API (e.g., a general traffic information API) to obtain the latest traffic information in real time. The obtained traffic information is stored in a database and becomes input data for the system.

[0807] Step 5:

[0808] The device uses an external information service (e.g., a rating site API) to obtain a list of cafes in the specified area, along with their ratings and business information. This data is input, and a list of highly rated cafes is output.

[0809] Step 6:

[0810] The server retrieves the user's past behavioral data and preferences (e.g., a list of cafes visited and their ratings) from the database as profile information. The retrieved profile information becomes input data for the system.

[0811] Step 7:

[0812] The device then proposes the optimal route based on the acquired profile information, the results of the emotion engine analysis, and traffic and cafe information. The device analyzes this data and outputs a personalized route for the user.

[0813] Step 8:

[0814] The device provides entertainment functions. Based on the analysis results of the emotion engine, it dynamically adjusts and provides music, quizzes, and chat functions that suit the user. This function outputs entertainment content optimized for the user's emotional state.

[0815] Step 9:

[0816] The device monitors the vehicle's sensor information in real time and adjusts the content and timing of safety alerts according to the user's emotional state. Alerts are output at appropriate times based on the sensor information and the user's emotional state.

[0817] Step 10:

[0818] The server uses generative AI technology (e.g., generative AI model) to generate a chatbot with an original character based on the user's profile. The generated character is used as an output to the system to interact with the user.

[0819] Step 11:

[0820] Users can share the generated character chatbots with other users or sell them on the marketplace. The server generates a sharing link and provides a sales function in cooperation with the marketplace. This function is provided as an output to the user.

[0821] Adding specific examples

[0822] For example, if a user says, "I want to go to a nearby relaxing cafe," the system converts the speech to text and then uses its emotion engine to interpret it as "tired." It then retrieves cafe information from a review site API and, taking into account the user's past review data, creates a list of the most suitable cafes and suggests them to the user along with a route. It also plays relaxing music and adjusts safety alerts while driving based on the user's emotional state.

[0823] Prompt Sentence Examples

[0824] "Based on the invention below, you will design an emotional car navigation app for self-driving cars. The app should include a function that allows the user to specify a destination by voice, and the emotion engine analyzes the user's emotions, suggests the optimal route, and provides entertainment."

[0825] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0826] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0827] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0828] [Third embodiment]

[0829] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0830] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0831] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0832] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0833] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0834] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0835] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0836] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0837] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0838] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0839] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0840] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0841] The present invention proposes a car navigation system with an interactive interface that allows users to intuitively specify where they want to go and where they want to stop. This not only realizes the comfortable and efficient driving that modern drivers desire, but also provides entertainment while traveling and support for safe driving. Specific embodiments of the present invention are described below.

[0842] 1. User Interface Design

[0843] Users can input instructions through a conversational interface using voice or text. For example, they can say, "I want to go to a nice cafe nearby." The device then analyzes this with a natural language processing engine and extracts keywords to understand the user's intent.

[0844] 2. Obtaining real-time traffic information

[0845] The server periodically retrieves the latest traffic information from external sources (e.g., traffic information API). This information includes congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time.

[0846] 3. Obtaining ratings and business information for restaurants and tourist spots

[0847] The device retrieves ratings and business information for restaurants and tourist spots in the specified area from external information services (e.g., rating site APIs). Based on this data, it creates a list of places that best meet the user's requirements.

[0848] 4. Personalized Route Suggestions

[0849] The server manages the user's past behavioral data and preferences as profile information, and based on this, the device suggests optimized routes to the user, for example, prioritizing cafes that have been highly rated in the past.

[0850] 5. Providing entertainment features

[0851] Users can make requests to their device, such as "play the next song," to enjoy entertainment such as music playback, quizzes, and chat functions. The device then plays music by calling an external music service (e.g., music distribution API). It also generates quizzes from a quiz database, providing entertainment while driving.

[0852] 6. Safe driving support

[0853] The device monitors vehicle data (e.g., speed, distance between vehicles) in real time and displays alerts to encourage safe driving. For example, if the distance between vehicles is too small, a message such as "There is insufficient distance between vehicles ahead. Please be careful" will be displayed.

[0854] 7. Creating a Character Chatbot

[0855] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile. This character is personalized based on the user's preferences and interests. The device then uses the generated character to engage in natural conversations with the user.

[0856] 8. Sharing and selling characters

[0857] Users can obtain a dedicated sharing link from the server to share the chatbot characters they have created with other users. They can also sell chatbot characters through the marketplace. The server provides this functionality through the network.

[0858] In this way, the present invention provides users with intuitive operation, real-time information and personalized suggestions, resulting in a comfortable and safe driving experience.

[0859] The processing flow will be explained below.

[0860] Step 1:

[0861] The user speaks to the device, saying, "I want to go to a nice cafe nearby." The device receives this command and uses a speech recognition engine to convert the voice data into text data.

[0862] Step 2:

[0863] The device passes the converted text data to a natural language processing engine, which extracts keywords such as "cafe," "nearby," and "delicious."

[0864] Step 3:

[0865] The server sends a request to an external information source (e.g., a traffic information API) to obtain real-time updated traffic information, and stores the obtained traffic information in a database.

[0866] Step 4:

[0867] The device obtains a list of local cafes, along with their ratings and business information, from an external information service (e.g., a rating site API). Based on this information, the device creates a list of the cafes with the highest ratings.

[0868] Step 5:

[0869] The device calls the Google Maps API to calculate the optimal route based on the traffic and cafe information obtained in steps 3 and 4. If there is traffic congestion, it suggests a detour route.

[0870] Step 6:

[0871] The server stores the user's past behavioral data and preferences as a profile, and the device accesses this profile information to prioritize suggestions for cafes that the user prefers.

[0872] Step 7:

[0873] Users can request "play my favorite music." The device sends the request to the music distribution API and plays music that matches the user's preferences. It also offers entertainment features such as quizzes.

[0874] Step 8:

[0875] The device monitors vehicle sensor information in real time and provides alerts if speed, distance, or improper driving is detected.

[0876] Step 9:

[0877] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile, and the device uses this character to interact with the user.

[0878] Step 10:

[0879] Users can share their chatbot characters with friends by obtaining a sharing link from the server. Characters can also be sold on the marketplace. The server manages transactions and generates sharing links.

[0880] Example 1

[0881] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0882] Conventional car navigation systems have issues with not being able to intuitively indicate where users want to go or stop, and it is difficult to efficiently provide real-time updated traffic information, local ratings, and business information. Furthermore, they lack the means to provide entertainment and safe driving support, failing to fully realize a comfortable and safe driving experience for users. Furthermore, there is a lack of means to provide personalized route suggestions based on user preferences and natural dialogue.

[0883] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0884] In this invention, the server includes means for providing an interactive interface that allows a user to intuitively indicate places they want to go and points they want to stop at, means for acquiring traffic information from external information sources that is updated in real time and proposing optimal routes that avoid congestion, means for acquiring and providing area-specific ratings and business information from external information services, means for personalizing routes based on the user's past behavioral data and preferences, means for providing entertainment functions, means for supporting safe driving based on vehicle data, means for analyzing voice input from the user and performing natural language processing, means for storing information acquired from external databases in a local database and updating it as needed, and means for generating a customized character chatbot using generative artificial intelligence technology to enable natural dialogue with the user.This allows the user to intuitively specify their destination, receive real-time information and personalized suggestions, and provide enhanced entertainment and support for safe driving, resulting in a comfortable and safe driving experience.

[0885] An "interactive interface" is a user interface that allows users to intuitively specify places they want to go or points they want to stop at using voice or text.

[0886] An "external information source" is an external data providing system or service that provides traffic information, rating information, business information, and the like.

[0887] "Traffic information" refers to real-time information about road conditions such as congestion, accidents, and road closures.

[0888] "Reviews and business information" refers to detailed information about reviews and business operations of restaurants, tourist spots, etc.

[0889] "Personalization" refers to individualization based on a user's past behavioral data and preferences.

[0890] "Entertainment functions" are functions that provide entertainment to users, such as playing music and asking quizzes.

[0891] "Safe Driving Support" is a support function that warns of dangers while driving based on vehicle data and encourages safe driving.

[0892] "Natural language processing" is a technology that analyzes voice and text input from users and understands their intent.

[0893] A "local database" is a database for storing information obtained from outside and updating it as necessary.

[0894] "Generative AI technology" is an AI technology for generating customized character chatbots based on user profile information.

[0895] A "character chatbot" is a virtual character generated using generative artificial intelligence technology to enable natural conversations with users.

[0896] The present invention provides a car navigation system with an interactive interface that allows users to intuitively specify destinations and stops. This system supports users in comfortable and efficient driving, and also provides entertainment while traveling and assists in safe driving.

[0897] First, the user inputs instructions via voice or text through a conversational interface. This interface converts the voice into text using a speech recognition engine (e.g., Google Cloud Speech-to-Text). Next, a natural language processing engine (e.g., Google Cloud Natural Language API) is used to analyze the user's instructions and extract keywords. For example, if the user says, "I want to go to a delicious cafe nearby," the system extracts the keywords "nearby" and "delicious cafe."

[0898] The server periodically retrieves the latest traffic information from external sources (e.g., HERE API) and stores information on congestion, accidents, road closures, etc. in a database (e.g., MySQL). This information is updated in real time and pushed to the device as needed.

[0899] The device also obtains ratings and business information for local restaurants and tourist spots from external information services (e.g., Yelp API) and stores this information in a local database. Based on this stored information, it lists the best places for the user and displays them on the screen. Furthermore, the server manages the user's past behavioral data as profile information, and the device calculates and suggests the best route based on this information. For example, it can prioritize routes to cafes that have been highly rated in the past.

[0900] As an entertainment feature, the user can make requests to the device, such as "Play the next song." The device will then call a music distribution service (e.g., Spotify API) and play the specified song. The device can also select questions from a quiz database and pose them to the user, providing entertainment while driving.

[0901] As a safe driving function, the device collects and analyzes data such as speed and following distance from various sensors in the vehicle in real time, and displays an alert to the user if danger is predicted. For example, it displays a warning message such as "There is insufficient distance between vehicles ahead. Please be careful."

[0902] Furthermore, the server uses generative AI technology (e.g., OpenAI GPT-4) to generate a customized character chatbot based on the user's profile. Using this chatbot, the device can have natural conversations with the user. Users can obtain a dedicated sharing link to share the chatbot character they created with other users, or sell the chatbot character through a marketplace.

[0903] Examples of prompts include:

[0904] "Tell me a recommended cafe nearby."

[0905] "Turn right at the next traffic light"

[0906] "Tell me the latest traffic information"

[0907] "Show me suggestions for cafes I've been to in the past."

[0908] "Play some Mozart"

[0909] "Show an alert if the following distance is too close"

[0910] "Generate a chatbot character based on user preferences"

[0911] "Get a link to share your generated character with other users."

[0912] In this way, the present invention allows users to intuitively operate the device and enjoy real-time information and personalized suggestions, resulting in a comfortable and safe driving experience.

[0913] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0914] Step 1:

[0915] The user inputs instructions through a conversational interface using voice or text. For example, the user might input, "I want to go to a nearby delicious cafe." The input voice data is sent to a speech recognition engine (e.g., Google Cloud Speech-to-Text) and converted into text data.

[0916] Input: User instructions via voice or text

[0917] Data processing: Speech to text conversion using a voice recognition engine

[0918] Output: Textual user instructions

[0919] Step 2:

[0920] The device sends the text data converted by the speech recognition engine to a natural language processing engine (e.g., Google Cloud Natural Language API) for analysis. Keywords are extracted to understand the user's intent. For example, keywords such as "nearby" and "delicious cafe" are extracted.

[0921] Input: Textual user instructions

[0922] Data Calculation: Keyword Extraction Using Natural Language Processing Engine

[0923] Output: Extracted keywords

[0924] Step 3:

[0925] The server periodically sends requests to external sources (e.g., HERE API) to obtain the latest traffic information. The obtained data is stored in a database (e.g., MySQL) and updated in real time. Push notifications are sent to the device when necessary.

[0926] Input: Request to external source

[0927] Data processing: Acquisition of traffic information and storage in a database

[0928] Output: Saved real-time traffic information data

[0929] Step 4:

[0930] The device sends a request to an external information service (e.g., Yelp API) to obtain ratings and business information for restaurants and tourist attractions in the area specified by the user. This information is stored in a local database.

[0931] Input: A request to an external information service

[0932] Data processing: Acquisition of evaluation information and storage in a local database

[0933] Output: Saved evaluation and sales information data

[0934] Step 5:

[0935] The device selects and lists the best places for the user based on the stored rating and business information data and real-time traffic information data. It then displays the list on the screen. For example, it presents the user with a list of "delicious cafes nearby."

[0936] Input: Rating and business information data, traffic information data

[0937] Data calculation: listing and optimized location suggestions

[0938] Output: A list of the best places listed by the user

[0939] Step 6:

[0940] The server manages the user's past behavior data (e.g., places rated in the past) as profile information. The device references this profile information and calculates and proposes the optimal route based on the user's past behavior. For example, it may prioritize routes to cafes that have been highly rated in the past.

[0941] Input: User's past behavior data

[0942] Data calculation: Referencing profile information and calculating routes

[0943] Output: Optimal route suggested to the user

[0944] Step 7:

[0945] The user inputs an entertainment request, such as "Play the next song," into the device. The device then calls a music distribution service (e.g., Spotify API) and plays the specified song. It also selects questions from a quiz database and presents them to the user in the form of a quiz.

[0946] Input: User entertainment request

[0947] Data calculation: Calling music distribution services and quiz questions

[0948] Output: Music playback and quiz

[0949] Step 8:

[0950] The device acquires and analyzes the vehicle's sensor data (e.g., speed and following distance) in real time. If there is a problem with safe driving, an alert will be displayed to the user. For example, if there is insufficient distance between the vehicle ahead, the device will display the message "Insufficient distance between the vehicle ahead. Please be careful."

[0951] Input: Vehicle sensor data

[0952] Data Calculation: Real-time analysis and safe driving alerts

[0953] Output: Alert the user

[0954] Step 9:

[0955] The server uses generative AI technology (e.g., OpenAI GPT-4) to generate a customized character chatbot based on the user's profile information. The generated chatbot is sent to the device and engages in natural conversations with the user.

[0956] Input: User profile information

[0957] Data Computation: Generating Character Chatbots with Generative AI Technology

[0958] Output: A chatbot that enables natural dialogue with users

[0959] Step 10:

[0960] Users can share their chatbot characters with other users by obtaining a dedicated sharing link from the server. Chatbot characters can also be sold through the marketplace.

[0961] Input: Generated chatbot character

[0962] Data processing: generating sharing links and selling on the marketplace

[0963] Output: Shared link and sales method

[0964] (Application example 1)

[0965] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0966] Today's drivers demand intuitive and efficient navigation systems. Especially important are interfaces that can be easily operated via voice, and the ability to obtain real-time traffic information and suggest optimal routes. However, few systems can simultaneously provide entertainment and safe driving support in addition to these functions. They also demand personalized suggestions based on individual user preferences and behavioral history. Furthermore, they also want chatbots that use generative AI technology to make the interface more user-friendly.

[0967] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0968] In this invention, the server includes: means for providing an interactive interface that allows a user to intuitively indicate places the user wants to go and points the user wants to stop at; means for acquiring traffic information from external information sources that is updated in real time and proposing optimal routes that avoid congestion; means for acquiring and providing area-specific ratings and business information from external information services; means for personalizing routes based on the user's past behavioral data and preferences; means for providing entertainment functions; means for supporting safe driving based on vehicle data; means for analyzing the user's voice commands using voice recognition technology and understanding the user's intentions using a natural language processing engine; means for displaying information in real time on the smart glasses; and means for generating an original character chatbot using generative artificial intelligence technology and having the chatbot interact with the user through the smart glasses. This allows the driver to intuitively operate the system and enjoy a comfortable and safe driving experience that combines real-time information with personalized suggestions.

[0969] - "User" means a person who uses a navigation system to reach a destination.

[0970] An "interactive interface" is a system that accepts and responds to instructions from the user through voice or text.

[0971] "External information sources" refer to external services or APIs that provide data such as traffic information and store information.

[0972] "Traffic information" refers to data on traffic conditions, including information on congestion, accidents, road closures, etc.

[0973] "Ratings and business information" is data showing user ratings and business status of stores and tourist attractions in the area.

[0974] "Personalization" refers to individual optimization based on a user's past behavioral data and preferences.

[0975] "Entertainment features" are features that allow users to enjoy themselves, such as music playback, quizzes, and chat.

[0976] "Vehicle data" is information about the vehicle's operation, such as speed and following distance.

[0977] "Voice recognition technology" is a technology that analyzes the user's voice and converts it into text data.

[0978] A "natural language processing engine" is a technology that understands the user's intent from text data and generates an appropriate response.

[0979] "Smart glasses" are glasses-type devices that have a built-in display and provide visual information when worn by the user.

[0980] "Generative artificial intelligence technology" is a technology that uses AI to generate new characters and content.

[0981] An "original character chatbot" is a conversational character created based on a user's profile using generative AI technology.

[0982] The present invention provides an interactive navigation system that allows users to intuitively specify where they want to go, provides real-time traffic information, and realizes a safe and comfortable driving experience. Specific embodiments for implementing this system are described below.

[0983] 1. User Interface Design

[0984] Users input commands by voice through a conversational interface, such as "I want to go to a nice cafe nearby." This voice command is converted into text data by the device's voice recognition technology (e.g., speech_recognition library), and then analyzed by a natural language processing engine (e.g., IntentRecognizer) to understand the user's intent.

[0985] 2. Obtaining real-time traffic information

[0986] The server periodically retrieves the latest traffic information from external sources (e.g., traffic information APIs), including traffic congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time. The device displays this information on the smart glasses display and suggests optimal routes for the user.

[0987] 3. Obtaining evaluations and sales information

[0988] The device obtains ratings and business information for restaurants and tourist spots in the area from external information services (e.g., rating site APIs), and based on this, it creates a list of places that best suit the user's needs.

[0989] 4. Personalized Route Suggestions

[0990] The server manages the user's past behavioral data and preferences as profile information. Based on this, the device suggests an optimized route to the user. By prioritizing cafes that have been highly rated in the past, the device can provide the user with a more satisfying experience.

[0991] 5. Providing entertainment features

[0992] Users can make requests to their device, such as "play the next song," to enjoy music playback, quizzes, and chat functions. The device then calls an external music service (e.g., music distribution API) to play music. It also generates quizzes from a quiz database, providing entertainment while driving.

[0993] 6. Safe driving support

[0994] The device monitors vehicle data (e.g., speed, distance between vehicles) in real time and displays alerts to encourage safe driving. For example, if the distance between vehicles is too small, a message such as "There is insufficient distance between vehicles ahead. Please be careful" will be displayed.

[0995] 7. Character Chatbot Generation and Interaction

[0996] The server uses generative AI technology to generate an original character chatbot based on the user's profile, which then interacts with the user through smart glasses, enabling natural communication.

[0997] 8. Sharing and selling characters

[0998] Users can share their chatbot characters with other users by obtaining a dedicated sharing link from the server, and can also sell their chatbot characters through the marketplace.

[0999] Examples and prompts

[1000] As a concrete example, the system functions can be realized by using the following prompt sentences:

[1001] Example prompt:

[1002] User: "I want to go to a recommended cafe nearby."

[1003] AI system: "Activate the speech recognition engine to understand the user's intent."

[1004] AI system: "Get information about nearby cafes from an external API."

[1005] AI system: "Recommended cafes will be displayed on the screen."

[1006] User: "Okay, let's go to the second cafe."

[1007] AI system: "Route guidance in progress. Turn right."

[1008] User: "Play some music."

[1009] AI system: "Playing your favorite song"

[1010] This allows users to enjoy a comfortable driving experience that is intuitive to operate and combines real-time information with personalized suggestions.

[1011] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1012] Step 1:

[1013] The user inputs instructions by voice, such as "I want to go to a nice cafe nearby." This voice input is captured through the device's microphone.

[1014] Step 2:

[1015] The device uses speech recognition technology (e.g., the speech_recognition library) to convert speech to text data. The input of this step is speech data, and the output is text data.

[1016] Step 3:

[1017] The device uses a natural language processing engine (e.g., IntentRecognizer) to analyze the user's intent from the text data. The input is text data, and the output is data indicating the user's intent (e.g., the intent to "find a cafe").

[1018] Step 4:

[1019] The device sends the user's intention to the server and queries the relevant local cafe information. The input is the user's intention data, and the output is the cafe information obtained through the external API.

[1020] Step 5:

[1021] The server obtains ratings and business information for restaurants and tourist attractions from external information sources (e.g., rating site APIs). This includes region-specific rating data and business information. The input is data indicating the region, and the output is the ratings and business information.

[1022] Step 6:

[1023] The server filters relevant information based on the user's past behavioral data and preferences to generate a personalized cafe list. The input is the user's past behavioral data and externally acquired ratings and sales information, and the output is a personalized list of cafes.

[1024] Step 7:

[1025] The terminal displays the cafe list obtained from the server on the display of the smart glasses. The input is the cafe list received from the server, and the output is the information displayed on the display of the smart glasses.

[1026] Step 8:

[1027] The user selects a cafe and then gives a voice command, such as "Let's go to the second cafe," which is also captured through the smart glasses.

[1028] Step 9:

[1029] The device again uses voice recognition technology and a natural language processing engine to analyze the user's new command: the input is voice data, and the output is the intent data: "Go to the second cafe."

[1030] Step 10:

[1031] The device retrieves optimal route information from the server and displays route guidance on the smart glasses. The input is the user's intention, current location, and route information received from the server, and the output is the display of route guidance.

[1032] Step 11:

[1033] The user types in a voice command requesting "play music," which is also captured through the smart glasses.

[1034] Step 12:

[1035] The device uses voice recognition technology and a natural language processing engine to analyze the music request and then calls an external music service (e.g., a music distribution API) to play the music. The input is audio data, and the output is the music being played.

[1036] Step 13:

[1037] The terminal monitors vehicle data (e.g., speed, distance between vehicles) in real time and displays safe driving alerts on the smart glasses. The input is vehicle data, and the output is the display of safe driving alerts.

[1038] Step 14:

[1039] The server uses generative AI technology to generate an original character chatbot based on the user's profile, and the device displays this character on the smart glasses and allows the user to interact with it. The input is the user profile data, and the output is the interaction with the generated character.

[1040] Step 15:

[1041] To share the chatbot character that a user has created with other users, the user obtains a dedicated sharing link from the server. The input is the generated character data, and the output is the sharing link.

[1042] This allows users to enjoy a comfortable driving experience that combines real-time information with personalized suggestions, with more intuitive operation than usual.

[1043] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1044] The present invention provides a car navigation system that has an interactive interface that allows users to intuitively indicate places they want to go or points they want to stop at, and that also combines an emotion engine that recognizes the user's emotions. This system allows the user to have a more personalized experience and realize a more comfortable driving experience. Specific embodiments of the present invention are described below.

[1045] 1. User Interface Design

[1046] Users can input instructions through a conversational interface using voice or text. For example, they can say, "I want to go to a nice cafe nearby." The device then uses a speech recognition engine to convert the voice data into text.

[1047] 2. Introducing the Emotion Engine

[1048] The device implements an emotion engine that analyzes emotions from the user's voice and text input. The emotion engine recognizes the user's emotional state (e.g., joy, anger, sadness, or happiness) from the user's tone of voice and text expressions, and saves the analysis results.

[1049] 3. Obtaining real-time traffic information

[1050] The server periodically retrieves the latest traffic information from external sources (e.g., traffic information API). This information includes congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time.

[1051] 4. Obtaining ratings and business information for restaurants and tourist spots

[1052] The device retrieves a list of cafes in the specified area, along with their ratings and business information, from an external information service (e.g., a rating site API). Based on this information, the device creates a list of the cafes with the highest ratings.

[1053] 5. Personalized Route Suggestions

[1054] The server manages the user's past behavioral data and preferences as profile information. The device accesses this profile information and suggests optimized routes to the user. For example, it may prioritize cafes that users have given high ratings to in the past or places based on their current emotional state.

[1055] 6. Provision of entertainment features

[1056] Users can enjoy entertainment such as music playback, quizzes, and chat functions. The device receives requests such as "play the next song" and calls an external music service (e.g., music distribution API) to play music. In addition, an emotion engine analyzes the user's emotions and adjusts appropriate entertainment content.

[1057] 7. Safe driving support

[1058] The device monitors vehicle sensor information in real time and provides alerts when speed, distance, or improper driving is detected. In addition, an emotion engine adjusts the content and timing of alerts based on the user's emotional state.

[1059] 8. Creating a Character Chatbot

[1060] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile. The device then uses this character to engage in natural conversations with the user. Using generative AI technology, the character's expressions also change dynamically to reflect the user's emotional state.

[1061] 9. Sharing and selling characters

[1062] Users can obtain a dedicated sharing link from the server to share the chatbot characters they have created with other users. They can also sell chatbot characters through the marketplace. The server provides this functionality through the network.

[1063] Specific examples

[1064] As a concrete example, consider the case where a user verbally commands, "I want to go to a nearby, relaxing cafe." The device converts this command into text using a speech recognition engine and analyzes it using a natural language processing engine. Next, the emotion engine analyzes the user's tone of voice and recognizes that the user is a little tired. The device obtains cafe information from an external information service, and the server uses past behavioral data and emotion information to create a list of highly rated cafes that are relaxing. The device then suggests the user the optimal route to that cafe. While driving, the device plays relaxing music that matches the user's preferences. The device also obtains safety data from vehicle sensors and works with the emotion engine to provide the user with safety alerts at the appropriate time.

[1065] In this way, the present invention makes it possible to realize a system that allows the user to enjoy a more comfortable and safe driving experience.

[1066] The processing flow will be explained below.

[1067] Step 1:

[1068] The user speaks to the device, saying, "I want to go to a quiet cafe nearby." The device receives this instruction and uses a speech recognition engine to convert the voice data into text data.

[1069] Step 2:

[1070] The device passes the converted text data to a natural language processing engine, which extracts keywords such as "cafe," "nearby," and "calm."

[1071] Step 3:

[1072] The device uses an emotion engine to analyze the user's emotional state from their voice and determines that they are "tired." The emotion engine then stores this information in their profile.

[1073] Step 4:

[1074] The server sends a request to an external information source (e.g., a traffic information API) to obtain the latest traffic information, and stores the obtained traffic information in a database.

[1075] Step 5:

[1076] The terminal obtains a list of cafes in a specified area, along with their ratings and business information, from an external information service (e.g., a rating site API).

[1077] Step 6:

[1078] The device will use the cafe information obtained in step 5 to create a list of cafes that are highly rated and have a relaxing atmosphere.

[1079] Step 7:

[1080] The device uses the Google Maps API to calculate the optimal route based on the traffic information obtained in step 4 and the cafe information listed in step 6.

[1081] Step 8:

[1082] Based on the user's past behavioral data and the emotional profile obtained in step 3, the server selects the cafe that best suits the user's preferences and current emotional state, and then personalizes and suggests a route to the cafe.

[1083] Step 9:

[1084] The user checks the suggested cafe and route on the device and confirms, "I want to go there."

[1085] Step 10:

[1086] The user requests, "Play some relaxing music." The device sends a request to the music distribution API and plays relaxing music.

[1087] Step 11:

[1088] While driving, the device monitors vehicle sensor information to check speed and following distance, and provides alerts if improper driving is detected.

[1089] Step 12:

[1090] The device uses an emotion engine to monitor the user's emotional state and adjust the entertainment content and timing of safe driving alerts as needed.

[1091] Step 13:

[1092] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile, and the device uses this character to interact with the user.

[1093] Step 14:

[1094] Users can share their chatbot characters with friends by obtaining a sharing link from the server. Characters can also be sold on the marketplace. The server manages transactions and generates sharing links.

[1095] In this way, users can operate intuitively and enjoy real-time information and personalized suggestions based on their emotional state, ensuring a comfortable and safe driving experience.

[1096] Example 2

[1097] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1098] Conventional car navigation systems simply provide routes to a destination and lack the ability to provide a personalized experience based on the user's emotional state and individual preferences. Furthermore, entertainment and safe driving support functions are limited, creating a need for an integrated solution to improve the overall driving experience. Furthermore, providing interactive character chatbots using generative AI technology and creating new user experiences based on them are also challenges.

[1099] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1100] In this invention, the server includes means for providing an interactive interface that allows the user to intuitively indicate places the user wants to go and points the user wants to stop at, means for acquiring traffic information from external information sources that is updated in real time and proposing optimal routes that avoid congestion, means for acquiring and providing area-specific ratings and business information from external information services, means for personalizing routes based on the user's past behavioral data and preferences, means for analyzing the user's emotional state from voice and text input, means for adjusting and providing entertainment functions according to the emotional state, means for supporting safe driving based on vehicle data and adjusting the content and timing of alerts based on the emotional state, and means for generating an original character chatbot based on the user's profile and dynamically changing and interacting with the chatbot. This enables optimal route suggestions, entertainment, and safe driving support based on the user's emotions and preferences, providing a more personalized and comfortable driving experience.

[1101] An "interactive interface" is an interface that allows a user to intuitively input instructions by voice or text, and the system returns an appropriate response to that input.

[1102] "External information sources" refer to information services and APIs that provide traffic information, traffic accident information, traffic congestion information, etc. from outside the system.

[1103] "Traffic Information" is real-time information about current road conditions, traffic congestion, traffic accidents, road closures, etc.

[1104] "Region-specific ratings" refers to user ratings and word-of-mouth information about places and facilities in a specific region.

[1105] "Business information" refers to information such as opening hours, regular holidays, and special opening days of restaurants, tourist attractions, etc.

[1106] "External information services" refer to services and APIs that provide evaluations and sales information.

[1107] "Personalizing your route" means suggesting the best route based on the user's past behavioral data and individual preferences.

[1108] "Emotional state" refers to the emotions such as joy, anger, sadness, and happiness that the user is currently feeling, and is analyzed from voice and text input.

[1109] "Entertainment features" are features that users can enjoy while driving, such as music playback, quizzes, and chat functions.

[1110] "Safe Driving Support" is a function that uses information from the vehicle's sensors to detect driving speed, vehicle distance, and inappropriate driving, and provides alerts to the user.

[1111] An "alert" is a notification that the system uses to warn or alert the user to encourage safe driving.

[1112] "Generative AI technology" is a technology that uses artificial intelligence to generate new content and characters.

[1113] "Profile Information" refers to information about a user, such as the user's past behavioral data, preferences, and emotional state.

[1114] A "character chatbot" is a virtual character created using generative AI technology that can engage in natural conversations with users.

[1115] "Dynamic change" means that the chatbot's behavior and responses change in real time depending on the user's emotional state and situation.

[1116] "Marketplace" means an online platform for sharing or selling user-generated chatbot characters.

[1117] This invention is a car navigation system that features an interactive interface that allows users to intuitively indicate where they want to go and where they want to stop, and provides real-time traffic information, emotion analysis, personalized route suggestions, entertainment functions, safe driving support, and a character chatbot that utilizes generative AI technology.

[1118] User Interface Design

[1119] Users input instructions to the system through voice or text. For example, if a user says, "I want to go to a nice cafe nearby," the device uses a speech recognition engine (e.g., Google Speech-to-Text) to convert the voice into text data. Then, a natural language processing engine (e.g., OpenAI GPT) analyzes this text data.

[1120] Introducing the Emotion Engine

[1121] The device implements an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the user's emotional state from voice and text. This engine analyzes the user's emotional state (e.g., joy, anger, sadness, or happiness) from the user's tone of voice and text expressions and saves the results.

[1122] Obtaining real-time traffic information

[1123] The server periodically retrieves the latest traffic information from external sources (e.g., Traffic API), including information on congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time.

[1124] Obtaining ratings and business information for restaurants and tourist spots

[1125] The device uses external information services (e.g., Yelp API) that provide location-specific ratings and business information to obtain a list of cafes and their ratings and business information, allowing it to present the location that best suits the user's request.

[1126] Personalized route suggestions

[1127] The server manages the user's past behavioral data and preferences as profile information. The device then uses this profile information to suggest the most suitable route for the user. Specifically, it prioritizes cafes that users have given high ratings to in the past and places that correspond to their current emotional state.

[1128] Providing entertainment features

[1129] Users can enjoy entertainment such as music playback, quizzes, and chat functions. The device receives requests such as "play the next song" and calls an external music service (e.g., Spotify API) to play music. In addition, an emotion engine analyzes the user's emotions and adjusts appropriate entertainment content.

[1130] Safe driving support

[1131] The device monitors vehicle sensor information in real time and provides alerts when speed, distance, or improper driving is detected. The emotion engine adjusts the content and timing of alerts based on the user's emotional state.

[1132] Character chatbot generation

[1133] The server uses generative AI technology (e.g., OpenAI GPT) to generate a chatbot with an original character based on the user's profile. The device then uses this character to engage in natural conversations with the user. Generative AI technology dynamically changes the character's expressions according to the user's emotional state.

[1134] Sharing and selling characters

[1135] Users can obtain a dedicated sharing link from the server to share their chatbot characters with other users, and can also sell chatbot characters through the marketplace.

[1136] Specific examples

[1137] Consider a scenario where a user verbally commands, "I want to go to a nearby relaxing cafe." The device converts this command into text using a speech recognition engine (e.g., Google Speech-to-Text) and analyzes it using a natural language processing engine (e.g., OpenAI GPT). Next, an emotion engine (e.g., IBM Watson Tone Analyzer) analyzes the user's tone of voice and recognizes that the user is slightly tired. The device obtains cafe information from an external information service (e.g., Yelp API), and the server uses past behavioral data and emotion information to create a list of highly rated cafes that are relaxing. The device then suggests the user the optimal route to the cafe. While driving, the device plays relaxing music using an external music service (e.g., Spotify API). The device also obtains safety data from vehicle sensors and, in cooperation with the emotion engine, provides the user with safety alerts at the appropriate time.

[1138] Prompt Sentence Examples

[1139] "If a user says, 'I want to go to a quiet cafe nearby,' what kind of processing and data is required?"

[1140] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1141] Step 1:

[1142] The user launches the device's conversational interface. The user inputs a command by voice or text, such as "I want to go to a nearby delicious cafe." In this case, the input is voice data, which is passed to the device. Specifically, the user performs input operations using a microphone or keyboard.

[1143] Step 2:

[1144] The device uses a speech recognition engine (e.g., a speech recognition engine API) to convert the user's voice data into text data. The output is text data, which is used as input data for the next processing step. The specific operation involves the process in which the speech recognition engine analyzes the voice waveform and converts it into corresponding text.

[1145] Step 3:

[1146] The device passes the converted text data to a natural language processing engine (e.g., natural language processing API) and analyzes the user's instructions. The input is text data, and the output is the intent data resulting from the analysis. For example, the intent extracted is "I want to go to a nearby delicious cafe." Specifically, the natural language processing engine understands the context of the text and concludes on the user's request.

[1147] Step 4:

[1148] The device uses an emotion engine (e.g., emotion analysis API) to analyze the user's emotional state from text data and voice tone. The input is text data and voice tone data, and the output is emotional state data. For example, it may recognize that the user is slightly tired. Specifically, the emotion engine analyzes the intonation of the voice and the tone of the text to determine the emotional state.

[1149] Step 5:

[1150] The server periodically obtains the latest traffic information from an external information source (e.g., a traffic information API). The input is an API request, and the output is traffic information data. Specifically, the server sends an API request and receives real-time data such as congestion and accident information.

[1151] Step 6:

[1152] The device obtains cafe ratings and business information from an external information service (e.g., a rating site API). The input is an API request, and the output is a list of cafes and their rating data. Specific operations include sending an API request, parsing the results, and generating a list of cafes.

[1153] Step 7:

[1154] The device accesses the user's profile information (e.g., past behavioral data, preferences) stored on the server and proposes the optimal route. The input is the profile data, and the output is optimal route guidance data. Specifically, the server evaluates candidate locations based on the past behavioral data and the user's preferences, and generates the optimal route.

[1155] Step 8:

[1156] When a user requests "play the next song," the device calls an external music service (e.g., a music distribution API) to play music. The input is the user's voice request, and the output is playback instruction data. Specifically, the device sends an API request to the music distribution service and streams the music data.

[1157] Step 9:

[1158] The device monitors vehicle sensor information in real time and provides alerts to support safe driving. The input is sensor information and the output is alert data. Specific operations include analyzing sensor information to evaluate speed and vehicle distance and detect inappropriate driving.

[1159] Step 10:

[1160] The server uses generative AI technology (e.g., generative AI model) to generate an original character chatbot based on the user's profile. The input is profile information, and the output is chatbot data. Specifically, the server uses generative AI technology to simulate the character's dialogue.

[1161] Step 11:

[1162] The server generates a dedicated sharing link so that users can share their chatbot characters with other users. The input is the chatbot data, and the output is the sharing link. Specifically, the server registers the chatbot data on the sharing platform and generates the link.

[1163] (Application example 2)

[1164] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1165] Conventional car navigation systems only provide simple route guidance without considering the user's emotions or entertainment needs. This makes it difficult to provide a comfortable and personalized driving experience for users. Furthermore, their safe driving support features are limited, and they do not provide appropriate alerts or entertainment tailored to the user's emotional state. This makes it difficult to reduce user stress and discomfort, and furthermore, ensures safe driving. The present invention aims to solve these problems of the conventional technology and provide users with a personalized, comfortable driving experience and safe driving support.

[1166] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for analyzing the user's emotional state and proposing entertainment content and routes based on the emotion, a means for monitoring the vehicle's sensor information in real time and adjusting the content and timing of alerts based on the user's emotional state, and a means for acquiring and proposing area-specific reviews and business information from external information services. This enables personalized entertainment and route suggestions that take the user's emotional state into consideration. Furthermore, safety alerts can be provided at appropriate times according to the user's emotional state, resulting in a comfortable and safe driving experience.

[1167] An "interactive interface" is an input method that allows users to intuitively specify their destination or points of interest using voice or text.

[1168] "External information sources updated in real time" refers to information provision services that provide traffic information, rating information, etc. with the latest data over time.

[1169] "Means of obtaining traffic information and proposing the optimal route to avoid congestion" refers to a function that imports traffic data in real time from external traffic information services and, based on that data, presents users with the optimal route to avoid congestion.

[1170] "Means for obtaining and providing region-specific ratings and business information from external information services" refers to a function that obtains ratings and business information on restaurants, tourist attractions, etc. related to a specified region from external information services and provides it to users.

[1171] "Means to personalize routes based on the user's past behavioral data and preferences" is a function that suggests individualized routes based on information such as the user's past travel history, favorite categories, and preferred locations.

[1172] "Means for providing entertainment functions" refers to functions that provide users with entertainment content such as music, quizzes, and chat.

[1173] "Means to support safe driving based on vehicle data" refers to a function that provides users with alerts and notifications to encourage safe driving based on data obtained from sensors, cameras, etc. installed in the vehicle.

[1174] "Means for analyzing the user's emotional state and suggesting entertainment content and routes based on that emotion" refers to a function that analyzes the user's emotional state (e.g., happy, tired) from voice and text data, and personalizes entertainment content and routes based on the analysis results.

[1175] "Means for monitoring vehicle sensor information in real time and adjusting the content and timing of alerts based on the user's emotional state" refers to a function that monitors data obtained in real time from various sensors installed in the vehicle and optimizes the content and timing of alerts according to the user's emotional state.

[1176] The system of the present invention includes an interactive interface that allows users to intuitively indicate where they want to go and where they want to stop, routes that are suggested based on traffic information obtained from external sources that is updated in real time, personalized entertainment content and route suggestions based on the user's emotional state, and functions that support safe driving.

[1177] 1. User Interface Design

[1178] A conversational interface uses a speech recognition engine (e.g., Google Speech Recognition API) to convert the user's speech into text. When a user speaks a command such as "I want to go to a nice cafe nearby," the speech data is converted into text by the recognition engine.

[1179] 2. Introducing the Emotion Engine

[1180] Using an emotion engine (e.g., EmotionRecognizer), the emotional state of the user is analyzed from voice and text, allowing accurate understanding of the user's emotional state, such as whether they are tired or happy.

[1181] 3. Obtaining real-time traffic information

[1182] The server periodically obtains the latest traffic information using a traffic information API (e.g., a general traffic information API), thereby obtaining real-time information on traffic congestion and accidents and storing it in a database.

[1183] 4. Obtaining ratings and business information for restaurants and tourist spots

[1184] The device retrieves a list of cafes in the specified area, along with their ratings and business information, from an external information service (e.g., a rating site API). Based on this information, the device creates a list of the cafes with the highest ratings.

[1185] 5. Personalized Route Suggestions

[1186] The server manages the user's past behavioral data and preferences (favorite restaurant trends and records of places visited) as profile information. The device then suggests the optimal route based on this profile information and the results of the emotion engine's analysis.

[1187] 6. Provision of entertainment features

[1188] The device provides features such as music playback (using external music service APIs), quizzes, and chat functions. The emotion engine analyzes the user's emotions and dynamically adjusts the entertainment content to suit their needs.

[1189] 7. Safe driving support

[1190] The device monitors vehicle sensor information in real time to support safe driving, and improves safety by adjusting the content and timing of safety alerts based on the user's emotional state.

[1191] 8. Creating a Character Chatbot

[1192] The server uses generative AI technology (e.g., generative AI model) to generate a chatbot with an original character based on the user's profile. The device then uses this character to engage in natural conversations with the user.

[1193] 9. Sharing and selling characters

[1194] Users can obtain a dedicated sharing link from the server to share their chatbot characters with other users, and can also sell their characters through the marketplace.

[1195] Specific examples

[1196] For example, if a user says, "I want to go to a nearby relaxing cafe," the device will convert this into text using a speech recognition engine and analyze the user's emotional state using an emotion engine. If the device detects that the user is tired, it will retrieve cafe information from a rating site API and create a list of highly rated cafes that are relaxing and calming based on past behavioral data and emotional information. It will then suggest the optimal route to the cafe to the user and play relaxing music. It will also retrieve safety data from vehicle sensors and link with the emotion engine to provide safety alerts at the appropriate time.

[1197] Prompt Sentence Examples

[1198] "Based on the invention below, you will design an emotional car navigation app for self-driving cars. The app should include a function that allows the user to specify a destination by voice, and the emotion engine analyzes the user's emotions, suggests the optimal route, and provides entertainment."

[1199] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1200] Step 1:

[1201] The user inputs instructions by voice. The user may say, "I want to go to a nice cafe nearby." This voice input becomes the input to the system.

[1202] Step 2:

[1203] The device uses a speech recognition engine (e.g., Google Speech Recognition API) to convert the voice data into text data. The speech recognition engine analyzes the voice data and outputs it as text data.

[1204] Step 3:

[1205] The device uses an emotion engine (e.g., EmotionRecognizer) to analyze the user's emotional state from text and voice data, and outputs the emotional state (e.g., tired, happy) obtained through this analysis.

[1206] Step 4:

[1207] The server uses a traffic information API (e.g., a general traffic information API) to obtain the latest traffic information in real time. The obtained traffic information is stored in a database and becomes input data for the system.

[1208] Step 5:

[1209] The device uses an external information service (e.g., a rating site API) to obtain a list of cafes in the specified area, along with their ratings and business information. This data is input, and a list of highly rated cafes is output.

[1210] Step 6:

[1211] The server retrieves the user's past behavioral data and preferences (e.g., a list of cafes visited and their ratings) from the database as profile information. The retrieved profile information becomes input data for the system.

[1212] Step 7:

[1213] The device then proposes the optimal route based on the acquired profile information, the results of the emotion engine analysis, and traffic and cafe information. The device analyzes this data and outputs a personalized route for the user.

[1214] Step 8:

[1215] The device provides entertainment functions. Based on the analysis results of the emotion engine, it dynamically adjusts and provides music, quizzes, and chat functions that suit the user. This function outputs entertainment content optimized for the user's emotional state.

[1216] Step 9:

[1217] The device monitors the vehicle's sensor information in real time and adjusts the content and timing of safety alerts according to the user's emotional state. Alerts are output at appropriate times based on the sensor information and the user's emotional state.

[1218] Step 10:

[1219] The server uses generative AI technology (e.g., generative AI model) to generate a chatbot with an original character based on the user's profile. The generated character is used as an output to the system to interact with the user.

[1220] Step 11:

[1221] Users can share the generated character chatbots with other users or sell them on the marketplace. The server generates a sharing link and provides a sales function in cooperation with the marketplace. This function is provided as an output to the user.

[1222] Adding specific examples

[1223] For example, if a user says, "I want to go to a nearby relaxing cafe," the system converts the speech to text and then uses its emotion engine to interpret it as "tired." It then retrieves cafe information from a review site API and, taking into account the user's past review data, creates a list of the most suitable cafes and suggests them to the user along with a route. It also plays relaxing music and adjusts safety alerts while driving based on the user's emotional state.

[1224] Prompt Sentence Examples

[1225] "Based on the invention below, you will design an emotional car navigation app for self-driving cars. The app should include a function that allows the user to specify a destination by voice, and the emotion engine analyzes the user's emotions, suggests the optimal route, and provides entertainment."

[1226] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1227] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1228] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1229] [Fourth embodiment]

[1230] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1231] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1232] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1233] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1234] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1235] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1236] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1237] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1238] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1239] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1240] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1241] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1242] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1243] The present invention proposes a car navigation system with an interactive interface that allows users to intuitively specify where they want to go and where they want to stop. This not only realizes the comfortable and efficient driving that modern drivers desire, but also provides entertainment while traveling and support for safe driving. Specific embodiments of the present invention are described below.

[1244] 1. User Interface Design

[1245] Users can input instructions through a conversational interface using voice or text. For example, they can say, "I want to go to a nice cafe nearby." The device then analyzes this with a natural language processing engine and extracts keywords to understand the user's intent.

[1246] 2. Obtaining real-time traffic information

[1247] The server periodically retrieves the latest traffic information from external sources (e.g., traffic information API). This information includes congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time.

[1248] 3. Obtaining ratings and business information for restaurants and tourist spots

[1249] The device retrieves ratings and business information for restaurants and tourist spots in the specified area from external information services (e.g., rating site APIs). Based on this data, it creates a list of places that best meet the user's requirements.

[1250] 4. Personalized Route Suggestions

[1251] The server manages the user's past behavioral data and preferences as profile information, and based on this, the device suggests optimized routes to the user, for example, prioritizing cafes that have been highly rated in the past.

[1252] 5. Providing entertainment features

[1253] Users can make requests to their device, such as "play the next song," to enjoy entertainment such as music playback, quizzes, and chat functions. The device then plays music by calling an external music service (e.g., music distribution API). It also generates quizzes from a quiz database, providing entertainment while driving.

[1254] 6. Safe driving support

[1255] The device monitors vehicle data (e.g., speed, distance between vehicles) in real time and displays alerts to encourage safe driving. For example, if the distance between vehicles is too small, a message such as "There is insufficient distance between vehicles ahead. Please be careful" will be displayed.

[1256] 7. Creating a Character Chatbot

[1257] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile. This character is personalized based on the user's preferences and interests. The device then uses the generated character to engage in natural conversations with the user.

[1258] 8. Sharing and selling characters

[1259] Users can obtain a dedicated sharing link from the server to share the chatbot characters they have created with other users. They can also sell chatbot characters through the marketplace. The server provides this functionality through the network.

[1260] In this way, the present invention provides users with intuitive operation, real-time information and personalized suggestions, resulting in a comfortable and safe driving experience.

[1261] The processing flow will be explained below.

[1262] Step 1:

[1263] The user speaks to the device, saying, "I want to go to a nice cafe nearby." The device receives this command and uses a speech recognition engine to convert the voice data into text data.

[1264] Step 2:

[1265] The device passes the converted text data to a natural language processing engine, which extracts keywords such as "cafe," "nearby," and "delicious."

[1266] Step 3:

[1267] The server sends a request to an external information source (e.g., a traffic information API) to obtain real-time updated traffic information, and stores the obtained traffic information in a database.

[1268] Step 4:

[1269] The device obtains a list of local cafes, along with their ratings and business information, from an external information service (e.g., a rating site API). Based on this information, the device creates a list of the cafes with the highest ratings.

[1270] Step 5:

[1271] The device calls the Google Maps API to calculate the optimal route based on the traffic and cafe information obtained in steps 3 and 4. If there is traffic congestion, it suggests a detour route.

[1272] Step 6:

[1273] The server stores the user's past behavioral data and preferences as a profile, and the device accesses this profile information to prioritize suggestions for cafes that the user prefers.

[1274] Step 7:

[1275] Users can request "play my favorite music." The device sends the request to the music distribution API and plays music that matches the user's preferences. It also offers entertainment features such as quizzes.

[1276] Step 8:

[1277] The device monitors vehicle sensor information in real time and provides alerts if speed, distance, or improper driving is detected.

[1278] Step 9:

[1279] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile, and the device uses this character to interact with the user.

[1280] Step 10:

[1281] Users can share their chatbot characters with friends by obtaining a sharing link from the server. Characters can also be sold on the marketplace. The server manages transactions and generates sharing links.

[1282] Example 1

[1283] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1284] Conventional car navigation systems have issues with not being able to intuitively indicate where users want to go or stop, and it is difficult to efficiently provide real-time updated traffic information, local ratings, and business information. Furthermore, they lack the means to provide entertainment and safe driving support, failing to fully realize a comfortable and safe driving experience for users. Furthermore, there is a lack of means to provide personalized route suggestions based on user preferences and natural dialogue.

[1285] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1286] In this invention, the server includes means for providing an interactive interface that allows a user to intuitively indicate places they want to go and points they want to stop at, means for acquiring traffic information from external information sources that is updated in real time and proposing optimal routes that avoid congestion, means for acquiring and providing area-specific ratings and business information from external information services, means for personalizing routes based on the user's past behavioral data and preferences, means for providing entertainment functions, means for supporting safe driving based on vehicle data, means for analyzing voice input from the user and performing natural language processing, means for storing information acquired from external databases in a local database and updating it as needed, and means for generating a customized character chatbot using generative artificial intelligence technology to enable natural dialogue with the user.This allows the user to intuitively specify their destination, receive real-time information and personalized suggestions, and provide enhanced entertainment and support for safe driving, resulting in a comfortable and safe driving experience.

[1287] An "interactive interface" is a user interface that allows users to intuitively specify places they want to go or points they want to stop at using voice or text.

[1288] An "external information source" is an external data providing system or service that provides traffic information, rating information, business information, and the like.

[1289] "Traffic information" refers to real-time information about road conditions such as congestion, accidents, and road closures.

[1290] "Reviews and business information" refers to detailed information about reviews and business operations of restaurants, tourist spots, etc.

[1291] "Personalization" refers to individualization based on a user's past behavioral data and preferences.

[1292] "Entertainment functions" are functions that provide entertainment to users, such as playing music and asking quizzes.

[1293] "Safe Driving Support" is a support function that warns of dangers while driving based on vehicle data and encourages safe driving.

[1294] "Natural language processing" is a technology that analyzes voice and text input from users and understands their intent.

[1295] A "local database" is a database for storing information obtained from outside and updating it as necessary.

[1296] "Generative AI technology" is an AI technology for generating customized character chatbots based on user profile information.

[1297] A "character chatbot" is a virtual character generated using generative artificial intelligence technology to enable natural conversations with users.

[1298] The present invention provides a car navigation system with an interactive interface that allows users to intuitively specify destinations and stops. This system supports users in comfortable and efficient driving, and also provides entertainment while traveling and assists in safe driving.

[1299] First, the user inputs instructions via voice or text through a conversational interface. This interface converts the voice into text using a speech recognition engine (e.g., Google Cloud Speech-to-Text). Next, a natural language processing engine (e.g., Google Cloud Natural Language API) is used to analyze the user's instructions and extract keywords. For example, if the user says, "I want to go to a delicious cafe nearby," the system extracts the keywords "nearby" and "delicious cafe."

[1300] The server periodically retrieves the latest traffic information from external sources (e.g., HERE API) and stores information on congestion, accidents, road closures, etc. in a database (e.g., MySQL). This information is updated in real time and pushed to the device as needed.

[1301] The device also obtains ratings and business information for local restaurants and tourist spots from external information services (e.g., Yelp API) and stores this information in a local database. Based on this stored information, it lists the best places for the user and displays them on the screen. Furthermore, the server manages the user's past behavioral data as profile information, and the device calculates and suggests the best route based on this information. For example, it can prioritize routes to cafes that have been highly rated in the past.

[1302] As an entertainment feature, the user can make requests to the device, such as "Play the next song." The device will then call a music distribution service (e.g., Spotify API) and play the specified song. The device can also select questions from a quiz database and pose them to the user, providing entertainment while driving.

[1303] As a safe driving function, the device collects and analyzes data such as speed and following distance from various sensors in the vehicle in real time, and displays an alert to the user if danger is predicted. For example, it displays a warning message such as "There is insufficient distance between vehicles ahead. Please be careful."

[1304] Furthermore, the server uses generative AI technology (e.g., OpenAI GPT-4) to generate a customized character chatbot based on the user's profile. Using this chatbot, the device can have natural conversations with the user. Users can obtain a dedicated sharing link to share the chatbot character they created with other users, or sell the chatbot character through a marketplace.

[1305] Examples of prompts include:

[1306] "Tell me a recommended cafe nearby."

[1307] "Turn right at the next traffic light"

[1308] "Tell me the latest traffic information"

[1309] "Show me suggestions for cafes I've been to in the past."

[1310] "Play some Mozart"

[1311] "Show an alert if the following distance is too close"

[1312] "Generate a chatbot character based on user preferences"

[1313] "Get a link to share your generated character with other users."

[1314] In this way, the present invention allows users to intuitively operate the device and enjoy real-time information and personalized suggestions, resulting in a comfortable and safe driving experience.

[1315] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1316] Step 1:

[1317] The user inputs instructions through a conversational interface using voice or text. For example, the user might input, "I want to go to a nearby delicious cafe." The input voice data is sent to a speech recognition engine (e.g., Google Cloud Speech-to-Text) and converted into text data.

[1318] Input: User instructions via voice or text

[1319] Data processing: Speech to text conversion using a voice recognition engine

[1320] Output: Textual user instructions

[1321] Step 2:

[1322] The device sends the text data converted by the speech recognition engine to a natural language processing engine (e.g., Google Cloud Natural Language API) for analysis. Keywords are extracted to understand the user's intent. For example, keywords such as "nearby" and "delicious cafe" are extracted.

[1323] Input: Textual user instructions

[1324] Data Calculation: Keyword Extraction Using Natural Language Processing Engine

[1325] Output: Extracted keywords

[1326] Step 3:

[1327] The server periodically sends requests to external sources (e.g., HERE API) to obtain the latest traffic information. The obtained data is stored in a database (e.g., MySQL) and updated in real time. Push notifications are sent to the device when necessary.

[1328] Input: Request to external source

[1329] Data processing: Acquisition of traffic information and storage in a database

[1330] Output: Saved real-time traffic information data

[1331] Step 4:

[1332] The device sends a request to an external information service (e.g., Yelp API) to obtain ratings and business information for restaurants and tourist attractions in the area specified by the user. This information is stored in a local database.

[1333] Input: A request to an external information service

[1334] Data processing: Acquisition of evaluation information and storage in a local database

[1335] Output: Saved evaluation and sales information data

[1336] Step 5:

[1337] The device selects and lists the best places for the user based on the stored rating and business information data and real-time traffic information data. It then displays the list on the screen. For example, it presents the user with a list of "delicious cafes nearby."

[1338] Input: Rating and business information data, traffic information data

[1339] Data calculation: listing and optimized location suggestions

[1340] Output: A list of the best places listed by the user

[1341] Step 6:

[1342] The server manages the user's past behavior data (e.g., places rated in the past) as profile information. The device references this profile information and calculates and proposes the optimal route based on the user's past behavior. For example, it may prioritize routes to cafes that have been highly rated in the past.

[1343] Input: User's past behavior data

[1344] Data calculation: Referencing profile information and calculating routes

[1345] Output: Optimal route suggested to the user

[1346] Step 7:

[1347] The user inputs an entertainment request, such as "Play the next song," into the device. The device then calls a music distribution service (e.g., Spotify API) and plays the specified song. It also selects questions from a quiz database and presents them to the user in the form of a quiz.

[1348] Input: User entertainment request

[1349] Data calculation: Calling music distribution services and quiz questions

[1350] Output: Music playback and quiz

[1351] Step 8:

[1352] The device acquires and analyzes the vehicle's sensor data (e.g., speed and following distance) in real time. If there is a problem with safe driving, an alert will be displayed to the user. For example, if there is insufficient distance between the vehicle ahead, the device will display the message "Insufficient distance between the vehicle ahead. Please be careful."

[1353] Input: Vehicle sensor data

[1354] Data Calculation: Real-time analysis and safe driving alerts

[1355] Output: Alert the user

[1356] Step 9:

[1357] The server uses generative AI technology (e.g., OpenAI GPT-4) to generate a customized character chatbot based on the user's profile information. The generated chatbot is sent to the device and engages in natural conversations with the user.

[1358] Input: User profile information

[1359] Data Computation: Generating Character Chatbots with Generative AI Technology

[1360] Output: A chatbot that enables natural dialogue with users

[1361] Step 10:

[1362] Users can share their chatbot characters with other users by obtaining a dedicated sharing link from the server. Chatbot characters can also be sold through the marketplace.

[1363] Input: Generated chatbot character

[1364] Data processing: generating sharing links and selling on the marketplace

[1365] Output: Shared link and sales method

[1366] (Application example 1)

[1367] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1368] Today's drivers demand intuitive and efficient navigation systems. Especially important are interfaces that can be easily operated via voice, and the ability to obtain real-time traffic information and suggest optimal routes. However, few systems can simultaneously provide entertainment and safe driving support in addition to these functions. They also demand personalized suggestions based on individual user preferences and behavioral history. Furthermore, they also want chatbots that use generative AI technology to make the interface more user-friendly.

[1369] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1370] In this invention, the server includes: means for providing an interactive interface that allows a user to intuitively indicate places the user wants to go and points the user wants to stop at; means for acquiring traffic information from external information sources that is updated in real time and proposing optimal routes that avoid congestion; means for acquiring and providing area-specific ratings and business information from external information services; means for personalizing routes based on the user's past behavioral data and preferences; means for providing entertainment functions; means for supporting safe driving based on vehicle data; means for analyzing the user's voice commands using voice recognition technology and understanding the user's intentions using a natural language processing engine; means for displaying information in real time on the smart glasses; and means for generating an original character chatbot using generative artificial intelligence technology and having the chatbot interact with the user through the smart glasses. This allows the driver to intuitively operate the system and enjoy a comfortable and safe driving experience that combines real-time information with personalized suggestions.

[1371] - "User" means a person who uses a navigation system to reach a destination.

[1372] An "interactive interface" is a system that accepts and responds to instructions from the user through voice or text.

[1373] "External information sources" refer to external services or APIs that provide data such as traffic information and store information.

[1374] "Traffic information" refers to data on traffic conditions, including information on congestion, accidents, road closures, etc.

[1375] "Ratings and business information" is data showing user ratings and business status of stores and tourist attractions in the area.

[1376] "Personalization" refers to individual optimization based on a user's past behavioral data and preferences.

[1377] "Entertainment features" are features that allow users to enjoy themselves, such as music playback, quizzes, and chat.

[1378] "Vehicle data" is information about the vehicle's operation, such as speed and following distance.

[1379] "Voice recognition technology" is a technology that analyzes the user's voice and converts it into text data.

[1380] A "natural language processing engine" is a technology that understands the user's intent from text data and generates an appropriate response.

[1381] "Smart glasses" are glasses-type devices that have a built-in display and provide visual information when worn by the user.

[1382] "Generative artificial intelligence technology" is a technology that uses AI to generate new characters and content.

[1383] An "original character chatbot" is a conversational character created based on a user's profile using generative AI technology.

[1384] The present invention provides an interactive navigation system that allows users to intuitively specify where they want to go, provides real-time traffic information, and realizes a safe and comfortable driving experience. Specific embodiments for implementing this system are described below.

[1385] 1. User Interface Design

[1386] Users input commands by voice through a conversational interface, such as "I want to go to a nice cafe nearby." This voice command is converted into text data by the device's voice recognition technology (e.g., speech_recognition library), and then analyzed by a natural language processing engine (e.g., IntentRecognizer) to understand the user's intent.

[1387] 2. Obtaining real-time traffic information

[1388] The server periodically retrieves the latest traffic information from external sources (e.g., traffic information APIs), including traffic congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time. The device displays this information on the smart glasses display and suggests optimal routes for the user.

[1389] 3. Obtaining evaluations and sales information

[1390] The device obtains ratings and business information for restaurants and tourist spots in the area from external information services (e.g., rating site APIs), and based on this, it creates a list of places that best suit the user's needs.

[1391] 4. Personalized Route Suggestions

[1392] The server manages the user's past behavioral data and preferences as profile information. Based on this, the device suggests an optimized route to the user. By prioritizing cafes that have been highly rated in the past, the device can provide the user with a more satisfying experience.

[1393] 5. Providing entertainment features

[1394] Users can make requests to their device, such as "play the next song," to enjoy music playback, quizzes, and chat functions. The device then calls an external music service (e.g., music distribution API) to play music. It also generates quizzes from a quiz database, providing entertainment while driving.

[1395] 6. Safe driving support

[1396] The device monitors vehicle data (e.g., speed, distance between vehicles) in real time and displays alerts to encourage safe driving. For example, if the distance between vehicles is too small, a message such as "There is insufficient distance between vehicles ahead. Please be careful" will be displayed.

[1397] 7. Character Chatbot Generation and Interaction

[1398] The server uses generative AI technology to generate an original character chatbot based on the user's profile, which then interacts with the user through smart glasses, enabling natural communication.

[1399] 8. Sharing and selling characters

[1400] Users can share their chatbot characters with other users by obtaining a dedicated sharing link from the server, and can also sell their chatbot characters through the marketplace.

[1401] Examples and prompts

[1402] As a concrete example, the system functions can be realized by using the following prompt sentences:

[1403] Example prompt:

[1404] User: "I want to go to a recommended cafe nearby."

[1405] AI system: "Activate the speech recognition engine to understand the user's intent."

[1406] AI system: "Get information about nearby cafes from an external API."

[1407] AI system: "Recommended cafes will be displayed on the screen."

[1408] User: "Okay, let's go to the second cafe."

[1409] AI system: "Route guidance in progress. Turn right."

[1410] User: "Play some music."

[1411] AI system: "Playing your favorite song"

[1412] This allows users to enjoy a comfortable driving experience that is intuitive to operate and combines real-time information with personalized suggestions.

[1413] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1414] Step 1:

[1415] The user inputs instructions by voice, such as "I want to go to a nice cafe nearby." This voice input is captured through the device's microphone.

[1416] Step 2:

[1417] The device uses speech recognition technology (e.g., the speech_recognition library) to convert speech to text data. The input of this step is speech data, and the output is text data.

[1418] Step 3:

[1419] The device uses a natural language processing engine (e.g., IntentRecognizer) to analyze the user's intent from the text data. The input is text data, and the output is data indicating the user's intent (e.g., the intent to "find a cafe").

[1420] Step 4:

[1421] The device sends the user's intention to the server and queries the relevant local cafe information. The input is the user's intention data, and the output is the cafe information obtained through the external API.

[1422] Step 5:

[1423] The server obtains ratings and business information for restaurants and tourist attractions from external information sources (e.g., rating site APIs). This includes region-specific rating data and business information. The input is data indicating the region, and the output is the ratings and business information.

[1424] Step 6:

[1425] The server filters relevant information based on the user's past behavioral data and preferences to generate a personalized cafe list. The input is the user's past behavioral data and externally acquired ratings and sales information, and the output is a personalized list of cafes.

[1426] Step 7:

[1427] The terminal displays the cafe list obtained from the server on the display of the smart glasses. The input is the cafe list received from the server, and the output is the information displayed on the display of the smart glasses.

[1428] Step 8:

[1429] The user selects a cafe and then gives a voice command, such as "Let's go to the second cafe," which is also captured through the smart glasses.

[1430] Step 9:

[1431] The device again uses voice recognition technology and a natural language processing engine to analyze the user's new command: the input is voice data, and the output is the intent data: "Go to the second cafe."

[1432] Step 10:

[1433] The device retrieves optimal route information from the server and displays route guidance on the smart glasses. The input is the user's intention, current location, and route information received from the server, and the output is the display of route guidance.

[1434] Step 11:

[1435] The user types in a voice command requesting "play music," which is also captured through the smart glasses.

[1436] Step 12:

[1437] The device uses voice recognition technology and a natural language processing engine to analyze the music request and then calls an external music service (e.g., a music distribution API) to play the music. The input is audio data, and the output is the music being played.

[1438] Step 13:

[1439] The terminal monitors vehicle data (e.g., speed, distance between vehicles) in real time and displays safe driving alerts on the smart glasses. The input is vehicle data, and the output is the display of safe driving alerts.

[1440] Step 14:

[1441] The server uses generative AI technology to generate an original character chatbot based on the user's profile, and the device displays this character on the smart glasses and allows the user to interact with it. The input is the user profile data, and the output is the interaction with the generated character.

[1442] Step 15:

[1443] To share the chatbot character that a user has created with other users, the user obtains a dedicated sharing link from the server. The input is the generated character data, and the output is the sharing link.

[1444] This allows users to enjoy a comfortable driving experience that combines real-time information with personalized suggestions, with more intuitive operation than usual.

[1445] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1446] The present invention provides a car navigation system that has an interactive interface that allows users to intuitively indicate places they want to go or points they want to stop at, and that also combines an emotion engine that recognizes the user's emotions. This system allows the user to have a more personalized experience and realize a more comfortable driving experience. Specific embodiments of the present invention are described below.

[1447] 1. User Interface Design

[1448] Users can input instructions through a conversational interface using voice or text. For example, they can say, "I want to go to a nice cafe nearby." The device then uses a speech recognition engine to convert the voice data into text.

[1449] 2. Introducing the Emotion Engine

[1450] The device implements an emotion engine that analyzes emotions from the user's voice and text input. The emotion engine recognizes the user's emotional state (e.g., joy, anger, sadness, or happiness) from the user's tone of voice and text expressions, and saves the analysis results.

[1451] 3. Obtaining real-time traffic information

[1452] The server periodically retrieves the latest traffic information from external sources (e.g., traffic information API). This information includes congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time.

[1453] 4. Obtaining ratings and business information for restaurants and tourist spots

[1454] The device retrieves a list of cafes in the specified area, along with their ratings and business information, from an external information service (e.g., a rating site API). Based on this information, the device creates a list of the cafes with the highest ratings.

[1455] 5. Personalized Route Suggestions

[1456] The server manages the user's past behavioral data and preferences as profile information. The device accesses this profile information and suggests optimized routes to the user. For example, it may prioritize cafes that users have given high ratings to in the past or places based on their current emotional state.

[1457] 6. Provision of entertainment features

[1458] Users can enjoy entertainment such as music playback, quizzes, and chat functions. The device receives requests such as "play the next song" and calls an external music service (e.g., music distribution API) to play music. In addition, an emotion engine analyzes the user's emotions and adjusts appropriate entertainment content.

[1459] 7. Safe driving support

[1460] The device monitors vehicle sensor information in real time and provides alerts when speed, distance, or improper driving is detected. In addition, an emotion engine adjusts the content and timing of alerts based on the user's emotional state.

[1461] 8. Creating a Character Chatbot

[1462] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile. The device then uses this character to engage in natural conversations with the user. Using generative AI technology, the character's expressions also change dynamically to reflect the user's emotional state.

[1463] 9. Sharing and selling characters

[1464] Users can obtain a dedicated sharing link from the server to share the chatbot characters they have created with other users. They can also sell chatbot characters through the marketplace. The server provides this functionality through the network.

[1465] Specific examples

[1466] As a concrete example, consider the case where a user verbally commands, "I want to go to a nearby, relaxing cafe." The device converts this command into text using a speech recognition engine and analyzes it using a natural language processing engine. Next, the emotion engine analyzes the user's tone of voice and recognizes that the user is a little tired. The device obtains cafe information from an external information service, and the server uses past behavioral data and emotion information to create a list of highly rated cafes that are relaxing. The device then suggests the user the optimal route to that cafe. While driving, the device plays relaxing music that matches the user's preferences. The device also obtains safety data from vehicle sensors and works with the emotion engine to provide the user with safety alerts at the appropriate time.

[1467] In this way, the present invention makes it possible to realize a system that allows the user to enjoy a more comfortable and safe driving experience.

[1468] The processing flow will be explained below.

[1469] Step 1:

[1470] The user speaks to the device, saying, "I want to go to a quiet cafe nearby." The device receives this instruction and uses a speech recognition engine to convert the voice data into text data.

[1471] Step 2:

[1472] The device passes the converted text data to a natural language processing engine, which extracts keywords such as "cafe," "nearby," and "calm."

[1473] Step 3:

[1474] The device uses an emotion engine to analyze the user's emotional state from their voice and determines that they are "tired." The emotion engine then stores this information in their profile.

[1475] Step 4:

[1476] The server sends a request to an external information source (e.g., a traffic information API) to obtain the latest traffic information, and stores the obtained traffic information in a database.

[1477] Step 5:

[1478] The terminal obtains a list of cafes in a specified area, along with their ratings and business information, from an external information service (e.g., a rating site API).

[1479] Step 6:

[1480] The device will use the cafe information obtained in step 5 to create a list of cafes that are highly rated and have a relaxing atmosphere.

[1481] Step 7:

[1482] The device uses the Google Maps API to calculate the optimal route based on the traffic information obtained in step 4 and the cafe information listed in step 6.

[1483] Step 8:

[1484] Based on the user's past behavioral data and the emotional profile obtained in step 3, the server selects the cafe that best suits the user's preferences and current emotional state, and then personalizes and suggests a route to the cafe.

[1485] Step 9:

[1486] The user checks the suggested cafe and route on the device and confirms, "I want to go there."

[1487] Step 10:

[1488] The user requests, "Play some relaxing music." The device sends a request to the music distribution API and plays relaxing music.

[1489] Step 11:

[1490] While driving, the device monitors vehicle sensor information to check speed and following distance, and provides alerts if improper driving is detected.

[1491] Step 12:

[1492] The device uses an emotion engine to monitor the user's emotional state and adjust the entertainment content and timing of safe driving alerts as needed.

[1493] Step 13:

[1494] The server uses generative AI technology to generate a chatbot with an original character based on the user's profile, and the device uses this character to interact with the user.

[1495] Step 14:

[1496] Users can share their chatbot characters with friends by obtaining a sharing link from the server. Characters can also be sold on the marketplace. The server manages transactions and generates sharing links.

[1497] In this way, users can operate intuitively and enjoy real-time information and personalized suggestions based on their emotional state, ensuring a comfortable and safe driving experience.

[1498] Example 2

[1499] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1500] Conventional car navigation systems simply provide routes to a destination and lack the ability to provide a personalized experience based on the user's emotional state and individual preferences. Furthermore, entertainment and safe driving support functions are limited, creating a need for an integrated solution to improve the overall driving experience. Furthermore, providing interactive character chatbots using generative AI technology and creating new user experiences based on them are also challenges.

[1501] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1502] In this invention, the server includes means for providing an interactive interface that allows the user to intuitively indicate places the user wants to go and points the user wants to stop at, means for acquiring traffic information from external information sources that is updated in real time and proposing optimal routes that avoid congestion, means for acquiring and providing area-specific ratings and business information from external information services, means for personalizing routes based on the user's past behavioral data and preferences, means for analyzing the user's emotional state from voice and text input, means for adjusting and providing entertainment functions according to the emotional state, means for supporting safe driving based on vehicle data and adjusting the content and timing of alerts based on the emotional state, and means for generating an original character chatbot based on the user's profile and dynamically changing and interacting with the chatbot. This enables optimal route suggestions, entertainment, and safe driving support based on the user's emotions and preferences, providing a more personalized and comfortable driving experience.

[1503] An "interactive interface" is an interface that allows a user to intuitively input instructions by voice or text, and the system returns an appropriate response to that input.

[1504] "External information sources" refer to information services and APIs that provide traffic information, traffic accident information, traffic congestion information, etc. from outside the system.

[1505] "Traffic Information" is real-time information about current road conditions, traffic congestion, traffic accidents, road closures, etc.

[1506] "Region-specific ratings" refers to user ratings and word-of-mouth information about places and facilities in a specific region.

[1507] "Business information" refers to information such as opening hours, regular holidays, and special opening days of restaurants, tourist attractions, etc.

[1508] "External information services" refer to services and APIs that provide evaluations and sales information.

[1509] "Personalizing your route" means suggesting the best route based on the user's past behavioral data and individual preferences.

[1510] "Emotional state" refers to the emotions such as joy, anger, sadness, and happiness that the user is currently feeling, and is analyzed from voice and text input.

[1511] "Entertainment features" are features that users can enjoy while driving, such as music playback, quizzes, and chat functions.

[1512] "Safe Driving Support" is a function that uses information from the vehicle's sensors to detect driving speed, vehicle distance, and inappropriate driving, and provides alerts to the user.

[1513] An "alert" is a notification that the system uses to warn or alert the user to encourage safe driving.

[1514] "Generative AI technology" is a technology that uses artificial intelligence to generate new content and characters.

[1515] "Profile Information" refers to information about a user, such as the user's past behavioral data, preferences, and emotional state.

[1516] A "character chatbot" is a virtual character created using generative AI technology that can engage in natural conversations with users.

[1517] "Dynamic change" means that the chatbot's behavior and responses change in real time depending on the user's emotional state and situation.

[1518] "Marketplace" means an online platform for sharing or selling user-generated chatbot characters.

[1519] This invention is a car navigation system that features an interactive interface that allows users to intuitively indicate where they want to go and where they want to stop, and provides real-time traffic information, emotion analysis, personalized route suggestions, entertainment functions, safe driving support, and a character chatbot that utilizes generative AI technology.

[1520] User Interface Design

[1521] Users input instructions to the system through voice or text. For example, if a user says, "I want to go to a nice cafe nearby," the device uses a speech recognition engine (e.g., Google Speech-to-Text) to convert the voice into text data. Then, a natural language processing engine (e.g., OpenAI GPT) analyzes this text data.

[1522] Introducing the Emotion Engine

[1523] The device implements an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the user's emotional state from voice and text. This engine analyzes the user's emotional state (e.g., joy, anger, sadness, or happiness) from the user's tone of voice and text expressions and saves the results.

[1524] Obtaining real-time traffic information

[1525] The server periodically retrieves the latest traffic information from external sources (e.g., Traffic API), including information on congestion, accidents, road closures, etc. The retrieved traffic information is stored in a database and updated in real time.

[1526] Obtaining ratings and business information for restaurants and tourist spots

[1527] The device uses external information services (e.g., Yelp API) that provide location-specific ratings and business information to obtain a list of cafes and their ratings and business information, allowing it to present the location that best suits the user's request.

[1528] Personalized route suggestions

[1529] The server manages the user's past behavioral data and preferences as profile information. The device then uses this profile information to suggest the most suitable route for the user. Specifically, it prioritizes cafes that users have given high ratings to in the past and places that correspond to their current emotional state.

[1530] Providing entertainment features

[1531] Users can enjoy entertainment such as music playback, quizzes, and chat functions. The device receives requests such as "play the next song" and calls an external music service (e.g., Spotify API) to play music. In addition, an emotion engine analyzes the user's emotions and adjusts appropriate entertainment content.

[1532] Safe driving support

[1533] The device monitors vehicle sensor information in real time and provides alerts when speed, distance, or improper driving is detected. The emotion engine adjusts the content and timing of alerts based on the user's emotional state.

[1534] Character chatbot generation

[1535] The server uses generative AI technology (e.g., OpenAI GPT) to generate a chatbot with an original character based on the user's profile. The device then uses this character to engage in natural conversations with the user. Generative AI technology dynamically changes the character's expressions according to the user's emotional state.

[1536] Sharing and selling characters

[1537] Users can obtain a dedicated sharing link from the server to share their chatbot characters with other users, and can also sell chatbot characters through the marketplace.

[1538] Specific examples

[1539] Consider a scenario where a user verbally commands, "I want to go to a nearby relaxing cafe." The device converts this command into text using a speech recognition engine (e.g., Google Speech-to-Text) and analyzes it using a natural language processing engine (e.g., OpenAI GPT). Next, an emotion engine (e.g., IBM Watson Tone Analyzer) analyzes the user's tone of voice and recognizes that the user is slightly tired. The device obtains cafe information from an external information service (e.g., Yelp API), and the server uses past behavioral data and emotion information to create a list of highly rated cafes that are relaxing. The device then suggests the user the optimal route to the cafe. While driving, the device plays relaxing music using an external music service (e.g., Spotify API). The device also obtains safety data from vehicle sensors and, in cooperation with the emotion engine, provides the user with safety alerts at the appropriate time.

[1540] Prompt Sentence Examples

[1541] "If a user says, 'I want to go to a quiet cafe nearby,' what kind of processing and data is required?"

[1542] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1543] Step 1:

[1544] The user launches the device's conversational interface. The user inputs a command by voice or text, such as "I want to go to a nearby delicious cafe." In this case, the input is voice data, which is passed to the device. Specifically, the user performs input operations using a microphone or keyboard.

[1545] Step 2:

[1546] The device uses a speech recognition engine (e.g., a speech recognition engine API) to convert the user's voice data into text data. The output is text data, which is used as input data for the next processing step. The specific operation involves the process in which the speech recognition engine analyzes the voice waveform and converts it into corresponding text.

[1547] Step 3:

[1548] The device passes the converted text data to a natural language processing engine (e.g., natural language processing API) and analyzes the user's instructions. The input is text data, and the output is the intent data resulting from the analysis. For example, the intent extracted is "I want to go to a nearby delicious cafe." Specifically, the natural language processing engine understands the context of the text and concludes on the user's request.

[1549] Step 4:

[1550] The device uses an emotion engine (e.g., emotion analysis API) to analyze the user's emotional state from text data and voice tone. The input is text data and voice tone data, and the output is emotional state data. For example, it may recognize that the user is slightly tired. Specifically, the emotion engine analyzes the intonation of the voice and the tone of the text to determine the emotional state.

[1551] Step 5:

[1552] The server periodically obtains the latest traffic information from an external information source (e.g., a traffic information API). The input is an API request, and the output is traffic information data. Specifically, the server sends an API request and receives real-time data such as congestion and accident information.

[1553] Step 6:

[1554] The device obtains cafe ratings and business information from an external information service (e.g., a rating site API). The input is an API request, and the output is a list of cafes and their rating data. Specific operations include sending an API request, parsing the results, and generating a list of cafes.

[1555] Step 7:

[1556] The device accesses the user's profile information (e.g., past behavioral data, preferences) stored on the server and proposes the optimal route. The input is the profile data, and the output is optimal route guidance data. Specifically, the server evaluates candidate locations based on the past behavioral data and the user's preferences, and generates the optimal route.

[1557] Step 8:

[1558] When a user requests "play the next song," the device calls an external music service (e.g., a music distribution API) to play music. The input is the user's voice request, and the output is playback instruction data. Specifically, the device sends an API request to the music distribution service and streams the music data.

[1559] Step 9:

[1560] The device monitors vehicle sensor information in real time and provides alerts to support safe driving. The input is sensor information and the output is alert data. Specific operations include analyzing sensor information to evaluate speed and vehicle distance and detect inappropriate driving.

[1561] Step 10:

[1562] The server uses generative AI technology (e.g., generative AI model) to generate an original character chatbot based on the user's profile. The input is profile information, and the output is chatbot data. Specifically, the server uses generative AI technology to simulate the character's dialogue.

[1563] Step 11:

[1564] The server generates a dedicated sharing link so that users can share their chatbot characters with other users. The input is the chatbot data, and the output is the sharing link. Specifically, the server registers the chatbot data on the sharing platform and generates the link.

[1565] (Application example 2)

[1566] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1567] Conventional car navigation systems only provide simple route guidance without considering the user's emotions or entertainment needs. This makes it difficult to provide a comfortable and personalized driving experience for users. Furthermore, their safe driving support features are limited, and they do not provide appropriate alerts or entertainment tailored to the user's emotional state. This makes it difficult to reduce user stress and discomfort, and furthermore, ensures safe driving. The present invention aims to solve these problems of the conventional technology and provide users with a personalized, comfortable driving experience and safe driving support.

[1568] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes a means for analyzing the user's emotional state and proposing entertainment content and routes based on the emotion, a means for monitoring the vehicle's sensor information in real time and adjusting the content and timing of alerts based on the user's emotional state, and a means for acquiring and proposing area-specific reviews and business information from external information services. This enables personalized entertainment and route suggestions that take the user's emotional state into consideration. Furthermore, safety alerts can be provided at appropriate times according to the user's emotional state, resulting in a comfortable and safe driving experience.

[1569] An "interactive interface" is an input method that allows users to intuitively specify their destination or points of interest using voice or text.

[1570] "External information sources updated in real time" refers to information provision services that provide traffic information, rating information, etc. with the latest data over time.

[1571] "Means of obtaining traffic information and proposing the optimal route to avoid congestion" refers to a function that imports traffic data in real time from external traffic information services and, based on that data, presents users with the optimal route to avoid congestion.

[1572] "Means for obtaining and providing region-specific ratings and business information from external information services" refers to a function that obtains ratings and business information on restaurants, tourist attractions, etc. related to a specified region from external information services and provides it to users.

[1573] "Means to personalize routes based on the user's past behavioral data and preferences" is a function that suggests individualized routes based on information such as the user's past travel history, favorite categories, and preferred locations.

[1574] "Means for providing entertainment functions" refers to functions that provide users with entertainment content such as music, quizzes, and chat.

[1575] "Means to support safe driving based on vehicle data" refers to a function that provides users with alerts and notifications to encourage safe driving based on data obtained from sensors, cameras, etc. installed in the vehicle.

[1576] "Means for analyzing the user's emotional state and suggesting entertainment content and routes based on that emotion" refers to a function that analyzes the user's emotional state (e.g., happy, tired) from voice and text data, and personalizes entertainment content and routes based on the analysis results.

[1577] "Means for monitoring vehicle sensor information in real time and adjusting the content and timing of alerts based on the user's emotional state" refers to a function that monitors data obtained in real time from various sensors installed in the vehicle and optimizes the content and timing of alerts according to the user's emotional state.

[1578] The system of the present invention includes an interactive interface that allows users to intuitively indicate where they want to go and where they want to stop, routes that are suggested based on traffic information obtained from external sources that is updated in real time, personalized entertainment content and route suggestions based on the user's emotional state, and functions that support safe driving.

[1579] 1. User Interface Design

[1580] A conversational interface uses a speech recognition engine (e.g., Google Speech Recognition API) to convert the user's speech into text. When a user speaks a command such as "I want to go to a nice cafe nearby," the speech data is converted into text by the recognition engine.

[1581] 2. Introducing the Emotion Engine

[1582] Using an emotion engine (e.g., EmotionRecognizer), the emotional state of the user is analyzed from voice and text, allowing accurate understanding of the user's emotional state, such as whether they are tired or happy.

[1583] 3. Obtaining real-time traffic information

[1584] The server periodically obtains the latest traffic information using a traffic information API (e.g., a general traffic information API), thereby obtaining real-time information on traffic congestion and accidents and storing it in a database.

[1585] 4. Obtaining ratings and business information for restaurants and tourist spots

[1586] The device retrieves a list of cafes in the specified area, along with their ratings and business information, from an external information service (e.g., a rating site API). Based on this information, the device creates a list of the cafes with the highest ratings.

[1587] 5. Personalized Route Suggestions

[1588] The server manages the user's past behavioral data and preferences (favorite restaurant trends and records of places visited) as profile information. The device then suggests the optimal route based on this profile information and the results of the emotion engine's analysis.

[1589] 6. Provision of entertainment features

[1590] The device provides features such as music playback (using external music service APIs), quizzes, and chat functions. The emotion engine analyzes the user's emotions and dynamically adjusts the entertainment content to suit their needs.

[1591] 7. Safe driving support

[1592] The device monitors vehicle sensor information in real time to support safe driving, and improves safety by adjusting the content and timing of safety alerts based on the user's emotional state.

[1593] 8. Creating a Character Chatbot

[1594] The server uses generative AI technology (e.g., generative AI model) to generate a chatbot with an original character based on the user's profile. The device then uses this character to engage in natural conversations with the user.

[1595] 9. Sharing and selling characters

[1596] Users can obtain a dedicated sharing link from the server to share their chatbot characters with other users, and can also sell their characters through the marketplace.

[1597] Specific examples

[1598] For example, if a user says, "I want to go to a nearby relaxing cafe," the device will convert this into text using a speech recognition engine and analyze the user's emotional state using an emotion engine. If the device detects that the user is tired, it will retrieve cafe information from a rating site API and create a list of highly rated cafes that are relaxing and calming based on past behavioral data and emotional information. It will then suggest the optimal route to the cafe to the user and play relaxing music. It will also retrieve safety data from vehicle sensors and link with the emotion engine to provide safety alerts at the appropriate time.

[1599] Prompt Sentence Examples

[1600] "Based on the invention below, you will design an emotional car navigation app for self-driving cars. The app should include a function that allows the user to specify a destination by voice, and the emotion engine analyzes the user's emotions, suggests the optimal route, and provides entertainment."

[1601] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1602] Step 1:

[1603] The user inputs instructions by voice. The user may say, "I want to go to a nice cafe nearby." This voice input becomes the input to the system.

[1604] Step 2:

[1605] The device uses a speech recognition engine (e.g., Google Speech Recognition API) to convert the voice data into text data. The speech recognition engine analyzes the voice data and outputs it as text data.

[1606] Step 3:

[1607] The device uses an emotion engine (e.g., EmotionRecognizer) to analyze the user's emotional state from text and voice data, and outputs the emotional state (e.g., tired, happy) obtained through this analysis.

[1608] Step 4:

[1609] The server uses a traffic information API (e.g., a general traffic information API) to obtain the latest traffic information in real time. The obtained traffic information is stored in a database and becomes input data for the system.

[1610] Step 5:

[1611] The device uses an external information service (e.g., a rating site API) to obtain a list of cafes in the specified area, along with their ratings and business information. This data is input, and a list of highly rated cafes is output.

[1612] Step 6:

[1613] The server retrieves the user's past behavioral data and preferences (e.g., a list of cafes visited and their ratings) from the database as profile information. The retrieved profile information becomes input data for the system.

[1614] Step 7:

[1615] The device then proposes the optimal route based on the acquired profile information, the results of the emotion engine analysis, and traffic and cafe information. The device analyzes this data and outputs a personalized route for the user.

[1616] Step 8:

[1617] The device provides entertainment functions. Based on the analysis results of the emotion engine, it dynamically adjusts and provides music, quizzes, and chat functions that suit the user. This function outputs entertainment content optimized for the user's emotional state.

[1618] Step 9:

[1619] The device monitors the vehicle's sensor information in real time and adjusts the content and timing of safety alerts according to the user's emotional state. Alerts are output at appropriate times based on the sensor information and the user's emotional state.

[1620] Step 10:

[1621] The server uses generative AI technology (e.g., generative AI model) to generate a chatbot with an original character based on the user's profile. The generated character is used as an output to the system to interact with the user.

[1622] Step 11:

[1623] Users can share the generated character chatbots with other users or sell them on the marketplace. The server generates a sharing link and provides a sales function in cooperation with the marketplace. This function is provided as an output to the user.

[1624] Adding specific examples

[1625] For example, if a user says, "I want to go to a nearby relaxing cafe," the system converts the speech to text and then uses its emotion engine to interpret it as "tired." It then retrieves cafe information from a review site API and, taking into account the user's past review data, creates a list of the most suitable cafes and suggests them to the user along with a route. It also plays relaxing music and adjusts safety alerts while driving based on the user's emotional state.

[1626] Prompt Sentence Examples

[1627] "Based on the invention below, you will design an emotional car navigation app for self-driving cars. The app should include a function that allows the user to specify a destination by voice, and the emotion engine analyzes the user's emotions, suggests the optimal route, and provides entertainment."

[1628] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1629] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1630] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1631] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1632] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1633] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1634] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1635] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1636] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1637] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1638] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1639] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1640] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1641] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1642] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1643] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1644] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1645] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way,...

Claims

1. A means for providing an interactive interface that allows users to intuitively indicate where they want to go or what points they want to stop at; A means of obtaining traffic information from external sources that is updated in real time and suggesting the best route to avoid congestion; A means for obtaining and providing region-specific evaluations and sales information from external information services; A way to personalize routes based on users' past behavioral data and preferences; a means for providing entertainment functionality; A system including means for supporting safe driving based on vehicle data.

2. The system of claim 1 further comprising means for generating an original character chatbot using generative artificial intelligence technology and having the chatbot interact with a user.

3. 10. The system of claim 1, further comprising means for enabling users to share or sell their created chatbot characters.

Images