system

Abstract

We provide the system. [Solution] Means for collecting and analyzing traffic data, weather data, and user profile data, A means for generating multiple candidate paths based on analyzed data and determining the optimal path according to evaluation criteria, A means of filtering and presenting information on places to visit based on user requests, A means for interpreting user input using speech recognition and performing the desired operation, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional car navigation system, it has been a problem that it cannot flexibly respond to real-time traffic conditions and weather changes, and the proposal of a route and detour spots according to the user's preference is insufficient. Furthermore, there has also been a problem that it is difficult to perform an operation considering safety during driving. These problems have increased the driver's stress and impaired the pleasure of driving.

Means for Solving the Problems

[0005] This invention solves the above problems by providing a navigation system using AI and voice recognition technology. This system has means for collecting and analyzing traffic condition data, weather data, and user profile data, and proposes the optimal route in real time in response to changing conditions. It also has a function to filter and present detour spots based on the user's preferences. Furthermore, it enables safe operation while driving using voice recognition functionality and can interpret user input to perform desired operations.

[0006] "Traffic condition data" refers to data that shows the current road conditions, such as traffic flow, congestion information, vehicle speed, and density.

[0007] "Weather data" refers to data about current and expected weather conditions, including meteorological conditions such as rainfall, temperature, and wind speed.

[0008] "User profile data" refers to data that includes information specific to each individual user, such as their past behavioral history, preferences, and settings.

[0009] An "AI algorithm" is a set of computational procedures and methods that use artificial intelligence, and is used for data analysis and the generation of predictive models.

[0010] A "candidate route" refers to multiple possible route patterns for reaching a destination, and represents the options suggested to the user.

[0011] "Detour spot information" refers to information about places you can stop at along the way, such as tourist attractions, restaurants, and rest areas, in addition to your original destination.

[0012] "Voice recognition functionality" is a technology that recognizes a user's speech as voice input and interprets it as a command. [Brief explanation of the drawing]

[0013] [Figure 1]This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] The car navigation system of the present invention functions by having a server collect traffic condition data, weather data, and user profile data, and analyzing this data using an AI algorithm. Based on these analysis results, the server generates multiple candidate routes and evaluates each route. This evaluation includes factors such as time, distance, user preferences, presence or absence of congestion, and road conditions. The system then delivers the optimal route to the user terminal, taking these factors into consideration.

[0035] The user terminal displays the received route information as a navigation display. Furthermore, it retrieves information on potential detours from the server and suggests spots that might interest the user. This includes specific tourist attractions, restaurants, and rest areas. For example, if the user sets their interest to "historical buildings," the system will list suitable detour spots.

[0036] While driving, users can interact with the system through voice recognition. If a user issues a voice command such as, "I want to try turning left instead of right," the user terminal sends the instruction to the server and requests that a new route be calculated. The server immediately sends the newly calculated route to the user terminal, enabling a quick route change while maintaining driving safety.

[0037] In this way, the present invention provides a personalized driving experience tailored to the individual needs of the user, guiding the user safely and comfortably to their destination.

[0038] The following describes the processing flow.

[0039] Step 1:

[0040] The server collects real-time data from sources such as traffic information providers, satellite data services, and weather information sources. This includes current traffic congestion, road construction information, and weather conditions.

[0041] Step 2:

[0042] The server uses the collected data to run AI algorithms and analyzes it in comparison with user profile data. This profile data includes the user's past route choices and preferences, forming the basis of the analysis.

[0043] Step 3:

[0044] The server generates multiple candidate routes based on the analysis results. Each candidate route is evaluated based on factors such as traffic conditions, estimated travel time, and scenic beauty.

[0045] Step 4:

[0046] The server selects the optimal route from the evaluated routes and sends that route information to the user's terminal.

[0047] Step 5:

[0048] The user terminal displays the received route information on the navigation screen, allowing the user to refer to it while driving. It also provides route guidance via voice prompts.

[0049] Step 6:

[0050] The server retrieves information on potential stops from a database and creates a list of suggested spots based on the user's settings and past activity history. This list includes highly-rated tourist attractions and restaurants.

[0051] Step 7:

[0052] Users can give instructions to the system regarding routes and detours via a voice assistant. The user's voice commands are processed by being converted to text on the user's terminal and sent to the server.

[0053] Step 8:

[0054] The server receives the voice command, recalculates the route as needed, and resends the latest route guidance to the user.

[0055] In this way, the entire system works together to provide users with a consistent and personalized navigation experience.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] This invention aims to solve the complex route selection problems faced by users in modern transportation systems. In particular, there is a need to provide optimal routes that take into account traffic congestion, weather changes, and individual user preferences. Furthermore, there is a lack of flexible navigation systems that allow users to operate the system and change routes via voice while ensuring safety during travel.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for acquiring and analyzing environmental information, weather information, and user attribute information; means for generating multiple alternative routes based on the analyzed information and selecting the optimal route according to indicators; and means for interpreting user instructions and performing desired operations using a voice input function. This enables users to select the optimal and safe route according to dynamically changing traffic conditions and personal preferences.

[0061] "Environmental information" refers to general information related to road conditions, such as road congestion, traffic restrictions, and road construction information.

[0062] "Weather information" refers to information about weather conditions such as rain, snow, wind, and temperature, including the impact of these factors on traffic conditions and movement.

[0063] "User attribute information" refers to information about users' preferences, past selection history, and destinations, and is used to provide personalized services to individual users.

[0064] An "alternative route" refers to a route suggested from among several possible routes to the destination, depending on the circumstances.

[0065] An "indicator" refers to a set of criteria used to evaluate a route, such as distance, travel time, road conditions, and user preferences.

[0066] "Voice input functionality" is a technology that allows a user's voice to be input as data, enabling the system to recognize that voice and perform operations.

[0067] "Desired operation" refers to requests made by the user to the system, such as changing the route or resetting the destination, and the system's response to these requests.

[0068] This invention relates to a traffic navigation system in which a server and a terminal work together to provide route guidance optimized for the user.

[0069] The server utilizes various APIs to collect environmental, weather, and user attribute information via the internet. Specifically, it obtains road congestion and regulation information through general map service APIs, and weather information is obtained using weather information service APIs. Based on this information, the server analyzes the data using Python and machine learning frameworks. For example, Scikit-Learn's machine learning algorithms are used to predict past traffic patterns.

[0070] The server generates alternative routes based on the analysis results and evaluates them based on distance, time, and user preferences. Based on this, the server selects the optimal route and sends the selected route information to the terminal.

[0071] The terminal displays route information received from the server on the navigation screen, presenting it in a user-friendly format for drivers. The terminal also sends requests to the server to retrieve relevant location information based on the user's preferences. These relevant locations may include tourist attractions or restaurants, and the system uses an API similar to Foursquare to collect location and associated facility information.

[0072] Users can easily give instructions such as changing routes or obtaining additional information through the terminal's voice input function. The voice input uses a common speech recognition service to provide accurate analysis and responses.

[0073] For example, if a user gives a voice command such as "Tell me a route that goes through a scenic area," the server will analyze and select a route that meets that request, and the terminal will display the result to the user. An example of a prompt would be, "Suggest the best route for a user who prefers scenic routes. The starting point is a major city, the destination is a tourist spot, and the detours should include natural landscapes."

[0074] Through this system, users can enjoy a flexible, safe, and comfortable driving experience tailored to their individual needs.

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] The server collects environmental information, weather information, and user attribute information. This data becomes input to the server, and data is retrieved using traffic information APIs and weather information APIs. Specifically, API requests are issued and the data is received in JSON format. This is stored in a temporary database inside the server and used in the next processing step.

[0078] Step 2:

[0079] The server performs data analysis based on the collected data. The input is the information obtained in Step 1, which is then analyzed by a Python program. Specifically, it uses a machine learning algorithm to analyze traffic patterns and predicts suitable routes based on user profiles. As a result of the calculations, multiple alternative routes are generated, which become the input for the next step.

[0080] Step 3:

[0081] The server evaluates the generated alternative routes. The input is the set of routes generated in step 2, and the evaluation criteria include distance, travel time, and user preference. Each alternative route is scored, and the route with the highest score is selected as the output. This is then sent to the terminal.

[0082] Step 4:

[0083] The terminal receives optimal route information from the server. This route information is input to the terminal, which then displays it on the navigation screen. Specifically, the display method involves providing real-time route guidance using a map application.

[0084] Step 5:

[0085] The terminal sends a request for relevant location information to the server. The input includes user preference information, which the server uses to search for relevant locations and outputs the results to the terminal. This includes providing a list of suggested locations and suggesting potential stops for the user.

[0086] Step 6:

[0087] The user uses the terminal's voice input function to instruct the terminal to change the route. Voice data is provided to the terminal as input, and the terminal converts it into text data through a speech recognition algorithm. The converted data is sent to the server, which calculates the new optimal route and outputs the result back to the terminal.

[0088] Overall, the system processes a large amount of dynamically changing data and provides users with appropriate route guidance, thereby creating a comfortable and safe driving experience.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] Modern autonomous vehicles require advanced navigation systems to ensure safe and efficient travel. However, conventional systems fail to fully utilize real-time traffic information and individual user preferences, making it difficult to provide optimal route guidance. Furthermore, they lack personalized suggestions for detours that take user preferences into account. As a result, there is a challenge in that the convenience of autonomous vehicles cannot be fully realized.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes means for collecting and analyzing traffic information, weather information, and user profile information; means for generating multiple candidate routes based on the analyzed information and determining the optimal route according to evaluation criteria; and means for filtering and presenting detour point information to the user upon request. This makes it possible to provide safe and efficient route guidance in autonomous vehicles, as well as to provide personalized suggestions of detour points according to the user's preferences.

[0094] "Traffic information" refers to information that shows real-time traffic conditions, such as the degree of congestion on roads, the occurrence of accidents, and road closures.

[0095] "Weather information" refers to data that indicates the weather conditions on and around roads, and includes temperature, precipitation, wind speed, etc.

[0096] "User profile information" refers to individual information about the user of the navigation system, including age, hobbies, and past driving history.

[0097] A "candidate route" refers to a list of multiple possible travel routes from the starting point to the destination.

[0098] "Evaluation criteria" are standards used to compare candidate routes and determine which route is optimal, and these include factors such as time, distance, and traffic congestion.

[0099] "Detour information" refers to information about tourist spots, restaurants, rest areas, and other places you can stop at on your way to your destination.

[0100] "Voice recognition functionality" is a technology that converts a user's speech into digital signals, analyzes the content, and understands its meaning.

[0101] An "autonomous vehicle" is a vehicle that, thanks to its installed technology, can drive autonomously without driver intervention.

[0102] "Real-time updates" means that information is continuously updated to the latest version.

[0103] A "voice assistant" is software that recognizes voice commands and performs tasks based on their content.

[0104] The system for implementing this invention mainly consists of a server, a user terminal, and an autonomous vehicle. The server is responsible for collecting traffic information, weather information, and user profile information, and analyzing this data using AI to provide the user with the optimal driving route. This analysis utilizes data acquisition services such as Google® Maps API and OpenWeatherMap API, as well as machine learning frameworks such as TENSORFLOW®.

[0105] Based on the analysis results, the server generates multiple candidate routes and evaluates them according to evaluation criteria. These criteria include route distance, time, user preferences, and road conditions. After the optimal route is determined, information about it is sent to the user's terminal.

[0106] The user terminal displays route information received from the server through a navigation system. This navigation system incorporates speech recognition functionality using the Google Cloud Speech API. When a user issues a voice command, the system converts the command into text data, sends it to the server, and performs further analysis.

[0107] For example, if a user requests by voice, "I want to stop by a delicious ramen shop along the way," the system will search for potential locations near their current location in real time and suggest the optimal route for stopping there.

[0108] By using generative AI models, the system can provide more accurate route predictions and suggestions. Examples of prompts include "Suggest ramen spots along the drive from Tokyo to Osaka" and "Recalculate the route; I want to turn left instead of right."

[0109] In this way, users can achieve safe and efficient travel via autonomous vehicles.

[0110] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0111] Step 1:

[0112] The server obtains traffic and weather information from their respective APIs (e.g., Google Maps API and OpenWeatherMap API). The input is the user's current location and destination. The server uses this information to perform real-time data analysis and visualize traffic flow and weather changes.

[0113] Step 2:

[0114] The server receives user profile information from the user's terminal. The input consists of user preference information, including the user's past driving history and preferred detour spots. The server analyzes this information and uses it to generate route recommendation models optimized for each individual user.

[0115] Step 3:

[0116] The server generates multiple candidate routes based on all the information it acquires. The data used includes traffic conditions, weather information, and user profile information. The output is a route list that includes a recommendation index for each route. An AI algorithm is used to perform data calculations that take into account time, distance, weather, and user preferences.

[0117] Step 4:

[0118] The server evaluates the generated candidate routes according to evaluation criteria and determines the optimal route. The evaluation criteria are based on user-defined priorities (e.g., shortest time, shortest distance, interest level of detour spots, etc.). The output is the route determined to be optimal.

[0119] Step 5:

[0120] The user terminal displays the optimal route received from the server on the navigation system. The input is the optimal route information from the server. The output is the route drawn on the map and its detailed information (e.g., information about the next intersection to turn at and suggestions for places to stop along the way). The user receives navigation support through visual and voice guidance.

[0121] Step 6:

[0122] Users can give instructions to the system through voice recognition. For example, they might request, "I want to stop at a ramen shop on the way." The input is a voice command, which the system converts into text data and sends to the server.

[0123] Step 7:

[0124] The server re-analyzes the data based on the voice command and dynamically recalculates the route. The input is a new request based on the voice command. The output is updated route information. A generative AI model is used to quickly generate a route that includes new detour options.

[0125] In this way, the system can provide users with safe and efficient routes.

[0126] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0127] The car navigation system incorporating the emotion engine of the present invention provides a more personalized navigation experience by having a server acquire traffic data, weather data, and user profile data, and further analyze emotion data. The server uses various sensors and cameras to analyze the user's voice, facial expressions, and body language to understand the user's emotional state.

[0128] Based on this emotion recognition result, the server generates optimal candidate routes and selects the route that best suits the user's mood. For example, if the user is feeling stressed, the navigation system can prioritize suggesting quieter routes or routes that offer a sense of nature. Similarly, when suggesting detours, the system filters and presents appropriate spots according to the user's current emotions. For example, if the system determines that the user is tired, it will recommend places suitable for rest, such as quiet cafes or parks.

[0129] While driving, users can use a voice assistant to request route changes or add detours. The voice assistant takes into account the output of its emotion engine and adjusts the tone and content of its responses according to the user's emotions. For example, if the user is angry, the voice assistant will speak in a calm tone and offer suggestions to facilitate problem resolution.

[0130] All information is displayed on the user's terminal and provided to the user visually and audibly through voice feedback. This allows the present invention to provide drivers with an emotionally conscious, safe, and comfortable driving experience, reducing stress and enhancing the enjoyment of driving.

[0131] The following describes the processing flow.

[0132] Step 1:

[0133] The server retrieves relevant data from traffic information providers, weather data services, and user profile databases. This allows it to collect information about current traffic conditions, weather conditions, and user preferences.

[0134] Step 2:

[0135] The server captures the user's voice, facial expressions, and body language from sensors such as cameras and microphones installed inside the vehicle. The emotion engine analyzes this data in real time to estimate the user's emotional state.

[0136] Step 3:

[0137] The server inputs collected sentiment data and traffic / weather data into an AI algorithm for analysis. This generates multiple candidate routes that are best suited to the user's current emotions.

[0138] Step 4:

[0139] The server selects the route that best suits the user's emotional state from the generated routes, according to evaluation criteria. These criteria include travel time, the pleasantness of the scenery, and weighting based on the user's preferences.

[0140] Step 5:

[0141] The user terminal displays the optimal route information received from the server on the navigation screen. It also provides route guidance and instructions to the user via voice guidance.

[0142] Step 6:

[0143] The server filters information on potential detour spots from its database based on the user's emotional state and presents relaxing spots and places suitable for a change of pace to the user's terminal.

[0144] Step 7:

[0145] The user gives instructions regarding route changes or detours through a voice assistant. The device uses voice recognition to convert the commands into text and sends the content to the server.

[0146] Step 8:

[0147] The server calculates a new route based on the user's voice commands and emotional state, and redistributes the updated route information to the user's terminal. During this process, the voice assistant's response is adjusted according to the user's emotions, providing appropriate feedback.

[0148] (Example 2)

[0149] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0150] Conventional car navigation systems primarily select routes based on external conditions such as traffic and weather, and do not offer personalized route suggestions that take into account the user's emotional state. As a result, it has been difficult to provide optimal routes for drivers who are stressed or fatigued. This invention aims to improve driving comfort and safety by providing navigation that takes the user's emotional state into consideration.

[0151] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0152] In this invention, the server includes means for collecting and analyzing traffic information, weather information, and user profile information; means for analyzing the user's voice, facial expressions, and body language using sensors and cameras to evaluate the user's emotional state; and means for adjusting the voice assistant's response according to the user's emotions using a generative AI model. This enables the suggestion of an optimal route that takes the user's emotional state into consideration and the filtering of detour spots.

[0153] "Traffic information" is a general term for data related to traffic, such as the flow of vehicles on roads, congestion levels, and road closures.

[0154] "Weather information" refers to data that shows the weather conditions at any given time, and specifically includes temperature, precipitation, wind speed, and visibility information.

[0155] "User profile information" refers to personal information about a specific user, including past driving history and individual preferences.

[0156] "Emotional state" refers to the user's current psychological state and includes emotions such as joy, anger, sadness, and stress.

[0157] "Candidate routes" refer to multiple possible route options from the starting point to the destination.

[0158] "Evaluation criteria" are a set of indicators used to evaluate each route option, based on distance, time, scenery, and the user's emotional state.

[0159] An "emotion engine" is an algorithm or system for detecting and analyzing a user's emotional state.

[0160] "Detour spot information" refers to information about places that users can visit along the way, including tourist attractions, restaurants, cafes, and rest facilities.

[0161] "Voice recognition functionality" refers to technology that processes a user's speech as digital data and interprets the intended command.

[0162] A "generative AI model" refers to an artificial intelligence algorithm that learns from large amounts of data and performs predictions and classifications.

[0163] In this invention, the system mainly consists of three elements: a server, a terminal, and a user. The server first accesses multiple data sources to collect traffic information, weather information, and user profile information. Specifically, it obtains this information by using a traffic information API, a weather information API, and a profile database. User profile information includes past driving history and individual preferences.

[0164] The server uses voice sensors and cameras installed in the vehicle to analyze the user's voice tone, facial expressions, and body language in real time. This allows it to determine the user's current emotional state and, using a generative AI model, suggest the optimal route tailored to the user's emotions. For example, if the user is seeking relaxation, the server will prioritize suggesting quiet routes.

[0165] The terminal's role is to notify the user of information provided by the server through the in-car display and voice output. The voice assistant dynamically adjusts its tone and content according to the user's emotional state. Specifically, it provides guidance in a calm tone to users who are feeling stressed.

[0166] Users can request route changes or detours via voice input using voice recognition technology. The server receives these requests, uses a generative AI model to quickly calculate new routes, and provides navigation tailored to the user's preferences.

[0167] For example, if a user requests to "add a quiet cafe nearby to the route," the server will quickly gather information, consider the user's current emotional state, and then suggest the most suitable cafe options.

[0168] An example of a prompt message might be, "If the user is feeling stressed, please suggest a place where they can feel relaxed." This allows the system to provide the most appropriate information for the user.

[0169] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0170] Step 1:

[0171] The server collects traffic information, weather information, and user profile information from various sources. It calls APIs to retrieve the latest traffic and weather data. Input data includes GPS location information, weather APIs, and a profile database. The server analyzes this data to obtain outputs such as traffic delay information and weather patterns.

[0172] Step 2:

[0173] The server uses in-vehicle sensors and cameras to collect and analyze the user's voice, facial expressions, and body language in real time. Voice data and image data are used as input. A generative AI model is used to estimate the user's current emotional state. The output is an emotional state (e.g., stress, relief, excitement).

[0174] Step 3:

[0175] The server generates multiple candidate routes based on collected data and emotional states. Each route is evaluated using evaluation criteria according to the user's preferences and emotions. The input data consists of the outputs from steps 1 and 2. Using a generative AI model, the server outputs the most suitable route and sends it to the terminal.

[0176] Step 4:

[0177] The terminal presents route information received from the server to the user via an in-vehicle display and audio output. The input is route information from the server. The terminal converts this information into a human-readable format and outputs it, for example, displaying the route in map format on the display and providing voice guidance.

[0178] Step 5:

[0179] Users request route changes or detours from the system via voice input. The voice recognition function receives this input and processes the data for transmission to the server. The output is the request itself (e.g., "Please add nearby cafes").

[0180] Step 6:

[0181] The server receives a request from the user, collects new information, and re-evaluates the route. Here, a generative AI model is used to calculate the optimal route based on the new conditions in real time and send it back to the terminal. The output is the updated route information.

[0182] Step 7:

[0183] The terminal re-presents the updated route information to the user, and the voice assistant provides guidance in an appropriate tone and content. The input is the newly suggested route information. The terminal immediately reflects this information and outputs it in a way that reassures the user.

[0184] (Application Example 2)

[0185] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0186] In autonomous vehicles, conventional navigation systems have a problem: they only optimize routes based on traffic conditions and weather, without considering the user's emotional state. This tends to result in a uniform user experience, particularly limiting improvements in comfort and safety. Therefore, there is a need for methods to provide personalized route selection and ride experiences that take into account the individual user's emotional state.

[0187] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0188] In this invention, the server includes means for collecting and analyzing traffic condition data, weather data, and user profile data; means for generating multiple candidate routes based on the analyzed data and determining the optimal route according to evaluation criteria; and means for analyzing the user's emotional state using an emotion engine and adjusting the route and ride experience based on the emotional state. This enables safer and more comfortable navigation and ride experience optimized for the user's emotional state.

[0189] "Traffic condition data" refers to information about traffic flow and conditions, such as road congestion and accident information.

[0190] "Weather data" refers to information that shows weather conditions in a specific region, such as temperature, precipitation, and wind speed.

[0191] "User profile data" refers to data that includes personal information about individual users, such as their age, gender, past behavioral history, and preferences.

[0192] "Means of analysis" refer to processing methods and technical means for collecting data and using it to derive useful information.

[0193] "Candidate routes" refer to multiple route options for reaching a destination, and are evaluated to determine the priority of each route.

[0194] The "emotion engine" is a technology that analyzes the user's voice, facial expressions, and body language to determine their emotions at any given moment.

[0195] "Means of adjusting routes and ride experiences" refers to systems or functions that change the selected route or the environment during the ride according to the user's feelings and preferences.

[0196] "Speech recognition functionality" is a technology that collects the user's speech as data and processes it as text data.

[0197] This invention relates to a navigation system for autonomous vehicles that personalizes routes and ride experiences based on the user's emotional state. The server collects traffic condition data, weather data, and user profile data in real time and analyzes them comprehensively. The analysis incorporates an emotion engine that includes facial recognition and voice analysis, thereby understanding the user's current emotional state. This emotional state is fed back into the route selection algorithm, which then selects the route best suited to the user.

[0198] Specifically, the server uses a camera and microphone to collect the user's facial expression and voice data, and processes this data using the image analysis library OpenCV and the speech recognition service Google Cloud Speech-to-Text. At the same time, it uses the Emotion API to analyze characteristics derived from facial expressions and voice and evaluate emotions.

[0199] Based on the processing results, the server generates multiple candidate routes and selects the optimal route in light of evaluation criteria and the user's emotional state. The selected route is displayed on the in-car tablet or smartphone, and a voice assistant guides the user in a tone adjusted according to their emotions. For example, if the user is relaxed, it provides a quiet environment with music playing and suggests a park with abundant nature as a break point.

[0200] An example of a prompt might be, "To provide a fun family ride, please consider the passenger's current emotions and suggest the optimal route and settings." Based on this prompt, the generative AI model suggests the optimal route and settings according to the user's emotions. This makes it possible to provide the user with a relaxed and comfortable driving experience.

[0201] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0202] Step 1:

[0203] The server uses a camera and microphone to collect user facial and audio data in real time. Still images or video data are acquired as input from the camera, and audio data is input directly from the microphone. This data is used as basic information for emotion analysis.

[0204] Step 2:

[0205] The server analyzes facial expression data acquired using OpenCV and quantifies the characteristics of the expressions. Specifically, it extracts landmark information from the face and infers emotional states such as joy, anger, sadness, and happiness from its shape. This analysis result is input into the Emotion API.

[0206] Step 3:

[0207] The server uses Google Cloud Speech-to-Text to convert speech data into text data. This text data is logged as a result of speech recognition and is also used for sentiment analysis. Information is also collected from the intonation and word choice of the speech to estimate emotions.

[0208] Step 4:

[0209] The server uses the Emotion API to comprehensively evaluate information obtained from quantified facial features and voice to determine the user's emotional state. The Emotion API analyzes the input feature data and outputs it as an emotion score. This emotion score is used for route optimization in the next step.

[0210] Step 5:

[0211] The server collects traffic condition data, weather data, and user profile data, and generates multiple candidate routes based on evaluation criteria. This data is obtained from external data sources and acts as an indicator of travel time and comfort for each candidate route.

[0212] Step 6:

[0213] The server uses a program model to select the optimal route based on emotional scores. This selection prioritizes routes with scenic views that allow the user to relax and routes that cause less stress. A generative AI model evaluates the score of each route, taking into account emotional and other input data, and outputs the most suitable route.

[0214] Step 7:

[0215] The server displays the selected optimal route on an in-car tablet or smartphone and guides the user via a voice assistant. The voice assistant provides feedback in a tone that matches the user's emotional state, adjusting to enhance feelings of security and comfort. The prompts included in the voice feedback also change according to the user's emotions.

[0216] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0217] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0218] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0219] [Second Embodiment]

[0220] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0221] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0222] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0223] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0224] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0225] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0226] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0227] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0228] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0229] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0230] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0231] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0232] The car navigation system of the present invention functions by having a server collect traffic condition data, weather data, and user profile data, and analyzing this data using an AI algorithm. Based on these analysis results, the server generates multiple candidate routes and evaluates each route. This evaluation includes factors such as time, distance, user preferences, presence or absence of congestion, and road conditions. The system then delivers the optimal route to the user terminal, taking these factors into consideration.

[0233] The user terminal displays the received route information as a navigation display. Furthermore, it retrieves information on potential detours from the server and suggests spots that might interest the user. This includes specific tourist attractions, restaurants, and rest areas. For example, if the user sets their interest to "historical buildings," the system will list suitable detour spots.

[0234] While driving, users can interact with the system through voice recognition. If a user issues a voice command such as, "I want to try turning left instead of right," the user terminal sends the instruction to the server and requests that a new route be calculated. The server immediately sends the newly calculated route to the user terminal, enabling a quick route change while maintaining driving safety.

[0235] In this way, the present invention provides a personalized driving experience tailored to the individual needs of the user, guiding the user safely and comfortably to their destination.

[0236] The following describes the processing flow.

[0237] Step 1:

[0238] The server collects real-time data from sources such as traffic information providers, satellite data services, and weather information sources. This includes current traffic congestion, road construction information, and weather conditions.

[0239] Step 2:

[0240] The server uses the collected data to run AI algorithms and analyzes it in comparison with user profile data. This profile data includes the user's past route choices and preferences, forming the basis of the analysis.

[0241] Step 3:

[0242] The server generates multiple candidate routes based on the analysis results. Each candidate route is evaluated based on factors such as traffic conditions, estimated travel time, and scenic beauty.

[0243] Step 4:

[0244] The server selects the optimal route from the evaluated routes and sends that route information to the user's terminal.

[0245] Step 5:

[0246] The user terminal displays the received route information on the navigation screen, allowing the user to refer to it while driving. It also provides route guidance via voice prompts.

[0247] Step 6:

[0248] The server retrieves information on potential stops from a database and creates a list of suggested spots based on the user's settings and past activity history. This list includes highly-rated tourist attractions and restaurants.

[0249] Step 7:

[0250] Users can give instructions to the system regarding routes and detours via a voice assistant. The user's voice commands are processed by being converted to text on the user's terminal and sent to the server.

[0251] Step 8:

[0252] The server receives the voice command, recalculates the route as needed, and resends the latest route guidance to the user.

[0253] In this way, the entire system works together to provide users with a consistent and personalized navigation experience.

[0254] (Example 1)

[0255] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0256] This invention aims to solve the complex route selection problems faced by users in modern transportation systems. In particular, there is a need to provide optimal routes that take into account traffic congestion, weather changes, and individual user preferences. Furthermore, there is a lack of flexible navigation systems that allow users to operate the system and change routes via voice while ensuring safety during travel.

[0257] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0258] In this invention, the server includes means for acquiring and analyzing environmental information, weather information, and user attribute information; means for generating multiple alternative routes based on the analyzed information and selecting the optimal route according to indicators; and means for interpreting user instructions and performing desired operations using a voice input function. This enables users to select the optimal and safe route according to dynamically changing traffic conditions and personal preferences.

[0259] "Environmental information" refers to general information related to road conditions, such as road congestion, traffic restrictions, and road construction information.

[0260] "Weather information" refers to information about weather conditions such as rain, snow, wind, and temperature, including the impact of these factors on traffic conditions and movement.

[0261] "User attribute information" refers to information about users' preferences, past selection history, and destinations, and is used to provide personalized services to individual users.

[0262] An "alternative route" refers to a route suggested from among several possible routes to the destination, depending on the circumstances.

[0263] An "indicator" refers to a set of criteria used to evaluate a route, such as distance, travel time, road conditions, and user preferences.

[0264] "Voice input functionality" is a technology that allows a user's voice to be input as data, enabling the system to recognize that voice and perform operations.

[0265] "Desired operation" refers to requests made by the user to the system, such as changing the route or resetting the destination, and the system's response to these requests.

[0266] This invention relates to a traffic navigation system in which a server and a terminal work together to provide route guidance optimized for the user.

[0267] The server utilizes various APIs to collect environmental, weather, and user attribute information via the internet. Specifically, it obtains road congestion and regulation information through general map service APIs, and weather information is obtained using weather information service APIs. Based on this information, the server analyzes the data using Python and machine learning frameworks. For example, Scikit-Learn's machine learning algorithms are used to predict past traffic patterns.

[0268] The server generates alternative routes based on the analysis results and evaluates them based on distance, time, and user preferences. Based on this, the server selects the optimal route and sends the selected route information to the terminal.

[0269] The terminal displays route information received from the server on the navigation screen, presenting it in a user-friendly format for drivers. The terminal also sends requests to the server to retrieve relevant location information based on the user's preferences. These relevant locations may include tourist attractions or restaurants, and the system uses an API similar to Foursquare to collect location and associated facility information.

[0270] Users can easily give instructions such as changing routes or obtaining additional information through the terminal's voice input function. The voice input uses a common speech recognition service to provide accurate analysis and responses.

[0271] For example, if a user gives a voice command such as "Tell me a route that goes through a scenic area," the server will analyze and select a route that meets that request, and the terminal will display the result to the user. An example of a prompt would be, "Suggest the best route for a user who prefers scenic routes. The starting point is a major city, the destination is a tourist spot, and the detours should include natural landscapes."

[0272] Through this system, users can enjoy a flexible, safe, and comfortable driving experience tailored to their individual needs.

[0273] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0274] Step 1:

[0275] The server collects environmental information, weather information, and user attribute information. This data becomes input to the server, and data is retrieved using traffic information APIs and weather information APIs. Specifically, API requests are issued and the data is received in JSON format. This is stored in a temporary database inside the server and used in the next processing step.

[0276] Step 2:

[0277] The server performs data analysis based on the collected data. The input is the information obtained in Step 1, which is then analyzed by a Python program. Specifically, it uses a machine learning algorithm to analyze traffic patterns and predicts suitable routes based on user profiles. As a result of the calculations, multiple alternative routes are generated, which become the input for the next step.

[0278] Step 3:

[0279] The server evaluates the generated alternative routes. The input is the set of routes generated in Step 2, and the evaluation criteria include distance, travel time, and user preferences. Scoring is performed for each alternative route, and the route with the highest score is selected as the output. This is then sent to the terminal next.

[0280] Step 4:

[0281] The terminal receives the optimal route information from the server. This route information is the input to the terminal, and the terminal displays it on the navigation screen. As a specific display method, real-time route guidance is performed using a map application.

[0282] Step 5:

[0283] The terminal sends a request for related location information to the server. The input includes the user preference information, and based on this information, the server searches for related locations and outputs the results to the terminal. This includes preparing a list of location candidates and presenting detour spots to the user.

[0284] Step 6:

[0285] The user uses the voice input function of the terminal to instruct a route change. Voice data is provided to the terminal as the input, and the terminal converts it into text data through a voice recognition algorithm. The converted data is sent to the server, and the server calculates a new optimal route and outputs the result to the terminal again.

[0286] Overall, the system processes a large number of dynamically changing data and provides appropriate route guidance to the user, thereby realizing a comfortable and safe driving experience.

[0287] (Application Example 1)

[0288] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0289] Modern autonomous vehicles require advanced navigation systems to ensure safe and efficient travel. However, conventional systems fail to fully utilize real-time traffic information and individual user preferences, making it difficult to provide optimal route guidance. Furthermore, they lack personalized suggestions for detours that take user preferences into account. As a result, there is a challenge in that the convenience of autonomous vehicles cannot be fully realized.

[0290] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0291] In this invention, the server includes means for collecting and analyzing traffic information, weather information, and user profile information; means for generating multiple candidate routes based on the analyzed information and determining the optimal route according to evaluation criteria; and means for filtering and presenting detour point information to the user upon request. This makes it possible to provide safe and efficient route guidance in autonomous vehicles, as well as to provide personalized suggestions of detour points according to the user's preferences.

[0292] "Traffic information" refers to information that shows real-time traffic conditions, such as the degree of congestion on roads, the occurrence of accidents, and road closures.

[0293] "Weather information" refers to data that indicates the weather conditions on and around roads, and includes temperature, precipitation, wind speed, etc.

[0294] "User profile information" refers to individual information about the user of the navigation system, including age, hobbies, and past driving history.

[0295] A "candidate route" refers to a list of multiple possible travel routes from the starting point to the destination.

[0296] "Evaluation criteria" are standards used to compare candidate routes and determine which route is optimal, and these include factors such as time, distance, and traffic congestion.

[0297] "Detour information" refers to information about tourist spots, restaurants, rest areas, and other places you can stop at on your way to your destination.

[0298] "Voice recognition functionality" is a technology that converts a user's speech into digital signals, analyzes the content, and understands its meaning.

[0299] An "autonomous vehicle" is a vehicle that, thanks to its installed technology, can drive autonomously without driver intervention.

[0300] "Real-time updates" means that information is continuously updated to the latest version.

[0301] A "voice assistant" is software that recognizes voice commands and performs tasks based on their content.

[0302] The system for implementing this invention mainly consists of a server, a user terminal, and an autonomous vehicle. The server is responsible for collecting traffic information, weather information, and user profile information, and analyzing this data using AI to provide the user with the optimal driving route. This analysis utilizes data acquisition services such as the Google Maps API and the OpenWeatherMap API, as well as machine learning frameworks such as TensorFlow.

[0303] Based on the analysis results, the server generates multiple candidate routes and evaluates them according to evaluation criteria. These criteria include route distance, time, user preferences, and road conditions. After the optimal route is determined, information about it is sent to the user's terminal.

[0304] The user terminal displays the route information received from the server through the navigation system. This navigation system incorporates a speech recognition function using the Google Cloud Speech API. When the user issues a voice command, the system converts the command into text data, sends it to the server, and performs new analysis.

[0305] As a specific example, when the user requests by voice "I want to stop by a delicious ramen shop on the way", the system searches for candidates around the current location in real time and proposes an optimal detour route.

[0306] By using a generative AI model, the system can perform more accurate route prediction and proposal. Examples of prompt sentences include "Propose ramen spots during a drive from Tokyo to Osaka" and "I want to turn left instead of right at the next intersection, so recalculate".

[0307] In this way, users can achieve safe and efficient movement via an autonomous vehicle.

[0308] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0309] Step 1:

[0310] The server obtains traffic situation information and weather information from their respective APIs (for example, Google Maps API and OpenWeatherMap API). The inputs are the user's current location and destination. The server uses this information to perform real-time data analysis and visualize the traffic flow and weather changes.

[0311] Step 2:

[0312] The server receives user profile information from the user's terminal. The input consists of user preference information, including the user's past driving history and preferred detour spots. The server analyzes this information and uses it to generate route recommendation models optimized for each individual user.

[0313] Step 3:

[0314] The server generates multiple candidate routes based on all the information it acquires. The data used includes traffic conditions, weather information, and user profile information. The output is a route list that includes a recommendation index for each route. An AI algorithm is used to perform data calculations that take into account time, distance, weather, and user preferences.

[0315] Step 4:

[0316] The server evaluates the generated candidate routes according to evaluation criteria and determines the optimal route. The evaluation criteria are based on user-defined priorities (e.g., shortest time, shortest distance, interest level of detour spots, etc.). The output is the route determined to be optimal.

[0317] Step 5:

[0318] The user terminal displays the optimal route received from the server on the navigation system. The input is the optimal route information from the server. The output is the route drawn on the map and its detailed information (e.g., information about the next intersection to turn at and suggestions for places to stop along the way). The user receives navigation support through visual and voice guidance.

[0319] Step 6:

[0320] Users can give instructions to the system through voice recognition. For example, they might request, "I want to stop at a ramen shop on the way." The input is a voice command, which the system converts into text data and sends to the server.

[0321] Step 7:

[0322] The server re-analyzes the data based on the voice command and dynamically recalculates the route. The input is a new request based on the voice command. The output is updated route information. A generative AI model is used to quickly generate a route that includes new detour options.

[0323] In this way, the system can provide users with safe and efficient routes.

[0324] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0325] The car navigation system incorporating the emotion engine of the present invention provides a more personalized navigation experience by having a server acquire traffic data, weather data, and user profile data, and further analyze emotion data. The server uses various sensors and cameras to analyze the user's voice, facial expressions, and body language to understand the user's emotional state.

[0326] Based on this emotion recognition result, the server generates optimal candidate routes and selects the route that best suits the user's mood. For example, if the user is feeling stressed, the navigation system can prioritize suggesting quieter routes or routes that offer a sense of nature. Similarly, when suggesting detours, the system filters and presents appropriate spots according to the user's current emotions. For example, if the system determines that the user is tired, it will recommend places suitable for rest, such as quiet cafes or parks.

[0327] While driving, users can use a voice assistant to request route changes or add detours. The voice assistant takes into account the output of its emotion engine and adjusts the tone and content of its responses according to the user's emotions. For example, if the user is angry, the voice assistant will speak in a calm tone and offer suggestions to facilitate problem resolution.

[0328] All information is displayed on the user's terminal and provided to the user visually and audibly through voice feedback. This allows the present invention to provide drivers with an emotionally conscious, safe, and comfortable driving experience, reducing stress and enhancing the enjoyment of driving.

[0329] The following describes the processing flow.

[0330] Step 1:

[0331] The server retrieves relevant data from traffic information providers, weather data services, and user profile databases. This allows it to collect information about current traffic conditions, weather conditions, and user preferences.

[0332] Step 2:

[0333] The server captures the user's voice, facial expressions, and body language from sensors such as cameras and microphones installed inside the vehicle. The emotion engine analyzes this data in real time to estimate the user's emotional state.

[0334] Step 3:

[0335] The server inputs collected sentiment data and traffic / weather data into an AI algorithm for analysis. This generates multiple candidate routes that are best suited to the user's current emotions.

[0336] Step 4:

[0337] The server selects the route that best suits the user's emotional state from the generated routes, according to evaluation criteria. These criteria include travel time, the pleasantness of the scenery, and weighting based on the user's preferences.

[0338] Step 5:

[0339] The user terminal displays the optimal route information received from the server on the navigation screen. It also provides route guidance and instructions to the user via voice guidance.

[0340] Step 6:

[0341] The server filters information on potential detour spots from its database based on the user's emotional state and presents relaxing spots and places suitable for a change of pace to the user's terminal.

[0342] Step 7:

[0343] The user gives instructions regarding route changes or detours through a voice assistant. The device uses voice recognition to convert the commands into text and sends the content to the server.

[0344] Step 8:

[0345] The server calculates a new route based on the user's voice commands and emotional state, and redistributes the updated route information to the user's terminal. During this process, the voice assistant's response is adjusted according to the user's emotions, providing appropriate feedback.

[0346] (Example 2)

[0347] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0348] Conventional car navigation systems primarily select routes based on external conditions such as traffic and weather, and do not offer personalized route suggestions that take into account the user's emotional state. As a result, it has been difficult to provide optimal routes for drivers who are stressed or fatigued. This invention aims to improve driving comfort and safety by providing navigation that takes the user's emotional state into consideration.

[0349] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0350] In this invention, the server includes means for collecting and analyzing traffic information, weather information, and user profile information; means for analyzing the user's voice, facial expressions, and body language using sensors and cameras to evaluate the user's emotional state; and means for adjusting the voice assistant's response according to the user's emotions using a generative AI model. This enables the suggestion of an optimal route that takes the user's emotional state into consideration and the filtering of detour spots.

[0351] "Traffic information" is a general term for data related to traffic, such as the flow of vehicles on roads, congestion levels, and road closures.

[0352] "Weather information" refers to data that shows the weather conditions at any given time, and specifically includes temperature, precipitation, wind speed, and visibility information.

[0353] "User profile information" refers to personal information about a specific user, including past driving history and individual preferences.

[0354] "Emotional state" refers to the user's current psychological state and includes emotions such as joy, anger, sadness, and stress.

[0355] "Candidate routes" refer to multiple possible route options from the starting point to the destination.

[0356] "Evaluation criteria" are a set of indicators used to evaluate each route option, based on distance, time, scenery, and the user's emotional state.

[0357] An "emotion engine" is an algorithm or system for detecting and analyzing a user's emotional state.

[0358] "Detour spot information" refers to information about places that users can visit along the way, including tourist attractions, restaurants, cafes, and rest facilities.

[0359] "Voice recognition functionality" refers to technology that processes a user's speech as digital data and interprets the intended command.

[0360] A "generative AI model" refers to an artificial intelligence algorithm that learns from large amounts of data and performs predictions and classifications.

[0361] In this invention, the system mainly consists of three elements: a server, a terminal, and a user. The server first accesses multiple data sources to collect traffic information, weather information, and user profile information. Specifically, it obtains this information by using a traffic information API, a weather information API, and a profile database. User profile information includes past driving history and individual preferences.

[0362] The server uses voice sensors and cameras installed in the vehicle to analyze the user's voice tone, facial expressions, and body language in real time. This allows it to determine the user's current emotional state and, using a generative AI model, suggest the optimal route tailored to the user's emotions. For example, if the user is seeking relaxation, the server will prioritize suggesting quiet routes.

[0363] The terminal's role is to notify the user of information provided by the server through the in-car display and voice output. The voice assistant dynamically adjusts its tone and content according to the user's emotional state. Specifically, it provides guidance in a calm tone to users who are feeling stressed.

[0364] Users can request route changes or detours via voice input using voice recognition technology. The server receives these requests, uses a generative AI model to quickly calculate new routes, and provides navigation tailored to the user's preferences.

[0365] For example, if a user requests to "add a quiet cafe nearby to the route," the server will quickly gather information, consider the user's current emotional state, and then suggest the most suitable cafe options.

[0366] An example of a prompt message might be, "If the user is feeling stressed, please suggest a place where they can feel relaxed." This allows the system to provide the most appropriate information for the user.

[0367] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0368] Step 1:

[0369] The server collects traffic information, weather information, and user profile information from various sources. It calls APIs to retrieve the latest traffic and weather data. Input data includes GPS location information, weather APIs, and a profile database. The server analyzes this data to obtain outputs such as traffic delay information and weather patterns.

[0370] Step 2:

[0371] The server uses in-vehicle sensors and cameras to collect and analyze the user's voice, facial expressions, and body language in real time. Voice data and image data are used as input. A generative AI model is used to estimate the user's current emotional state. The output is an emotional state (e.g., stress, relief, excitement).

[0372] Step 3:

[0373] The server generates multiple candidate routes based on collected data and emotional states. Each route is evaluated using evaluation criteria according to the user's preferences and emotions. The input data consists of the outputs from steps 1 and 2. Using a generative AI model, the server outputs the most suitable route and sends it to the terminal.

[0374] Step 4:

[0375] The terminal presents route information received from the server to the user via an in-vehicle display and audio output. The input is route information from the server. The terminal converts this information into a human-readable format and outputs it, for example, displaying the route in map format on the display and providing voice guidance.

[0376] Step 5:

[0377] Users request route changes or detours from the system via voice input. The voice recognition function receives this input and processes the data for transmission to the server. The output is the request itself (e.g., "Please add nearby cafes").

[0378] Step 6:

[0379] The server receives a request from the user, collects new information, and re-evaluates the route. Here, a generative AI model is used to calculate the optimal route based on the new conditions in real time and send it back to the terminal. The output is the updated route information.

[0380] Step 7:

[0381] The terminal re-presents the updated route information to the user, and the voice assistant provides guidance in an appropriate tone and content. The input is the newly suggested route information. The terminal immediately reflects this information and outputs it in a way that reassures the user.

[0382] (Application Example 2)

[0383] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0384] In autonomous vehicles, conventional navigation systems have a problem: they only optimize routes based on traffic conditions and weather, without considering the user's emotional state. This tends to result in a uniform user experience, particularly limiting improvements in comfort and safety. Therefore, there is a need for methods to provide personalized route selection and ride experiences that take into account the individual user's emotional state.

[0385] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0386] In this invention, the server includes means for collecting and analyzing traffic condition data, weather data, and user profile data; means for generating multiple candidate routes based on the analyzed data and determining the optimal route according to evaluation criteria; and means for analyzing the user's emotional state using an emotion engine and adjusting the route and ride experience based on the emotional state. This enables safer and more comfortable navigation and ride experience optimized for the user's emotional state.

[0387] "Traffic condition data" refers to information about traffic flow and conditions, such as road congestion and accident information.

[0388] "Weather data" refers to information that shows weather conditions in a specific region, such as temperature, precipitation, and wind speed.

[0389] "User profile data" refers to data that includes personal information about individual users, such as their age, gender, past behavioral history, and preferences.

[0390] "Means of analysis" refer to processing methods and technical means for collecting data and using it to derive useful information.

[0391] "Candidate routes" refer to multiple route options for reaching a destination, and are evaluated to determine the priority of each route.

[0392] The "emotion engine" is a technology that analyzes the user's voice, facial expressions, and body language to determine their emotions at any given moment.

[0393] "Means of adjusting routes and ride experiences" refers to systems or functions that change the selected route or the environment during the ride according to the user's feelings and preferences.

[0394] "Speech recognition functionality" is a technology that collects the user's speech as data and processes it as text data.

[0395] This invention relates to a navigation system for autonomous vehicles that personalizes routes and ride experiences based on the user's emotional state. The server collects traffic condition data, weather data, and user profile data in real time and analyzes them comprehensively. The analysis incorporates an emotion engine that includes facial recognition and voice analysis, thereby understanding the user's current emotional state. This emotional state is fed back into the route selection algorithm, which then selects the route best suited to the user.

[0396] Specifically, the server uses a camera and microphone to collect the user's facial expression and voice data, and processes this data using the image analysis library OpenCV and the speech recognition service Google Cloud Speech-to-Text. At the same time, it uses the Emotion API to analyze characteristics derived from facial expressions and voice and evaluate emotions.

[0397] Based on the processing results, the server generates multiple candidate routes and selects the optimal route in light of evaluation criteria and the user's emotional state. The selected route is displayed on the in-car tablet or smartphone, and a voice assistant guides the user in a tone adjusted according to their emotions. For example, if the user is relaxed, it provides a quiet environment with music playing and suggests a park with abundant nature as a break point.

[0398] An example of a prompt might be, "To provide a fun family ride, please consider the passenger's current emotions and suggest the optimal route and settings." Based on this prompt, the generative AI model suggests the optimal route and settings according to the user's emotions. This makes it possible to provide the user with a relaxed and comfortable driving experience.

[0399] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0400] Step 1:

[0401] The server uses a camera and microphone to collect user facial and audio data in real time. Still images or video data are acquired as input from the camera, and audio data is input directly from the microphone. This data is used as basic information for emotion analysis.

[0402] Step 2:

[0403] The server analyzes facial expression data acquired using OpenCV and quantifies the characteristics of the expressions. Specifically, it extracts landmark information from the face and infers emotional states such as joy, anger, sadness, and happiness from its shape. This analysis result is input into the Emotion API.

[0404] Step 3:

[0405] The server uses Google Cloud Speech-to-Text to convert speech data into text data. This text data is logged as a result of speech recognition and is also used for sentiment analysis. Information is also collected from the intonation and word choice of the speech to estimate emotions.

[0406] Step 4:

[0407] The server uses the Emotion API to comprehensively evaluate information obtained from quantified facial features and voice to determine the user's emotional state. The Emotion API analyzes the input feature data and outputs it as an emotion score. This emotion score is used for route optimization in the next step.

[0408] Step 5:

[0409] The server collects traffic condition data, weather data, and user profile data, and generates multiple candidate routes based on evaluation criteria. This data is obtained from external data sources and acts as an indicator of travel time and comfort for each candidate route.

[0410] Step 6:

[0411] The server uses a program model to select the optimal route based on emotional scores. This selection prioritizes routes with scenic views that allow the user to relax and routes that cause less stress. A generative AI model evaluates the score of each route, taking into account emotional and other input data, and outputs the most suitable route.

[0412] Step 7:

[0413] The server displays the selected optimal route on an in-car tablet or smartphone and guides the user via a voice assistant. The voice assistant provides feedback in a tone that matches the user's emotional state, adjusting to enhance feelings of security and comfort. The prompts included in the voice feedback also change according to the user's emotions.

[0414] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0415] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0416] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0417] [Third Embodiment]

[0418] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0419] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0420] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0421] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0422] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0423] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0424] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0425] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0426] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0427] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0428] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0429] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0430] The car navigation system of the present invention functions by having a server collect traffic condition data, weather data, and user profile data, and analyzing this data using an AI algorithm. Based on these analysis results, the server generates multiple candidate routes and evaluates each route. This evaluation includes factors such as time, distance, user preferences, presence or absence of congestion, and road conditions. The system then delivers the optimal route to the user terminal, taking these factors into consideration.

[0431] The user terminal displays the received route information as a navigation display. Furthermore, it retrieves information on potential detours from the server and suggests spots that might interest the user. This includes specific tourist attractions, restaurants, and rest areas. For example, if the user sets their interest to "historical buildings," the system will list suitable detour spots.

[0432] While driving, users can interact with the system through voice recognition. If a user issues a voice command such as, "I want to try turning left instead of right," the user terminal sends the instruction to the server and requests that a new route be calculated. The server immediately sends the newly calculated route to the user terminal, enabling a quick route change while maintaining driving safety.

[0433] In this way, the present invention provides a personalized driving experience tailored to the individual needs of the user, guiding the user safely and comfortably to their destination.

[0434] The following describes the processing flow.

[0435] Step 1:

[0436] The server collects real-time data from sources such as traffic information providers, satellite data services, and weather information sources. This includes current traffic congestion, road construction information, and weather conditions.

[0437] Step 2:

[0438] The server uses the collected data to run AI algorithms and analyzes it in comparison with user profile data. This profile data includes the user's past route choices and preferences, forming the basis of the analysis.

[0439] Step 3:

[0440] The server generates multiple candidate routes based on the analysis results. Each candidate route is evaluated based on factors such as traffic conditions, estimated travel time, and scenic beauty.

[0441] Step 4:

[0442] The server selects the optimal route from the evaluated routes and sends that route information to the user's terminal.

[0443] Step 5:

[0444] The user terminal displays the received route information on the navigation screen, allowing the user to refer to it while driving. It also provides route guidance via voice prompts.

[0445] Step 6:

[0446] The server retrieves information on potential stops from a database and creates a list of suggested spots based on the user's settings and past activity history. This list includes highly-rated tourist attractions and restaurants.

[0447] Step 7:

[0448] Users can give instructions to the system regarding routes and detours via a voice assistant. The user's voice commands are processed by being converted to text on the user's terminal and sent to the server.

[0449] Step 8:

[0450] The server receives the voice command, recalculates the route as needed, and resends the latest route guidance to the user.

[0451] In this way, the entire system works together to provide users with a consistent and personalized navigation experience.

[0452] (Example 1)

[0453] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0454] This invention aims to solve the complex route selection problems faced by users in modern transportation systems. In particular, there is a need to provide optimal routes that take into account traffic congestion, weather changes, and individual user preferences. Furthermore, there is a lack of flexible navigation systems that allow users to operate the system and change routes via voice while ensuring safety during travel.

[0455] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0456] In this invention, the server includes means for acquiring and analyzing environmental information, weather information, and user attribute information; means for generating multiple alternative routes based on the analyzed information and selecting the optimal route according to indicators; and means for interpreting user instructions and performing desired operations using a voice input function. This enables users to select the optimal and safe route according to dynamically changing traffic conditions and personal preferences.

[0457] "Environmental information" refers to general information related to road conditions, such as road congestion, traffic restrictions, and road construction information.

[0458] "Weather information" refers to information about weather conditions such as rain, snow, wind, and temperature, including the impact of these factors on traffic conditions and movement.

[0459] "User attribute information" refers to information about users' preferences, past selection history, and destinations, and is used to provide personalized services to individual users.

[0460] An "alternative route" refers to a route suggested from among several possible routes to the destination, depending on the circumstances.

[0461] An "indicator" refers to a set of criteria used to evaluate a route, such as distance, travel time, road conditions, and user preferences.

[0462] "Voice input functionality" is a technology that allows a user's voice to be input as data, enabling the system to recognize that voice and perform operations.

[0463] "Desired operation" refers to requests made by the user to the system, such as changing the route or resetting the destination, and the system's response to these requests.

[0464] This invention relates to a traffic navigation system in which a server and a terminal work together to provide route guidance optimized for the user.

[0465] The server utilizes various APIs to collect environmental, weather, and user attribute information via the internet. Specifically, it obtains road congestion and regulation information through general map service APIs, and weather information is obtained using weather information service APIs. Based on this information, the server analyzes the data using Python and machine learning frameworks. For example, Scikit-Learn's machine learning algorithms are used to predict past traffic patterns.

[0466] The server generates alternative routes based on the analysis results and evaluates them based on distance, time, and user preferences. Based on this, the server selects the optimal route and sends the selected route information to the terminal.

[0467] The terminal displays route information received from the server on the navigation screen, presenting it in a user-friendly format for drivers. The terminal also sends requests to the server to retrieve relevant location information based on the user's preferences. These relevant locations may include tourist attractions or restaurants, and the system uses an API similar to Foursquare to collect location and associated facility information.

[0468] Users can easily give instructions such as changing routes or obtaining additional information through the terminal's voice input function. The voice input uses a common speech recognition service to provide accurate analysis and responses.

[0469] For example, if a user gives a voice command such as "Tell me a route that goes through a scenic area," the server will analyze and select a route that meets that request, and the terminal will display the result to the user. An example of a prompt would be, "Suggest the best route for a user who prefers scenic routes. The starting point is a major city, the destination is a tourist spot, and the detours should include natural landscapes."

[0470] Through this system, users can enjoy a flexible, safe, and comfortable driving experience tailored to their individual needs.

[0471] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0472] Step 1:

[0473] The server collects environmental information, weather information, and user attribute information. This data becomes input to the server, and data is retrieved using traffic information APIs and weather information APIs. Specifically, API requests are issued and the data is received in JSON format. This is stored in a temporary database inside the server and used in the next processing step.

[0474] Step 2:

[0475] The server performs data analysis based on the collected data. The input is the information obtained in Step 1, which is then analyzed by a Python program. Specifically, it uses a machine learning algorithm to analyze traffic patterns and predicts suitable routes based on user profiles. As a result of the calculations, multiple alternative routes are generated, which become the input for the next step.

[0476] Step 3:

[0477] The server evaluates the generated alternative routes. The input is the set of routes generated in step 2, and the evaluation criteria include distance, travel time, and user preference. Each alternative route is scored, and the route with the highest score is selected as the output. This is then sent to the terminal.

[0478] Step 4:

[0479] The terminal receives optimal route information from the server. This route information is input to the terminal, which then displays it on the navigation screen. Specifically, the display method involves providing real-time route guidance using a map application.

[0480] Step 5:

[0481] The terminal sends a request for relevant location information to the server. The input includes user preference information, which the server uses to search for relevant locations and outputs the results to the terminal. This includes providing a list of suggested locations and suggesting potential stops for the user.

[0482] Step 6:

[0483] The user uses the terminal's voice input function to instruct the terminal to change the route. Voice data is provided to the terminal as input, and the terminal converts it into text data through a speech recognition algorithm. The converted data is sent to the server, which calculates the new optimal route and outputs the result back to the terminal.

[0484] Overall, the system processes a large amount of dynamically changing data and provides users with appropriate route guidance, thereby creating a comfortable and safe driving experience.

[0485] (Application Example 1)

[0486] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0487] Modern autonomous vehicles require advanced navigation systems to ensure safe and efficient travel. However, conventional systems fail to fully utilize real-time traffic information and individual user preferences, making it difficult to provide optimal route guidance. Furthermore, they lack personalized suggestions for detours that take user preferences into account. As a result, there is a challenge in that the convenience of autonomous vehicles cannot be fully realized.

[0488] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0489] In this invention, the server includes means for collecting and analyzing traffic information, weather information, and user profile information; means for generating multiple candidate routes based on the analyzed information and determining the optimal route according to evaluation criteria; and means for filtering and presenting detour point information to the user upon request. This makes it possible to provide safe and efficient route guidance in autonomous vehicles, as well as to provide personalized suggestions of detour points according to the user's preferences.

[0490] "Traffic information" refers to information that shows real-time traffic conditions, such as the degree of congestion on roads, the occurrence of accidents, and road closures.

[0491] "Weather information" refers to data that indicates the weather conditions on and around roads, and includes temperature, precipitation, wind speed, etc.

[0492] "User profile information" refers to individual information about the user of the navigation system, including age, hobbies, and past driving history.

[0493] A "candidate route" refers to a list of multiple possible travel routes from the starting point to the destination.

[0494] "Evaluation criteria" are standards used to compare candidate routes and determine which route is optimal, and these include factors such as time, distance, and traffic congestion.

[0495] "Detour information" refers to information about tourist spots, restaurants, rest areas, and other places you can stop at on your way to your destination.

[0496] "Voice recognition functionality" is a technology that converts a user's speech into digital signals, analyzes the content, and understands its meaning.

[0497] An "autonomous vehicle" is a vehicle that, thanks to its installed technology, can drive autonomously without driver intervention.

[0498] "Real-time updates" means that information is continuously updated to the latest version.

[0499] A "voice assistant" is software that recognizes voice commands and performs tasks based on their content.

[0500] The system for implementing this invention mainly consists of a server, a user terminal, and an autonomous vehicle. The server is responsible for collecting traffic information, weather information, and user profile information, and analyzing this data using AI to provide the user with the optimal driving route. This analysis utilizes data acquisition services such as the Google Maps API and the OpenWeatherMap API, as well as machine learning frameworks such as TensorFlow.

[0501] Based on the analysis results, the server generates multiple candidate routes and evaluates them according to evaluation criteria. These criteria include route distance, time, user preferences, and road conditions. After the optimal route is determined, information about it is sent to the user's terminal.

[0502] The user terminal displays route information received from the server through a navigation system. This navigation system incorporates speech recognition functionality using the Google Cloud Speech API. When a user issues a voice command, the system converts the command into text data, sends it to the server, and performs further analysis.

[0503] For example, if a user requests by voice, "I want to stop by a delicious ramen shop along the way," the system will search for potential locations near their current location in real time and suggest the optimal route for stopping there.

[0504] By using generative AI models, the system can provide more accurate route predictions and suggestions. Examples of prompts include "Suggest ramen spots along the drive from Tokyo to Osaka" and "Recalculate the route; I want to turn left instead of right."

[0505] In this way, users can achieve safe and efficient travel via autonomous vehicles.

[0506] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0507] Step 1:

[0508] The server obtains traffic and weather information from their respective APIs (e.g., Google Maps API and OpenWeatherMap API). The input is the user's current location and destination. The server uses this information to perform real-time data analysis and visualize traffic flow and weather changes.

[0509] Step 2:

[0510] The server receives user profile information from the user's terminal. The input consists of user preference information, including the user's past driving history and preferred detour spots. The server analyzes this information and uses it to generate route recommendation models optimized for each individual user.

[0511] Step 3:

[0512] The server generates multiple candidate routes based on all the information it acquires. The data used includes traffic conditions, weather information, and user profile information. The output is a route list that includes a recommendation index for each route. An AI algorithm is used to perform data calculations that take into account time, distance, weather, and user preferences.

[0513] Step 4:

[0514] The server evaluates the generated candidate routes according to evaluation criteria and determines the optimal route. The evaluation criteria are based on user-defined priorities (e.g., shortest time, shortest distance, interest level of detour spots, etc.). The output is the route determined to be optimal.

[0515] Step 5:

[0516] The user terminal displays the optimal route received from the server on the navigation system. The input is the optimal route information from the server. The output is the route drawn on the map and its detailed information (e.g., information about the next intersection to turn at and suggestions for places to stop along the way). The user receives navigation support through visual and voice guidance.

[0517] Step 6:

[0518] Users can give instructions to the system through voice recognition. For example, they might request, "I want to stop at a ramen shop on the way." The input is a voice command, which the system converts into text data and sends to the server.

[0519] Step 7:

[0520] The server re-analyzes the data based on the voice command and dynamically recalculates the route. The input is a new request based on the voice command. The output is updated route information. A generative AI model is used to quickly generate a route that includes new detour options.

[0521] In this way, the system can provide users with safe and efficient routes.

[0522] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0523] The car navigation system incorporating the emotion engine of the present invention provides a more personalized navigation experience by having a server acquire traffic data, weather data, and user profile data, and further analyze emotion data. The server uses various sensors and cameras to analyze the user's voice, facial expressions, and body language to understand the user's emotional state.

[0524] Based on this emotion recognition result, the server generates optimal candidate routes and selects the route that best suits the user's mood. For example, if the user is feeling stressed, the navigation system can prioritize suggesting quieter routes or routes that offer a sense of nature. Similarly, when suggesting detours, the system filters and presents appropriate spots according to the user's current emotions. For example, if the system determines that the user is tired, it will recommend places suitable for rest, such as quiet cafes or parks.

[0525] While driving, users can use a voice assistant to request route changes or add detours. The voice assistant takes into account the output of its emotion engine and adjusts the tone and content of its responses according to the user's emotions. For example, if the user is angry, the voice assistant will speak in a calm tone and offer suggestions to facilitate problem resolution.

[0526] All information is displayed on the user's terminal and provided to the user visually and audibly through voice feedback. This allows the present invention to provide drivers with an emotionally conscious, safe, and comfortable driving experience, reducing stress and enhancing the enjoyment of driving.

[0527] The following describes the processing flow.

[0528] Step 1:

[0529] The server retrieves relevant data from traffic information providers, weather data services, and user profile databases. This allows it to collect information about current traffic conditions, weather conditions, and user preferences.

[0530] Step 2:

[0531] The server captures the user's voice, facial expressions, and body language from sensors such as cameras and microphones installed inside the vehicle. The emotion engine analyzes this data in real time to estimate the user's emotional state.

[0532] Step 3:

[0533] The server inputs collected sentiment data and traffic / weather data into an AI algorithm for analysis. This generates multiple candidate routes that are best suited to the user's current emotions.

[0534] Step 4:

[0535] The server selects the route that best suits the user's emotional state from the generated routes, according to evaluation criteria. These criteria include travel time, the pleasantness of the scenery, and weighting based on the user's preferences.

[0536] Step 5:

[0537] The user terminal displays the optimal route information received from the server on the navigation screen. It also provides route guidance and instructions to the user via voice guidance.

[0538] Step 6:

[0539] The server filters information on potential detour spots from its database based on the user's emotional state and presents relaxing spots and places suitable for a change of pace to the user's terminal.

[0540] Step 7:

[0541] The user gives instructions regarding route changes or detours through a voice assistant. The device uses voice recognition to convert the commands into text and sends the content to the server.

[0542] Step 8:

[0543] The server calculates a new route based on the user's voice commands and emotional state, and redistributes the updated route information to the user's terminal. During this process, the voice assistant's response is adjusted according to the user's emotions, providing appropriate feedback.

[0544] (Example 2)

[0545] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0546] Conventional car navigation systems primarily select routes based on external conditions such as traffic and weather, and do not offer personalized route suggestions that take into account the user's emotional state. As a result, it has been difficult to provide optimal routes for drivers who are stressed or fatigued. This invention aims to improve driving comfort and safety by providing navigation that takes the user's emotional state into consideration.

[0547] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0548] In this invention, the server includes means for collecting and analyzing traffic information, weather information, and user profile information; means for analyzing the user's voice, facial expressions, and body language using sensors and cameras to evaluate the user's emotional state; and means for adjusting the voice assistant's response according to the user's emotions using a generative AI model. This enables the suggestion of an optimal route that takes the user's emotional state into consideration and the filtering of detour spots.

[0549] "Traffic information" is a general term for data related to traffic, such as the flow of vehicles on roads, congestion levels, and road closures.

[0550] "Weather information" refers to data that shows the weather conditions at any given time, and specifically includes temperature, precipitation, wind speed, and visibility information.

[0551] "User profile information" refers to personal information about a specific user, including past driving history and individual preferences.

[0552] "Emotional state" refers to the user's current psychological state and includes emotions such as joy, anger, sadness, and stress.

[0553] "Candidate routes" refer to multiple possible route options from the starting point to the destination.

[0554] "Evaluation criteria" are a set of indicators used to evaluate each route option, based on distance, time, scenery, and the user's emotional state.

[0555] An "emotion engine" is an algorithm or system for detecting and analyzing a user's emotional state.

[0556] "Detour spot information" refers to information about places that users can visit along the way, including tourist attractions, restaurants, cafes, and rest facilities.

[0557] "Voice recognition functionality" refers to technology that processes a user's speech as digital data and interprets the intended command.

[0558] A "generative AI model" refers to an artificial intelligence algorithm that learns from large amounts of data and performs predictions and classifications.

[0559] In this invention, the system mainly consists of three elements: a server, a terminal, and a user. The server first accesses multiple data sources to collect traffic information, weather information, and user profile information. Specifically, it obtains this information by using a traffic information API, a weather information API, and a profile database. User profile information includes past driving history and individual preferences.

[0560] The server uses voice sensors and cameras installed in the vehicle to analyze the user's voice tone, facial expressions, and body language in real time. This allows it to determine the user's current emotional state and, using a generative AI model, suggest the optimal route tailored to the user's emotions. For example, if the user is seeking relaxation, the server will prioritize suggesting quiet routes.

[0561] The terminal's role is to notify the user of information provided by the server through the in-car display and voice output. The voice assistant dynamically adjusts its tone and content according to the user's emotional state. Specifically, it provides guidance in a calm tone to users who are feeling stressed.

[0562] Users can request route changes or detours via voice input using voice recognition technology. The server receives these requests, uses a generative AI model to quickly calculate new routes, and provides navigation tailored to the user's preferences.

[0563] For example, if a user requests to "add a quiet cafe nearby to the route," the server will quickly gather information, consider the user's current emotional state, and then suggest the most suitable cafe options.

[0564] An example of a prompt message might be, "If the user is feeling stressed, please suggest a place where they can feel relaxed." This allows the system to provide the most appropriate information for the user.

[0565] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0566] Step 1:

[0567] The server collects traffic information, weather information, and user profile information from various sources. It calls APIs to retrieve the latest traffic and weather data. Input data includes GPS location information, weather APIs, and a profile database. The server analyzes this data to obtain outputs such as traffic delay information and weather patterns.

[0568] Step 2:

[0569] The server uses in-vehicle sensors and cameras to collect and analyze the user's voice, facial expressions, and body language in real time. Voice data and image data are used as input. A generative AI model is used to estimate the user's current emotional state. The output is an emotional state (e.g., stress, relief, excitement).

[0570] Step 3:

[0571] The server generates multiple candidate routes based on collected data and emotional states. Each route is evaluated using evaluation criteria according to the user's preferences and emotions. The input data consists of the outputs from steps 1 and 2. Using a generative AI model, the server outputs the most suitable route and sends it to the terminal.

[0572] Step 4:

[0573] The terminal presents route information received from the server to the user via an in-vehicle display and audio output. The input is route information from the server. The terminal converts this information into a human-readable format and outputs it, for example, displaying the route in map format on the display and providing voice guidance.

[0574] Step 5:

[0575] Users request route changes or detours from the system via voice input. The voice recognition function receives this input and processes the data for transmission to the server. The output is the request itself (e.g., "Please add nearby cafes").

[0576] Step 6:

[0577] The server receives a request from the user, collects new information, and re-evaluates the route. Here, a generative AI model is used to calculate the optimal route based on the new conditions in real time and send it back to the terminal. The output is the updated route information.

[0578] Step 7:

[0579] The terminal re-presents the updated route information to the user, and the voice assistant provides guidance in an appropriate tone and content. The input is the newly suggested route information. The terminal immediately reflects this information and outputs it in a way that reassures the user.

[0580] (Application Example 2)

[0581] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0582] In autonomous vehicles, conventional navigation systems have a problem: they only optimize routes based on traffic conditions and weather, without considering the user's emotional state. This tends to result in a uniform user experience, particularly limiting improvements in comfort and safety. Therefore, there is a need for methods to provide personalized route selection and ride experiences that take into account the individual user's emotional state.

[0583] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0584] In this invention, the server includes means for collecting and analyzing traffic condition data, weather data, and user profile data; means for generating multiple candidate routes based on the analyzed data and determining the optimal route according to evaluation criteria; and means for analyzing the user's emotional state using an emotion engine and adjusting the route and ride experience based on the emotional state. This enables safer and more comfortable navigation and ride experience optimized for the user's emotional state.

[0585] "Traffic condition data" refers to information about traffic flow and conditions, such as road congestion and accident information.

[0586] "Weather data" refers to information that shows weather conditions in a specific region, such as temperature, precipitation, and wind speed.

[0587] "User profile data" refers to data that includes personal information about individual users, such as their age, gender, past behavioral history, and preferences.

[0588] "Means of analysis" refer to processing methods and technical means for collecting data and using it to derive useful information.

[0589] "Candidate routes" refer to multiple route options for reaching a destination, and are evaluated to determine the priority of each route.

[0590] The "emotion engine" is a technology that analyzes the user's voice, facial expressions, and body language to determine their emotions at any given moment.

[0591] "Means of adjusting routes and ride experiences" refers to systems or functions that change the selected route or the environment during the ride according to the user's feelings and preferences.

[0592] "Speech recognition functionality" is a technology that collects the user's speech as data and processes it as text data.

[0593] This invention relates to a navigation system for autonomous vehicles that personalizes routes and ride experiences based on the user's emotional state. The server collects traffic condition data, weather data, and user profile data in real time and analyzes them comprehensively. The analysis incorporates an emotion engine that includes facial recognition and voice analysis, thereby understanding the user's current emotional state. This emotional state is fed back into the route selection algorithm, which then selects the route best suited to the user.

[0594] Specifically, the server uses a camera and microphone to collect the user's facial expression and voice data, and processes this data using the image analysis library OpenCV and the speech recognition service Google Cloud Speech-to-Text. At the same time, it uses the Emotion API to analyze characteristics derived from facial expressions and voice and evaluate emotions.

[0595] Based on the processing results, the server generates multiple candidate routes and selects the optimal route in light of evaluation criteria and the user's emotional state. The selected route is displayed on the in-car tablet or smartphone, and a voice assistant guides the user in a tone adjusted according to their emotions. For example, if the user is relaxed, it provides a quiet environment with music playing and suggests a park with abundant nature as a break point.

[0596] An example of a prompt might be, "To provide a fun family ride, please consider the passenger's current emotions and suggest the optimal route and settings." Based on this prompt, the generative AI model suggests the optimal route and settings according to the user's emotions. This makes it possible to provide the user with a relaxed and comfortable driving experience.

[0597] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0598] Step 1:

[0599] The server uses a camera and microphone to collect user facial and audio data in real time. Still images or video data are acquired as input from the camera, and audio data is input directly from the microphone. This data is used as basic information for emotion analysis.

[0600] Step 2:

[0601] The server analyzes facial expression data acquired using OpenCV and quantifies the characteristics of the expressions. Specifically, it extracts landmark information from the face and infers emotional states such as joy, anger, sadness, and happiness from its shape. This analysis result is input into the Emotion API.

[0602] Step 3:

[0603] The server uses Google Cloud Speech-to-Text to convert speech data into text data. This text data is logged as a result of speech recognition and is also used for sentiment analysis. Information is also collected from the intonation and word choice of the speech to estimate emotions.

[0604] Step 4:

[0605] The server uses the Emotion API to comprehensively evaluate information obtained from quantified facial features and voice to determine the user's emotional state. The Emotion API analyzes the input feature data and outputs it as an emotion score. This emotion score is used for route optimization in the next step.

[0606] Step 5:

[0607] The server collects traffic condition data, weather data, and user profile data, and generates multiple candidate routes based on evaluation criteria. This data is obtained from external data sources and acts as an indicator of travel time and comfort for each candidate route.

[0608] Step 6:

[0609] The server uses a program model to select the optimal route based on emotional scores. This selection prioritizes routes with scenic views that allow the user to relax and routes that cause less stress. A generative AI model evaluates the score of each route, taking into account emotional and other input data, and outputs the most suitable route.

[0610] Step 7:

[0611] The server displays the selected optimal route on an in-car tablet or smartphone and guides the user via a voice assistant. The voice assistant provides feedback in a tone that matches the user's emotional state, adjusting to enhance feelings of security and comfort. The prompts included in the voice feedback also change according to the user's emotions.

[0612] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0613] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0614] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0615] [Fourth Embodiment]

[0616] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0617] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0618] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0619] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0620] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0621] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0622] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0623] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0624] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0625] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0626] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0627] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0628] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0629] The car navigation system of the present invention functions by having a server collect traffic condition data, weather data, and user profile data, and analyzing this data using an AI algorithm. Based on these analysis results, the server generates multiple candidate routes and evaluates each route. This evaluation includes factors such as time, distance, user preferences, presence or absence of congestion, and road conditions. The system then delivers the optimal route to the user terminal, taking these factors into consideration.

[0630] The user terminal displays the received route information as a navigation display. Furthermore, it retrieves information on potential detours from the server and suggests spots that might interest the user. This includes specific tourist attractions, restaurants, and rest areas. For example, if the user sets their interest to "historical buildings," the system will list suitable detour spots.

[0631] While driving, users can interact with the system through voice recognition. If a user issues a voice command such as, "I want to try turning left instead of right," the user terminal sends the instruction to the server and requests that a new route be calculated. The server immediately sends the newly calculated route to the user terminal, enabling a quick route change while maintaining driving safety.

[0632] In this way, the present invention provides a personalized driving experience tailored to the individual needs of the user, guiding the user safely and comfortably to their destination.

[0633] The following describes the processing flow.

[0634] Step 1:

[0635] The server collects real-time data from sources such as traffic information providers, satellite data services, and weather information sources. This includes current traffic congestion, road construction information, and weather conditions.

[0636] Step 2:

[0637] The server uses the collected data to run AI algorithms and analyzes it in comparison with user profile data. This profile data includes the user's past route choices and preferences, forming the basis of the analysis.

[0638] Step 3:

[0639] The server generates multiple candidate routes based on the analysis results. Each candidate route is evaluated based on factors such as traffic conditions, estimated travel time, and scenic beauty.

[0640] Step 4:

[0641] The server selects the optimal route from the evaluated routes and sends that route information to the user's terminal.

[0642] Step 5:

[0643] The user terminal displays the received route information on the navigation screen, allowing the user to refer to it while driving. It also provides route guidance via voice prompts.

[0644] Step 6:

[0645] The server retrieves information on potential stops from a database and creates a list of suggested spots based on the user's settings and past activity history. This list includes highly-rated tourist attractions and restaurants.

[0646] Step 7:

[0647] Users can give instructions to the system regarding routes and detours via a voice assistant. The user's voice commands are processed by being converted to text on the user's terminal and sent to the server.

[0648] Step 8:

[0649] The server receives the voice command, recalculates the route as needed, and resends the latest route guidance to the user.

[0650] In this way, the entire system works together to provide users with a consistent and personalized navigation experience.

[0651] (Example 1)

[0652] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0653] This invention aims to solve the complex route selection problems faced by users in modern transportation systems. In particular, there is a need to provide optimal routes that take into account traffic congestion, weather changes, and individual user preferences. Furthermore, there is a lack of flexible navigation systems that allow users to operate the system and change routes via voice while ensuring safety during travel.

[0654] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0655] In this invention, the server includes means for acquiring and analyzing environmental information, weather information, and user attribute information; means for generating multiple alternative routes based on the analyzed information and selecting the optimal route according to indicators; and means for interpreting user instructions and performing desired operations using a voice input function. This enables users to select the optimal and safe route according to dynamically changing traffic conditions and personal preferences.

[0656] "Environmental information" refers to general information related to road conditions, such as road congestion, traffic restrictions, and road construction information.

[0657] "Weather information" refers to information about weather conditions such as rain, snow, wind, and temperature, including the impact of these factors on traffic conditions and movement.

[0658] "User attribute information" refers to information about users' preferences, past selection history, and destinations, and is used to provide personalized services to individual users.

[0659] An "alternative route" refers to a route suggested from among several possible routes to the destination, depending on the circumstances.

[0660] An "indicator" refers to a set of criteria used to evaluate a route, such as distance, travel time, road conditions, and user preferences.

[0661] "Voice input functionality" is a technology that allows a user's voice to be input as data, enabling the system to recognize that voice and perform operations.

[0662] "Desired operation" refers to requests made by the user to the system, such as changing the route or resetting the destination, and the system's response to these requests.

[0663] This invention relates to a traffic navigation system in which a server and a terminal work together to provide route guidance optimized for the user.

[0664] The server utilizes various APIs to collect environmental, weather, and user attribute information via the internet. Specifically, it obtains road congestion and regulation information through general map service APIs, and weather information is obtained using weather information service APIs. Based on this information, the server analyzes the data using Python and machine learning frameworks. For example, Scikit-Learn's machine learning algorithms are used to predict past traffic patterns.

[0665] The server generates alternative routes based on the analysis results and evaluates them based on distance, time, and user preferences. Based on this, the server selects the optimal route and sends the selected route information to the terminal.

[0666] The terminal displays route information received from the server on the navigation screen, presenting it in a user-friendly format for drivers. The terminal also sends requests to the server to retrieve relevant location information based on the user's preferences. These relevant locations may include tourist attractions or restaurants, and the system uses an API similar to Foursquare to collect location and associated facility information.

[0667] Users can easily give instructions such as changing routes or obtaining additional information through the terminal's voice input function. The voice input uses a common speech recognition service to provide accurate analysis and responses.

[0668] For example, if a user gives a voice command such as "Tell me a route that goes through a scenic area," the server will analyze and select a route that meets that request, and the terminal will display the result to the user. An example of a prompt would be, "Suggest the best route for a user who prefers scenic routes. The starting point is a major city, the destination is a tourist spot, and the detours should include natural landscapes."

[0669] Through this system, users can enjoy a flexible, safe, and comfortable driving experience tailored to their individual needs.

[0670] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0671] Step 1:

[0672] The server collects environmental information, weather information, and user attribute information. This data becomes input to the server, and data is retrieved using traffic information APIs and weather information APIs. Specifically, API requests are issued and the data is received in JSON format. This is stored in a temporary database inside the server and used in the next processing step.

[0673] Step 2:

[0674] The server performs data analysis based on the collected data. The input is the information obtained in Step 1, which is then analyzed by a Python program. Specifically, it uses a machine learning algorithm to analyze traffic patterns and predicts suitable routes based on user profiles. As a result of the calculations, multiple alternative routes are generated, which become the input for the next step.

[0675] Step 3:

[0676] The server evaluates the generated alternative routes. The input is the set of routes generated in step 2, and the evaluation criteria include distance, travel time, and user preference. Each alternative route is scored, and the route with the highest score is selected as the output. This is then sent to the terminal.

[0677] Step 4:

[0678] The terminal receives optimal route information from the server. This route information is input to the terminal, which then displays it on the navigation screen. Specifically, the display method involves providing real-time route guidance using a map application.

[0679] Step 5:

[0680] The terminal sends a request for relevant location information to the server. The input includes user preference information, which the server uses to search for relevant locations and outputs the results to the terminal. This includes providing a list of suggested locations and suggesting potential stops for the user.

[0681] Step 6:

[0682] The user uses the terminal's voice input function to instruct the terminal to change the route. Voice data is provided to the terminal as input, and the terminal converts it into text data through a speech recognition algorithm. The converted data is sent to the server, which calculates the new optimal route and outputs the result back to the terminal.

[0683] Overall, the system processes a large amount of dynamically changing data and provides users with appropriate route guidance, thereby creating a comfortable and safe driving experience.

[0684] (Application Example 1)

[0685] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0686] Modern autonomous vehicles require advanced navigation systems to ensure safe and efficient travel. However, conventional systems fail to fully utilize real-time traffic information and individual user preferences, making it difficult to provide optimal route guidance. Furthermore, they lack personalized suggestions for detours that take user preferences into account. As a result, there is a challenge in that the convenience of autonomous vehicles cannot be fully realized.

[0687] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0688] In this invention, the server includes means for collecting and analyzing traffic information, weather information, and user profile information; means for generating multiple candidate routes based on the analyzed information and determining the optimal route according to evaluation criteria; and means for filtering and presenting detour point information to the user upon request. This makes it possible to provide safe and efficient route guidance in autonomous vehicles, as well as to provide personalized suggestions of detour points according to the user's preferences.

[0689] "Traffic information" refers to information that shows real-time traffic conditions, such as the degree of congestion on roads, the occurrence of accidents, and road closures.

[0690] "Weather information" refers to data that indicates the weather conditions on and around roads, and includes temperature, precipitation, wind speed, etc.

[0691] "User profile information" refers to individual information about the user of the navigation system, including age, hobbies, and past driving history.

[0692] A "candidate route" refers to a list of multiple possible travel routes from the starting point to the destination.

[0693] "Evaluation criteria" are standards used to compare candidate routes and determine which route is optimal, and these include factors such as time, distance, and traffic congestion.

[0694] "Detour information" refers to information about tourist spots, restaurants, rest areas, and other places you can stop at on your way to your destination.

[0695] "Voice recognition functionality" is a technology that converts a user's speech into digital signals, analyzes the content, and understands its meaning.

[0696] An "autonomous vehicle" is a vehicle that, thanks to its installed technology, can drive autonomously without driver intervention.

[0697] "Real-time updates" means that information is continuously updated to the latest version.

[0698] A "voice assistant" is software that recognizes voice commands and performs tasks based on their content.

[0699] The system for implementing this invention mainly consists of a server, a user terminal, and an autonomous vehicle. The server is responsible for collecting traffic information, weather information, and user profile information, and analyzing this data using AI to provide the user with the optimal driving route. This analysis utilizes data acquisition services such as the Google Maps API and the OpenWeatherMap API, as well as machine learning frameworks such as TensorFlow.

[0700] Based on the analysis results, the server generates multiple candidate routes and evaluates them according to evaluation criteria. These criteria include route distance, time, user preferences, and road conditions. After the optimal route is determined, information about it is sent to the user's terminal.

[0701] The user terminal displays route information received from the server through a navigation system. This navigation system incorporates speech recognition functionality using the Google Cloud Speech API. When a user issues a voice command, the system converts the command into text data, sends it to the server, and performs further analysis.

[0702] For example, if a user requests by voice, "I want to stop by a delicious ramen shop along the way," the system will search for potential locations near their current location in real time and suggest the optimal route for stopping there.

[0703] By using generative AI models, the system can provide more accurate route predictions and suggestions. Examples of prompts include "Suggest ramen spots along the drive from Tokyo to Osaka" and "Recalculate the route; I want to turn left instead of right."

[0704] In this way, users can achieve safe and efficient travel via autonomous vehicles.

[0705] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0706] Step 1:

[0707] The server obtains traffic and weather information from their respective APIs (e.g., Google Maps API and OpenWeatherMap API). The input is the user's current location and destination. The server uses this information to perform real-time data analysis and visualize traffic flow and weather changes.

[0708] Step 2:

[0709] The server receives user profile information from the user's terminal. The input consists of user preference information, including the user's past driving history and preferred detour spots. The server analyzes this information and uses it to generate route recommendation models optimized for each individual user.

[0710] Step 3:

[0711] The server generates multiple candidate routes based on all the information it acquires. The data used includes traffic conditions, weather information, and user profile information. The output is a route list that includes a recommendation index for each route. An AI algorithm is used to perform data calculations that take into account time, distance, weather, and user preferences.

[0712] Step 4:

[0713] The server evaluates the generated candidate routes according to evaluation criteria and determines the optimal route. The evaluation criteria are based on user-defined priorities (e.g., shortest time, shortest distance, interest level of detour spots, etc.). The output is the route determined to be optimal.

[0714] Step 5:

[0715] The user terminal displays the optimal route received from the server on the navigation system. The input is the optimal route information from the server. The output is the route drawn on the map and its detailed information (e.g., information about the next intersection to turn at and suggestions for places to stop along the way). The user receives navigation support through visual and voice guidance.

[0716] Step 6:

[0717] Users can give instructions to the system through voice recognition. For example, they might request, "I want to stop at a ramen shop on the way." The input is a voice command, which the system converts into text data and sends to the server.

[0718] Step 7:

[0719] The server re-analyzes the data based on the voice command and dynamically recalculates the route. The input is a new request based on the voice command. The output is updated route information. A generative AI model is used to quickly generate a route that includes new detour options.

[0720] In this way, the system can provide users with safe and efficient routes.

[0721] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0722] The car navigation system incorporating the emotion engine of the present invention provides a more personalized navigation experience by having a server acquire traffic data, weather data, and user profile data, and further analyze emotion data. The server uses various sensors and cameras to analyze the user's voice, facial expressions, and body language to understand the user's emotional state.

[0723] Based on this emotion recognition result, the server generates optimal candidate routes and selects the route that best suits the user's mood. For example, if the user is feeling stressed, the navigation system can prioritize suggesting quieter routes or routes that offer a sense of nature. Similarly, when suggesting detours, the system filters and presents appropriate spots according to the user's current emotions. For example, if the system determines that the user is tired, it will recommend places suitable for rest, such as quiet cafes or parks.

[0724] While driving, users can use a voice assistant to request route changes or add detours. The voice assistant takes into account the output of its emotion engine and adjusts the tone and content of its responses according to the user's emotions. For example, if the user is angry, the voice assistant will speak in a calm tone and offer suggestions to facilitate problem resolution.

[0725] All information is displayed on the user's terminal and provided to the user visually and audibly through voice feedback. This allows the present invention to provide drivers with an emotionally conscious, safe, and comfortable driving experience, reducing stress and enhancing the enjoyment of driving.

[0726] The following describes the processing flow.

[0727] Step 1:

[0728] The server retrieves relevant data from traffic information providers, weather data services, and user profile databases. This allows it to collect information about current traffic conditions, weather conditions, and user preferences.

[0729] Step 2:

[0730] The server captures the user's voice, facial expressions, and body language from sensors such as cameras and microphones installed inside the vehicle. The emotion engine analyzes this data in real time to estimate the user's emotional state.

[0731] Step 3:

[0732] The server inputs collected sentiment data and traffic / weather data into an AI algorithm for analysis. This generates multiple candidate routes that are best suited to the user's current emotions.

[0733] Step 4:

[0734] The server selects the route that best suits the user's emotional state from the generated routes, according to evaluation criteria. These criteria include travel time, the pleasantness of the scenery, and weighting based on the user's preferences.

[0735] Step 5:

[0736] The user terminal displays the optimal route information received from the server on the navigation screen. It also provides route guidance and instructions to the user via voice guidance.

[0737] Step 6:

[0738] The server filters information on potential detour spots from its database based on the user's emotional state and presents relaxing spots and places suitable for a change of pace to the user's terminal.

[0739] Step 7:

[0740] The user gives instructions regarding route changes or detours through a voice assistant. The device uses voice recognition to convert the commands into text and sends the content to the server.

[0741] Step 8:

[0742] The server calculates a new route based on the user's voice commands and emotional state, and redistributes the updated route information to the user's terminal. During this process, the voice assistant's response is adjusted according to the user's emotions, providing appropriate feedback.

[0743] (Example 2)

[0744] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0745] Conventional car navigation systems primarily select routes based on external conditions such as traffic and weather, and do not offer personalized route suggestions that take into account the user's emotional state. As a result, it has been difficult to provide optimal routes for drivers who are stressed or fatigued. This invention aims to improve driving comfort and safety by providing navigation that takes the user's emotional state into consideration.

[0746] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0747] In this invention, the server includes means for collecting and analyzing traffic information, weather information, and user profile information; means for analyzing the user's voice, facial expressions, and body language using sensors and cameras to evaluate the user's emotional state; and means for adjusting the voice assistant's response according to the user's emotions using a generative AI model. This enables the suggestion of an optimal route that takes the user's emotional state into consideration and the filtering of detour spots.

[0748] "Traffic information" is a general term for data related to traffic, such as the flow of vehicles on roads, congestion levels, and road closures.

[0749] "Weather information" refers to data that shows the weather conditions at any given time, and specifically includes temperature, precipitation, wind speed, and visibility information.

[0750] "User profile information" refers to personal information about a specific user, including past driving history and individual preferences.

[0751] "Emotional state" refers to the user's current psychological state and includes emotions such as joy, anger, sadness, and stress.

[0752] "Candidate routes" refer to multiple possible route options from the starting point to the destination.

[0753] "Evaluation criteria" are a set of indicators used to evaluate each route option, based on distance, time, scenery, and the user's emotional state.

[0754] An "emotion engine" is an algorithm or system for detecting and analyzing a user's emotional state.

[0755] "Detour spot information" refers to information about places that users can visit along the way, including tourist attractions, restaurants, cafes, and rest facilities.

[0756] "Voice recognition functionality" refers to technology that processes a user's speech as digital data and interprets the intended command.

[0757] A "generative AI model" refers to an artificial intelligence algorithm that learns from large amounts of data and performs predictions and classifications.

[0758] In this invention, the system mainly consists of three elements: a server, a terminal, and a user. The server first accesses multiple data sources to collect traffic information, weather information, and user profile information. Specifically, it obtains this information by using a traffic information API, a weather information API, and a profile database. User profile information includes past driving history and individual preferences.

[0759] The server uses voice sensors and cameras installed in the vehicle to analyze the user's voice tone, facial expressions, and body language in real time. This allows it to determine the user's current emotional state and, using a generative AI model, suggest the optimal route tailored to the user's emotions. For example, if the user is seeking relaxation, the server will prioritize suggesting quiet routes.

[0760] The terminal's role is to notify the user of information provided by the server through the in-car display and voice output. The voice assistant dynamically adjusts its tone and content according to the user's emotional state. Specifically, it provides guidance in a calm tone to users who are feeling stressed.

[0761] Users can request route changes or detours via voice input using voice recognition technology. The server receives these requests, uses a generative AI model to quickly calculate new routes, and provides navigation tailored to the user's preferences.

[0762] For example, if a user requests to "add a quiet cafe nearby to the route," the server will quickly gather information, consider the user's current emotional state, and then suggest the most suitable cafe options.

[0763] An example of a prompt message might be, "If the user is feeling stressed, please suggest a place where they can feel relaxed." This allows the system to provide the most appropriate information for the user.

[0764] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0765] Step 1:

[0766] The server collects traffic information, weather information, and user profile information from various sources. It calls APIs to retrieve the latest traffic and weather data. Input data includes GPS location information, weather APIs, and a profile database. The server analyzes this data to obtain outputs such as traffic delay information and weather patterns.

[0767] Step 2:

[0768] The server uses in-vehicle sensors and cameras to collect and analyze the user's voice, facial expressions, and body language in real time. Voice data and image data are used as input. A generative AI model is used to estimate the user's current emotional state. The output is an emotional state (e.g., stress, relief, excitement).

[0769] Step 3:

[0770] The server generates multiple candidate routes based on collected data and emotional states. Each route is evaluated using evaluation criteria according to the user's preferences and emotions. The input data consists of the outputs from steps 1 and 2. Using a generative AI model, the server outputs the most suitable route and sends it to the terminal.

[0771] Step 4:

[0772] The terminal presents route information received from the server to the user via an in-vehicle display and audio output. The input is route information from the server. The terminal converts this information into a human-readable format and outputs it, for example, displaying the route in map format on the display and providing voice guidance.

[0773] Step 5:

[0774] Users request route changes or detours from the system via voice input. The voice recognition function receives this input and processes the data for transmission to the server. The output is the request itself (e.g., "Please add nearby cafes").

[0775] Step 6:

[0776] The server receives a request from the user, collects new information, and re-evaluates the route. Here, a generative AI model is used to calculate the optimal route based on the new conditions in real time and send it back to the terminal. The output is the updated route information.

[0777] Step 7:

[0778] The terminal re-presents the updated route information to the user, and the voice assistant provides guidance in an appropriate tone and content. The input is the newly suggested route information. The terminal immediately reflects this information and outputs it in a way that reassures the user.

[0779] (Application Example 2)

[0780] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0781] In autonomous vehicles, conventional navigation systems have a problem: they only optimize routes based on traffic conditions and weather, without considering the user's emotional state. This tends to result in a uniform user experience, particularly limiting improvements in comfort and safety. Therefore, there is a need for methods to provide personalized route selection and ride experiences that take into account the individual user's emotional state.

[0782] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0783] In this invention, the server includes means for collecting and analyzing traffic condition data, weather data, and user profile data; means for generating multiple candidate routes based on the analyzed data and determining the optimal route according to evaluation criteria; and means for analyzing the user's emotional state using an emotion engine and adjusting the route and ride experience based on the emotional state. This enables safer and more comfortable navigation and ride experience optimized for the user's emotional state.

[0784] "Traffic condition data" refers to information about traffic flow and conditions, such as road congestion and accident information.

[0785] "Weather data" refers to information that shows weather conditions in a specific region, such as temperature, precipitation, and wind speed.

[0786] "User profile data" refers to data that includes personal information about individual users, such as their age, gender, past behavioral history, and preferences.

[0787] "Means of analysis" refer to processing methods and technical means for collecting data and using it to derive useful information.

[0788] "Candidate routes" refer to multiple route options for reaching a destination, and are evaluated to determine the priority of each route.

[0789] The "emotion engine" is a technology that analyzes the user's voice, facial expressions, and body language to determine their emotions at any given moment.

[0790] "Means of adjusting routes and ride experiences" refers to systems or functions that change the selected route or the environment during the ride according to the user's feelings and preferences.

[0791] "Speech recognition functionality" is a technology that collects the user's speech as data and processes it as text data.

[0792] This invention relates to a navigation system for autonomous vehicles that personalizes routes and ride experiences based on the user's emotional state. The server collects traffic condition data, weather data, and user profile data in real time and analyzes them comprehensively. The analysis incorporates an emotion engine that includes facial recognition and voice analysis, thereby understanding the user's current emotional state. This emotional state is fed back into the route selection algorithm, which then selects the route best suited to the user.

[0793] Specifically, the server uses a camera and microphone to collect the user's facial expression and voice data, and processes this data using the image analysis library OpenCV and the speech recognition service Google Cloud Speech-to-Text. At the same time, it uses the Emotion API to analyze characteristics derived from facial expressions and voice and evaluate emotions.

[0794] Based on the processing results, the server generates multiple candidate routes and selects the optimal route in light of evaluation criteria and the user's emotional state. The selected route is displayed on the in-car tablet or smartphone, and a voice assistant guides the user in a tone adjusted according to their emotions. For example, if the user is relaxed, it provides a quiet environment with music playing and suggests a park with abundant nature as a break point.

[0795] An example of a prompt might be, "To provide a fun family ride, please consider the passenger's current emotions and suggest the optimal route and settings." Based on this prompt, the generative AI model suggests the optimal route and settings according to the user's emotions. This makes it possible to provide the user with a relaxed and comfortable driving experience.

[0796] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0797] Step 1:

[0798] The server uses a camera and microphone to collect user facial and audio data in real time. Still images or video data are acquired as input from the camera, and audio data is input directly from the microphone. This data is used as basic information for emotion analysis.

[0799] Step 2:

[0800] The server analyzes facial expression data acquired using OpenCV and quantifies the characteristics of the expressions. Specifically, it extracts landmark information from the face and infers emotional states such as joy, anger, sadness, and happiness from its shape. This analysis result is input into the Emotion API.

[0801] Step 3:

[0802] The server uses Google Cloud Speech-to-Text to convert speech data into text data. This text data is logged as a result of speech recognition and is also used for sentiment analysis. Information is also collected from the intonation and word choice of the speech to estimate emotions.

[0803] Step 4:

[0804] The server uses the Emotion API to comprehensively evaluate information obtained from quantified facial features and voice to determine the user's emotional state. The Emotion API analyzes the input feature data and outputs it as an emotion score. This emotion score is used for route optimization in the next step.

[0805] Step 5:

[0806] The server collects traffic condition data, weather data, and user profile data, and generates multiple candidate routes based on evaluation criteria. This data is obtained from external data sources and acts as an indicator of travel time and comfort for each candidate route.

[0807] Step 6:

[0808] The server uses a program model to select the optimal route based on emotional scores. This selection prioritizes routes with scenic views that allow the user to relax and routes that cause less stress. A generative AI model evaluates the score of each route, taking into account emotional and other input data, and outputs the most suitable route.

[0809] Step 7:

[0810] The server displays the selected optimal route on an in-car tablet or smartphone and guides the user via a voice assistant. The voice assistant provides feedback in a tone that matches the user's emotional state, adjusting to enhance feelings of security and comfort. The prompts included in the voice feedback also change according to the user's emotions.

[0811] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0812] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0813] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0814] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0815] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0816] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0817] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0818] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0819] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0820] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0821] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0822] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0823] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0824] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0825] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0826] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0827] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0828] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0829] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0830] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0831] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0832] The following is further disclosed regarding the embodiments described above.

[0833] (Claim 1)

[0834] Means for collecting and analyzing traffic data, weather data, and user profile data,

[0835] A means for generating multiple candidate paths based on analyzed data and determining the optimal path according to evaluation criteria,

[0836] A means of filtering and presenting information on places to visit based on user requests,

[0837] A means for interpreting user input using speech recognition and performing the desired operation,

[0838] A system that includes this.

[0839] (Claim 2)

[0840] The system according to claim 1, which uses data updated in real time to optimize routes in response to changes in traffic conditions.

[0841] (Claim 3)

[0842] The system according to claim 1, which can be safely operated even while driving via a voice assistant.

[0843] "Example 1"

[0844] (Claim 1)

[0845] Means for acquiring and analyzing environmental information, weather information, and user attribute information,

[0846] A means for generating multiple alternative routes based on the analyzed information and selecting the optimal route according to the indicators,

[0847] A means of selecting and presenting relevant location information to the user according to the user's settings,

[0848] A means of interpreting user instructions using voice input and performing the desired operation,

[0849] A method for making predictions that take past traffic patterns into account using machine learning technology,

[0850] A system that includes this.

[0851] (Claim 2)

[0852] The system according to claim 1, which utilizes dynamically updated information to select a route in accordance with changes in traffic information.

[0853] (Claim 3)

[0854] The system according to claim 1, which can be safely operated even while on the move via voice functions.

[0855] "Application Example 1"

[0856] (Claim 1)

[0857] A means for collecting and analyzing traffic information, weather information, and user profile information,

[0858] A means for generating multiple candidate paths based on analyzed information and determining the optimal path according to evaluation criteria,

[0859] A means of filtering and presenting detour information to users according to their requests,

[0860] A means for interpreting user input using voice recognition and executing the desired operation,

[0861] A means of providing safe and comfortable route guidance for autonomous vehicles, and suggesting the optimal driving route including detours,

[0862] A system that includes this.

[0863] (Claim 2)

[0864] The system according to claim 1, which uses real-time updated information to optimize the route in response to changes in travel conditions and ensures safe driving for autonomous vehicles.

[0865] (Claim 3)

[0866] The system according to claim 1, which makes safe situational judgments via a voice assistant and can be operated without interruption even while the user is driving.

[0867] "Example 2 of combining an emotion engine"

[0868] (Claim 1)

[0869] A means for collecting and analyzing traffic information, weather information, and user profile information,

[0870] A means for generating multiple candidate paths, including the user's emotional state, based on the analyzed information, and determining the optimal path according to evaluation criteria,

[0871] A means of analyzing the user's voice, facial expressions, and body language using sensors and cameras to evaluate the user's emotional state,

[0872] A means of using an emotion engine to filter and present detour spot information to the user according to the user's current emotional state,

[0873] A means for interpreting user input using voice recognition and executing the desired operation,

[0874] A means of adjusting the voice assistant's response according to the user's emotions using a generative AI model,

[0875] A system that includes this.

[0876] (Claim 2)

[0877] The system according to claim 1, which uses real-time updated information to optimize routes in response to changes in traffic conditions and the emotional state of users.

[0878] (Claim 3)

[0879] The system according to claim 1, which allows for safe operation even while driving, while taking into consideration the user's emotional state, via a voice assistant.

[0880] "Application example 2 when combining with an emotional engine"

[0881] (Claim 1)

[0882] Means for collecting and analyzing traffic condition data, weather data, and user profile data,

[0883] A means for generating multiple candidate paths based on analyzed data and determining the optimal path according to evaluation criteria,

[0884] A means of filtering and presenting detour information to users according to their requests,

[0885] A means for interpreting user input using voice recognition and executing the desired operation,

[0886] A means of analyzing the user's emotional state using an emotion engine and adjusting the route and ride experience based on that emotional state,

[0887] A system that includes this.

[0888] (Claim 2)

[0889] The system according to claim 1, which uses real-time updated data to optimize routes in response to changes in traffic conditions and the emotional state of users.

[0890] (Claim 3)

[0891] The system according to claim 1, which allows for safe operation even while driving via a voice assistant and adjusts voice feedback according to the user's emotions. [Explanation of symbols]

[0892] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for collecting and analyzing traffic data, weather data, and user profile data, A means for generating multiple candidate paths based on analyzed data and determining the optimal path according to evaluation criteria, A means of filtering and presenting information on places to visit based on user requests, A means for interpreting user input using speech recognition and performing the desired operation, A system that includes this.

2. The system according to claim 1, which uses data updated in real time to optimize routes in response to changes in traffic conditions.

3. The system according to claim 1, which can be safely operated even while driving via a voice assistant.

Images