system

Abstract

We provide the system. [Solution] A means of converting voice input into text data, A method for analyzing text data using a generative model to set destinations and waypoints, A means of acquiring traffic information and proposing alternative routes in response to changes in traffic conditions, A means of notifying users of suggestions via voice and receiving user feedback, A means of updating route information based on user instructions, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Current navigation systems require setting destinations and waypoints through screen operations, which may distract visual attention during driving. Also, when traffic conditions change, it is difficult to quickly propose an alternative route, and there is a problem that flexible route changes cannot be made according to the user's intention. Therefore, improvement in safety and convenience is demanded.

Means for Solving the Problems

[0005] This invention solves the above problems by providing a system that converts voice input into text data and sets the user's destination and waypoints using a generative model. Furthermore, it achieves safe and flexible navigation by acquiring traffic information in real time, suggesting alternative routes and rest stops to the user via voice in response to changes in traffic conditions, and updating route information based on user feedback.

[0006] "Voice input" is a method of conveying information to a system by having the user speak.

[0007] "Text data" refers to digital information that represents voice input as text.

[0008] A "generative model" is an algorithm that uses machine learning to analyze vast amounts of data and generate new information or predictions.

[0009] The "destination" is the final destination that the user wishes to reach through navigation.

[0010] A "stopover point" is a place you plan to visit on your way to your destination.

[0011] "Traffic information" refers to real-time data on road congestion, traffic jams, accidents, and other related issues.

[0012] An "alternative route" is a route different from the usual route, suggested depending on traffic conditions and other factors.

[0013] A "user" is an individual who uses this navigation system.

[0014] "Feedback" refers to instructions or opinions that users provide to a system.

[0015] "Route information" refers to digital data about the route from the current location to the destination. [Brief explanation of the drawing]

[0016] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0018] First, let's explain the terminology used in the following explanation.

[0019] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0020] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention provides a flexible navigation system based on voice input. Users can specify destinations and waypoints to the navigation system via voice. For example, suppose a user says, "My destination is Tokyo Station, and I'd like to stop in Yokohama on the way."

[0038] The terminal converts the user's voice into text data and sends that data to the server. The server analyzes the received text data using a generative model and calculates the optimal route based on the specified destination and waypoints. In this calculation, it refers to a map database and takes into account the location information and distance of each point.

[0039] The server further obtains real-time traffic information via external traffic information services. This allows it to exclusively collect and analyze information such as congestion and road construction occurring on the specified route. Based on this analysis, the server generates suggestions for alternative routes or rest stops as needed.

[0040] The terminal notifies the user of information obtained from the server via voice. For example, it might say, "The Tomei Expressway to Tokyo Station is congested, so we recommend taking an alternative route using the Daisan Keihin Expressway." Based on this information, the user can provide voice feedback to the terminal, such as, "Please take that route."

[0041] User feedback is sent back from the terminal to the server, which updates the route information. Finally, the terminal continuously provides navigation while offering the updated route information to the user. In this way, the present invention minimizes visual input during driving and achieves safe and efficient navigation through natural interaction via voice.

[0042] The following describes the processing flow.

[0043] Step 1:

[0044] The user inputs by voice, "My destination is Tokyo Station, and I'd like to stop in Yokohama on the way."

[0045] Step 2:

[0046] The terminal receives voice input and converts it into text data using speech recognition technology. The converted text data is then sent to the server.

[0047] Step 3:

[0048] The server analyzes the received text data using a generative model to identify the destination and waypoints. It then refers to a map database to calculate the optimal route.

[0049] Step 4:

[0050] The server obtains real-time traffic information for the route via an external traffic information service.

[0051] Step 5:

[0052] The server uses the acquired traffic information to generate suggestions for alternative routes and rest stops as needed. It then sends this information to the terminal.

[0053] Step 6:

[0054] The terminal notifies the user of suggestions received from the server via voice and requests their feedback.

[0055] Step 7:

[0056] Users respond to suggestions by voice, instructing to adopt new routes or making other requests.

[0057] Step 8:

[0058] The device sends user feedback to the server, which then updates the route information based on the instructions.

[0059] Step 9:

[0060] The updated route information is sent to the terminal, and the terminal provides the user with continuous voice navigation.

[0061] (Example 1)

[0062] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0063] Conventional navigation systems have faced challenges in providing flexible route planning based on voice instructions and in rapidly changing routes in response to real-time traffic conditions. This makes it difficult to reach one's destination safely and efficiently.

[0064] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0065] In this invention, the server includes means for converting voice input into digital data, means for analyzing the digital data using a generative model to set locations and routes, and means for acquiring route information and proposing options in response to changes in route conditions. This allows users to easily set routes via voice and to dynamically update routes based on real-time information.

[0066] "Voice input" is a method of transmitting information to a system by having the user speak.

[0067] "Digital data" refers to data that represents information in a format that can be processed by a computer.

[0068] A "generative model" is an algorithm that uses artificial intelligence technology to analyze data and find specific patterns.

[0069] "Analysis" is the process of examining data and extracting meaningful information.

[0070] A "location" is information that indicates a specific geographical position.

[0071] A "route" is information that indicates the path between points.

[0072] "Route information" refers to information that shows the directions or geographical route to a destination.

[0073] "To propose" means to present options or solutions and encourage their implementation.

[0074] "User" is a term that refers to a person who operates the system.

[0075] "Response" refers to information such as replies and feedback from users.

[0076] A "server" is a computer system that processes information and provides data to other devices.

[0077] This invention provides a flexible navigation system based on voice input. Users can specify destinations and waypoints to the navigation system via voice. For example, suppose a user says, "My destination is the central station, and I'd like to stop at a park along the way."

[0078] The terminal receives voice from the user and converts it into digital data using speech recognition software. This process can utilize common speech recognition tools. The converted digital data is transmitted to a server via a communication network. The server analyzes the text data using a generative AI model and extracts destination and transit point information. Ideally, analysis software suitable for the generative AI model should be used for this analysis.

[0079] The server uses a geographic information system to calculate the optimal route. This is achieved by using the server's processing equipment to refer to map data and consider distance and travel time. Furthermore, the server obtains real-time traffic information from external traffic information services and suggests alternative routes as needed based on the obtained information. This service works in conjunction with the server's data processing equipment to make the latest traffic data available.

[0080] The terminal uses speech synthesis software to provide voice notifications to the user based on information provided by the server. This allows the user to navigate efficiently to their destination without relying on visual input, even while driving. The user receives the guidance and provides voice feedback, such as "Please take that route," which is then sent back to the server via the terminal. The server updates the route information based on the feedback, and the terminal provides it to the user. Through this process, the navigation system can always provide the most up-to-date information.

[0081] An example of a prompt message is: "Calculate the optimal route based on the user's specified location and route, and provide options based on real-time traffic information."

[0082] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0083] Step 1:

[0084] Users specify their destination and intermediate stops by voice. For example, they might say, "My destination is the central station, and I'd like to stop at a park along the way." This input is transmitted to the terminal as an audio signal.

[0085] Step 2:

[0086] The terminal converts the received audio signal into digital data using speech recognition software. The input is an audio signal, and the output is text data. This conversion process includes the operation of analyzing the audio signal and converting it into a corresponding string.

[0087] Step 3:

[0088] The terminal sends the converted text data to the server. Here, the text data becomes the input, and communication processing to the server is performed. The output is the text information passed to the server.

[0089] Step 4:

[0090] The server analyzes the received text data using a generative AI model. The input is text data, and location and route information is extracted through the analysis process. This analysis utilizes natural language processing techniques. The output is a set of location information.

[0091] Step 5:

[0092] The server references a geographic information system and calculates the optimal route based on location information. The input is a set of location information, and the server executes a route calculation algorithm based on this data. The output is detailed information about the optimal route. This step includes evaluation of distance and time.

[0093] Step 6:

[0094] The server obtains real-time traffic information from an external traffic information service. The input is a traffic information acquisition request, and the server obtains the latest traffic condition data through communication with the external service. The output is traffic condition data. Based on the acquired data, route adjustments are made.

[0095] Step 7:

[0096] The server proposes appropriate alternative routes based on traffic information. Inputs include detailed information on the optimal route and traffic condition data. This data is integrated to generate alternative routes. The output is the proposed alternative route information.

[0097] Step 8:

[0098] The terminal uses speech synthesis software to notify the user of route information provided by the server. The input is alternative route information, and speech synthesis generates voice guidance. The output is a voice message conveyed to the user. Specifically, it would announce, "As you head towards the central station, you will now pass through the park."

[0099] Step 9:

[0100] The user, upon receiving directions, provides voice feedback such as, "Please take that route." This voice feedback is entered into the device.

[0101] Step 10:

[0102] The terminal converts the user's voice feedback into text and sends it back to the server. The input is voice feedback, and the output is the converted text data. The server receives this information and updates the final route information.

[0103] (Application Example 1)

[0104] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0105] Navigation in autonomous vehicles requires the provision of appropriate routes in real time through natural voice interaction, rather than relying on visual information. However, conventional systems lack sufficient capabilities to set routes and respond to changing traffic conditions, and in particular, they lack efficient means to reduce the burden on the driver. This can lead to problems with safety and convenience.

[0106] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0107] In this invention, the server includes means for converting voice input into data, means for analyzing the data using a generative model to set points and waypoints, and means for acquiring information and proposing alternative routes in response to changes in circumstances. This enables voice-based route guidance in autonomous vehicles.

[0108] "Voice input" is a method of acquiring data from the voice spoken by the user.

[0109] Converting "sound" to "data" means analyzing a human voice and converting the audio signal into digital information.

[0110] A "generative model" is an algorithm used to analyze data and generate new information or predictions.

[0111] A "location" refers to a specific geographical position.

[0112] A "waypoint" is a point that you plan to pass through on your way to your destination.

[0113] "Acquiring information" means gathering necessary data and knowledge from external databases and services.

[0114] "Changes in circumstances" refers to factors that change over time or due to environmental factors.

[0115] An "alternative route" is a different path from several options for reaching a specified destination.

[0116] "Notification means" refers to methods or devices for conveying necessary information to the user.

[0117] "Receiving feedback" means receiving feedback and instructions from users and incorporating them into the system.

[0118] "Route information" refers to geographical and traffic data necessary for traveling between specific points.

[0119] An "autonomous vehicle" is a vehicle that operates autonomously through a system without human intervention.

[0120] "Voice-based route guidance" refers to notifying users of information about their travel route using voice output.

[0121] This invention provides a system for realizing flexible voice-based navigation in autonomous vehicles. The system includes a voice recognition device, a communication module, a server, a generative AI model, a map database, and a real-time traffic information service.

[0122] The speech recognition device will acquire the user's voice with high accuracy within the autonomous vehicle and convert this voice into digital data. Possible technologies to be used include Google® Cloud Speech-to-Text API and Azure® Speech Service.

[0123] The converted voice data is transmitted to the server via a communication module. The server analyzes the voice data using an OpenAI® generative AI model to accurately understand the destination and waypoints specified by the user. Based on the analysis results, the server calculates the optimal route while referring to a map database. The map database used includes the Google Maps API. Furthermore, the server utilizes real-time traffic information services such as the Here Maps API to adjust the route to reflect current traffic conditions.

[0124] The optimal route calculated by the server is communicated to the user via voice through an interface within the autonomous vehicle. For this voice synthesis, which requires speed and accuracy, technologies such as Amazon Polly and Microsoft® Azure Cognitive Services are used.

[0125] For example, if a user gives voice instructions during a long-distance family road trip, such as "I want to leave Tokyo, see Mt. Fuji along the way, and then go to Osaka," the system will analyze the user's intent and provide the optimal route. This route will also include alternative routes depending on traffic conditions.

[0126] An example of a prompt in this invention is, "If the user requests voice navigation from Tokyo to Osaka, please tell me the best route." In this way, the user can navigate efficiently and safely through natural conversation without visual interaction.

[0127] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0128] Step 1:

[0129] Users use a voice input device inside the autonomous vehicle to give voice instructions for destinations and waypoints. The voice input device converts this voice signal into digital data. The output data is a digital representation of the voice waveform.

[0130] Step 2:

[0131] The device receives the converted audio data. Using speech recognition technology, it converts the audio waveform into text data. This process utilizes the Google Cloud Speech-to-Text API and Azure Speech Service. The output is a text representation of the user's voice commands.

[0132] Step 3:

[0133] The terminal sends text data to the server via a communication module. The server uses an OpenAI generative AI model to analyze the input text data and identify the specified destination and waypoints. The output is structured data for route planning.

[0134] Step 4:

[0135] The server consults a map database and calculates the optimal route based on the specified destination and waypoints. This process utilizes services such as the Google Maps API. The output is information about the optimal route to the destination.

[0136] Step 5:

[0137] The server obtains real-time traffic information via the Here Maps API and other means, and incorporates it into the calculated route. It sets alternative routes as needed, taking into account traffic conditions and construction information. The output is an adjusted route that takes real-time information into account.

[0138] Step 6:

[0139] The server sends the optimized routing information to the terminal. The terminal uses speech synthesis technology to notify the user of the routing information as a voice message. At this stage, Amazon Polly or Azure Cognitive Services are used. The output is the voice guidance transmitted to the user.

[0140] Step 7:

[0141] The user operates the autonomous vehicle according to the guided route. If necessary, they provide voice feedback, and the system updates the route information accordingly. This ensures the user continuously receives optimal navigation in real time.

[0142] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0143] This invention combines a voice-input-based, flexible navigation system with an emotion engine to achieve user-responsive interaction. Users can specify destinations and waypoints by voice. Furthermore, the system recognizes emotions from the user's voice and provides feedback and suggestions accordingly. For example, consider a scenario where the user says, "My destination is Tokyo Station, but I'm worried about traffic."

[0144] The device converts the user's voice into text data and simultaneously analyzes the emotions from that voice using an emotion engine. The analyzed text data and emotion information are sent to the server. The server analyzes the text data using a generative model, identifies the specified destination and waypoints, and calculates the optimal route.

[0145] Furthermore, the server obtains real-time traffic information from external traffic information services and generates route and rest stop suggestions that take into account the user's emotional state. For example, if the server detects that the user is feeling stressed, it can recommend relaxing music or suggest a comfortable rest stop.

[0146] The terminal notifies the user of suggestions from the server via voice and provides feedback based on sentiment recognition results. The user can respond to the suggestions via voice and decide whether to accept or reject the new route or the suggestion. For example, they can give instructions such as "That route is fine" or "Is there a more comfortable route?"

[0147] The server updates route information based on user instructions and sends the updated information to the terminal. The terminal continuously provides the user with voice guidance that reflects the route information. In this way, the present invention realizes safer and more comfortable navigation by providing driving assistance that takes user emotions into consideration.

[0148] The following describes the processing flow.

[0149] Step 1:

[0150] The user inputs by voice, "My destination is Tokyo Station, and I'm worried about traffic."

[0151] Step 2:

[0152] The device receives voice input, converts it into text data using speech recognition technology, and simultaneously analyzes the emotions using an emotion engine. The emotional state and text data are then sent to the server.

[0153] Step 3:

[0154] The server analyzes the received text data using a generative model to identify the destination and intermediate points. It also obtains real-time traffic conditions using an external traffic information service.

[0155] Step 4:

[0156] The server takes into account the user's emotional state and traffic information to calculate the optimal route and rest stops, and includes relaxing music and guidance in its suggestions as needed.

[0157] Step 5:

[0158] The server sends the generated suggestions to the terminal.

[0159] Step 6:

[0160] The terminal notifies the user via voice of suggestions received from the server. For example, it might say, "There is currently traffic congestion to Tokyo Station, so we recommend an alternative route. Also, would you like us to play some music to help you relax?"

[0161] Step 7:

[0162] Users respond to suggestions via voice, instructing the system to adopt new routes or play music that matches their emotions.

[0163] Step 8:

[0164] The device sends user feedback to the server, which then updates route information and suggestions based on that feedback.

[0165] Step 9:

[0166] Updated route information and suggestions are sent to the device, which then provides the user with continuous voice navigation and emotionally sensitive service.

[0167] (Example 2)

[0168] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0169] Conventional navigation systems allow route setting via voice input, but they lack flexible interaction that takes user emotions into consideration. As a result, it is difficult to provide appropriate route guidance and alternative route suggestions while reducing user stress. Therefore, there is a need to develop navigation systems that take the user's emotional state into account and provide more comfortable and intuitive guidance.

[0170] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0171] In this invention, the server includes processing means for converting voice input into text data, processing means for analyzing the text data using a generative model and setting destinations and waypoints, and processing means for analyzing the user's emotions from the user's voice data. This makes it possible to propose the optimal route in real time while taking the user's emotions into consideration, enabling more comfortable and less stressful navigation.

[0172] "Voice input" is an input method that captures the content of what the user says as data.

[0173] "Text data" refers to character information converted from voice input.

[0174] A "generative model" is an algorithm or program that analyzes input text data to identify destinations and configuration information.

[0175] "Emotional analysis" is a technology that identifies a user's emotional state from their voice.

[0176] "Traffic information" is a general term for data that includes road congestion, accident information, traffic restrictions, and other operational conditions.

[0177] An "alternative route" is a different travel route proposed based on current traffic conditions and user requests.

[0178] "User feedback" refers to users' reactions and instructions to suggestions and guidance from the system.

[0179] "Route information" refers to guidance information for reaching a designated destination or intermediate points.

[0180] In an embodiment of the present invention, the user provides voice input through a terminal. The terminal uses a built-in microphone to acquire the voice input. The acquired voice data is converted into text data using speech recognition software. For example, a speech recognition API can be used for this process.

[0181] Furthermore, the device incorporates a generative AI model to analyze the converted text data and identify destinations and intermediate points. This generative AI model might include, for example, a large-scale language model used for natural language processing. Additionally, an emotion engine is used to analyze the user's emotions from their voice. This emotion analysis employs machine learning algorithms to determine the user's level of stress and comfort.

[0182] The analyzed text data and sentiment information are sent to the server. The server obtains real-time traffic information using an external traffic information service. This traffic information is obtained, for example, via an API from a map service provider. The server then considers the user's sentiment state and generates suggestions for the optimal route, waypoints, and rest stops.

[0183] The terminal notifies the user of the received suggestions using speech synthesis software. The user can respond to the suggestions by voice, accepting them or giving further instructions. Based on the user's voice feedback, the server updates the route information and sends optimized, up-to-date directions to the terminal.

[0184] As a concrete example, a user can enter a prompt message into the terminal such as, "My current destination is Tokyo Skytree. I'd like to stop at a cafe along the way; do you have any recommendations? I'd like to avoid crowded places," and receive appropriate route and cafe suggestions. This enables flexible and emotionally sensitive driving assistance.

[0185] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0186] Step 1:

[0187] The user specifies their destination and intermediate stops by voice. This voice data is input to the terminal. The terminal uses its built-in microphone to acquire the user's voice data, and then uses voice recognition software to convert the voice into text data. As a result, the voice input is output as text data.

[0188] Step 2:

[0189] The device inputs the converted text data into a generating AI model. This model uses natural language processing technology to analyze the text data and extract information about the destination and intermediate points. Simultaneously, it analyzes the user's emotions using an emotion engine based on their voice data. As a result, the analyzed destination information and emotion information are output.

[0190] Step 3:

[0191] The terminal sends the text data and sentiment information obtained in the previous step to the server. The server receives this data and also obtains real-time traffic information via an external traffic information service. Using the obtained traffic information, the server generates the optimal route, taking into account the destination, intermediate stops, and sentiment state. In this process, a suggested route based on traffic information and the user's state is output.

[0192] Step 4:

[0193] The server sends the generated optimal route and suggestions to the terminal. The terminal uses speech synthesis software to inform the user of the suggestions verbally. The user responds based on the verbal guidance. This response is input into the terminal and sent to the server.

[0194] Step 5:

[0195] The server analyzes voice feedback from the user and updates route information as needed. The updated route information is sent to the terminal. The terminal receives this information and continuously provides voice guidance to the user based on the updated information. In this process, the user receives the most up-to-date route guidance, resulting in more comfortable and appropriate navigation.

[0196] (Application Example 2)

[0197] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0198] In autonomous vehicles and navigation systems, simply guiding users to their destination without considering their emotions or feelings presents a challenge in adequately enhancing safety and comfort. In this context, the ability to flexibly set and suggest routes that respond to user emotions is believed to provide higher levels of satisfaction and safety.

[0199] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0200] In this invention, the server includes means for converting voice input into information data, means for analyzing the information data using a generative model and setting location information and waypoints, and means for analyzing the emotional state and proposing alternative routes and stopping points according to the user's emotions. This makes it possible to propose navigation that takes the user's emotions into consideration.

[0201] "Voice input" is the process of converting audio signals into digital information.

[0202] "Information data" refers to digital data used to store or transmit processed information.

[0203] A "generative model" is a machine learning model that generates a specific output based on input data.

[0204] "Location information" refers to digital data that indicates the geographical location of an object.

[0205] A "waypoint" is a point that must be passed through in order to reach the destination.

[0206] "Emotional state" is a state indicator that shows the user's emotions.

[0207] An "alternative route" is a different travel route that takes place instead of the primary route.

[0208] A "stopping point" is a location set during navigation for the purpose of temporarily stopping or resting the vehicle.

[0209] To realize this invention, it is necessary to build a system that converts voice input into information data, analyzes emotional states, and provides navigation suggestions. For this purpose, the terminal receives voice input and uses a microphone and software to convert the voice signal into digital information. Specifically, the Google Cloud Speech-to-Text API is used to convert the voice into text information.

[0210] Next, the server uses a generative AI model to analyze information obtained from the voice data. This generative model can identify the user's emotional state using IBM Watson's (registered trademark) sentiment analysis API. The analyzed information is used to set location information and waypoints.

[0211] Furthermore, the server can obtain situational information from external services, including the Google Maps API. Based on the information obtained in real time, the server suggests alternative routes and stopping points that are tailored to the user's emotions.

[0212] The device uses its speaker to notify the user of these suggestions via voice output. Furthermore, the interaction continues by receiving user feedback again as voice and feeding it back into the analysis cycle.

[0213] For example, if a user says, "I'm in a hurry to get to my destination, but I'm worried about traffic," the system will calculate a fast route while also suggesting a route that includes comfortable spots to alleviate the user's stress. An example of a prompt might be, "Please suggest a quieter route that avoids highways. The user wants to relax."

[0214] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0215] Step 1:

[0216] The device uses a microphone to receive voice input from the user. The input voice is converted into text data using the Google Cloud Speech-to-Text API. This text data is used as the basis for subsequent processing.

[0217] Step 2:

[0218] The server receives the text data acquired in step 1 and analyzes the user's emotional state using IBM Watson's emotion analysis API. This emotion analysis determines the type and intensity of the emotion and outputs it as emotional state data.

[0219] Step 3:

[0220] The server identifies user-defined location information and waypoints based on text data. A generative AI model is used to calculate the optimal route based on this information. The output consists of location information and the calculated route data.

[0221] Step 4:

[0222] The server uses the Google Maps API in real time to retrieve the latest situation information from an external source. This situation information is combined with the data from step 3 to suggest alternative routes and stopping points that take emotional states into account. This suggested data is then sent to the next process.

[0223] Step 5:

[0224] The terminal notifies the user of the suggested data generated in step 4 through an audio output device. A speaker is used for this purpose, and the user listens to the audio guidance and provides feedback.

[0225] Step 6:

[0226] The user provides voice feedback, which the device receives again as voice input. This feedback is used in the next process cycle, and the system recalculates to update the route and suggestions based on the new information.

[0227] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0228] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0229] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0230] [Second Embodiment]

[0231] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0232] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0233] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0234] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0235] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0236] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0237] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0238] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0239] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0240] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0241] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0242] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0243] This invention provides a flexible navigation system based on voice input. Users can specify destinations and waypoints to the navigation system via voice. For example, suppose a user says, "My destination is Tokyo Station, and I'd like to stop in Yokohama on the way."

[0244] The terminal converts the user's voice into text data and sends that data to the server. The server analyzes the received text data using a generative model and calculates the optimal route based on the specified destination and waypoints. In this calculation, it refers to a map database and takes into account the location information and distance of each point.

[0245] The server further obtains real-time traffic information via external traffic information services. This allows it to exclusively collect and analyze information such as congestion and road construction occurring on the specified route. Based on this analysis, the server generates suggestions for alternative routes or rest stops as needed.

[0246] The terminal notifies the user of information obtained from the server via voice. For example, it might say, "The Tomei Expressway to Tokyo Station is congested, so we recommend taking an alternative route using the Daisan Keihin Expressway." Based on this information, the user can provide voice feedback to the terminal, such as, "Please take that route."

[0247] User feedback is sent back from the terminal to the server, which updates the route information. Finally, the terminal continuously provides navigation while offering the updated route information to the user. In this way, the present invention minimizes visual input during driving and achieves safe and efficient navigation through natural interaction via voice.

[0248] The following describes the processing flow.

[0249] Step 1:

[0250] The user inputs by voice, "My destination is Tokyo Station, and I'd like to stop in Yokohama on the way."

[0251] Step 2:

[0252] The terminal receives voice input and converts it into text data using speech recognition technology. The converted text data is then sent to the server.

[0253] Step 3:

[0254] The server analyzes the received text data using a generative model to identify the destination and waypoints. It then refers to a map database to calculate the optimal route.

[0255] Step 4:

[0256] The server obtains real-time traffic information for the route via an external traffic information service.

[0257] Step 5:

[0258] The server uses the acquired traffic information to generate suggestions for alternative routes and rest stops as needed. It then sends this information to the terminal.

[0259] Step 6:

[0260] The terminal notifies the user of suggestions received from the server via voice and requests their feedback.

[0261] Step 7:

[0262] Users respond to suggestions by voice, instructing to adopt new routes or making other requests.

[0263] Step 8:

[0264] The device sends user feedback to the server, which then updates the route information based on the instructions.

[0265] Step 9:

[0266] The updated route information is sent to the terminal, and the terminal provides the user with continuous voice navigation.

[0267] (Example 1)

[0268] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0269] Conventional navigation systems have faced challenges in providing flexible route planning based on voice instructions and in rapidly changing routes in response to real-time traffic conditions. This makes it difficult to reach one's destination safely and efficiently.

[0270] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0271] In this invention, the server includes means for converting voice input into digital data, means for analyzing the digital data using a generative model to set locations and routes, and means for acquiring route information and proposing options in response to changes in route conditions. This allows users to easily set routes via voice and to dynamically update routes based on real-time information.

[0272] "Voice input" is a method of transmitting information to a system by having the user speak.

[0273] "Digital data" refers to data that represents information in a format that can be processed by a computer.

[0274] A "generative model" is an algorithm that uses artificial intelligence technology to analyze data and find specific patterns.

[0275] "Analysis" is the process of examining data and extracting meaningful information.

[0276] A "location" is information that indicates a specific geographical position.

[0277] A "route" is information that indicates the path between points.

[0278] "Route information" refers to information that shows the directions or geographical route to a destination.

[0279] "To propose" means to show options or improvement measures and encourage implementation.

[0280] "User" is a term that refers to a person who operates a system.

[0281] "Response" refers to information that is a reply or feedback from the user.

[0282] "Server" is a computer system that processes information and provides data to other devices.

[0283] The present invention provides a flexible navigation system based on voice input. The user can specify a destination or a transit point to the navigation system through voice. For example, suppose the user says "The destination is Central Station, and I want to stop by the park on the way."

[0284] The terminal receives the voice from the user and converts it into digital data using voice recognition software. General voice recognition tools can be used for this process. The converted digital data is transmitted to the server via a communication network. The server analyzes the text data using a generative AI model and extracts information on the destination and transit point. It is desirable to use analysis software suitable for the generative AI model for the analysis.

[0285] The server calculates the optimal route using a geographic information system. This is achieved by the server's processing device considering distance and travel time while referring to map data. Furthermore, the server obtains real-time traffic conditions from an external traffic information service and proposes an alternative route if necessary based on the obtained information. This service cooperates with the server's data processing device to make the latest traffic data available.

[0286] Based on the information provided by the server, the terminal uses voice synthesis software to send voice notifications to the user. As a result, the user can achieve efficient navigation to the destination without relying on visual inputs even while driving. The user provides voice feedback such as "Please use that route" upon receiving the guidance and sends it to the server again through the terminal. The server updates the route information based on the feedback, and the terminal provides it to the user. Through such a process, the navigation system can continuously provide the latest information.

[0287] An example of the prompt text is "Please calculate the optimal route based on the location and route specified by the user and provide options based on real-time traffic information."

[0288] The flow of the specific process in Example 1 will be described using FIG. 11.

[0289] Step 1:

[0290] The user specifies the destination and intermediate stops verbally. For example, the user makes a voice input such as "The destination is Central Station, and I want to stop by the park on the way." This input is transmitted to the terminal as a voice signal.

[0291] Step 2:

[0292] The terminal converts the received voice signal into digital data using voice recognition software. The input is a voice signal, and the output is text data. This conversion process includes the operation of converting the voice signal into the corresponding character string through analysis.

[0293] Step 3:

[0294] The terminal sends the converted text data to the server. Here, the text data is the input, and the communication process to the server is executed. The output is the text information passed to the server.

[0295] Step 4:

[0296] The server analyzes the received text data using a generative AI model. The input is text data, and location and route information is extracted through the analysis process. This analysis utilizes natural language processing techniques. The output is a set of location information.

[0297] Step 5:

[0298] The server references a geographic information system and calculates the optimal route based on location information. The input is a set of location information, and the server executes a route calculation algorithm based on this data. The output is detailed information about the optimal route. This step includes evaluation of distance and time.

[0299] Step 6:

[0300] The server obtains real-time traffic information from an external traffic information service. The input is a traffic information acquisition request, and the server obtains the latest traffic condition data through communication with the external service. The output is traffic condition data. Based on the acquired data, route adjustments are made.

[0301] Step 7:

[0302] The server proposes appropriate alternative routes based on traffic information. Inputs include detailed information on the optimal route and traffic condition data. This data is integrated to generate alternative routes. The output is the proposed alternative route information.

[0303] Step 8:

[0304] The terminal uses speech synthesis software to notify the user of route information provided by the server. The input is alternative route information, and speech synthesis generates voice guidance. The output is a voice message conveyed to the user. Specifically, it would announce, "As you head towards the central station, you will now pass through the park."

[0305] Step 9:

[0306] The user provides voice feedback saying "Please use that route" upon receiving the guidance. This voice feedback is input into the terminal.

[0307] Step 10:

[0308] The terminal converts the voice feedback from the user into text and sends it to the server again. The input is the voice feedback, and the output is the converted text data. The server receives this information and updates the final route information.

[0309] (Application Example 1)

[0310] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0311] In the navigation of an autonomous vehicle, it is required to provide an appropriate route in real time through natural voice interaction without relying on visual information. However, in conventional systems, the ability to handle route settings and changes in traffic conditions is not sufficient, and there is a lack of efficient means to reduce the driver's burden. This may cause problems in safety and convenience.

[0312] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0313] In this invention, the server includes means for converting voice input into data, means for analyzing the data using a generation model to set points and waypoints, and means for obtaining information and proposing an alternative route according to changes in the situation. Thereby, voice-based route guidance in an autonomous vehicle becomes possible.

[0314] "Voice input" is a method of acquiring the voice uttered by the user as data.

[0315] Converting "sound" to "data" means analyzing a human voice and converting the audio signal into digital information.

[0316] A "generative model" is an algorithm used to analyze data and generate new information or predictions.

[0317] A "location" refers to a specific geographical position.

[0318] A "waypoint" is a point that you plan to pass through on your way to your destination.

[0319] "Acquiring information" means gathering necessary data and knowledge from external databases and services.

[0320] "Changes in circumstances" refers to factors that change over time or due to environmental factors.

[0321] An "alternative route" is a different path from several options for reaching a specified destination.

[0322] "Notification means" refers to methods or devices for conveying necessary information to the user.

[0323] "Receiving feedback" means receiving feedback and instructions from users and incorporating them into the system.

[0324] "Route information" refers to geographical and traffic data necessary for traveling between specific points.

[0325] An "autonomous vehicle" is a vehicle that operates autonomously through a system without human intervention.

[0326] "Voice-based route guidance" refers to notifying users of information about their travel route using voice output.

[0327] This invention provides a system for realizing flexible voice-based navigation in autonomous vehicles. The system includes a voice recognition device, a communication module, a server, a generative AI model, a map database, and a real-time traffic information service.

[0328] The speech recognition device will capture the user's voice with high accuracy within the autonomous vehicle and convert this voice into digital data. Possible technologies to be used include Google Cloud Speech-to-Text API and Azure Speech Service.

[0329] The converted voice data is sent to the server via a communication module. The server analyzes the voice data using OpenAI's generative AI model to accurately understand the destination and waypoints specified by the user. Based on the analysis results, the server calculates the optimal route while referring to a map database. The map database used includes the Google Maps API. Furthermore, the server utilizes real-time traffic information services such as the Here Maps API to adjust the route to reflect current traffic conditions.

[0330] The optimal route calculated by the server is communicated to the user via voice through an interface within the autonomous vehicle. For this voice synthesis, which requires speed and accuracy, technologies such as Amazon Polly or Microsoft Azure Cognitive Services are used.

[0331] For example, if a user gives voice instructions during a long-distance family road trip, such as "I want to leave Tokyo, see Mt. Fuji along the way, and then go to Osaka," the system will analyze the user's intent and provide the optimal route. This route will also include alternative routes depending on traffic conditions.

[0332] An example of a prompt in this invention is, "If the user requests voice navigation from Tokyo to Osaka, please tell me the best route." In this way, the user can navigate efficiently and safely through natural conversation without visual interaction.

[0333] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0334] Step 1:

[0335] Users use a voice input device inside the autonomous vehicle to give voice instructions for destinations and waypoints. The voice input device converts this voice signal into digital data. The output data is a digital representation of the voice waveform.

[0336] Step 2:

[0337] The device receives the converted audio data. Using speech recognition technology, it converts the audio waveform into text data. This process utilizes the Google Cloud Speech-to-Text API and Azure Speech Service. The output is a text representation of the user's voice commands.

[0338] Step 3:

[0339] The terminal sends text data to the server via a communication module. The server uses an OpenAI generative AI model to analyze the input text data and identify the specified destination and waypoints. The output is structured data for route planning.

[0340] Step 4:

[0341] The server consults a map database and calculates the optimal route based on the specified destination and waypoints. This process utilizes services such as the Google Maps API. The output is information about the optimal route to the destination.

[0342] Step 5:

[0343] The server obtains real-time traffic information via the Here Maps API and other means, and incorporates it into the calculated route. It sets alternative routes as needed, taking into account traffic conditions and construction information. The output is an adjusted route that takes real-time information into account.

[0344] Step 6:

[0345] The server sends the optimized routing information to the terminal. The terminal uses speech synthesis technology to notify the user of the routing information as a voice message. At this stage, Amazon Polly or Azure Cognitive Services are used. The output is the voice guidance transmitted to the user.

[0346] Step 7:

[0347] The user operates the autonomous vehicle according to the guided route. If necessary, they provide voice feedback, and the system updates the route information accordingly. This ensures the user continuously receives optimal navigation in real time.

[0348] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0349] This invention combines a voice-input-based, flexible navigation system with an emotion engine to achieve user-responsive interaction. Users can specify destinations and waypoints by voice. Furthermore, the system recognizes emotions from the user's voice and provides feedback and suggestions accordingly. For example, consider a scenario where the user says, "My destination is Tokyo Station, but I'm worried about traffic."

[0350] The device converts the user's voice into text data and simultaneously analyzes the emotions from that voice using an emotion engine. The analyzed text data and emotion information are sent to the server. The server analyzes the text data using a generative model, identifies the specified destination and waypoints, and calculates the optimal route.

[0351] Furthermore, the server obtains real-time traffic information from external traffic information services and generates route and rest stop suggestions that take into account the user's emotional state. For example, if the server detects that the user is feeling stressed, it can recommend relaxing music or suggest a comfortable rest stop.

[0352] The terminal notifies the user of suggestions from the server via voice and provides feedback based on sentiment recognition results. The user can respond to the suggestions via voice and decide whether to accept or reject the new route or the suggestion. For example, they can give instructions such as "That route is fine" or "Is there a more comfortable route?"

[0353] The server updates route information based on user instructions and sends the updated information to the terminal. The terminal continuously provides the user with voice guidance that reflects the route information. In this way, the present invention realizes safer and more comfortable navigation by providing driving assistance that takes user emotions into consideration.

[0354] The following describes the processing flow.

[0355] Step 1:

[0356] The user inputs by voice, "My destination is Tokyo Station, and I'm worried about traffic."

[0357] Step 2:

[0358] The device receives voice input, converts it into text data using speech recognition technology, and simultaneously analyzes the emotions using an emotion engine. The emotional state and text data are then sent to the server.

[0359] Step 3:

[0360] The server analyzes the received text data using a generative model to identify the destination and intermediate points. It also obtains real-time traffic conditions using an external traffic information service.

[0361] Step 4:

[0362] The server takes into account the user's emotional state and traffic information to calculate the optimal route and rest stops, and includes relaxing music and guidance in its suggestions as needed.

[0363] Step 5:

[0364] The server sends the generated suggestions to the terminal.

[0365] Step 6:

[0366] The terminal notifies the user via voice of suggestions received from the server. For example, it might say, "There is currently traffic congestion to Tokyo Station, so we recommend an alternative route. Also, would you like us to play some music to help you relax?"

[0367] Step 7:

[0368] Users respond to suggestions via voice, instructing the system to adopt new routes or play music that matches their emotions.

[0369] Step 8:

[0370] The device sends user feedback to the server, which then updates route information and suggestions based on that feedback.

[0371] Step 9:

[0372] Updated route information and suggestions are sent to the device, which then provides the user with continuous voice navigation and emotionally sensitive service.

[0373] (Example 2)

[0374] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0375] Conventional navigation systems allow route setting via voice input, but they lack flexible interaction that takes user emotions into consideration. As a result, it is difficult to provide appropriate route guidance and alternative route suggestions while reducing user stress. Therefore, there is a need to develop navigation systems that take the user's emotional state into account and provide more comfortable and intuitive guidance.

[0376] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0377] In this invention, the server includes processing means for converting voice input into text data, processing means for analyzing the text data using a generative model and setting destinations and waypoints, and processing means for analyzing the user's emotions from the user's voice data. This makes it possible to propose the optimal route in real time while taking the user's emotions into consideration, enabling more comfortable and less stressful navigation.

[0378] "Voice input" is an input method that captures the content of what the user says as data.

[0379] "Text data" refers to character information converted from voice input.

[0380] A "generative model" is an algorithm or program that analyzes input text data to identify destinations and configuration information.

[0381] "Emotional analysis" is a technology that identifies a user's emotional state from their voice.

[0382] "Traffic information" is a general term for data that includes road congestion, accident information, traffic restrictions, and other operational conditions.

[0383] An "alternative route" is a different travel route proposed based on current traffic conditions and user requests.

[0384] "User feedback" refers to users' reactions and instructions to suggestions and guidance from the system.

[0385] "Route information" refers to guidance information for reaching a designated destination or intermediate points.

[0386] In an embodiment of the present invention, the user provides voice input through a terminal. The terminal uses a built-in microphone to acquire the voice input. The acquired voice data is converted into text data using speech recognition software. For example, a speech recognition API can be used for this process.

[0387] Furthermore, the device incorporates a generative AI model to analyze the converted text data and identify destinations and intermediate points. This generative AI model might include, for example, a large-scale language model used for natural language processing. Additionally, an emotion engine is used to analyze the user's emotions from their voice. This emotion analysis employs machine learning algorithms to determine the user's level of stress and comfort.

[0388] The analyzed text data and sentiment information are sent to the server. The server obtains real-time traffic information using an external traffic information service. This traffic information is obtained, for example, via an API from a map service provider. The server then considers the user's sentiment state and generates suggestions for the optimal route, waypoints, and rest stops.

[0389] The terminal notifies the user of the received suggestions using speech synthesis software. The user can respond to the suggestions by voice, accepting them or giving further instructions. Based on the user's voice feedback, the server updates the route information and sends optimized, up-to-date directions to the terminal.

[0390] As a concrete example, a user can enter a prompt message into the terminal such as, "My current destination is Tokyo Skytree. I'd like to stop at a cafe along the way; do you have any recommendations? I'd like to avoid crowded places," and receive appropriate route and cafe suggestions. This enables flexible and emotionally sensitive driving assistance.

[0391] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0392] Step 1:

[0393] The user specifies their destination and intermediate stops by voice. This voice data is input to the terminal. The terminal uses its built-in microphone to acquire the user's voice data, and then uses voice recognition software to convert the voice into text data. As a result, the voice input is output as text data.

[0394] Step 2:

[0395] The device inputs the converted text data into a generating AI model. This model uses natural language processing technology to analyze the text data and extract information about the destination and intermediate points. Simultaneously, it analyzes the user's emotions using an emotion engine based on their voice data. As a result, the analyzed destination information and emotion information are output.

[0396] Step 3:

[0397] The terminal sends the text data and sentiment information obtained in the previous step to the server. The server receives this data and also obtains real-time traffic information via an external traffic information service. Using the obtained traffic information, the server generates the optimal route, taking into account the destination, intermediate stops, and sentiment state. In this process, a suggested route based on traffic information and the user's state is output.

[0398] Step 4:

[0399] The server sends the generated optimal route and suggestions to the terminal. The terminal uses speech synthesis software to inform the user of the suggestions verbally. The user responds based on the verbal guidance. This response is input into the terminal and sent to the server.

[0400] Step 5:

[0401] The server analyzes voice feedback from the user and updates route information as needed. The updated route information is sent to the terminal. The terminal receives this information and continuously provides voice guidance to the user based on the updated information. In this process, the user receives the most up-to-date route guidance, resulting in more comfortable and appropriate navigation.

[0402] (Application Example 2)

[0403] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0404] In autonomous vehicles and navigation systems, simply guiding users to their destination without considering their emotions or feelings presents a challenge in adequately enhancing safety and comfort. In this context, the ability to flexibly set and suggest routes that respond to user emotions is believed to provide higher levels of satisfaction and safety.

[0405] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0406] In this invention, the server includes means for converting voice input into information data, means for analyzing the information data using a generative model and setting location information and waypoints, and means for analyzing the emotional state and proposing alternative routes and stopping points according to the user's emotions. This makes it possible to propose navigation that takes the user's emotions into consideration.

[0407] "Voice input" is the process of converting audio signals into digital information.

[0408] "Information data" refers to digital data used to store or transmit processed information.

[0409] A "generative model" is a machine learning model that generates a specific output based on input data.

[0410] "Location information" refers to digital data that indicates the geographical location of an object.

[0411] A "waypoint" is a point that must be passed through in order to reach the destination.

[0412] "Emotional state" is a state indicator that shows the user's emotions.

[0413] An "alternative route" is a different travel route that takes place instead of the primary route.

[0414] A "stopping point" is a location set during navigation for the purpose of temporarily stopping or resting the vehicle.

[0415] To realize this invention, it is necessary to build a system that converts voice input into information data, analyzes emotional states, and provides navigation suggestions. For this purpose, the terminal receives voice input and uses a microphone and software to convert the voice signal into digital information. Specifically, the Google Cloud Speech-to-Text API is used to convert the voice into text information.

[0416] Next, the server uses a generative AI model to analyze the information obtained from the voice data. This generative model can identify the user's emotional state using IBM Watson's emotion analysis API. The analyzed information is used to set location information and waypoints.

[0417] Furthermore, the server can obtain situational information from external services, including the Google Maps API. Based on the information obtained in real time, the server suggests alternative routes and stopping points that are tailored to the user's emotions.

[0418] The device uses its speaker to notify the user of these suggestions via voice output. Furthermore, the interaction continues by receiving user feedback again as voice and feeding it back into the analysis cycle.

[0419] For example, if a user says, "I'm in a hurry to get to my destination, but I'm worried about traffic," the system will calculate a fast route while also suggesting a route that includes comfortable spots to alleviate the user's stress. An example of a prompt might be, "Please suggest a quieter route that avoids highways. The user wants to relax."

[0420] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0421] Step 1:

[0422] The device uses a microphone to receive voice input from the user. The input voice is converted into text data using the Google Cloud Speech-to-Text API. This text data is used as the basis for subsequent processing.

[0423] Step 2:

[0424] The server receives the text data acquired in step 1 and analyzes the user's emotional state using IBM Watson's emotion analysis API. This emotion analysis determines the type and intensity of the emotion and outputs it as emotional state data.

[0425] Step 3:

[0426] The server identifies user-defined location information and waypoints based on text data. A generative AI model is used to calculate the optimal route based on this information. The output consists of location information and the calculated route data.

[0427] Step 4:

[0428] The server uses the Google Maps API in real time to retrieve the latest situation information from an external source. This situation information is combined with the data from step 3 to suggest alternative routes and stopping points that take emotional states into account. This suggested data is then sent to the next process.

[0429] Step 5:

[0430] The terminal notifies the user of the suggested data generated in step 4 through an audio output device. A speaker is used for this purpose, and the user listens to the audio guidance and provides feedback.

[0431] Step 6:

[0432] The user provides voice feedback, which the device receives again as voice input. This feedback is used in the next process cycle, and the system recalculates to update the route and suggestions based on the new information.

[0433] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0434] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0435] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0436] [Third Embodiment]

[0437] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0438] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0439] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0440] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0441] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0442] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0443] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0444] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0445] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0446] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0447] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0448] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0449] This invention provides a flexible navigation system based on voice input. Users can specify destinations and waypoints to the navigation system via voice. For example, suppose a user says, "My destination is Tokyo Station, and I'd like to stop in Yokohama on the way."

[0450] The terminal converts the user's voice into text data and sends that data to the server. The server analyzes the received text data using a generative model and calculates the optimal route based on the specified destination and waypoints. In this calculation, it refers to a map database and takes into account the location information and distance of each point.

[0451] The server further obtains real-time traffic information via external traffic information services. This allows it to exclusively collect and analyze information such as congestion and road construction occurring on the specified route. Based on this analysis, the server generates suggestions for alternative routes or rest stops as needed.

[0452] The terminal notifies the user of information obtained from the server via voice. For example, it might say, "The Tomei Expressway to Tokyo Station is congested, so we recommend taking an alternative route using the Daisan Keihin Expressway." Based on this information, the user can provide voice feedback to the terminal, such as, "Please take that route."

[0453] User feedback is sent back from the terminal to the server, which updates the route information. Finally, the terminal continuously provides navigation while offering the updated route information to the user. In this way, the present invention minimizes visual input during driving and achieves safe and efficient navigation through natural interaction via voice.

[0454] The following describes the processing flow.

[0455] Step 1:

[0456] The user inputs by voice, "My destination is Tokyo Station, and I'd like to stop in Yokohama on the way."

[0457] Step 2:

[0458] The terminal receives voice input and converts it into text data using speech recognition technology. The converted text data is then sent to the server.

[0459] Step 3:

[0460] The server analyzes the received text data using a generative model to identify the destination and waypoints. It then refers to a map database to calculate the optimal route.

[0461] Step 4:

[0462] The server obtains real-time traffic information for the route via an external traffic information service.

[0463] Step 5:

[0464] The server uses the acquired traffic information to generate suggestions for alternative routes and rest stops as needed. It then sends this information to the terminal.

[0465] Step 6:

[0466] The terminal notifies the user of suggestions received from the server via voice and requests their feedback.

[0467] Step 7:

[0468] Users respond to suggestions by voice, instructing to adopt new routes or making other requests.

[0469] Step 8:

[0470] The device sends user feedback to the server, which then updates the route information based on the instructions.

[0471] Step 9:

[0472] The updated route information is sent to the terminal, and the terminal provides the user with continuous voice navigation.

[0473] (Example 1)

[0474] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0475] Conventional navigation systems have faced challenges in providing flexible route planning based on voice instructions and in rapidly changing routes in response to real-time traffic conditions. This makes it difficult to reach one's destination safely and efficiently.

[0476] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0477] In this invention, the server includes means for converting voice input into digital data, means for analyzing the digital data using a generative model to set locations and routes, and means for acquiring route information and proposing options in response to changes in route conditions. This allows users to easily set routes via voice and to dynamically update routes based on real-time information.

[0478] "Voice input" is a method of transmitting information to a system by having the user speak.

[0479] "Digital data" refers to data that represents information in a format that can be processed by a computer.

[0480] A "generative model" is an algorithm that uses artificial intelligence technology to analyze data and find specific patterns.

[0481] "Analysis" is the process of examining data and extracting meaningful information.

[0482] A "location" is information that indicates a specific geographical position.

[0483] A "route" is information that indicates the path between points.

[0484] "Route information" refers to information that shows the directions or geographical route to a destination.

[0485] "To propose" means to present options or solutions and encourage their implementation.

[0486] "User" is a term that refers to a person who operates the system.

[0487] "Response" refers to information such as replies and feedback from users.

[0488] A "server" is a computer system that processes information and provides data to other devices.

[0489] This invention provides a flexible navigation system based on voice input. Users can specify destinations and waypoints to the navigation system via voice. For example, suppose a user says, "My destination is the central station, and I'd like to stop at a park along the way."

[0490] The terminal receives voice from the user and converts it into digital data using speech recognition software. This process can utilize common speech recognition tools. The converted digital data is transmitted to a server via a communication network. The server analyzes the text data using a generative AI model and extracts destination and transit point information. Ideally, analysis software suitable for the generative AI model should be used for this analysis.

[0491] The server uses a geographic information system to calculate the optimal route. This is achieved by using the server's processing equipment to refer to map data and consider distance and travel time. Furthermore, the server obtains real-time traffic information from external traffic information services and suggests alternative routes as needed based on the obtained information. This service works in conjunction with the server's data processing equipment to make the latest traffic data available.

[0492] The terminal uses speech synthesis software to provide voice notifications to the user based on information provided by the server. This allows the user to navigate efficiently to their destination without relying on visual input, even while driving. The user receives the guidance and provides voice feedback, such as "Please take that route," which is then sent back to the server via the terminal. The server updates the route information based on the feedback, and the terminal provides it to the user. Through this process, the navigation system can always provide the most up-to-date information.

[0493] An example of a prompt message is: "Calculate the optimal route based on the user's specified location and route, and provide options based on real-time traffic information."

[0494] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0495] Step 1:

[0496] Users specify their destination and intermediate stops by voice. For example, they might say, "My destination is the central station, and I'd like to stop at a park along the way." This input is transmitted to the terminal as an audio signal.

[0497] Step 2:

[0498] The terminal converts the received audio signal into digital data using speech recognition software. The input is an audio signal, and the output is text data. This conversion process includes the operation of analyzing the audio signal and converting it into a corresponding string.

[0499] Step 3:

[0500] The terminal sends the converted text data to the server. Here, the text data becomes the input, and communication processing to the server is performed. The output is the text information passed to the server.

[0501] Step 4:

[0502] The server analyzes the received text data using a generative AI model. The input is text data, and location and route information is extracted through the analysis process. This analysis utilizes natural language processing techniques. The output is a set of location information.

[0503] Step 5:

[0504] The server references a geographic information system and calculates the optimal route based on location information. The input is a set of location information, and the server executes a route calculation algorithm based on this data. The output is detailed information about the optimal route. This step includes evaluation of distance and time.

[0505] Step 6:

[0506] The server obtains real-time traffic information from an external traffic information service. The input is a traffic information acquisition request, and the server obtains the latest traffic condition data through communication with the external service. The output is traffic condition data. Based on the acquired data, route adjustments are made.

[0507] Step 7:

[0508] The server proposes appropriate alternative routes based on traffic information. Inputs include detailed information on the optimal route and traffic condition data. This data is integrated to generate alternative routes. The output is the proposed alternative route information.

[0509] Step 8:

[0510] The terminal uses speech synthesis software to notify the user of route information provided by the server. The input is alternative route information, and speech synthesis generates voice guidance. The output is a voice message conveyed to the user. Specifically, it would announce, "As you head towards the central station, you will now pass through the park."

[0511] Step 9:

[0512] The user, upon receiving directions, provides voice feedback such as, "Please take that route." This voice feedback is entered into the device.

[0513] Step 10:

[0514] The terminal converts the user's voice feedback into text and sends it back to the server. The input is voice feedback, and the output is the converted text data. The server receives this information and updates the final route information.

[0515] (Application Example 1)

[0516] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0517] Navigation in autonomous vehicles requires the provision of appropriate routes in real time through natural voice interaction, rather than relying on visual information. However, conventional systems lack sufficient capabilities to set routes and respond to changing traffic conditions, and in particular, they lack efficient means to reduce the burden on the driver. This can lead to problems with safety and convenience.

[0518] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0519] In this invention, the server includes means for converting voice input into data, means for analyzing the data using a generative model to set points and waypoints, and means for acquiring information and proposing alternative routes in response to changes in circumstances. This enables voice-based route guidance in autonomous vehicles.

[0520] "Voice input" is a method of acquiring data from the voice spoken by the user.

[0521] Converting "sound" to "data" means analyzing a human voice and converting the audio signal into digital information.

[0522] A "generative model" is an algorithm used to analyze data and generate new information or predictions.

[0523] A "location" refers to a specific geographical position.

[0524] A "waypoint" is a point that you plan to pass through on your way to your destination.

[0525] "Acquiring information" means gathering necessary data and knowledge from external databases and services.

[0526] "Changes in circumstances" refers to factors that change over time or due to environmental factors.

[0527] An "alternative route" is a different path from several options for reaching a specified destination.

[0528] "Notification means" refers to methods or devices for conveying necessary information to the user.

[0529] "Receiving feedback" means receiving feedback and instructions from users and incorporating them into the system.

[0530] "Route information" refers to geographical and traffic data necessary for traveling between specific points.

[0531] An "autonomous vehicle" is a vehicle that operates autonomously through a system without human intervention.

[0532] "Voice-based route guidance" refers to notifying users of information about their travel route using voice output.

[0533] This invention provides a system for realizing flexible voice-based navigation in autonomous vehicles. The system includes a voice recognition device, a communication module, a server, a generative AI model, a map database, and a real-time traffic information service.

[0534] The speech recognition device will capture the user's voice with high accuracy within the autonomous vehicle and convert this voice into digital data. Possible technologies to be used include Google Cloud Speech-to-Text API and Azure Speech Service.

[0535] The converted voice data is sent to the server via a communication module. The server analyzes the voice data using OpenAI's generative AI model to accurately understand the destination and waypoints specified by the user. Based on the analysis results, the server calculates the optimal route while referring to a map database. The map database used includes the Google Maps API. Furthermore, the server utilizes real-time traffic information services such as the Here Maps API to adjust the route to reflect current traffic conditions.

[0536] The optimal route calculated by the server is communicated to the user via voice through an interface within the autonomous vehicle. For this voice synthesis, which requires speed and accuracy, technologies such as Amazon Polly or Microsoft Azure Cognitive Services are used.

[0537] For example, if a user gives voice instructions during a long-distance family road trip, such as "I want to leave Tokyo, see Mt. Fuji along the way, and then go to Osaka," the system will analyze the user's intent and provide the optimal route. This route will also include alternative routes depending on traffic conditions.

[0538] An example of a prompt in this invention is, "If the user requests voice navigation from Tokyo to Osaka, please tell me the best route." In this way, the user can navigate efficiently and safely through natural conversation without visual interaction.

[0539] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0540] Step 1:

[0541] Users use a voice input device inside the autonomous vehicle to give voice instructions for destinations and waypoints. The voice input device converts this voice signal into digital data. The output data is a digital representation of the voice waveform.

[0542] Step 2:

[0543] The device receives the converted audio data. Using speech recognition technology, it converts the audio waveform into text data. This process utilizes the Google Cloud Speech-to-Text API and Azure Speech Service. The output is a text representation of the user's voice commands.

[0544] Step 3:

[0545] The terminal sends text data to the server via a communication module. The server uses an OpenAI generative AI model to analyze the input text data and identify the specified destination and waypoints. The output is structured data for route planning.

[0546] Step 4:

[0547] The server consults a map database and calculates the optimal route based on the specified destination and waypoints. This process utilizes services such as the Google Maps API. The output is information about the optimal route to the destination.

[0548] Step 5:

[0549] The server obtains real-time traffic information via the Here Maps API and other means, and incorporates it into the calculated route. It sets alternative routes as needed, taking into account traffic conditions and construction information. The output is an adjusted route that takes real-time information into account.

[0550] Step 6:

[0551] The server sends the optimized routing information to the terminal. The terminal uses speech synthesis technology to notify the user of the routing information as a voice message. At this stage, Amazon Polly or Azure Cognitive Services are used. The output is the voice guidance transmitted to the user.

[0552] Step 7:

[0553] The user operates the autonomous vehicle according to the guided route. If necessary, they provide voice feedback, and the system updates the route information accordingly. This ensures the user continuously receives optimal navigation in real time.

[0554] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0555] This invention combines a voice-input-based, flexible navigation system with an emotion engine to achieve user-responsive interaction. Users can specify destinations and waypoints by voice. Furthermore, the system recognizes emotions from the user's voice and provides feedback and suggestions accordingly. For example, consider a scenario where the user says, "My destination is Tokyo Station, but I'm worried about traffic."

[0556] The device converts the user's voice into text data and simultaneously analyzes the emotions from that voice using an emotion engine. The analyzed text data and emotion information are sent to the server. The server analyzes the text data using a generative model, identifies the specified destination and waypoints, and calculates the optimal route.

[0557] Furthermore, the server obtains real-time traffic information from external traffic information services and generates route and rest stop suggestions that take into account the user's emotional state. For example, if the server detects that the user is feeling stressed, it can recommend relaxing music or suggest a comfortable rest stop.

[0558] The terminal notifies the user of suggestions from the server via voice and provides feedback based on sentiment recognition results. The user can respond to the suggestions via voice and decide whether to accept or reject the new route or the suggestion. For example, they can give instructions such as "That route is fine" or "Is there a more comfortable route?"

[0559] The server updates route information based on user instructions and sends the updated information to the terminal. The terminal continuously provides the user with voice guidance that reflects the route information. In this way, the present invention realizes safer and more comfortable navigation by providing driving assistance that takes user emotions into consideration.

[0560] The following describes the processing flow.

[0561] Step 1:

[0562] The user inputs by voice, "My destination is Tokyo Station, and I'm worried about traffic."

[0563] Step 2:

[0564] The device receives voice input, converts it into text data using speech recognition technology, and simultaneously analyzes the emotions using an emotion engine. The emotional state and text data are then sent to the server.

[0565] Step 3:

[0566] The server analyzes the received text data using a generative model to identify the destination and intermediate points. It also obtains real-time traffic conditions using an external traffic information service.

[0567] Step 4:

[0568] The server takes into account the user's emotional state and traffic information to calculate the optimal route and rest stops, and includes relaxing music and guidance in its suggestions as needed.

[0569] Step 5:

[0570] The server sends the generated suggestions to the terminal.

[0571] Step 6:

[0572] The terminal notifies the user via voice of suggestions received from the server. For example, it might say, "There is currently traffic congestion to Tokyo Station, so we recommend an alternative route. Also, would you like us to play some music to help you relax?"

[0573] Step 7:

[0574] Users respond to suggestions via voice, instructing the system to adopt new routes or play music that matches their emotions.

[0575] Step 8:

[0576] The device sends user feedback to the server, which then updates route information and suggestions based on that feedback.

[0577] Step 9:

[0578] Updated route information and suggestions are sent to the device, which then provides the user with continuous voice navigation and emotionally sensitive service.

[0579] (Example 2)

[0580] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0581] Conventional navigation systems allow route setting via voice input, but they lack flexible interaction that takes user emotions into consideration. As a result, it is difficult to provide appropriate route guidance and alternative route suggestions while reducing user stress. Therefore, there is a need to develop navigation systems that take the user's emotional state into account and provide more comfortable and intuitive guidance.

[0582] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0583] In this invention, the server includes processing means for converting voice input into text data, processing means for analyzing the text data using a generative model and setting destinations and waypoints, and processing means for analyzing the user's emotions from the user's voice data. This makes it possible to propose the optimal route in real time while taking the user's emotions into consideration, enabling more comfortable and less stressful navigation.

[0584] "Voice input" is an input method that captures the content of what the user says as data.

[0585] "Text data" refers to character information converted from voice input.

[0586] A "generative model" is an algorithm or program that analyzes input text data to identify destinations and configuration information.

[0587] "Emotional analysis" is a technology that identifies a user's emotional state from their voice.

[0588] "Traffic information" is a general term for data that includes road congestion, accident information, traffic restrictions, and other operational conditions.

[0589] An "alternative route" is a different travel route proposed based on current traffic conditions and user requests.

[0590] "User feedback" refers to users' reactions and instructions to suggestions and guidance from the system.

[0591] "Route information" refers to guidance information for reaching a designated destination or intermediate points.

[0592] In an embodiment of the present invention, the user provides voice input through a terminal. The terminal uses a built-in microphone to acquire the voice input. The acquired voice data is converted into text data using speech recognition software. For example, a speech recognition API can be used for this process.

[0593] Furthermore, the device incorporates a generative AI model to analyze the converted text data and identify destinations and intermediate points. This generative AI model might include, for example, a large-scale language model used for natural language processing. Additionally, an emotion engine is used to analyze the user's emotions from their voice. This emotion analysis employs machine learning algorithms to determine the user's level of stress and comfort.

[0594] The analyzed text data and sentiment information are sent to the server. The server obtains real-time traffic information using an external traffic information service. This traffic information is obtained, for example, via an API from a map service provider. The server then considers the user's sentiment state and generates suggestions for the optimal route, waypoints, and rest stops.

[0595] The terminal notifies the user of the received suggestions using speech synthesis software. The user can respond to the suggestions by voice, accepting them or giving further instructions. Based on the user's voice feedback, the server updates the route information and sends optimized, up-to-date directions to the terminal.

[0596] As a concrete example, a user can enter a prompt message into the terminal such as, "My current destination is Tokyo Skytree. I'd like to stop at a cafe along the way; do you have any recommendations? I'd like to avoid crowded places," and receive appropriate route and cafe suggestions. This enables flexible and emotionally sensitive driving assistance.

[0597] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0598] Step 1:

[0599] The user specifies their destination and intermediate stops by voice. This voice data is input to the terminal. The terminal uses its built-in microphone to acquire the user's voice data, and then uses voice recognition software to convert the voice into text data. As a result, the voice input is output as text data.

[0600] Step 2:

[0601] The device inputs the converted text data into a generating AI model. This model uses natural language processing technology to analyze the text data and extract information about the destination and intermediate points. Simultaneously, it analyzes the user's emotions using an emotion engine based on their voice data. As a result, the analyzed destination information and emotion information are output.

[0602] Step 3:

[0603] The terminal sends the text data and sentiment information obtained in the previous step to the server. The server receives this data and also obtains real-time traffic information via an external traffic information service. Using the obtained traffic information, the server generates the optimal route, taking into account the destination, intermediate stops, and sentiment state. In this process, a suggested route based on traffic information and the user's state is output.

[0604] Step 4:

[0605] The server sends the generated optimal route and suggestions to the terminal. The terminal uses speech synthesis software to inform the user of the suggestions verbally. The user responds based on the verbal guidance. This response is input into the terminal and sent to the server.

[0606] Step 5:

[0607] The server analyzes voice feedback from the user and updates route information as needed. The updated route information is sent to the terminal. The terminal receives this information and continuously provides voice guidance to the user based on the updated information. In this process, the user receives the most up-to-date route guidance, resulting in more comfortable and appropriate navigation.

[0608] (Application Example 2)

[0609] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0610] In autonomous vehicles and navigation systems, simply guiding users to their destination without considering their emotions or feelings presents a challenge in adequately enhancing safety and comfort. In this context, the ability to flexibly set and suggest routes that respond to user emotions is believed to provide higher levels of satisfaction and safety.

[0611] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0612] In this invention, the server includes means for converting voice input into information data, means for analyzing the information data using a generative model and setting location information and waypoints, and means for analyzing the emotional state and proposing alternative routes and stopping points according to the user's emotions. This makes it possible to propose navigation that takes the user's emotions into consideration.

[0613] "Voice input" is the process of converting audio signals into digital information.

[0614] "Information data" refers to digital data used to store or transmit processed information.

[0615] A "generative model" is a machine learning model that generates a specific output based on input data.

[0616] "Location information" refers to digital data that indicates the geographical location of an object.

[0617] A "waypoint" is a point that must be passed through in order to reach the destination.

[0618] "Emotional state" is a state indicator that shows the user's emotions.

[0619] An "alternative route" is a different travel route that takes place instead of the primary route.

[0620] A "stopping point" is a location set during navigation for the purpose of temporarily stopping or resting the vehicle.

[0621] To realize this invention, it is necessary to build a system that converts voice input into information data, analyzes emotional states, and provides navigation suggestions. For this purpose, the terminal receives voice input and uses a microphone and software to convert the voice signal into digital information. Specifically, the Google Cloud Speech-to-Text API is used to convert the voice into text information.

[0622] Next, the server uses a generative AI model to analyze the information obtained from the voice data. This generative model can identify the user's emotional state using IBM Watson's emotion analysis API. The analyzed information is used to set location information and waypoints.

[0623] Furthermore, the server can obtain situational information from external services, including the Google Maps API. Based on the information obtained in real time, the server suggests alternative routes and stopping points that are tailored to the user's emotions.

[0624] The device uses its speaker to notify the user of these suggestions via voice output. Furthermore, the interaction continues by receiving user feedback again as voice and feeding it back into the analysis cycle.

[0625] For example, if a user says, "I'm in a hurry to get to my destination, but I'm worried about traffic," the system will calculate a fast route while also suggesting a route that includes comfortable spots to alleviate the user's stress. An example of a prompt might be, "Please suggest a quieter route that avoids highways. The user wants to relax."

[0626] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0627] Step 1:

[0628] The device uses a microphone to receive voice input from the user. The input voice is converted into text data using the Google Cloud Speech-to-Text API. This text data is used as the basis for subsequent processing.

[0629] Step 2:

[0630] The server receives the text data acquired in step 1 and analyzes the user's emotional state using IBM Watson's emotion analysis API. This emotion analysis determines the type and intensity of the emotion and outputs it as emotional state data.

[0631] Step 3:

[0632] The server identifies user-defined location information and waypoints based on text data. A generative AI model is used to calculate the optimal route based on this information. The output consists of location information and the calculated route data.

[0633] Step 4:

[0634] The server uses the Google Maps API in real time to retrieve the latest situation information from an external source. This situation information is combined with the data from step 3 to suggest alternative routes and stopping points that take emotional states into account. This suggested data is then sent to the next process.

[0635] Step 5:

[0636] The terminal notifies the user of the suggested data generated in step 4 through an audio output device. A speaker is used for this purpose, and the user listens to the audio guidance and provides feedback.

[0637] Step 6:

[0638] The user provides voice feedback, which the device receives again as voice input. This feedback is used in the next process cycle, and the system recalculates to update the route and suggestions based on the new information.

[0639] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0640] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0641] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0642] [Fourth Embodiment]

[0643] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0644] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0645] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0646] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0647] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0648] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0649] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0650] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0651] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0652] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0653] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0654] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0655] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0656] This invention provides a flexible navigation system based on voice input. Users can specify destinations and waypoints to the navigation system via voice. For example, suppose a user says, "My destination is Tokyo Station, and I'd like to stop in Yokohama on the way."

[0657] The terminal converts the user's voice into text data and sends that data to the server. The server analyzes the received text data using a generative model and calculates the optimal route based on the specified destination and waypoints. In this calculation, it refers to a map database and takes into account the location information and distance of each point.

[0658] The server further obtains real-time traffic information via external traffic information services. This allows it to exclusively collect and analyze information such as congestion and road construction occurring on the specified route. Based on this analysis, the server generates suggestions for alternative routes or rest stops as needed.

[0659] The terminal notifies the user of information obtained from the server via voice. For example, it might say, "The Tomei Expressway to Tokyo Station is congested, so we recommend taking an alternative route using the Daisan Keihin Expressway." Based on this information, the user can provide voice feedback to the terminal, such as, "Please take that route."

[0660] User feedback is sent back from the terminal to the server, which updates the route information. Finally, the terminal continuously provides navigation while offering the updated route information to the user. In this way, the present invention minimizes visual input during driving and achieves safe and efficient navigation through natural interaction via voice.

[0661] The following describes the processing flow.

[0662] Step 1:

[0663] The user inputs by voice, "My destination is Tokyo Station, and I'd like to stop in Yokohama on the way."

[0664] Step 2:

[0665] The terminal receives voice input and converts it into text data using speech recognition technology. The converted text data is then sent to the server.

[0666] Step 3:

[0667] The server analyzes the received text data using a generative model to identify the destination and waypoints. It then refers to a map database to calculate the optimal route.

[0668] Step 4:

[0669] The server obtains real-time traffic information for the route via an external traffic information service.

[0670] Step 5:

[0671] The server uses the acquired traffic information to generate suggestions for alternative routes and rest stops as needed. It then sends this information to the terminal.

[0672] Step 6:

[0673] The terminal notifies the user of suggestions received from the server via voice and requests their feedback.

[0674] Step 7:

[0675] Users respond to suggestions by voice, instructing to adopt new routes or making other requests.

[0676] Step 8:

[0677] The device sends user feedback to the server, which then updates the route information based on the instructions.

[0678] Step 9:

[0679] The updated route information is sent to the terminal, and the terminal provides the user with continuous voice navigation.

[0680] (Example 1)

[0681] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0682] Conventional navigation systems have faced challenges in providing flexible route planning based on voice instructions and in rapidly changing routes in response to real-time traffic conditions. This makes it difficult to reach one's destination safely and efficiently.

[0683] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0684] In this invention, the server includes means for converting voice input into digital data, means for analyzing the digital data using a generative model to set locations and routes, and means for acquiring route information and proposing options in response to changes in route conditions. This allows users to easily set routes via voice and to dynamically update routes based on real-time information.

[0685] "Voice input" is a method of transmitting information to a system by having the user speak.

[0686] "Digital data" refers to data that represents information in a format that can be processed by a computer.

[0687] A "generative model" is an algorithm that uses artificial intelligence technology to analyze data and find specific patterns.

[0688] "Analysis" is the process of examining data and extracting meaningful information.

[0689] A "location" is information that indicates a specific geographical position.

[0690] A "route" is information that indicates the path between points.

[0691] "Route information" refers to information that shows the directions or geographical route to a destination.

[0692] "To propose" means to present options or solutions and encourage their implementation.

[0693] "User" is a term that refers to a person who operates the system.

[0694] "Response" refers to information such as replies and feedback from users.

[0695] A "server" is a computer system that processes information and provides data to other devices.

[0696] This invention provides a flexible navigation system based on voice input. Users can specify destinations and waypoints to the navigation system via voice. For example, suppose a user says, "My destination is the central station, and I'd like to stop at a park along the way."

[0697] The terminal receives voice from the user and converts it into digital data using speech recognition software. This process can utilize common speech recognition tools. The converted digital data is transmitted to a server via a communication network. The server analyzes the text data using a generative AI model and extracts destination and transit point information. Ideally, analysis software suitable for the generative AI model should be used for this analysis.

[0698] The server uses a geographic information system to calculate the optimal route. This is achieved by using the server's processing equipment to refer to map data and consider distance and travel time. Furthermore, the server obtains real-time traffic information from external traffic information services and suggests alternative routes as needed based on the obtained information. This service works in conjunction with the server's data processing equipment to make the latest traffic data available.

[0699] The terminal uses speech synthesis software to provide voice notifications to the user based on information provided by the server. This allows the user to navigate efficiently to their destination without relying on visual input, even while driving. The user receives the guidance and provides voice feedback, such as "Please take that route," which is then sent back to the server via the terminal. The server updates the route information based on the feedback, and the terminal provides it to the user. Through this process, the navigation system can always provide the most up-to-date information.

[0700] An example of a prompt message is: "Calculate the optimal route based on the user's specified location and route, and provide options based on real-time traffic information."

[0701] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0702] Step 1:

[0703] Users specify their destination and intermediate stops by voice. For example, they might say, "My destination is the central station, and I'd like to stop at a park along the way." This input is transmitted to the terminal as an audio signal.

[0704] Step 2:

[0705] The terminal converts the received audio signal into digital data using speech recognition software. The input is an audio signal, and the output is text data. This conversion process includes the operation of analyzing the audio signal and converting it into a corresponding string.

[0706] Step 3:

[0707] The terminal sends the converted text data to the server. Here, the text data becomes the input, and communication processing to the server is performed. The output is the text information passed to the server.

[0708] Step 4:

[0709] The server analyzes the received text data using a generative AI model. The input is text data, and location and route information is extracted through the analysis process. This analysis utilizes natural language processing techniques. The output is a set of location information.

[0710] Step 5:

[0711] The server references a geographic information system and calculates the optimal route based on location information. The input is a set of location information, and the server executes a route calculation algorithm based on this data. The output is detailed information about the optimal route. This step includes evaluation of distance and time.

[0712] Step 6:

[0713] The server obtains real-time traffic information from an external traffic information service. The input is a traffic information acquisition request, and the server obtains the latest traffic condition data through communication with the external service. The output is traffic condition data. Based on the acquired data, route adjustments are made.

[0714] Step 7:

[0715] The server proposes appropriate alternative routes based on traffic information. Inputs include detailed information on the optimal route and traffic condition data. This data is integrated to generate alternative routes. The output is the proposed alternative route information.

[0716] Step 8:

[0717] The terminal uses speech synthesis software to notify the user of route information provided by the server. The input is alternative route information, and speech synthesis generates voice guidance. The output is a voice message conveyed to the user. Specifically, it would announce, "As you head towards the central station, you will now pass through the park."

[0718] Step 9:

[0719] The user, upon receiving directions, provides voice feedback such as, "Please take that route." This voice feedback is entered into the device.

[0720] Step 10:

[0721] The terminal converts the user's voice feedback into text and sends it back to the server. The input is voice feedback, and the output is the converted text data. The server receives this information and updates the final route information.

[0722] (Application Example 1)

[0723] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0724] Navigation in autonomous vehicles requires the provision of appropriate routes in real time through natural voice interaction, rather than relying on visual information. However, conventional systems lack sufficient capabilities to set routes and respond to changing traffic conditions, and in particular, they lack efficient means to reduce the burden on the driver. This can lead to problems with safety and convenience.

[0725] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0726] In this invention, the server includes means for converting voice input into data, means for analyzing the data using a generative model to set points and waypoints, and means for acquiring information and proposing alternative routes in response to changes in circumstances. This enables voice-based route guidance in autonomous vehicles.

[0727] "Voice input" is a method of acquiring data from the voice spoken by the user.

[0728] Converting "sound" to "data" means analyzing a human voice and converting the audio signal into digital information.

[0729] A "generative model" is an algorithm used to analyze data and generate new information or predictions.

[0730] A "location" refers to a specific geographical position.

[0731] A "waypoint" is a point that you plan to pass through on your way to your destination.

[0732] "Acquiring information" means gathering necessary data and knowledge from external databases and services.

[0733] "Changes in circumstances" refers to factors that change over time or due to environmental factors.

[0734] An "alternative route" is a different path from several options for reaching a specified destination.

[0735] "Notification means" refers to methods or devices for conveying necessary information to the user.

[0736] "Receiving feedback" means receiving feedback and instructions from users and incorporating them into the system.

[0737] "Route information" refers to geographical and traffic data necessary for traveling between specific points.

[0738] An "autonomous vehicle" is a vehicle that operates autonomously through a system without human intervention.

[0739] "Voice-based route guidance" refers to notifying users of information about their travel route using voice output.

[0740] This invention provides a system for realizing flexible voice-based navigation in autonomous vehicles. The system includes a voice recognition device, a communication module, a server, a generative AI model, a map database, and a real-time traffic information service.

[0741] The speech recognition device will capture the user's voice with high accuracy within the autonomous vehicle and convert this voice into digital data. Possible technologies to be used include Google Cloud Speech-to-Text API and Azure Speech Service.

[0742] The converted voice data is sent to the server via a communication module. The server analyzes the voice data using OpenAI's generative AI model to accurately understand the destination and waypoints specified by the user. Based on the analysis results, the server calculates the optimal route while referring to a map database. The map database used includes the Google Maps API. Furthermore, the server utilizes real-time traffic information services such as the Here Maps API to adjust the route to reflect current traffic conditions.

[0743] The optimal route calculated by the server is communicated to the user via voice through an interface within the autonomous vehicle. For this voice synthesis, which requires speed and accuracy, technologies such as Amazon Polly or Microsoft Azure Cognitive Services are used.

[0744] For example, if a user gives voice instructions during a long-distance family road trip, such as "I want to leave Tokyo, see Mt. Fuji along the way, and then go to Osaka," the system will analyze the user's intent and provide the optimal route. This route will also include alternative routes depending on traffic conditions.

[0745] An example of a prompt in this invention is, "If the user requests voice navigation from Tokyo to Osaka, please tell me the best route." In this way, the user can navigate efficiently and safely through natural conversation without visual interaction.

[0746] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0747] Step 1:

[0748] Users use a voice input device inside the autonomous vehicle to give voice instructions for destinations and waypoints. The voice input device converts this voice signal into digital data. The output data is a digital representation of the voice waveform.

[0749] Step 2:

[0750] The device receives the converted audio data. Using speech recognition technology, it converts the audio waveform into text data. This process utilizes the Google Cloud Speech-to-Text API and Azure Speech Service. The output is a text representation of the user's voice commands.

[0751] Step 3:

[0752] The terminal sends text data to the server via a communication module. The server uses an OpenAI generative AI model to analyze the input text data and identify the specified destination and waypoints. The output is structured data for route planning.

[0753] Step 4:

[0754] The server consults a map database and calculates the optimal route based on the specified destination and waypoints. This process utilizes services such as the Google Maps API. The output is information about the optimal route to the destination.

[0755] Step 5:

[0756] The server obtains real-time traffic information via the Here Maps API and other means, and incorporates it into the calculated route. It sets alternative routes as needed, taking into account traffic conditions and construction information. The output is an adjusted route that takes real-time information into account.

[0757] Step 6:

[0758] The server sends the optimized routing information to the terminal. The terminal uses speech synthesis technology to notify the user of the routing information as a voice message. At this stage, Amazon Polly or Azure Cognitive Services are used. The output is the voice guidance transmitted to the user.

[0759] Step 7:

[0760] The user operates the autonomous vehicle according to the guided route. If necessary, they provide voice feedback, and the system updates the route information accordingly. This ensures the user continuously receives optimal navigation in real time.

[0761] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0762] This invention combines a voice-input-based, flexible navigation system with an emotion engine to achieve user-responsive interaction. Users can specify destinations and waypoints by voice. Furthermore, the system recognizes emotions from the user's voice and provides feedback and suggestions accordingly. For example, consider a scenario where the user says, "My destination is Tokyo Station, but I'm worried about traffic."

[0763] The device converts the user's voice into text data and simultaneously analyzes the emotions from that voice using an emotion engine. The analyzed text data and emotion information are sent to the server. The server analyzes the text data using a generative model, identifies the specified destination and waypoints, and calculates the optimal route.

[0764] Furthermore, the server obtains real-time traffic information from external traffic information services and generates route and rest stop suggestions that take into account the user's emotional state. For example, if the server detects that the user is feeling stressed, it can recommend relaxing music or suggest a comfortable rest stop.

[0765] The terminal notifies the user of suggestions from the server via voice and provides feedback based on sentiment recognition results. The user can respond to the suggestions via voice and decide whether to accept or reject the new route or the suggestion. For example, they can give instructions such as "That route is fine" or "Is there a more comfortable route?"

[0766] The server updates route information based on user instructions and sends the updated information to the terminal. The terminal continuously provides the user with voice guidance that reflects the route information. In this way, the present invention realizes safer and more comfortable navigation by providing driving assistance that takes user emotions into consideration.

[0767] The following describes the processing flow.

[0768] Step 1:

[0769] The user inputs by voice, "My destination is Tokyo Station, and I'm worried about traffic."

[0770] Step 2:

[0771] The device receives voice input, converts it into text data using speech recognition technology, and simultaneously analyzes the emotions using an emotion engine. The emotional state and text data are then sent to the server.

[0772] Step 3:

[0773] The server analyzes the received text data using a generative model to identify the destination and intermediate points. It also obtains real-time traffic conditions using an external traffic information service.

[0774] Step 4:

[0775] The server takes into account the user's emotional state and traffic information to calculate the optimal route and rest stops, and includes relaxing music and guidance in its suggestions as needed.

[0776] Step 5:

[0777] The server sends the generated suggestions to the terminal.

[0778] Step 6:

[0779] The terminal notifies the user via voice of suggestions received from the server. For example, it might say, "There is currently traffic congestion to Tokyo Station, so we recommend an alternative route. Also, would you like us to play some music to help you relax?"

[0780] Step 7:

[0781] Users respond to suggestions via voice, instructing the system to adopt new routes or play music that matches their emotions.

[0782] Step 8:

[0783] The device sends user feedback to the server, which then updates route information and suggestions based on that feedback.

[0784] Step 9:

[0785] Updated route information and suggestions are sent to the device, which then provides the user with continuous voice navigation and emotionally sensitive service.

[0786] (Example 2)

[0787] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0788] Conventional navigation systems allow route setting via voice input, but they lack flexible interaction that takes user emotions into consideration. As a result, it is difficult to provide appropriate route guidance and alternative route suggestions while reducing user stress. Therefore, there is a need to develop navigation systems that take the user's emotional state into account and provide more comfortable and intuitive guidance.

[0789] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0790] In this invention, the server includes processing means for converting voice input into text data, processing means for analyzing the text data using a generative model and setting destinations and waypoints, and processing means for analyzing the user's emotions from the user's voice data. This makes it possible to propose the optimal route in real time while taking the user's emotions into consideration, enabling more comfortable and less stressful navigation.

[0791] "Voice input" is an input method that captures the content of what the user says as data.

[0792] "Text data" refers to character information converted from voice input.

[0793] A "generative model" is an algorithm or program that analyzes input text data to identify destinations and configuration information.

[0794] "Emotional analysis" is a technology that identifies a user's emotional state from their voice.

[0795] "Traffic information" is a general term for data that includes road congestion, accident information, traffic restrictions, and other operational conditions.

[0796] An "alternative route" is a different travel route proposed based on current traffic conditions and user requests.

[0797] "User feedback" refers to users' reactions and instructions to suggestions and guidance from the system.

[0798] "Route information" refers to guidance information for reaching a designated destination or intermediate points.

[0799] In an embodiment of the present invention, the user provides voice input through a terminal. The terminal uses a built-in microphone to acquire the voice input. The acquired voice data is converted into text data using speech recognition software. For example, a speech recognition API can be used for this process.

[0800] Furthermore, the device incorporates a generative AI model to analyze the converted text data and identify destinations and intermediate points. This generative AI model might include, for example, a large-scale language model used for natural language processing. Additionally, an emotion engine is used to analyze the user's emotions from their voice. This emotion analysis employs machine learning algorithms to determine the user's level of stress and comfort.

[0801] The analyzed text data and sentiment information are sent to the server. The server obtains real-time traffic information using an external traffic information service. This traffic information is obtained, for example, via an API from a map service provider. The server then considers the user's sentiment state and generates suggestions for the optimal route, waypoints, and rest stops.

[0802] The terminal notifies the user of the received suggestions using speech synthesis software. The user can respond to the suggestions by voice, accepting them or giving further instructions. Based on the user's voice feedback, the server updates the route information and sends optimized, up-to-date directions to the terminal.

[0803] As a concrete example, a user can enter a prompt message into the terminal such as, "My current destination is Tokyo Skytree. I'd like to stop at a cafe along the way; do you have any recommendations? I'd like to avoid crowded places," and receive appropriate route and cafe suggestions. This enables flexible and emotionally sensitive driving assistance.

[0804] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0805] Step 1:

[0806] The user specifies their destination and intermediate stops by voice. This voice data is input to the terminal. The terminal uses its built-in microphone to acquire the user's voice data, and then uses voice recognition software to convert the voice into text data. As a result, the voice input is output as text data.

[0807] Step 2:

[0808] The device inputs the converted text data into a generating AI model. This model uses natural language processing technology to analyze the text data and extract information about the destination and intermediate points. Simultaneously, it analyzes the user's emotions using an emotion engine based on their voice data. As a result, the analyzed destination information and emotion information are output.

[0809] Step 3:

[0810] The terminal sends the text data and sentiment information obtained in the previous step to the server. The server receives this data and also obtains real-time traffic information via an external traffic information service. Using the obtained traffic information, the server generates the optimal route, taking into account the destination, intermediate stops, and sentiment state. In this process, a suggested route based on traffic information and the user's state is output.

[0811] Step 4:

[0812] The server sends the generated optimal route and suggestions to the terminal. The terminal uses speech synthesis software to inform the user of the suggestions verbally. The user responds based on the verbal guidance. This response is input into the terminal and sent to the server.

[0813] Step 5:

[0814] The server analyzes voice feedback from the user and updates route information as needed. The updated route information is sent to the terminal. The terminal receives this information and continuously provides voice guidance to the user based on the updated information. In this process, the user receives the most up-to-date route guidance, resulting in more comfortable and appropriate navigation.

[0815] (Application Example 2)

[0816] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0817] In autonomous vehicles and navigation systems, simply guiding users to their destination without considering their emotions or feelings presents a challenge in adequately enhancing safety and comfort. In this context, the ability to flexibly set and suggest routes that respond to user emotions is believed to provide higher levels of satisfaction and safety.

[0818] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0819] In this invention, the server includes means for converting voice input into information data, means for analyzing the information data using a generative model and setting location information and waypoints, and means for analyzing the emotional state and proposing alternative routes and stopping points according to the user's emotions. This makes it possible to propose navigation that takes the user's emotions into consideration.

[0820] "Voice input" is the process of converting audio signals into digital information.

[0821] "Information data" refers to digital data used to store or transmit processed information.

[0822] A "generative model" is a machine learning model that generates a specific output based on input data.

[0823] "Location information" refers to digital data that indicates the geographical location of an object.

[0824] A "waypoint" is a point that must be passed through in order to reach the destination.

[0825] "Emotional state" is a state indicator that shows the user's emotions.

[0826] An "alternative route" is a different travel route that takes place instead of the primary route.

[0827] A "stopping point" is a location set during navigation for the purpose of temporarily stopping or resting the vehicle.

[0828] To realize this invention, it is necessary to build a system that converts voice input into information data, analyzes emotional states, and provides navigation suggestions. For this purpose, the terminal receives voice input and uses a microphone and software to convert the voice signal into digital information. Specifically, the Google Cloud Speech-to-Text API is used to convert the voice into text information.

[0829] Next, the server uses a generative AI model to analyze the information obtained from the voice data. This generative model can identify the user's emotional state using IBM Watson's emotion analysis API. The analyzed information is used to set location information and waypoints.

[0830] Furthermore, the server can obtain situational information from external services, including the Google Maps API. Based on the information obtained in real time, the server suggests alternative routes and stopping points that are tailored to the user's emotions.

[0831] The device uses its speaker to notify the user of these suggestions via voice output. Furthermore, the interaction continues by receiving user feedback again as voice and feeding it back into the analysis cycle.

[0832] For example, if a user says, "I'm in a hurry to get to my destination, but I'm worried about traffic," the system will calculate a fast route while also suggesting a route that includes comfortable spots to alleviate the user's stress. An example of a prompt might be, "Please suggest a quieter route that avoids highways. The user wants to relax."

[0833] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0834] Step 1:

[0835] The device uses a microphone to receive voice input from the user. The input voice is converted into text data using the Google Cloud Speech-to-Text API. This text data is used as the basis for subsequent processing.

[0836] Step 2:

[0837] The server receives the text data acquired in step 1 and analyzes the user's emotional state using IBM Watson's emotion analysis API. This emotion analysis determines the type and intensity of the emotion and outputs it as emotional state data.

[0838] Step 3:

[0839] The server identifies user-defined location information and waypoints based on text data. A generative AI model is used to calculate the optimal route based on this information. The output consists of location information and the calculated route data.

[0840] Step 4:

[0841] The server uses the Google Maps API in real time to retrieve the latest situation information from an external source. This situation information is combined with the data from step 3 to suggest alternative routes and stopping points that take emotional states into account. This suggested data is then sent to the next process.

[0842] Step 5:

[0843] The terminal notifies the user of the suggested data generated in step 4 through an audio output device. A speaker is used for this purpose, and the user listens to the audio guidance and provides feedback.

[0844] Step 6:

[0845] The user provides voice feedback, which the device receives again as voice input. This feedback is used in the next process cycle, and the system recalculates to update the route and suggestions based on the new information.

[0846] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0847] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0848] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0849] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0850] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0851] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0852] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0853] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0854] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0855] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0856] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0857] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0858] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0859] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0860] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0861] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0862] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0863] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0864] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0865] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0866] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0867] The following is further disclosed regarding the embodiments described above.

[0868] (Claim 1)

[0869] A means of converting voice input into text data,

[0870] A method for analyzing text data using a generative model to set destinations and waypoints,

[0871] A means of acquiring traffic information and proposing alternative routes in response to changes in traffic conditions,

[0872] A means of notifying users of suggestions via voice and receiving user feedback,

[0873] A means of updating route information based on user instructions,

[0874] A system that includes this.

[0875] (Claim 2)

[0876] The system according to claim 1 for acquiring traffic information in real time.

[0877] (Claim 3)

[0878] The system according to claim 1, which suggests rest locations based on the user's voice input.

[0879] "Example 1"

[0880] (Claim 1)

[0881] A means of converting voice input into digital data,

[0882] A method for analyzing digital data using a generative model to set locations and routes,

[0883] A means of acquiring route information and proposing options in response to changes in route conditions,

[0884] A means of notifying users of suggestions via voice and receiving user responses,

[0885] A means of updating route information based on user instructions,

[0886] A system characterized by continuously performing the following procedure,

[0887] A system that includes this.

[0888] (Claim 2)

[0889] The system according to claim 1 for acquiring route information in real time.

[0890] (Claim 3)

[0891] The system according to claim 1, which suggests relay points based on the user's voice input.

[0892] "Application Example 1"

[0893] (Claim 1)

[0894] A means of converting voice input into data,

[0895] A means of analyzing data using a generative model and setting points and waypoints,

[0896] A means of acquiring information and proposing alternative routes in response to changing circumstances,

[0897] A means of communicating a proposal using a notification means and receiving a response,

[0898] A means of updating route information based on instructions,

[0899] A means of providing voice-guided route guidance in an autonomous vehicle,

[0900] A system that includes this.

[0901] (Claim 2)

[0902] The system according to claim 1 for acquiring information in real time.

[0903] (Claim 3)

[0904] The system according to claim 1, which suggests intermediate points based on voice input.

[0905] "Example 2 of combining an emotion engine"

[0906] (Claim 1)

[0907] A processing means for converting voice input into text data,

[0908] A processing method that analyzes text data using a generative model and sets destinations and waypoints,

[0909] A processing method for analyzing emotions from user voice data,

[0910] A processing means that acquires traffic information and proposes alternative routes in response to changes in traffic conditions and user sentiment,

[0911] A processing method that notifies the user of a suggestion via voice and receives user feedback,

[0912] A processing means for updating route information based on user instructions,

[0913] A system that includes this.

[0914] (Claim 2)

[0915] The system according to claim 1 for acquiring traffic information in real time.

[0916] (Claim 3)

[0917] The system according to claim 1, which suggests rest locations based on the user's voice input and emotion analysis.

[0918] "Application example 2 when combining with an emotional engine"

[0919] (Claim 1)

[0920] A means of converting voice input into information data,

[0921] A means of analyzing information data using a generative model and setting location information and waypoints,

[0922] A means of analyzing emotional states and suggesting alternative routes and stopping points that correspond to the user's emotions,

[0923] A means of obtaining situation information from external services and proposing alternative routes in response to changes in the situation,

[0924] A means of notifying the user of a proposal via voice output and receiving the user's response,

[0925] A means of updating route information based on user instructions,

[0926] A system that includes this.

[0927] (Claim 2)

[0928] The system according to claim 1 for acquiring situational information in real time.

[0929] (Claim 3)

[0930] The system according to claim 1, which suggests a stopping point based on the user's voice input. [Explanation of symbols]

[0931] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of converting voice input into text data, A method for analyzing text data using a generative model to set destinations and waypoints, A means of acquiring traffic information and proposing alternative routes in response to changes in traffic conditions, A means of notifying users of suggestions via voice and receiving user feedback, A means of updating route information based on user instructions, A system that includes this.

2. The system according to claim 1 for acquiring traffic information in real time.

3. The system according to claim 1, which suggests rest locations based on the user's voice input.

Images