system

Abstract

Provide a system. A method for obtaining voice input from a user includes: means for converting voice input into text data; means for analyzing the text data to identify a user's intention; means for acquiring destination information and calculating a route based on the user's intention; means for providing the acquired information to the user by voice and visual means; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional car navigation systems require manual operations such as setting destinations, changing routes, checking traffic congestion information, and searching for nearby stores, and performing these tasks while driving can potentially compromise safety. Even existing systems equipped with voice recognition functions can sometimes have difficulty accurately understanding the user's intentions and providing appropriate information. Therefore, there is a demand for car navigation systems that effectively integrate voice interaction functions to improve the intuitiveness and safety of operation and enable users to easily set destinations and search for information. [Means for solving the problem]

[0005] To solve this problem, the present invention provides a car navigation system including: means for acquiring voice input from a user; means for converting the voice input into text data; means for analyzing the text data to identify the user's intent; means for acquiring destination information and calculating a route based on the user's intent; and means for providing the acquired information to the user through voice and visual guidance. Specifically, the system converts voice into text using a voice recognition engine, analyzes the text data using natural language processing technology, and identifies the user's intent. This allows various functions, such as setting a destination, searching for nearby stores, and providing traffic information, to be performed solely through voice. Furthermore, the system acquires destination information and calculates a route using a location information database and traffic condition data, providing the user with optimal information. This improves the intuitiveness and convenience of operation while ensuring safety during driving.

[0006] The "user" is the driver or passenger who operates the car navigation system.

[0007] "Voice input" refers to voice commands or queries made by a user through a microphone or other input device.

[0008] "Text data" is voice input converted into digital format as a string of characters.

[0009] A "voice recognition engine" refers to software or hardware that analyzes voice data and converts it into corresponding text data.

[0010] "Natural language processing technology" refers to technology that analyzes text data, understands grammar and context, and identifies the user's intent.

[0011] "Destination information" is information relating to the location and name of the destination set by the user.

[0012] "Route calculation means" refers to algorithms or software that calculate the optimal route from the current location to the destination based on traffic conditions and road information.

[0013] A "location information database" is a database that holds geographical location information and provides data corresponding to each location.

[0014] "Traffic condition data" refers to real-time or near-real-time traffic information on roads, such as current traffic flow, accident information, and congestion information.

[0015] A "speech synthesis engine" refers to a technology that analyzes text data and generates corresponding natural-sounding speech.

[0016] "Visual guidance" refers to the display of visual information such as maps and routes on a user interface. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This system allows a user to operate the functions of a car navigation system by voice. An embodiment of this system will be described below.

[0039] The system includes a means for capturing voice input and converting it into text using a speech recognition engine. The text data is then sent to a server, which uses natural language processing techniques to identify the user's intent, retrieves destination information, calculates routes, and finally provides the retrieved route information to the user via audio and visual guidance.

[0040] Program processing

[0041] 1. Acquiring voice input

[0042] The user issues a voice command such as "Tell me the way to Tokyo Tower." The device uses a microphone to capture the voice and converts it into text data using a built-in speech recognition engine. At this stage, the text data "Tell me the way to Tokyo Tower" is generated.

[0043] 2. Parsing the command

[0044] The device sends the text data to the server, which uses a natural language processing engine to analyze the text data and determine that the user's intent is to "set a destination."

[0045] 3. Database integration

[0046] The server queries the location database and API to obtain the coordinate data of Tokyo Tower, and calculates the optimal route using traffic condition data. This determines the optimal route taking real-time traffic information into account.

[0047] 4. Generate and send results

[0048] The server returns the route information it has acquired and calculated to the terminal. The terminal then uses a speech synthesis engine to convert the received route information into voice guidance data, and provides audible guidance to the user, saying, "The route to Tokyo Tower has been set." The route information is then displayed as visual guidance on the terminal screen.

[0049] Specific examples

[0050] Example 1: Setting a destination

[0051] The user says "Navigate to Tokyo Station" while in the car. The device receives the speech and uses a speech recognition engine to convert it into text data: "Navigate to Tokyo Station." The device then sends the text data to the server. The server analyzes the text data and determines that the user's intention is to "set a destination." The server queries a location database and API to obtain the coordinate data of Tokyo Station, and then calculates the optimal route taking into account real-time traffic information. The calculation results are sent back to the device, which then provides voice guidance to the user saying, "Route to Tokyo Station has been set," and displays the route on the screen.

[0052] Example 2: Search for nearby stores

[0053] The user says, "Find a nearby restaurant." The device receives the speech and converts it into text data saying, "Find a nearby restaurant" using a speech recognition engine. The text data is sent to the server, which uses a natural language processing engine to identify the user's intent as "Search for nearby stores." The server uses a location database and API to obtain information about restaurants near the current location and sends the results back to the device. The device then provides voice guidance saying, "There are three nearby restaurants: A, B, and C," and displays the locations of the restaurants on the screen.

[0054] summary

[0055] This system allows users to intuitively operate various car navigation functions using only their voice, improving safety while driving and increasing convenience of operation.

[0056] The processing flow will be explained below.

[0057] Step 1:

[0058] A user says, "Tell me the route to Tokyo Tower."

[0059] Step 2:

[0060] The device uses a built-in microphone to obtain the user's voice input.

[0061] Step 3:

[0062] The device calls a speech recognition engine and converts the acquired voice input into text data. For example, text data such as "Tell me the way to Tokyo Tower" is generated.

[0063] Step 4:

[0064] The device sends the generated text data to the server using the HTTP / HTTPS communication protocol.

[0065] Step 5:

[0066] The server receives the text data and invokes a natural language processing engine to analyze the text data.

[0067] Step 6:

[0068] The server identifies the user's intention from the analysis results. In this case, the user's intention is determined to be "setting a destination."

[0069] Step 7:

[0070] The server queries a location database or API to obtain the coordinate data of Tokyo Tower.

[0071] Step 8:

[0072] The server obtains traffic data and uses this data to calculate the optimal route from the current location to Tokyo Tower.

[0073] Step 9:

[0074] The server organizes the calculated route information and sends it back to the device, including route guidance steps and map data.

[0075] Step 10:

[0076] The route information received by the terminal is passed to a voice synthesis engine and converted into voice data for voice guidance.

[0077] Step 11:

[0078] The terminal uses the voice data to provide a voice message to the user saying, "A route to Tokyo Tower has been set."

[0079] Step 12:

[0080] The device displays a visual guide to Tokyo Tower on the screen, plotting the route on a map and indicating necessary guidance points.

[0081] Example 1

[0082] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0083] Conventional car navigation systems require users to manually input their destination or navigate complex menus, posing safety and operational challenges while driving. In particular, when voice control is inefficient, it can be difficult to correctly understand the user's intentions, leading to the inability to provide appropriate route guidance. Therefore, a system is needed that can more accurately and quickly analyze voice input and provide the optimal route based on the user's intentions.

[0084] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0085] In this invention, the server includes means for acquiring voice input from a user, means for converting the voice input into text data, means for analyzing the text data to identify the user's intention, means for acquiring destination information and calculating a route based on the user's intention, means for providing the acquired information to the user aurally and visually, means for analyzing the voice input converted into a digital signal using a voice recognition engine, means for analyzing the text data using natural language processing technology to identify the user's intention, and means for calculating an optimal route using a location information database and traffic condition data. This allows the user to intuitively operate the car navigation function using only voice, improving safety while driving and enabling faster and more accurate route guidance.

[0086] A "means for obtaining audio input" is a device or method that captures a user's voice and converts it into a digital signal.

[0087] The "means for converting voice input into text data" refers to the technology and engine that analyzes acquired voice data and converts it into text format data.

[0088] "Means for identifying user intent by analyzing text data" refers to a method for analyzing text data using natural language processing technology to clearly understand the content and purpose of the user's instructions.

[0089] The "means for acquiring destination information and calculating a route" refers to a database and analytical technology for acquiring the coordinates of a target point based on the user's intentions and calculating the optimal route.

[0090] The "means for providing audio and visual guidance to the user" refers to a method for visually and audibly informing the user of the generated route information using voice synthesis and display.

[0091] "Means for analyzing voice input converted into a digital signal using a voice recognition engine" refers to a technology that analyzes voice input with high accuracy and processes the voice data captured as a digital signal.

[0092] "Means for analyzing text data using natural language processing technology to identify user intent" refers to a method for analyzing text data using advanced natural language processing algorithms to accurately understand user intent.

[0093] "Means for calculating optimal routes using location information databases and traffic data" refers to systems and technologies that calculate the most efficient route for a user based on location information and real-time traffic information.

[0094] This invention is a system that allows users to operate car navigation functions by voice input. This system can acquire voice input and automatically perform recognition, analysis, destination setting, and route guidance.

[0095] Hardware and software used

[0096] To implement this system, the following hardware and software are used:

[0097] Hardware:

[0098] Microphone: Captures voice input from the user.

[0099] Display: Provides visual guidance to the user.

[0100] Processor: Performs speech recognition and data analysis.

[0101] Speaker: Provides audio guidance to the user.

[0102] software:

[0103] Speech recognition engine: For example, a general speech recognition API (such as Google® Speech-to-Text API) is used to convert voice input into text data.

[0104] Natural language processing engine: For example, using a common natural language processing API (such as Google Natural Language API) to analyze text data and identify user intent.

[0105] Location database: For example, use a common geographic information API (such as Google Maps API) to obtain coordinate data of the destination.

[0106] Traffic information API: For example, use a common traffic information API (such as Waze API) to obtain real-time traffic conditions.

[0107] Speech synthesis engine: For example, use a common speech synthesis API (such as Google Text-to-Speech API) to convert text data into speech data.

[0108] Example

[0109] Example 1: Setting a destination

[0110] The user says "Navigate to Tokyo Station" while in the car. The device captures the speech and uses a speech recognition engine to convert it into text data saying "Navigate to Tokyo Station." The device then sends this text data to the server. The server uses a natural language processing engine to analyze the text data and identify the user's intent as "set destination." The server queries the location information database and traffic information API to obtain coordinate data for Tokyo Station and real-time traffic information. Based on this, the server calculates the optimal route and sends this information back to the device. The device then uses a speech synthesis engine to announce "Route to Tokyo Station has been set" and displays the route information on the display.

[0111] Example 2: Search for nearby stores

[0112] The user says, "Find a nearby restaurant." The device receives this speech and converts it into text data saying, "Find a nearby restaurant" using a speech recognition engine. The device then sends the text data to the server. The server uses a natural language processing engine to determine that the user's intent is to "search for nearby stores." The server then queries the location information database and traffic information API to obtain information about restaurants near the current location. The server sends the results back to the device, and the device announces, "Nearby restaurants are A, B, and C," and displays the restaurant's location information on the display.

[0113] Prompt Sentence Examples

[0114] "Navigate to Tokyo Station"

[0115] "Find a nearby restaurant"

[0116] This invention allows a user to intuitively operate car navigation functions using only voice, improving safety while driving and enabling faster and more accurate route guidance.

[0117] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0118] Step 1:

[0119] The user inputs voice. For example, "Tell me the way to Tokyo Tower." The device uses a microphone to capture this voice. The input is the user's voice signal, which is converted into a digital format. The output is digital voice data.

[0120] Step 2:

[0121] The device uses a speech recognition engine to convert digital voice data into text data. The input is digital voice data, and the engine performs phonetic analysis and parses the voice content into text format. The output is text data such as "Tell me the way to Tokyo Tower."

[0122] Step 3:

[0123] The terminal sends the generated text data to the server. The input is the text data, and a secure communication protocol (e.g., HTTPS) is used to securely transport this data to the server. The output is the server's confirmation of receipt.

[0124] Step 4:

[0125] The server receives the text data and analyzes it using a natural language processing engine. The input is text data, and the engine performs grammatical analysis and keyword extraction to identify the user's intent. The output is the intent information, "Set destination."

[0126] Step 5:

[0127] Based on the intent information, the server accesses the location information database to obtain the coordinate data of the destination. It also queries the traffic data API for real-time traffic conditions. The input is the destination name "Tokyo Tower" and the intent information "set destination." The database returns the geographic coordinates, and the traffic data API returns the latest traffic conditions. The output is the destination coordinate data and traffic condition data.

[0128] Step 6:

[0129] The server calculates the optimal route based on coordinate data and traffic condition data. The input is the destination coordinate data and traffic condition data, and the server uses a route calculation algorithm to derive the shortest and most efficient route. The output is the optimal route information.

[0130] Step 7:

[0131] The server sends the optimal route information to the terminal. The input is the optimal route information, which is sent to the terminal via a secure communication protocol. The output is the terminal's confirmation of receipt.

[0132] Step 8:

[0133] The terminal receives the optimal route information and generates voice guidance using a speech synthesis engine. The input is the optimal route information, and the speech synthesis engine converts the text-format route information into voice data. The output is voice data saying, "A route to Tokyo Tower has been set."

[0134] Step 9:

[0135] The terminal provides the generated voice data to the user through a speaker and simultaneously displays visual route guidance on the display. The input is voice data and visual guidance data, and the output is voice guidance and display.

[0136] (Application example 1)

[0137] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0138] Conventional car navigation systems often require manual operation by the user, resulting in issues of safety and convenience while driving. Furthermore, in autonomous vehicles, there is no easy way for users to give complex voice instructions, limiting operability. To solve this issue, there is a demand for a system that allows users to intuitively and efficiently navigate and operate autonomous vehicles and other vehicle operations using only voice input.

[0139] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0140] In this invention, the server includes means for acquiring voice input from a user, means for converting the voice input into text data, means for analyzing the text data to identify the user's intention, means for acquiring destination information and calculating a route based on the user's intention, means for providing the acquired route information to the user via voice and visual means, and means for operating the autonomous vehicle using voice, thereby minimizing manual operations while driving and enabling safe and efficient operation of the autonomous vehicle.

[0141] "Voice input" refers to a user providing words or instructions through a voice input device such as a microphone.

[0142] "Text data" refers to data that is expressed as text information after analyzing voice input using a voice recognition engine or the like.

[0143] "Means for identifying user intent" refers to a method for analyzing acquired text data using natural language processing technology, etc., to understand the user's requests and instructions.

[0144] A "location information database" is a database that stores location information such as geographic coordinates and addresses, and is used to locate specific locations and calculate routes.

[0145] "Traffic condition data" refers to data that indicates current road conditions and traffic information, and is used for route calculation and navigation.

[0146] "Route information" is information that includes specific directions and directional instructions for reaching the user's destination.

[0147] "Audio and visual guidance means" refers to a method of providing route information and instructions to the user both audibly and visually using a speech synthesis engine, a display, etc.

[0148] "Means for operating an autonomous vehicle using voice" refers to a method for controlling and adjusting various functions and settings within an autonomous vehicle in response to voice instructions from the user.

[0149] System Configuration

[0150] This system allows users to navigate and operate an autonomous vehicle through voice input. The specific configuration and processing steps of the system are shown below.

[0151] Hardware and Software

[0152] Smartphone: Equipped with a microphone and a display, it allows voice input and displays results.

[0153] Server: Maintains the voice recognition engine, natural language processing engine, location information database, and traffic situation database, and performs data analysis and calculations.

[0154] Specifically, the following technologies are used:

[0155] Speech recognition engine (e.g. Google Speech-to-Text API)

[0156] Natural language processing engine (e.g. Google Cloud Natural Language API)

[0157] Speech synthesis engine (e.g. Google Text-to-Speech API)

[0158] Data acquisition and processing

[0159] Acquiring voice input

[0160] The user inputs voice instructions through the smartphone's microphone, for example, "Turn right at the next traffic light."

[0161] Speech recognition and text conversion

[0162] The smartphone uses a speech recognition engine to convert the spoken input into text data in the form of "Turn right at the next traffic light."

[0163] Analysis using natural language processing

[0164] Sending to the server

[0165] The text data is sent to a server and analyzed by a natural language processing engine, which determines that the user's intent is to "turn right."

[0166] Route calculation and information acquisition

[0167] The server accesses the location information database and the traffic condition database to calculate the optimal right turn position based on the user's current location.

[0168] Audio and visual guidance

[0169] Server response and routing information transmission

[0170] The server sends the calculation results and route information to the smartphone, which then uses a speech synthesis engine to provide voice guidance such as "Turn right at the next traffic light." The smartphone also visually displays the right-turn location on the display.

[0171] Examples and prompts

[0172] Example 1: Reaching a highway entrance / exit

[0173] When a user says, "Take me to the next exit," the system analyzes their intent and guides them to the optimal route to the exit.

[0174] Example 2: Car charging station search

[0175] When a user says, "Tell me where the nearest charging station is," the system takes into account the current battery status and location information and provides directions to the best charging station.

[0176] Prompt Sentence Examples

[0177] The following prompts can be fed into the generative AI model to perform proper speech recognition and route calculation:

[0178] "Voice input: Turn right at the next traffic light."

[0179] "prompt:

[0180] 1. Convert speech to text.

[0181] 2. Identify the right turn indication from the text.

[0182] 3. Calculate the best right turn location from your current location.

[0183] 4. Please output visual and audio right turn instructions.

[0184] In this way, a system is realized that allows users to easily and safely navigate and operate an autonomous vehicle using only voice input.

[0185] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0186] Step 1:

[0187] The user performs voice input. The user speaks into the smartphone microphone, saying, "Turn right at the next traffic light." Voice data is generated as input.

[0188] Step 2:

[0189] The smartphone converts the voice into text data. It then uses a speech recognition engine (e.g., Google Speech-to-Text API) to analyze the voice data and convert it into text data such as "Turn right at the next traffic light." This generates text data as output.

[0190] Step 3:

[0191] The smartphone sends text data to the server. The sent text data arrives at the server, where it is analyzed to determine the user's intent using a natural language processing engine (e.g., Google Cloud Natural Language API). The input is the text data, and the output is the intent to "turn right."

[0192] Step 4:

[0193] The server obtains the current location information and calculates the optimal route. The server accesses the location information database and traffic condition database and calculates the route taking into account the user's current location and destination (right turn position at the next traffic light). The input is the user's current location and right turn instructions, and the output is right turn route information.

[0194] Step 5:

[0195] The server sends route information to the smartphone. The route information is sent from the server and reaches the smartphone. The route information is sent as input, and the information necessary for route guidance reaches the smartphone.

[0196] Step 6:

[0197] The smartphone provides route information through voice and visual guidance. The smartphone uses a speech synthesis engine (e.g., Google Text-to-Speech API) to provide voice guidance such as "Turn right at the next traffic light." The route information is also displayed visually on the display. Route information is input, and voice guidance and visual displays are generated as output.

[0198] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0199] The present invention allows a user to operate car navigation functions by voice, while at the same time recognizing the user's emotional state and providing an appropriate guidance method based on that emotion. The following describes an embodiment of the present invention.

[0200] The system includes a means for receiving voice input and converting it into text data using a voice recognition engine. The text data is then sent to a server, which uses natural language processing technology to identify the user's intent. The server also uses an emotion engine to analyze the user's emotional state and provides appropriate guidance based on this. Specifically, the system first converts the voice into text using a voice recognition engine, analyzes the user's intent using natural language processing technology, and then identifies the user's emotion using the emotion engine. Based on this information, the system obtains destination information, calculates a route, and provides audio and visual guidance to the user.

[0201] Program processing

[0202] 1. Acquiring voice input

[0203] The user issues a voice command such as "Tell me the way to Tokyo Tower." The device uses a microphone to capture the voice and converts it into text data using a built-in speech recognition engine. At this stage, the text data "Tell me the way to Tokyo Tower" is generated.

[0204] 2. Command analysis and emotion recognition

[0205] The device sends text data to the server. The server uses a natural language processing engine to analyze the text data and determine that the user's intent is to "set a destination." At the same time, it calls an emotion engine to determine the user's emotional state from their voice characteristics. For example, if the user is excited, the emotion engine will recognize this as "excited."

[0206] 3. Database integration

[0207] The server queries the location database and API to obtain the coordinate data of Tokyo Tower, and calculates the optimal route using traffic condition data. This determines the optimal route taking real-time traffic information into account.

[0208] 4. Emotion-based guidance adjustment

[0209] The server organizes the route information it obtains and calculates, and adjusts the guidance method based on the user's emotional state. For example, if the server detects that the user is tired, it will provide guidance in a calmer tone.

[0210] 5. Generate and send results

[0211] The server returns the organized route information to the terminal. The terminal uses a speech synthesis engine to convert the received route information into voice data for voice guidance, and provides audible guidance to the user, saying, "The route to Tokyo Tower has been set." The route information is then displayed as visual guidance on the terminal screen.

[0212] Specific examples

[0213] Example 1: Destination setting and emotion recognition

[0214] The user says "Navigate to Tokyo Station" while in the car. The device captures the speech and uses a speech recognition engine to convert it into text data: "Navigate to Tokyo Station." The device then sends the text data to the server. The server analyzes the text data and determines that the user's intention is to "set a destination." At the same time, the emotion engine analyzes the user's voice and determines, for example, that the user is in a relaxed state. The server queries a location database and API to obtain coordinate data for Tokyo Station, and then calculates the optimal route taking real-time traffic information into consideration. The calculation results are sent back to the device, which then provides voice guidance in a relaxed tone, saying "Route to Tokyo Station has been set," while displaying the route on the screen.

[0215] Example 2: Nearby store search and emotion recognition

[0216] The user says, "Find a nearby restaurant." The device captures the speech and converts it into text data saying, "Find a nearby restaurant" using a speech recognition engine. The text data is sent to the server, which uses a natural language processing engine to identify the user's intent as "search for nearby stores." At the same time, the emotion engine identifies the user's emotional state, recognizing, for example, that the user is excited. The server uses a location database and API to obtain information about restaurants near the current location and sends the results back to the device. The device then provides voice guidance in a soft tone, saying, "There are three nearby restaurants: A, B, and C," and displays the locations of the restaurants on the screen.

[0217] In this way, by combining the system with an emotion engine, it becomes possible to provide flexible guidance that corresponds to the user's emotional state, improving the convenience of operation and the user experience.

[0218] The processing flow will be explained below.

[0219] Step 1:

[0220] A user says, "Tell me the route to Tokyo Tower."

[0221] Step 2:

[0222] The device uses a built-in microphone to obtain the user's voice input.

[0223] Step 3:

[0224] The device calls a speech recognition engine and converts the acquired voice input into text data. For example, text data such as "Tell me the way to Tokyo Tower" is generated.

[0225] Step 4:

[0226] The device sends the generated text data to the server using the HTTP / HTTPS communication protocol.

[0227] Step 5:

[0228] The server receives the text data and calls an emotion engine to identify the user's emotional state from the user's voice characteristics. For example, emotions such as "excited" or "relaxed" can be recognized from the tone, speed, and rhythm of the user's voice.

[0229] Step 6:

[0230] The server calls a natural language processing engine to analyze the text data, and identifies the user's intent as "setting a destination" from the text data.

[0231] Step 7:

[0232] The server queries a location database or API to obtain the coordinate data of Tokyo Tower.

[0233] Step 8:

[0234] The server collects traffic data and uses it to calculate the optimal route from the user's current location to Tokyo Tower. Traffic information includes information on traffic congestion and accidents.

[0235] Step 9:

[0236] The server then organizes the calculated route information and adjusts the guidance based on the user's emotional state. For example, if the user is "excited," the server will set the guidance to be in a calm tone.

[0237] Step 10:

[0238] The server returns organized route information to the device, including each step of the route guidance and map data.

[0239] Step 11:

[0240] The route information received by the terminal is passed to a voice synthesis engine and converted into voice data for voice guidance.

[0241] Step 12:

[0242] The device uses the voice data to guide the user by voice, saying, "A route to Tokyo Tower has been set." The tone of the voice guidance is adjusted according to the user's emotional state.

[0243] Step 13:

[0244] The device displays a visual guide to Tokyo Tower on the screen, drawing the route on a map and highlighting the next guide point.

[0245] Step 14:

[0246] The user continues driving to the destination using the audio and visual guidance as a reference.

[0247] Example 2

[0248] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0249] Conventional car navigation systems set destinations and provide route guidance based on user voice instructions, but they are unable to consider the user's emotional state, resulting in issues with ease of operation and user experience.In addition, when the user's emotional state changes, the system may not provide an appropriate tone or guidance method, which can be stressful.

[0250] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0251] In this invention, the server includes a means for converting voice input into text data, a means for analyzing the user's intention and emotional state, and a means for determining a guidance method based on the analysis results, thereby making it possible to provide flexible and appropriate guidance according to the user's intention and emotional state.

[0252] "Means for obtaining voice input" refers to devices and programs for collecting the user's voice in digital form.

[0253] The "means for converting voice input into text data" refers to a device and program that has the function of analyzing collected voice and converting it into text information.

[0254] "Means for analyzing text data to identify user intent" refers to devices and programs that have the function of analyzing converted text information using natural language processing technology and understanding the user's requests and instructions.

[0255] The "means for analyzing the user's emotional state" refers to a device or program that has the function of detecting the user's emotions based on the characteristics of voice or text and identifying that state.

[0256] The "means for determining an appropriate guidance method" refers to a device and a program that have the function of selecting the most appropriate guidance method based on the analyzed user's intention and emotional state.

[0257] The "means for acquiring destination information and calculating a route" refers to a device and a program that has the function of acquiring information about a specified destination and calculating the optimal route to that point.

[0258] The "means for providing the user with the acquired information by voice and visually" refers to a device and a program that has the function of providing the user with calculated route information by voice and visually.

[0259] The present invention allows a user to operate car navigation functions by voice, while at the same time recognizing the emotional state of the user and providing an appropriate guidance method based on that emotion.

[0260] Acquiring voice input

[0261] When a user says, "Tell me the way to Tokyo Tower," the device picks up the voice using a microphone installed in the car. The acquired voice data is converted into text data via a built-in voice recognition engine (e.g., general-purpose voice recognition software). At this stage, the text data "Tell me the way to Tokyo Tower" is generated.

[0262] Command parsing and emotion recognition

[0263] The text data generated by the device is sent to the server. The server then uses a natural language processing engine (e.g., natural language processing software) to analyze the text data and determine that the user's intent is to "set a destination." At the same time, it invokes an emotion engine (e.g., emotion analysis software) to determine the user's emotional state from the user's voice characteristics. For example, if the user is excited, the emotion engine will recognize this as "excited."

[0264] Database integration

[0265] The server queries a location database (e.g., location API) or other APIs to obtain the coordinate data of Tokyo Tower. It also uses a traffic condition API to collect real-time traffic information and calculates the optimal route. This determines the optimal route taking real-time traffic information into account.

[0266] Emotion-based guidance adjustment

[0267] The server organizes the route information it obtains and calculates, and adjusts the guidance method based on the user's emotional state. For example, if the emotion engine determines that the user is tired, the guidance method will be adjusted to be more calming.

[0268] Generate and send results

[0269] The server returns the organized route information to the terminal. The terminal uses a speech synthesis engine (for example, speech synthesis software) to convert the received route information into audio data for voice guidance, and provides audible guidance to the user, such as "The route to Tokyo Tower has been set." The terminal also displays the route information on the screen as visual guidance.

[0270] Specific examples

[0271] Example 1: Destination setting and emotion recognition

[0272] The user says "Navigate to Tokyo Station" while in the car. The device uses a microphone to capture the speech and converts it into text data, "Navigate to Tokyo Station," using a speech recognition engine. The device then sends the text data to a server. The server analyzes the speech using natural language processing software and determines that the user's intent is "set destination." At the same time, emotion analysis software analyzes the user's voice and determines that the user is relaxed. The server queries location information APIs and other APIs to obtain coordinate data for Tokyo Station, and calculates the optimal route taking real-time traffic information into account. The calculation results are sent back to the device, which then provides voice guidance in a relaxed tone, saying "Route to Tokyo Station has been set," while displaying the route on the screen.

[0273] Example 2: Nearby store search and emotion recognition

[0274] The user says, "Find a nearby restaurant." The device captures the speech and converts it into text data, "Find a nearby restaurant," using a speech recognition engine. The text data is sent to the server. The server uses natural language processing software to identify the user's intent as "search for nearby restaurants." At the same time, emotion analysis software identifies the user's emotional state and recognizes that they are excited. The server uses location APIs and other APIs to obtain information about restaurants near the current location and sends the results back to the device. The device then provides voice guidance in a softer tone to calm the user's excitement, saying, "There are three restaurants nearby: A, B, and C," and displays the restaurant locations on the screen.

[0275] This system enables flexible and appropriate guidance to be provided according to the user's emotional state, improving operational convenience and user experience.

[0276] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0277] Step 1:

[0278] Acquiring voice input

[0279] The user says, "Tell me the way to Tokyo Tower." The device uses a microphone to capture the user's voice. The captured voice data is input and output.

[0280] Step 2:

[0281] Converting speech to text data

[0282] The acquired voice data is input into a voice recognition engine (for example, general-purpose voice recognition software). This engine converts the voice data into text data and outputs it. Specifically, the text data generated is "Tell me the way to Tokyo Tower."

[0283] Step 3:

[0284] Parsing commands

[0285] The device sends text data to the server. The text data is input into the server's natural language processing engine (e.g., natural language processing software). The server analyzes the text data, identifies the user's intention, and outputs the intention data "set destination."

[0286] Step 4:

[0287] Emotion recognition

[0288] The server invokes an emotion engine (e.g., emotion analysis software) to analyze the characteristics of the voice data, which determines the user's emotional state and outputs the emotion data (e.g., "excited").

[0289] Step 5:

[0290] Database integration

[0291] The server communicates with a location information database (e.g., location information API) to obtain coordinate data for a specific destination (e.g., Tokyo Tower). As a result of querying the API, the destination coordinate data is output. Furthermore, the server queries the traffic conditions API for real-time traffic information, calculates the optimal route, and outputs this route data.

[0292] Step 6:

[0293] Emotion-based guidance adjustment

[0294] The server determines an appropriate guidance method based on the user's emotional data and route data. For example, if the server determines that the user is tired, it adjusts the guidance method to provide guidance in a calmer tone. This adjusted guidance method data is output.

[0295] Step 7:

[0296] Generate and send results

[0297] The server returns the organized route information and the adjusted guidance method to the terminal. The terminal receives them and uses a speech synthesis engine (e.g., speech synthesis software) to generate voice data for voice guidance. This voice data and visual guidance data are output.

[0298] Step 8:

[0299] User Information

[0300] The device uses voice data to announce, "The route to Tokyo Tower has been set." It also displays visual guidance on the screen, providing the user with a visual of the route.

[0301] (Application example 2)

[0302] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0303] While voice navigation is intuitive and convenient, it can be stressful depending on the user's emotional state, resulting in a poor user experience. For example, when the user is tired or excited, it can be difficult to receive appropriate guidance. Furthermore, many navigation systems provide uniform guidance without taking the user's emotional state into account, resulting in a lack of optimal guidance tailored to the situation. There is a need to solve these problems and provide optimal guidance tailored to the user's emotional state.

[0304] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0305] In this invention, the server includes means for acquiring voice input from the user, means for converting the voice input into text data, means for identifying the user's intention, means for recognizing the user's emotional state, means for acquiring destination information and calculating a route based on the user's intention and emotional state, means for providing the acquired information to the user by voice and visual means, and means for providing guidance in a tone corresponding to the emotional state, thereby enabling flexible and optimal navigation guidance according to the user's emotional state.

[0306] "Means for obtaining voice input" refers to a device or function that recognizes a user's voice commands and captures them as digital data.

[0307] "Means for converting voice input into text data" refers to software or hardware for analyzing acquired voice data and converting it into text data.

[0308] "Means for identifying user intent" refers to a device or function that uses natural language processing techniques to analyze the converted text data and understand what the user is requesting.

[0309] "Means for recognizing a user's emotional state" refers to technology for analyzing a user's voice characteristics and other input data to identify the user's emotions (e.g., tired, excited, etc.).

[0310] "Means for obtaining destination information and calculating a route" refers to a database and algorithm for obtaining destination information based on the user's intentions and calculating the optimal route to that destination in real time.

[0311] "Means for providing the user with audio and visual guidance of the acquired information" refers to a device or function that provides the user with calculated route information and other navigation information using voice synthesis technology and a display.

[0312] "Means for providing guidance in a tone appropriate to the emotional state" refers to techniques and algorithms for providing guidance in a tone or format appropriate to the emotional state of the user.

[0313] The present invention provides a system that allows a user to operate a car navigation function by voice and provides appropriate guidance based on the user's emotional state. Hereinafter, an embodiment of the present invention will be described in detail.

[0314] The overall system includes means for capturing voice input, analyzing the user's intent, recognizing the user's emotional state, and providing guidance based on that. First, the user issues a voice command in the car. The device captures the voice through a microphone and converts the voice data into text data using a speech recognition engine, which could be the Google Speech-to-Text API.

[0315] The acquired text data is then sent to a server, where natural language processing technology is used to analyze the user's intent. This part is implemented using the "Custom NLP API." The server then uses the "Custom Emotion Recognition API" to identify the user's emotional state from their voice characteristics.

[0316] Based on the analysis results, the server uses a location database and traffic data to obtain destination information and calculates the optimal route. Specifically, the process is as follows: The server obtains the destination coordinate data using the "Custom Location API," and then calculates the optimal route taking real-time traffic information into account using the "Custom Route API."

[0317] The server organizes the acquired and calculated route information and adjusts the guidance method based on the user's emotional state. For example, if the emotion engine recognizes that the user is tired, guidance will be provided in a calm tone. Finally, the resulting route information is sent back to the device, where it is converted into voice data by a speech synthesis engine and presented to the user. Visual guidance is also displayed on the device screen.

[0318] Specific examples

[0319] Destination settings:

[0320] The user says, "Navigate to Tokyo Station." The device receives the speech and converts it into text data, "Navigate to Tokyo Station," via a voice recognition engine. The server analyzes the text data and determines that the user's intention is to "set a destination," while the emotion engine simultaneously determines that the user is in a relaxed state. The server queries the location database and API to obtain the coordinate data of Tokyo Station, and calculates the optimal route taking into account real-time traffic information. The calculation results are sent back to the device, where a voice guide in a relaxed tone announces, "Route to Tokyo Station has been set," and the route is displayed on the screen.

[0321] Example prompt sentence:

[0322] "Taking voice data as input when a user says, 'Tell me the way to Tokyo Tower.' Analyze the user's intent, identify 'Tokyo Tower' as the destination preference, and check the user's emotional state. If the emotion engine determines that this user is tired, what are the steps to provide directions in a calmer tone?"

[0323] This system can provide flexible and optimal navigation guidance according to the user's emotional state, thereby improving the user's driving experience.

[0324] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0325] Step 1:

[0326] The user specifies the destination by voice.

[0327] Input: User's voice data

[0328] Specific operation: The user gives instructions in the car, such as "Navigate to Tokyo Tower."

[0329] Step 2:

[0330] The device receives the voice data and converts it into text data using a voice recognition engine.

[0331] Input: User's voice data

[0332] Data calculation: Converting voice data into text data using the Google Speech-to-Text API

[0333] Output: Text data "Navigate to Tokyo Tower"

[0334] Step 3:

[0335] The device sends text data to a server, which then uses natural language processing technology to analyze the user's intent.

[0336] Input: Text data

[0337] Data calculation: Using Custom NLP API, analyze the user's intent from the text data and identify "destination setting."

[0338] Output: User intent is "Set destination"

[0339] Step 4:

[0340] The server analyzes the user's emotional state from their voice characteristics.

[0341] Input: User's voice data

[0342] Data Computing: Identifying a user's emotional state from audio data using the Custom Emotion Recognition API

[0343] Output: User's emotional state (e.g., relaxed)

[0344] Step 5:

[0345] The server obtains the destination information and calculates the route.

[0346] Input: User intent (destination setting), user emotional state

[0347] Data calculation: Obtain destination coordinate data using the Custom Location API and calculate the optimal route using the Custom Route API

[0348] Output: Destination coordinates, optimal route

[0349] Step 6:

[0350] The server organizes the information it obtains and adjusts the guidance method according to the emotional state.

[0351] Input: Optimal route, user's emotional state

[0352] Data calculations: Adjusting the tone of guidance based on the user's emotional state (e.g., a relaxed tone)

[0353] Output: Adjusted guidance information

[0354] Step 7:

[0355] The server transmits the adjusted guidance information to the terminal.

[0356] Input: Adjusted guidance information

[0357] Data Computing: Packaging and transmitting information

[0358] Output: Guidance information sent to the terminal

[0359] Step 8:

[0360] The terminal uses a speech synthesis engine and a display to provide audio and visual guidance to the user.

[0361] Input: Guidance information sent to the terminal

[0362] Specific operation: The device generates a guidance message using a speech synthesis engine and announces in a relaxed tone, "A route to Tokyo Tower has been set." Visual information is also displayed on the device's display.

[0363] Output: Guidance information provided to the user

[0364] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0365] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0366] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0367] [Second embodiment]

[0368] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0369] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0370] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0371] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0372] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0373] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0374] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0375] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0376] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0377] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0378] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0379] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0380] This system allows a user to operate the functions of a car navigation system by voice. An embodiment of this system will be described below.

[0381] The system includes a means for capturing voice input and converting it into text using a speech recognition engine. The text data is then sent to a server, which uses natural language processing techniques to identify the user's intent, retrieves destination information, calculates routes, and finally provides the retrieved route information to the user via audio and visual guidance.

[0382] Program processing

[0383] 1. Acquiring voice input

[0384] The user issues a voice command such as "Tell me the way to Tokyo Tower." The device uses a microphone to capture the voice and converts it into text data using a built-in speech recognition engine. At this stage, the text data "Tell me the way to Tokyo Tower" is generated.

[0385] 2. Parsing the command

[0386] The device sends the text data to the server, which uses a natural language processing engine to analyze the text data and determine that the user's intent is to "set a destination."

[0387] 3. Database integration

[0388] The server queries the location database and API to obtain the coordinate data of Tokyo Tower, and calculates the optimal route using traffic condition data. This determines the optimal route taking real-time traffic information into account.

[0389] 4. Generate and send results

[0390] The server returns the route information it has acquired and calculated to the terminal. The terminal then uses a speech synthesis engine to convert the received route information into voice guidance data, and provides audible guidance to the user, saying, "The route to Tokyo Tower has been set." The route information is then displayed as visual guidance on the terminal screen.

[0391] Specific examples

[0392] Example 1: Setting a destination

[0393] The user says "Navigate to Tokyo Station" while in the car. The device receives the speech and uses a speech recognition engine to convert it into text data: "Navigate to Tokyo Station." The device then sends the text data to the server. The server analyzes the text data and determines that the user's intention is to "set a destination." The server queries a location database and API to obtain the coordinate data of Tokyo Station, and then calculates the optimal route taking into account real-time traffic information. The calculation results are sent back to the device, which then provides voice guidance to the user saying, "Route to Tokyo Station has been set," and displays the route on the screen.

[0394] Example 2: Search for nearby stores

[0395] The user says, "Find a nearby restaurant." The device receives the speech and converts it into text data saying, "Find a nearby restaurant" using a speech recognition engine. The text data is sent to the server, which uses a natural language processing engine to identify the user's intent as "Search for nearby stores." The server uses a location database and API to obtain information about restaurants near the current location and sends the results back to the device. The device then provides voice guidance saying, "There are three nearby restaurants: A, B, and C," and displays the locations of the restaurants on the screen.

[0396] summary

[0397] This system allows users to intuitively operate various car navigation functions using only their voice, improving safety while driving and increasing convenience of operation.

[0398] The processing flow will be explained below.

[0399] Step 1:

[0400] A user says, "Tell me the route to Tokyo Tower."

[0401] Step 2:

[0402] The device uses a built-in microphone to obtain the user's voice input.

[0403] Step 3:

[0404] The device calls a speech recognition engine and converts the acquired voice input into text data. For example, text data such as "Tell me the way to Tokyo Tower" is generated.

[0405] Step 4:

[0406] The device sends the generated text data to the server using the HTTP / HTTPS communication protocol.

[0407] Step 5:

[0408] The server receives the text data and invokes a natural language processing engine to analyze the text data.

[0409] Step 6:

[0410] The server identifies the user's intention from the analysis results. In this case, the user's intention is determined to be "setting a destination."

[0411] Step 7:

[0412] The server queries a location database or API to obtain the coordinate data of Tokyo Tower.

[0413] Step 8:

[0414] The server obtains traffic data and uses this data to calculate the optimal route from the current location to Tokyo Tower.

[0415] Step 9:

[0416] The server organizes the calculated route information and sends it back to the device, including route guidance steps and map data.

[0417] Step 10:

[0418] The route information received by the terminal is passed to a voice synthesis engine and converted into voice data for voice guidance.

[0419] Step 11:

[0420] The terminal uses the voice data to provide a voice message to the user saying, "A route to Tokyo Tower has been set."

[0421] Step 12:

[0422] The device displays a visual guide to Tokyo Tower on the screen, plotting the route on a map and indicating necessary guidance points.

[0423] Example 1

[0424] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0425] Conventional car navigation systems require users to manually input their destination or navigate complex menus, posing safety and operational challenges while driving. In particular, when voice control is inefficient, it can be difficult to correctly understand the user's intentions, leading to the inability to provide appropriate route guidance. Therefore, a system is needed that can more accurately and quickly analyze voice input and provide the optimal route based on the user's intentions.

[0426] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0427] In this invention, the server includes means for acquiring voice input from a user, means for converting the voice input into text data, means for analyzing the text data to identify the user's intention, means for acquiring destination information and calculating a route based on the user's intention, means for providing the acquired information to the user aurally and visually, means for analyzing the voice input converted into a digital signal using a voice recognition engine, means for analyzing the text data using natural language processing technology to identify the user's intention, and means for calculating an optimal route using a location information database and traffic condition data. This allows the user to intuitively operate the car navigation function using only voice, improving safety while driving and enabling faster and more accurate route guidance.

[0428] A "means for obtaining audio input" is a device or method that captures a user's voice and converts it into a digital signal.

[0429] The "means for converting voice input into text data" refers to the technology and engine that analyzes acquired voice data and converts it into text format data.

[0430] "Means for identifying user intent by analyzing text data" refers to a method for analyzing text data using natural language processing technology to clearly understand the content and purpose of the user's instructions.

[0431] The "means for acquiring destination information and calculating a route" refers to a database and analytical technology for acquiring the coordinates of a target point based on the user's intentions and calculating the optimal route.

[0432] The "means for providing audio and visual guidance to the user" refers to a method for visually and audibly informing the user of the generated route information using voice synthesis and display.

[0433] "Means for analyzing voice input converted into a digital signal using a voice recognition engine" refers to a technology that analyzes voice input with high accuracy and processes the voice data captured as a digital signal.

[0434] "Means for analyzing text data using natural language processing technology to identify user intent" refers to a method for analyzing text data using advanced natural language processing algorithms to accurately understand user intent.

[0435] "Means for calculating optimal routes using location information databases and traffic data" refers to systems and technologies that calculate the most efficient route for a user based on location information and real-time traffic information.

[0436] This invention is a system that allows users to operate car navigation functions by voice input. This system can acquire voice input and automatically perform recognition, analysis, destination setting, and route guidance.

[0437] Hardware and software used

[0438] To implement this system, the following hardware and software are used:

[0439] Hardware:

[0440] Microphone: Captures voice input from the user.

[0441] Display: Provides visual guidance to the user.

[0442] Processor: Performs speech recognition and data analysis.

[0443] Speaker: Provides audio guidance to the user.

[0444] software:

[0445] Speech recognition engine: For example, using a common speech recognition API (such as Google Speech-to-Text API), converts voice input into text data.

[0446] Natural language processing engine: For example, using a common natural language processing API (such as Google Natural Language API) to analyze text data and identify user intent.

[0447] Location database: For example, use a common geographic information API (such as Google Maps API) to obtain coordinate data of the destination.

[0448] Traffic information API: For example, use a common traffic information API (such as Waze API) to obtain real-time traffic conditions.

[0449] Speech synthesis engine: For example, use a common speech synthesis API (such as Google Text-to-Speech API) to convert text data into speech data.

[0450] Example

[0451] Example 1: Setting a destination

[0452] The user says "Navigate to Tokyo Station" while in the car. The device captures the speech and uses a speech recognition engine to convert it into text data saying "Navigate to Tokyo Station." The device then sends this text data to the server. The server uses a natural language processing engine to analyze the text data and identify the user's intent as "set destination." The server queries the location information database and traffic information API to obtain coordinate data for Tokyo Station and real-time traffic information. Based on this, the server calculates the optimal route and sends this information back to the device. The device then uses a speech synthesis engine to announce "Route to Tokyo Station has been set" and displays the route information on the display.

[0453] Example 2: Search for nearby stores

[0454] The user says, "Find a nearby restaurant." The device receives this speech and converts it into text data saying, "Find a nearby restaurant" using a speech recognition engine. The device then sends the text data to the server. The server uses a natural language processing engine to determine that the user's intent is to "search for nearby stores." The server then queries the location information database and traffic information API to obtain information about restaurants near the current location. The server sends the results back to the device, and the device announces, "Nearby restaurants are A, B, and C," and displays the restaurant's location information on the display.

[0455] Prompt Sentence Examples

[0456] "Navigate to Tokyo Station"

[0457] "Find a nearby restaurant"

[0458] This invention allows a user to intuitively operate car navigation functions using only voice, improving safety while driving and enabling faster and more accurate route guidance.

[0459] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0460] Step 1:

[0461] The user inputs voice. For example, "Tell me the way to Tokyo Tower." The device uses a microphone to capture this voice. The input is the user's voice signal, which is converted into a digital format. The output is digital voice data.

[0462] Step 2:

[0463] The device uses a speech recognition engine to convert digital voice data into text data. The input is digital voice data, and the engine performs phonetic analysis and parses the voice content into text format. The output is text data such as "Tell me the way to Tokyo Tower."

[0464] Step 3:

[0465] The terminal sends the generated text data to the server. The input is the text data, and a secure communication protocol (e.g., HTTPS) is used to securely transport this data to the server. The output is the server's confirmation of receipt.

[0466] Step 4:

[0467] The server receives the text data and analyzes it using a natural language processing engine. The input is text data, and the engine performs grammatical analysis and keyword extraction to identify the user's intent. The output is the intent information, "Set destination."

[0468] Step 5:

[0469] Based on the intent information, the server accesses the location information database to obtain the coordinate data of the destination. It also queries the traffic data API for real-time traffic conditions. The input is the destination name "Tokyo Tower" and the intent information "set destination." The database returns the geographic coordinates, and the traffic data API returns the latest traffic conditions. The output is the destination coordinate data and traffic condition data.

[0470] Step 6:

[0471] The server calculates the optimal route based on coordinate data and traffic condition data. The input is the destination coordinate data and traffic condition data, and the server uses a route calculation algorithm to derive the shortest and most efficient route. The output is the optimal route information.

[0472] Step 7:

[0473] The server sends the optimal route information to the terminal. The input is the optimal route information, which is sent to the terminal via a secure communication protocol. The output is the terminal's confirmation of receipt.

[0474] Step 8:

[0475] The terminal receives the optimal route information and generates voice guidance using a speech synthesis engine. The input is the optimal route information, and the speech synthesis engine converts the text-format route information into voice data. The output is voice data saying, "A route to Tokyo Tower has been set."

[0476] Step 9:

[0477] The terminal provides the generated voice data to the user through a speaker and simultaneously displays visual route guidance on the display. The input is voice data and visual guidance data, and the output is voice guidance and display.

[0478] (Application example 1)

[0479] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0480] Conventional car navigation systems often require manual operation by the user, resulting in issues of safety and convenience while driving. Furthermore, in autonomous vehicles, there is no easy way for users to give complex voice instructions, limiting operability. To solve this issue, there is a demand for a system that allows users to intuitively and efficiently navigate and operate autonomous vehicles and other vehicle operations using only voice input.

[0481] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0482] In this invention, the server includes means for acquiring voice input from a user, means for converting the voice input into text data, means for analyzing the text data to identify the user's intention, means for acquiring destination information and calculating a route based on the user's intention, means for providing the acquired route information to the user via voice and visual means, and means for operating the autonomous vehicle using voice, thereby minimizing manual operations while driving and enabling safe and efficient operation of the autonomous vehicle.

[0483] "Voice input" refers to a user providing words or instructions through a voice input device such as a microphone.

[0484] "Text data" refers to data that is expressed as text information after analyzing voice input using a voice recognition engine or the like.

[0485] "Means for identifying user intent" refers to a method for analyzing acquired text data using natural language processing technology, etc., to understand the user's requests and instructions.

[0486] A "location information database" is a database that stores location information such as geographic coordinates and addresses, and is used to locate specific locations and calculate routes.

[0487] "Traffic condition data" refers to data that indicates current road conditions and traffic information, and is used for route calculation and navigation.

[0488] "Route information" is information that includes specific directions and directional instructions for reaching the user's destination.

[0489] "Audio and visual guidance means" refers to a method of providing route information and instructions to the user both audibly and visually using a speech synthesis engine, a display, etc.

[0490] "Means for operating an autonomous vehicle using voice" refers to a method for controlling and adjusting various functions and settings within an autonomous vehicle in response to voice instructions from the user.

[0491] System Configuration

[0492] This system allows users to navigate and operate an autonomous vehicle through voice input. The specific configuration and processing steps of the system are shown below.

[0493] Hardware and Software

[0494] Smartphone: Equipped with a microphone and a display, it allows voice input and displays results.

[0495] Server: Maintains the voice recognition engine, natural language processing engine, location information database, and traffic situation database, and performs data analysis and calculations.

[0496] Specifically, the following technologies are used:

[0497] Speech recognition engine (e.g. Google Speech-to-Text API)

[0498] Natural language processing engine (e.g. Google Cloud Natural Language API)

[0499] Speech synthesis engine (e.g. Google Text-to-Speech API)

[0500] Data acquisition and processing

[0501] Acquiring voice input

[0502] The user inputs voice instructions through the smartphone's microphone, for example, "Turn right at the next traffic light."

[0503] Speech recognition and text conversion

[0504] The smartphone uses a speech recognition engine to convert the spoken input into text data in the form of "Turn right at the next traffic light."

[0505] Analysis using natural language processing

[0506] Sending to the server

[0507] The text data is sent to a server and analyzed by a natural language processing engine, which determines that the user's intent is to "turn right."

[0508] Route calculation and information acquisition

[0509] The server accesses the location information database and the traffic condition database to calculate the optimal right turn position based on the user's current location.

[0510] Audio and visual guidance

[0511] Server response and routing information transmission

[0512] The server sends the calculation results and route information to the smartphone, which then uses a speech synthesis engine to provide voice guidance such as "Turn right at the next traffic light." The smartphone also visually displays the right-turn location on the display.

[0513] Examples and prompts

[0514] Example 1: Reaching a highway entrance / exit

[0515] When a user says, "Take me to the next exit," the system analyzes their intent and guides them to the optimal route to the exit.

[0516] Example 2: Car charging station search

[0517] When a user says, "Tell me where the nearest charging station is," the system takes into account the current battery status and location information and provides directions to the best charging station.

[0518] Prompt Sentence Examples

[0519] The following prompts can be fed into the generative AI model to perform proper speech recognition and route calculation:

[0520] "Voice input: Turn right at the next traffic light."

[0521] "prompt:

[0522] 1. Convert speech to text.

[0523] 2. Identify the right turn indication from the text.

[0524] 3. Calculate the best right turn location from your current location.

[0525] 4. Please output visual and audio right turn instructions.

[0526] In this way, a system is realized that allows users to easily and safely navigate and operate an autonomous vehicle using only voice input.

[0527] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0528] Step 1:

[0529] The user performs voice input. The user speaks into the smartphone microphone, saying, "Turn right at the next traffic light." Voice data is generated as input.

[0530] Step 2:

[0531] The smartphone converts the voice into text data. It then uses a speech recognition engine (e.g., Google Speech-to-Text API) to analyze the voice data and convert it into text data such as "Turn right at the next traffic light." This generates text data as output.

[0532] Step 3:

[0533] The smartphone sends text data to the server. The sent text data arrives at the server, where it is analyzed to determine the user's intent using a natural language processing engine (e.g., Google Cloud Natural Language API). The input is the text data, and the output is the intent to "turn right."

[0534] Step 4:

[0535] The server obtains the current location information and calculates the optimal route. The server accesses the location information database and traffic condition database and calculates the route taking into account the user's current location and destination (right turn position at the next traffic light). The input is the user's current location and right turn instructions, and the output is right turn route information.

[0536] Step 5:

[0537] The server sends route information to the smartphone. The route information is sent from the server and reaches the smartphone. The route information is sent as input, and the information necessary for route guidance reaches the smartphone.

[0538] Step 6:

[0539] The smartphone provides route information through voice and visual guidance. The smartphone uses a speech synthesis engine (e.g., Google Text-to-Speech API) to provide voice guidance such as "Turn right at the next traffic light." The route information is also displayed visually on the display. Route information is input, and voice guidance and visual displays are generated as output.

[0540] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0541] The present invention allows a user to operate car navigation functions by voice, while at the same time recognizing the user's emotional state and providing an appropriate guidance method based on that emotion. The following describes an embodiment of the present invention.

[0542] The system includes a means for receiving voice input and converting it into text data using a voice recognition engine. The text data is then sent to a server, which uses natural language processing technology to identify the user's intent. The server also uses an emotion engine to analyze the user's emotional state and provides appropriate guidance based on this. Specifically, the system first converts the voice into text using a voice recognition engine, analyzes the user's intent using natural language processing technology, and then identifies the user's emotion using the emotion engine. Based on this information, the system obtains destination information, calculates a route, and provides audio and visual guidance to the user.

[0543] Program processing

[0544] 1. Acquiring voice input

[0545] The user issues a voice command such as "Tell me the way to Tokyo Tower." The device uses a microphone to capture the voice and converts it into text data using a built-in speech recognition engine. At this stage, the text data "Tell me the way to Tokyo Tower" is generated.

[0546] 2. Command analysis and emotion recognition

[0547] The device sends text data to the server. The server uses a natural language processing engine to analyze the text data and determine that the user's intent is to "set a destination." At the same time, it calls an emotion engine to determine the user's emotional state from their voice characteristics. For example, if the user is excited, the emotion engine will recognize this as "excited."

[0548] 3. Database integration

[0549] The server queries the location database and API to obtain the coordinate data of Tokyo Tower, and calculates the optimal route using traffic condition data. This determines the optimal route taking real-time traffic information into account.

[0550] 4. Emotion-based guidance adjustment

[0551] The server organizes the route information it obtains and calculates, and adjusts the guidance method based on the user's emotional state. For example, if the server detects that the user is tired, it will provide guidance in a calmer tone.

[0552] 5. Generate and send results

[0553] The server returns the organized route information to the terminal. The terminal uses a speech synthesis engine to convert the received route information into voice data for voice guidance, and provides audible guidance to the user, saying, "The route to Tokyo Tower has been set." The route information is then displayed as visual guidance on the terminal screen.

[0554] Specific examples

[0555] Example 1: Destination setting and emotion recognition

[0556] The user says "Navigate to Tokyo Station" while in the car. The device captures the speech and uses a speech recognition engine to convert it into text data: "Navigate to Tokyo Station." The device then sends the text data to the server. The server analyzes the text data and determines that the user's intention is to "set a destination." At the same time, the emotion engine analyzes the user's voice and determines, for example, that the user is in a relaxed state. The server queries a location database and API to obtain coordinate data for Tokyo Station, and then calculates the optimal route taking real-time traffic information into consideration. The calculation results are sent back to the device, which then provides voice guidance in a relaxed tone, saying "Route to Tokyo Station has been set," while displaying the route on the screen.

[0557] Example 2: Nearby store search and emotion recognition

[0558] The user says, "Find a nearby restaurant." The device captures the speech and converts it into text data saying, "Find a nearby restaurant" using a speech recognition engine. The text data is sent to the server, which uses a natural language processing engine to identify the user's intent as "search for nearby stores." At the same time, the emotion engine identifies the user's emotional state, recognizing, for example, that the user is excited. The server uses a location database and API to obtain information about restaurants near the current location and sends the results back to the device. The device then provides voice guidance in a soft tone, saying, "There are three nearby restaurants: A, B, and C," and displays the locations of the restaurants on the screen.

[0559] In this way, by combining the system with an emotion engine, it becomes possible to provide flexible guidance that corresponds to the user's emotional state, improving the convenience of operation and the user experience.

[0560] The processing flow will be explained below.

[0561] Step 1:

[0562] A user says, "Tell me the route to Tokyo Tower."

[0563] Step 2:

[0564] The device uses a built-in microphone to obtain the user's voice input.

[0565] Step 3:

[0566] The device calls a speech recognition engine and converts the acquired voice input into text data. For example, text data such as "Tell me the way to Tokyo Tower" is generated.

[0567] Step 4:

[0568] The device sends the generated text data to the server using the HTTP / HTTPS communication protocol.

[0569] Step 5:

[0570] The server receives the text data and calls an emotion engine to identify the user's emotional state from the user's voice characteristics. For example, emotions such as "excited" or "relaxed" can be recognized from the tone, speed, and rhythm of the user's voice.

[0571] Step 6:

[0572] The server calls a natural language processing engine to analyze the text data, and identifies the user's intent as "setting a destination" from the text data.

[0573] Step 7:

[0574] The server queries a location database or API to obtain the coordinate data of Tokyo Tower.

[0575] Step 8:

[0576] The server collects traffic data and uses it to calculate the optimal route from the user's current location to Tokyo Tower. Traffic information includes information on traffic congestion and accidents.

[0577] Step 9:

[0578] The server then organizes the calculated route information and adjusts the guidance based on the user's emotional state. For example, if the user is "excited," the server will set the guidance to be in a calm tone.

[0579] Step 10:

[0580] The server returns organized route information to the device, including each step of the route guidance and map data.

[0581] Step 11:

[0582] The route information received by the terminal is passed to a voice synthesis engine and converted into voice data for voice guidance.

[0583] Step 12:

[0584] The device uses the voice data to guide the user by voice, saying, "A route to Tokyo Tower has been set." The tone of the voice guidance is adjusted according to the user's emotional state.

[0585] Step 13:

[0586] The device displays a visual guide to Tokyo Tower on the screen, drawing the route on a map and highlighting the next guide point.

[0587] Step 14:

[0588] The user continues driving to the destination using the audio and visual guidance as a reference.

[0589] Example 2

[0590] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0591] Conventional car navigation systems set destinations and provide route guidance based on user voice instructions, but they are unable to consider the user's emotional state, resulting in issues with ease of operation and user experience.In addition, when the user's emotional state changes, the system may not provide an appropriate tone or guidance method, which can be stressful.

[0592] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0593] In this invention, the server includes a means for converting voice input into text data, a means for analyzing the user's intention and emotional state, and a means for determining a guidance method based on the analysis results, thereby making it possible to provide flexible and appropriate guidance according to the user's intention and emotional state.

[0594] "Means for obtaining voice input" refers to devices and programs for collecting the user's voice in digital form.

[0595] The "means for converting voice input into text data" refers to a device and program that has the function of analyzing collected voice and converting it into text information.

[0596] "Means for analyzing text data to identify user intent" refers to devices and programs that have the function of analyzing converted text information using natural language processing technology and understanding the user's requests and instructions.

[0597] The "means for analyzing the user's emotional state" refers to a device or program that has the function of detecting the user's emotions based on the characteristics of voice or text and identifying that state.

[0598] The "means for determining an appropriate guidance method" refers to a device and a program that have the function of selecting the most appropriate guidance method based on the analyzed user's intention and emotional state.

[0599] The "means for acquiring destination information and calculating a route" refers to a device and a program that has the function of acquiring information about a specified destination and calculating the optimal route to that point.

[0600] The "means for providing the user with the acquired information by voice and visually" refers to a device and a program that has the function of providing the user with calculated route information by voice and visually.

[0601] The present invention allows a user to operate car navigation functions by voice, while at the same time recognizing the emotional state of the user and providing an appropriate guidance method based on that emotion.

[0602] Acquiring voice input

[0603] When a user says, "Tell me the way to Tokyo Tower," the device picks up the voice using a microphone installed in the car. The acquired voice data is converted into text data via a built-in voice recognition engine (e.g., general-purpose voice recognition software). At this stage, the text data "Tell me the way to Tokyo Tower" is generated.

[0604] Command parsing and emotion recognition

[0605] The text data generated by the device is sent to the server. The server then uses a natural language processing engine (e.g., natural language processing software) to analyze the text data and determine that the user's intent is to "set a destination." At the same time, it invokes an emotion engine (e.g., emotion analysis software) to determine the user's emotional state from the user's voice characteristics. For example, if the user is excited, the emotion engine will recognize this as "excited."

[0606] Database integration

[0607] The server queries a location database (e.g., location API) or other APIs to obtain the coordinate data of Tokyo Tower. It also uses a traffic condition API to collect real-time traffic information and calculates the optimal route. This determines the optimal route taking real-time traffic information into account.

[0608] Emotion-based guidance adjustment

[0609] The server organizes the route information it obtains and calculates, and adjusts the guidance method based on the user's emotional state. For example, if the emotion engine determines that the user is tired, the guidance method will be adjusted to be more calming.

[0610] Generate and send results

[0611] The server returns the organized route information to the terminal. The terminal uses a speech synthesis engine (for example, speech synthesis software) to convert the received route information into audio data for voice guidance, and provides audible guidance to the user, such as "The route to Tokyo Tower has been set." The terminal also displays the route information on the screen as visual guidance.

[0612] Specific examples

[0613] Example 1: Destination setting and emotion recognition

[0614] The user says "Navigate to Tokyo Station" while in the car. The device uses a microphone to capture the speech and converts it into text data, "Navigate to Tokyo Station," using a speech recognition engine. The device then sends the text data to a server. The server analyzes the speech using natural language processing software and determines that the user's intent is "set destination." At the same time, emotion analysis software analyzes the user's voice and determines that the user is relaxed. The server queries location information APIs and other APIs to obtain coordinate data for Tokyo Station, and calculates the optimal route taking real-time traffic information into account. The calculation results are sent back to the device, which then provides voice guidance in a relaxed tone, saying "Route to Tokyo Station has been set," while displaying the route on the screen.

[0615] Example 2: Nearby store search and emotion recognition

[0616] The user says, "Find a nearby restaurant." The device captures the speech and converts it into text data, "Find a nearby restaurant," using a speech recognition engine. The text data is sent to the server. The server uses natural language processing software to identify the user's intent as "search for nearby restaurants." At the same time, emotion analysis software identifies the user's emotional state and recognizes that they are excited. The server uses location APIs and other APIs to obtain information about restaurants near the current location and sends the results back to the device. The device then provides voice guidance in a softer tone to calm the user's excitement, saying, "There are three restaurants nearby: A, B, and C," and displays the restaurant locations on the screen.

[0617] This system enables flexible and appropriate guidance to be provided according to the user's emotional state, improving operational convenience and user experience.

[0618] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0619] Step 1:

[0620] Acquiring voice input

[0621] The user says, "Tell me the way to Tokyo Tower." The device uses a microphone to capture the user's voice. The captured voice data is input and output.

[0622] Step 2:

[0623] Converting speech to text data

[0624] The acquired voice data is input into a voice recognition engine (for example, general-purpose voice recognition software). This engine converts the voice data into text data and outputs it. Specifically, the text data generated is "Tell me the way to Tokyo Tower."

[0625] Step 3:

[0626] Parsing commands

[0627] The device sends text data to the server. The text data is input into the server's natural language processing engine (e.g., natural language processing software). The server analyzes the text data, identifies the user's intention, and outputs the intention data "set destination."

[0628] Step 4:

[0629] Emotion recognition

[0630] The server invokes an emotion engine (e.g., emotion analysis software) to analyze the characteristics of the voice data, which determines the user's emotional state and outputs the emotion data (e.g., "excited").

[0631] Step 5:

[0632] Database integration

[0633] The server communicates with a location information database (e.g., location information API) to obtain coordinate data for a specific destination (e.g., Tokyo Tower). As a result of querying the API, the destination coordinate data is output. Furthermore, the server queries the traffic conditions API for real-time traffic information, calculates the optimal route, and outputs this route data.

[0634] Step 6:

[0635] Emotion-based guidance adjustment

[0636] The server determines an appropriate guidance method based on the user's emotional data and route data. For example, if the server determines that the user is tired, it adjusts the guidance method to provide guidance in a calmer tone. This adjusted guidance method data is output.

[0637] Step 7:

[0638] Generate and send results

[0639] The server returns the organized route information and the adjusted guidance method to the terminal. The terminal receives them and uses a speech synthesis engine (e.g., speech synthesis software) to generate voice data for voice guidance. This voice data and visual guidance data are output.

[0640] Step 8:

[0641] User Information

[0642] The device uses voice data to announce, "The route to Tokyo Tower has been set." It also displays visual guidance on the screen, providing the user with a visual of the route.

[0643] (Application example 2)

[0644] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0645] While voice navigation is intuitive and convenient, it can be stressful depending on the user's emotional state, resulting in a poor user experience. For example, when the user is tired or excited, it can be difficult to receive appropriate guidance. Furthermore, many navigation systems provide uniform guidance without taking the user's emotional state into account, resulting in a lack of optimal guidance tailored to the situation. There is a need to solve these problems and provide optimal guidance tailored to the user's emotional state.

[0646] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0647] In this invention, the server includes means for acquiring voice input from the user, means for converting the voice input into text data, means for identifying the user's intention, means for recognizing the user's emotional state, means for acquiring destination information and calculating a route based on the user's intention and emotional state, means for providing the acquired information to the user by voice and visual means, and means for providing guidance in a tone corresponding to the emotional state, thereby enabling flexible and optimal navigation guidance according to the user's emotional state.

[0648] "Means for obtaining voice input" refers to a device or function that recognizes a user's voice commands and captures them as digital data.

[0649] "Means for converting voice input into text data" refers to software or hardware for analyzing acquired voice data and converting it into text data.

[0650] "Means for identifying user intent" refers to a device or function that uses natural language processing techniques to analyze the converted text data and understand what the user is requesting.

[0651] "Means for recognizing a user's emotional state" refers to technology for analyzing a user's voice characteristics and other input data to identify the user's emotions (e.g., tired, excited, etc.).

[0652] "Means for obtaining destination information and calculating a route" refers to a database and algorithm for obtaining destination information based on the user's intentions and calculating the optimal route to that destination in real time.

[0653] "Means for providing the user with audio and visual guidance of the acquired information" refers to a device or function that provides the user with calculated route information and other navigation information using voice synthesis technology and a display.

[0654] "Means for providing guidance in a tone appropriate to the emotional state" refers to techniques and algorithms for providing guidance in a tone or format appropriate to the emotional state of the user.

[0655] The present invention provides a system that allows a user to operate a car navigation function by voice and provides appropriate guidance based on the user's emotional state. Hereinafter, an embodiment of the present invention will be described in detail.

[0656] The overall system includes means for capturing voice input, analyzing the user's intent, recognizing the user's emotional state, and providing guidance based on that. First, the user issues a voice command in the car. The device captures the voice through a microphone and converts the voice data into text data using a speech recognition engine, which could be the Google Speech-to-Text API.

[0657] The acquired text data is then sent to a server, where natural language processing technology is used to analyze the user's intent. This part is implemented using the "Custom NLP API." The server then uses the "Custom Emotion Recognition API" to identify the user's emotional state from their voice characteristics.

[0658] Based on the analysis results, the server uses a location database and traffic data to obtain destination information and calculates the optimal route. Specifically, the process is as follows: The server obtains the destination coordinate data using the "Custom Location API," and then calculates the optimal route taking real-time traffic information into account using the "Custom Route API."

[0659] The server organizes the acquired and calculated route information and adjusts the guidance method based on the user's emotional state. For example, if the emotion engine recognizes that the user is tired, guidance will be provided in a calm tone. Finally, the resulting route information is sent back to the device, where it is converted into voice data by a speech synthesis engine and presented to the user. Visual guidance is also displayed on the device screen.

[0660] Specific examples

[0661] Destination settings:

[0662] The user says, "Navigate to Tokyo Station." The device receives the speech and converts it into text data, "Navigate to Tokyo Station," via a voice recognition engine. The server analyzes the text data and determines that the user's intention is to "set a destination," while the emotion engine simultaneously determines that the user is in a relaxed state. The server queries the location database and API to obtain the coordinate data of Tokyo Station, and calculates the optimal route taking into account real-time traffic information. The calculation results are sent back to the device, where a voice guide in a relaxed tone announces, "Route to Tokyo Station has been set," and the route is displayed on the screen.

[0663] Example prompt sentence:

[0664] "Taking voice data as input when a user says, 'Tell me the way to Tokyo Tower.' Analyze the user's intent, identify 'Tokyo Tower' as the destination preference, and check the user's emotional state. If the emotion engine determines that this user is tired, what are the steps to provide directions in a calmer tone?"

[0665] This system can provide flexible and optimal navigation guidance according to the user's emotional state, thereby improving the user's driving experience.

[0666] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0667] Step 1:

[0668] The user specifies the destination by voice.

[0669] Input: User's voice data

[0670] Specific operation: The user gives instructions in the car, such as "Navigate to Tokyo Tower."

[0671] Step 2:

[0672] The device receives the voice data and converts it into text data using a voice recognition engine.

[0673] Input: User's voice data

[0674] Data calculation: Converting voice data into text data using the Google Speech-to-Text API

[0675] Output: Text data "Navigate to Tokyo Tower"

[0676] Step 3:

[0677] The device sends text data to a server, which then uses natural language processing technology to analyze the user's intent.

[0678] Input: Text data

[0679] Data calculation: Using Custom NLP API, analyze the user's intent from the text data and identify "destination setting."

[0680] Output: User intent is "Set destination"

[0681] Step 4:

[0682] The server analyzes the user's emotional state from their voice characteristics.

[0683] Input: User's voice data

[0684] Data Computing: Identifying a user's emotional state from audio data using the Custom Emotion Recognition API

[0685] Output: User's emotional state (e.g., relaxed)

[0686] Step 5:

[0687] The server obtains the destination information and calculates the route.

[0688] Input: User intent (destination setting), user emotional state

[0689] Data calculation: Obtain destination coordinate data using the Custom Location API and calculate the optimal route using the Custom Route API

[0690] Output: Destination coordinates, optimal route

[0691] Step 6:

[0692] The server organizes the information it obtains and adjusts the guidance method according to the emotional state.

[0693] Input: Optimal route, user's emotional state

[0694] Data calculations: Adjusting the tone of guidance based on the user's emotional state (e.g., a relaxed tone)

[0695] Output: Adjusted guidance information

[0696] Step 7:

[0697] The server transmits the adjusted guidance information to the terminal.

[0698] Input: Adjusted guidance information

[0699] Data Computing: Packaging and transmitting information

[0700] Output: Guidance information sent to the terminal

[0701] Step 8:

[0702] The terminal uses a speech synthesis engine and a display to provide audio and visual guidance to the user.

[0703] Input: Guidance information sent to the terminal

[0704] Specific operation: The device generates a guidance message using a speech synthesis engine and announces in a relaxed tone, "A route to Tokyo Tower has been set." Visual information is also displayed on the device's display.

[0705] Output: Guidance information provided to the user

[0706] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0707] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0708] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0709] [Third embodiment]

[0710] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0711] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0712] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0713] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0714] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0715] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0716] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0717] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0718] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0719] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0720] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0721] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0722] This system allows a user to operate the functions of a car navigation system by voice. An embodiment of this system will be described below.

[0723] The system includes a means for capturing voice input and converting it into text using a speech recognition engine. The text data is then sent to a server, which uses natural language processing techniques to identify the user's intent, retrieves destination information, calculates routes, and finally provides the retrieved route information to the user via audio and visual guidance.

[0724] Program processing

[0725] 1. Acquiring voice input

[0726] The user issues a voice command such as "Tell me the way to Tokyo Tower." The device uses a microphone to capture the voice and converts it into text data using a built-in speech recognition engine. At this stage, the text data "Tell me the way to Tokyo Tower" is generated.

[0727] 2. Parsing the command

[0728] The device sends the text data to the server, which uses a natural language processing engine to analyze the text data and determine that the user's intent is to "set a destination."

[0729] 3. Database integration

[0730] The server queries the location database and API to obtain the coordinate data of Tokyo Tower, and calculates the optimal route using traffic condition data. This determines the optimal route taking real-time traffic information into account.

[0731] 4. Generate and send results

[0732] The server returns the route information it has acquired and calculated to the terminal. The terminal then uses a speech synthesis engine to convert the received route information into voice guidance data, and provides audible guidance to the user, saying, "The route to Tokyo Tower has been set." The route information is then displayed as visual guidance on the terminal screen.

[0733] Specific examples

[0734] Example 1: Setting a destination

[0735] The user says "Navigate to Tokyo Station" while in the car. The device receives the speech and uses a speech recognition engine to convert it into text data: "Navigate to Tokyo Station." The device then sends the text data to the server. The server analyzes the text data and determines that the user's intention is to "set a destination." The server queries a location database and API to obtain the coordinate data of Tokyo Station, and then calculates the optimal route taking into account real-time traffic information. The calculation results are sent back to the device, which then provides voice guidance to the user saying, "Route to Tokyo Station has been set," and displays the route on the screen.

[0736] Example 2: Search for nearby stores

[0737] The user says, "Find a nearby restaurant." The device receives the speech and converts it into text data saying, "Find a nearby restaurant" using a speech recognition engine. The text data is sent to the server, which uses a natural language processing engine to identify the user's intent as "Search for nearby stores." The server uses a location database and API to obtain information about restaurants near the current location and sends the results back to the device. The device then provides voice guidance saying, "There are three nearby restaurants: A, B, and C," and displays the locations of the restaurants on the screen.

[0738] summary

[0739] This system allows users to intuitively operate various car navigation functions using only their voice, improving safety while driving and increasing convenience of operation.

[0740] The processing flow will be explained below.

[0741] Step 1:

[0742] A user says, "Tell me the route to Tokyo Tower."

[0743] Step 2:

[0744] The device uses a built-in microphone to obtain the user's voice input.

[0745] Step 3:

[0746] The device calls a speech recognition engine and converts the acquired voice input into text data. For example, text data such as "Tell me the way to Tokyo Tower" is generated.

[0747] Step 4:

[0748] The device sends the generated text data to the server using the HTTP / HTTPS communication protocol.

[0749] Step 5:

[0750] The server receives the text data and invokes a natural language processing engine to analyze the text data.

[0751] Step 6:

[0752] The server identifies the user's intention from the analysis results. In this case, the user's intention is determined to be "setting a destination."

[0753] Step 7:

[0754] The server queries a location database or API to obtain the coordinate data of Tokyo Tower.

[0755] Step 8:

[0756] The server obtains traffic data and uses this data to calculate the optimal route from the current location to Tokyo Tower.

[0757] Step 9:

[0758] The server organizes the calculated route information and sends it back to the device, including route guidance steps and map data.

[0759] Step 10:

[0760] The route information received by the terminal is passed to a voice synthesis engine and converted into voice data for voice guidance.

[0761] Step 11:

[0762] The terminal uses the voice data to provide a voice message to the user saying, "A route to Tokyo Tower has been set."

[0763] Step 12:

[0764] The device displays a visual guide to Tokyo Tower on the screen, plotting the route on a map and indicating necessary guidance points.

[0765] Example 1

[0766] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0767] Conventional car navigation systems require users to manually input their destination or navigate complex menus, posing safety and operational challenges while driving. In particular, when voice control is inefficient, it can be difficult to correctly understand the user's intentions, leading to the inability to provide appropriate route guidance. Therefore, a system is needed that can more accurately and quickly analyze voice input and provide the optimal route based on the user's intentions.

[0768] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0769] In this invention, the server includes means for acquiring voice input from a user, means for converting the voice input into text data, means for analyzing the text data to identify the user's intention, means for acquiring destination information and calculating a route based on the user's intention, means for providing the acquired information to the user aurally and visually, means for analyzing the voice input converted into a digital signal using a voice recognition engine, means for analyzing the text data using natural language processing technology to identify the user's intention, and means for calculating an optimal route using a location information database and traffic condition data. This allows the user to intuitively operate the car navigation function using only voice, improving safety while driving and enabling faster and more accurate route guidance.

[0770] A "means for obtaining audio input" is a device or method that captures a user's voice and converts it into a digital signal.

[0771] The "means for converting voice input into text data" refers to the technology and engine that analyzes acquired voice data and converts it into text format data.

[0772] "Means for identifying user intent by analyzing text data" refers to a method for analyzing text data using natural language processing technology to clearly understand the content and purpose of the user's instructions.

[0773] The "means for acquiring destination information and calculating a route" refers to a database and analytical technology for acquiring the coordinates of a target point based on the user's intentions and calculating the optimal route.

[0774] The "means for providing audio and visual guidance to the user" refers to a method for visually and audibly informing the user of the generated route information using voice synthesis and display.

[0775] "Means for analyzing voice input converted into a digital signal using a voice recognition engine" refers to a technology that analyzes voice input with high accuracy and processes the voice data captured as a digital signal.

[0776] "Means for analyzing text data using natural language processing technology to identify user intent" refers to a method for analyzing text data using advanced natural language processing algorithms to accurately understand user intent.

[0777] "Means for calculating optimal routes using location information databases and traffic data" refers to systems and technologies that calculate the most efficient route for a user based on location information and real-time traffic information.

[0778] This invention is a system that allows users to operate car navigation functions by voice input. This system can acquire voice input and automatically perform recognition, analysis, destination setting, and route guidance.

[0779] Hardware and software used

[0780] To implement this system, the following hardware and software are used:

[0781] Hardware:

[0782] Microphone: Captures voice input from the user.

[0783] Display: Provides visual guidance to the user.

[0784] Processor: Performs speech recognition and data analysis.

[0785] Speaker: Provides audio guidance to the user.

[0786] software:

[0787] Speech recognition engine: For example, using a common speech recognition API (such as Google Speech-to-Text API), converts voice input into text data.

[0788] Natural language processing engine: For example, using a common natural language processing API (such as Google Natural Language API) to analyze text data and identify user intent.

[0789] Location database: For example, use a common geographic information API (such as Google Maps API) to obtain coordinate data of the destination.

[0790] Traffic information API: For example, use a common traffic information API (such as Waze API) to obtain real-time traffic conditions.

[0791] Speech synthesis engine: For example, use a common speech synthesis API (such as Google Text-to-Speech API) to convert text data into speech data.

[0792] Example

[0793] Example 1: Setting a destination

[0794] The user says "Navigate to Tokyo Station" while in the car. The device captures the speech and uses a speech recognition engine to convert it into text data saying "Navigate to Tokyo Station." The device then sends this text data to the server. The server uses a natural language processing engine to analyze the text data and identify the user's intent as "set destination." The server queries the location information database and traffic information API to obtain coordinate data for Tokyo Station and real-time traffic information. Based on this, the server calculates the optimal route and sends this information back to the device. The device then uses a speech synthesis engine to announce "Route to Tokyo Station has been set" and displays the route information on the display.

[0795] Example 2: Search for nearby stores

[0796] The user says, "Find a nearby restaurant." The device receives this speech and converts it into text data saying, "Find a nearby restaurant" using a speech recognition engine. The device then sends the text data to the server. The server uses a natural language processing engine to determine that the user's intent is to "search for nearby stores." The server then queries the location information database and traffic information API to obtain information about restaurants near the current location. The server sends the results back to the device, and the device announces, "Nearby restaurants are A, B, and C," and displays the restaurant's location information on the display.

[0797] Prompt Sentence Examples

[0798] "Navigate to Tokyo Station"

[0799] "Find a nearby restaurant"

[0800] This invention allows a user to intuitively operate car navigation functions using only voice, improving safety while driving and enabling faster and more accurate route guidance.

[0801] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0802] Step 1:

[0803] The user inputs voice. For example, "Tell me the way to Tokyo Tower." The device uses a microphone to capture this voice. The input is the user's voice signal, which is converted into a digital format. The output is digital voice data.

[0804] Step 2:

[0805] The device uses a speech recognition engine to convert digital voice data into text data. The input is digital voice data, and the engine performs phonetic analysis and parses the voice content into text format. The output is text data such as "Tell me the way to Tokyo Tower."

[0806] Step 3:

[0807] The terminal sends the generated text data to the server. The input is the text data, and a secure communication protocol (e.g., HTTPS) is used to securely transport this data to the server. The output is the server's confirmation of receipt.

[0808] Step 4:

[0809] The server receives the text data and analyzes it using a natural language processing engine. The input is text data, and the engine performs grammatical analysis and keyword extraction to identify the user's intent. The output is the intent information, "Set destination."

[0810] Step 5:

[0811] Based on the intent information, the server accesses the location information database to obtain the coordinate data of the destination. It also queries the traffic data API for real-time traffic conditions. The input is the destination name "Tokyo Tower" and the intent information "set destination." The database returns the geographic coordinates, and the traffic data API returns the latest traffic conditions. The output is the destination coordinate data and traffic condition data.

[0812] Step 6:

[0813] The server calculates the optimal route based on coordinate data and traffic condition data. The input is the destination coordinate data and traffic condition data, and the server uses a route calculation algorithm to derive the shortest and most efficient route. The output is the optimal route information.

[0814] Step 7:

[0815] The server sends the optimal route information to the terminal. The input is the optimal route information, which is sent to the terminal via a secure communication protocol. The output is the terminal's confirmation of receipt.

[0816] Step 8:

[0817] The terminal receives the optimal route information and generates voice guidance using a speech synthesis engine. The input is the optimal route information, and the speech synthesis engine converts the text-format route information into voice data. The output is voice data saying, "A route to Tokyo Tower has been set."

[0818] Step 9:

[0819] The terminal provides the generated voice data to the user through a speaker and simultaneously displays visual route guidance on the display. The input is voice data and visual guidance data, and the output is voice guidance and display.

[0820] (Application example 1)

[0821] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0822] Conventional car navigation systems often require manual operation by the user, resulting in issues of safety and convenience while driving. Furthermore, in autonomous vehicles, there is no easy way for users to give complex voice instructions, limiting operability. To solve this issue, there is a demand for a system that allows users to intuitively and efficiently navigate and operate autonomous vehicles and other vehicle operations using only voice input.

[0823] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0824] In this invention, the server includes means for acquiring voice input from a user, means for converting the voice input into text data, means for analyzing the text data to identify the user's intention, means for acquiring destination information and calculating a route based on the user's intention, means for providing the acquired route information to the user via voice and visual means, and means for operating the autonomous vehicle using voice, thereby minimizing manual operations while driving and enabling safe and efficient operation of the autonomous vehicle.

[0825] "Voice input" refers to a user providing words or instructions through a voice input device such as a microphone.

[0826] "Text data" refers to data that is expressed as text information after analyzing voice input using a voice recognition engine or the like.

[0827] "Means for identifying user intent" refers to a method for analyzing acquired text data using natural language processing technology, etc., to understand the user's requests and instructions.

[0828] A "location information database" is a database that stores location information such as geographic coordinates and addresses, and is used to locate specific locations and calculate routes.

[0829] "Traffic condition data" refers to data that indicates current road conditions and traffic information, and is used for route calculation and navigation.

[0830] "Route information" is information that includes specific directions and directional instructions for reaching the user's destination.

[0831] "Audio and visual guidance means" refers to a method of providing route information and instructions to the user both audibly and visually using a speech synthesis engine, a display, etc.

[0832] "Means for operating an autonomous vehicle using voice" refers to a method for controlling and adjusting various functions and settings within an autonomous vehicle in response to voice instructions from the user.

[0833] System Configuration

[0834] This system allows users to navigate and operate an autonomous vehicle through voice input. The specific configuration and processing steps of the system are shown below.

[0835] Hardware and Software

[0836] Smartphone: Equipped with a microphone and a display, it allows voice input and displays results.

[0837] Server: Maintains the voice recognition engine, natural language processing engine, location information database, and traffic situation database, and performs data analysis and calculations.

[0838] Specifically, the following technologies are used:

[0839] Speech recognition engine (e.g. Google Speech-to-Text API)

[0840] Natural language processing engine (e.g. Google Cloud Natural Language API)

[0841] Speech synthesis engine (e.g. Google Text-to-Speech API)

[0842] Data acquisition and processing

[0843] Acquiring voice input

[0844] The user inputs voice instructions through the smartphone's microphone, for example, "Turn right at the next traffic light."

[0845] Speech recognition and text conversion

[0846] The smartphone uses a speech recognition engine to convert the spoken input into text data in the form of "Turn right at the next traffic light."

[0847] Analysis using natural language processing

[0848] Sending to the server

[0849] The text data is sent to a server and analyzed by a natural language processing engine, which determines that the user's intent is to "turn right."

[0850] Route calculation and information acquisition

[0851] The server accesses the location information database and the traffic condition database to calculate the optimal right turn position based on the user's current location.

[0852] Audio and visual guidance

[0853] Server response and routing information transmission

[0854] The server sends the calculation results and route information to the smartphone, which then uses a speech synthesis engine to provide voice guidance such as "Turn right at the next traffic light." The smartphone also visually displays the right-turn location on the display.

[0855] Examples and prompts

[0856] Example 1: Reaching a highway entrance / exit

[0857] When a user says, "Take me to the next exit," the system analyzes their intent and guides them to the optimal route to the exit.

[0858] Example 2: Car charging station search

[0859] When a user says, "Tell me where the nearest charging station is," the system takes into account the current battery status and location information and provides directions to the best charging station.

[0860] Prompt Sentence Examples

[0861] The following prompts can be fed into the generative AI model to perform proper speech recognition and route calculation:

[0862] "Voice input: Turn right at the next traffic light."

[0863] "prompt:

[0864] 1. Convert speech to text.

[0865] 2. Identify the right turn indication from the text.

[0866] 3. Calculate the best right turn location from your current location.

[0867] 4. Please output visual and audio right turn instructions.

[0868] In this way, a system is realized that allows users to easily and safely navigate and operate an autonomous vehicle using only voice input.

[0869] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0870] Step 1:

[0871] The user performs voice input. The user speaks into the smartphone microphone, saying, "Turn right at the next traffic light." Voice data is generated as input.

[0872] Step 2:

[0873] The smartphone converts the voice into text data. It then uses a speech recognition engine (e.g., Google Speech-to-Text API) to analyze the voice data and convert it into text data such as "Turn right at the next traffic light." This generates text data as output.

[0874] Step 3:

[0875] The smartphone sends text data to the server. The sent text data arrives at the server, where it is analyzed to determine the user's intent using a natural language processing engine (e.g., Google Cloud Natural Language API). The input is the text data, and the output is the intent to "turn right."

[0876] Step 4:

[0877] The server obtains the current location information and calculates the optimal route. The server accesses the location information database and traffic condition database and calculates the route taking into account the user's current location and destination (right turn position at the next traffic light). The input is the user's current location and right turn instructions, and the output is right turn route information.

[0878] Step 5:

[0879] The server sends route information to the smartphone. The route information is sent from the server and reaches the smartphone. The route information is sent as input, and the information necessary for route guidance reaches the smartphone.

[0880] Step 6:

[0881] The smartphone provides route information through voice and visual guidance. The smartphone uses a speech synthesis engine (e.g., Google Text-to-Speech API) to provide voice guidance such as "Turn right at the next traffic light." The route information is also displayed visually on the display. Route information is input, and voice guidance and visual displays are generated as output.

[0882] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0883] The present invention allows a user to operate car navigation functions by voice, while at the same time recognizing the user's emotional state and providing an appropriate guidance method based on that emotion. The following describes an embodiment of the present invention.

[0884] The system includes a means for receiving voice input and converting it into text data using a voice recognition engine. The text data is then sent to a server, which uses natural language processing technology to identify the user's intent. The server also uses an emotion engine to analyze the user's emotional state and provides appropriate guidance based on this. Specifically, the system first converts the voice into text using a voice recognition engine, analyzes the user's intent using natural language processing technology, and then identifies the user's emotion using the emotion engine. Based on this information, the system obtains destination information, calculates a route, and provides audio and visual guidance to the user.

[0885] Program processing

[0886] 1. Acquiring voice input

[0887] The user issues a voice command such as "Tell me the way to Tokyo Tower." The device uses a microphone to capture the voice and converts it into text data using a built-in speech recognition engine. At this stage, the text data "Tell me the way to Tokyo Tower" is generated.

[0888] 2. Command analysis and emotion recognition

[0889] The device sends text data to the server. The server uses a natural language processing engine to analyze the text data and determine that the user's intent is to "set a destination." At the same time, it calls an emotion engine to determine the user's emotional state from their voice characteristics. For example, if the user is excited, the emotion engine will recognize this as "excited."

[0890] 3. Database integration

[0891] The server queries the location database and API to obtain the coordinate data of Tokyo Tower, and calculates the optimal route using traffic condition data. This determines the optimal route taking real-time traffic information into account.

[0892] 4. Emotion-based guidance adjustment

[0893] The server organizes the route information it obtains and calculates, and adjusts the guidance method based on the user's emotional state. For example, if the server detects that the user is tired, it will provide guidance in a calmer tone.

[0894] 5. Generate and send results

[0895] The server returns the organized route information to the terminal. The terminal uses a speech synthesis engine to convert the received route information into voice data for voice guidance, and provides audible guidance to the user, saying, "The route to Tokyo Tower has been set." The route information is then displayed as visual guidance on the terminal screen.

[0896] Specific examples

[0897] Example 1: Destination setting and emotion recognition

[0898] The user says "Navigate to Tokyo Station" while in the car. The device captures the speech and uses a speech recognition engine to convert it into text data: "Navigate to Tokyo Station." The device then sends the text data to the server. The server analyzes the text data and determines that the user's intention is to "set a destination." At the same time, the emotion engine analyzes the user's voice and determines, for example, that the user is in a relaxed state. The server queries a location database and API to obtain coordinate data for Tokyo Station, and then calculates the optimal route taking real-time traffic information into consideration. The calculation results are sent back to the device, which then provides voice guidance in a relaxed tone, saying "Route to Tokyo Station has been set," while displaying the route on the screen.

[0899] Example 2: Nearby store search and emotion recognition

[0900] The user says, "Find a nearby restaurant." The device captures the speech and converts it into text data saying, "Find a nearby restaurant" using a speech recognition engine. The text data is sent to the server, which uses a natural language processing engine to identify the user's intent as "search for nearby stores." At the same time, the emotion engine identifies the user's emotional state, recognizing, for example, that the user is excited. The server uses a location database and API to obtain information about restaurants near the current location and sends the results back to the device. The device then provides voice guidance in a soft tone, saying, "There are three nearby restaurants: A, B, and C," and displays the locations of the restaurants on the screen.

[0901] In this way, by combining the system with an emotion engine, it becomes possible to provide flexible guidance that corresponds to the user's emotional state, improving the convenience of operation and the user experience.

[0902] The processing flow will be explained below.

[0903] Step 1:

[0904] A user says, "Tell me the route to Tokyo Tower."

[0905] Step 2:

[0906] The device uses a built-in microphone to obtain the user's voice input.

[0907] Step 3:

[0908] The device calls a speech recognition engine and converts the acquired voice input into text data. For example, text data such as "Tell me the way to Tokyo Tower" is generated.

[0909] Step 4:

[0910] The device sends the generated text data to the server using the HTTP / HTTPS communication protocol.

[0911] Step 5:

[0912] The server receives the text data and calls an emotion engine to identify the user's emotional state from the user's voice characteristics. For example, emotions such as "excited" or "relaxed" can be recognized from the tone, speed, and rhythm of the user's voice.

[0913] Step 6:

[0914] The server calls a natural language processing engine to analyze the text data, and identifies the user's intent as "setting a destination" from the text data.

[0915] Step 7:

[0916] The server queries a location database or API to obtain the coordinate data of Tokyo Tower.

[0917] Step 8:

[0918] The server collects traffic data and uses it to calculate the optimal route from the user's current location to Tokyo Tower. Traffic information includes information on traffic congestion and accidents.

[0919] Step 9:

[0920] The server then organizes the calculated route information and adjusts the guidance based on the user's emotional state. For example, if the user is "excited," the server will set the guidance to be in a calm tone.

[0921] Step 10:

[0922] The server returns organized route information to the device, including each step of the route guidance and map data.

[0923] Step 11:

[0924] The route information received by the terminal is passed to a voice synthesis engine and converted into voice data for voice guidance.

[0925] Step 12:

[0926] The device uses the voice data to guide the user by voice, saying, "A route to Tokyo Tower has been set." The tone of the voice guidance is adjusted according to the user's emotional state.

[0927] Step 13:

[0928] The device displays a visual guide to Tokyo Tower on the screen, drawing the route on a map and highlighting the next guide point.

[0929] Step 14:

[0930] The user continues driving to the destination using the audio and visual guidance as a reference.

[0931] Example 2

[0932] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0933] Conventional car navigation systems set destinations and provide route guidance based on user voice instructions, but they are unable to consider the user's emotional state, resulting in issues with ease of operation and user experience.In addition, when the user's emotional state changes, the system may not provide an appropriate tone or guidance method, which can be stressful.

[0934] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0935] In this invention, the server includes a means for converting voice input into text data, a means for analyzing the user's intention and emotional state, and a means for determining a guidance method based on the analysis results, thereby making it possible to provide flexible and appropriate guidance according to the user's intention and emotional state.

[0936] "Means for obtaining voice input" refers to devices and programs for collecting the user's voice in digital form.

[0937] The "means for converting voice input into text data" refers to a device and program that has the function of analyzing collected voice and converting it into text information.

[0938] "Means for analyzing text data to identify user intent" refers to devices and programs that have the function of analyzing converted text information using natural language processing technology and understanding the user's requests and instructions.

[0939] The "means for analyzing the user's emotional state" refers to a device or program that has the function of detecting the user's emotions based on the characteristics of voice or text and identifying that state.

[0940] The "means for determining an appropriate guidance method" refers to a device and a program that have the function of selecting the most appropriate guidance method based on the analyzed user's intention and emotional state.

[0941] The "means for acquiring destination information and calculating a route" refers to a device and a program that has the function of acquiring information about a specified destination and calculating the optimal route to that point.

[0942] The "means for providing the user with the acquired information by voice and visually" refers to a device and a program that has the function of providing the user with calculated route information by voice and visually.

[0943] The present invention allows a user to operate car navigation functions by voice, while at the same time recognizing the emotional state of the user and providing an appropriate guidance method based on that emotion.

[0944] Acquiring voice input

[0945] When a user says, "Tell me the way to Tokyo Tower," the device picks up the voice using a microphone installed in the car. The acquired voice data is converted into text data via a built-in voice recognition engine (e.g., general-purpose voice recognition software). At this stage, the text data "Tell me the way to Tokyo Tower" is generated.

[0946] Command parsing and emotion recognition

[0947] The text data generated by the device is sent to the server. The server then uses a natural language processing engine (e.g., natural language processing software) to analyze the text data and determine that the user's intent is to "set a destination." At the same time, it invokes an emotion engine (e.g., emotion analysis software) to determine the user's emotional state from the user's voice characteristics. For example, if the user is excited, the emotion engine will recognize this as "excited."

[0948] Database integration

[0949] The server queries a location database (e.g., location API) or other APIs to obtain the coordinate data of Tokyo Tower. It also uses a traffic condition API to collect real-time traffic information and calculates the optimal route. This determines the optimal route taking real-time traffic information into account.

[0950] Emotion-based guidance adjustment

[0951] The server organizes the route information it obtains and calculates, and adjusts the guidance method based on the user's emotional state. For example, if the emotion engine determines that the user is tired, the guidance method will be adjusted to be more calming.

[0952] Generate and send results

[0953] The server returns the organized route information to the terminal. The terminal uses a speech synthesis engine (for example, speech synthesis software) to convert the received route information into audio data for voice guidance, and provides audible guidance to the user, such as "The route to Tokyo Tower has been set." The terminal also displays the route information on the screen as visual guidance.

[0954] Specific examples

[0955] Example 1: Destination setting and emotion recognition

[0956] The user says "Navigate to Tokyo Station" while in the car. The device uses a microphone to capture the speech and converts it into text data, "Navigate to Tokyo Station," using a speech recognition engine. The device then sends the text data to a server. The server analyzes the speech using natural language processing software and determines that the user's intent is "set destination." At the same time, emotion analysis software analyzes the user's voice and determines that the user is relaxed. The server queries location information APIs and other APIs to obtain coordinate data for Tokyo Station, and calculates the optimal route taking real-time traffic information into account. The calculation results are sent back to the device, which then provides voice guidance in a relaxed tone, saying "Route to Tokyo Station has been set," while displaying the route on the screen.

[0957] Example 2: Nearby store search and emotion recognition

[0958] The user says, "Find a nearby restaurant." The device captures the speech and converts it into text data, "Find a nearby restaurant," using a speech recognition engine. The text data is sent to the server. The server uses natural language processing software to identify the user's intent as "search for nearby restaurants." At the same time, emotion analysis software identifies the user's emotional state and recognizes that they are excited. The server uses location APIs and other APIs to obtain information about restaurants near the current location and sends the results back to the device. The device then provides voice guidance in a softer tone to calm the user's excitement, saying, "There are three restaurants nearby: A, B, and C," and displays the restaurant locations on the screen.

[0959] This system enables flexible and appropriate guidance to be provided according to the user's emotional state, improving operational convenience and user experience.

[0960] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0961] Step 1:

[0962] Acquiring voice input

[0963] The user says, "Tell me the way to Tokyo Tower." The device uses a microphone to capture the user's voice. The captured voice data is input and output.

[0964] Step 2:

[0965] Converting speech to text data

[0966] The acquired voice data is input into a voice recognition engine (for example, general-purpose voice recognition software). This engine converts the voice data into text data and outputs it. Specifically, the text data generated is "Tell me the way to Tokyo Tower."

[0967] Step 3:

[0968] Parsing commands

[0969] The device sends text data to the server. The text data is input into the server's natural language processing engine (e.g., natural language processing software). The server analyzes the text data, identifies the user's intention, and outputs the intention data "set destination."

[0970] Step 4:

[0971] Emotion recognition

[0972] The server invokes an emotion engine (e.g., emotion analysis software) to analyze the characteristics of the voice data, which determines the user's emotional state and outputs the emotion data (e.g., "excited").

[0973] Step 5:

[0974] Database integration

[0975] The server communicates with a location information database (e.g., location information API) to obtain coordinate data for a specific destination (e.g., Tokyo Tower). As a result of querying the API, the destination coordinate data is output. Furthermore, the server queries the traffic conditions API for real-time traffic information, calculates the optimal route, and outputs this route data.

[0976] Step 6:

[0977] Emotion-based guidance adjustment

[0978] The server determines an appropriate guidance method based on the user's emotional data and route data. For example, if the server determines that the user is tired, it adjusts the guidance method to provide guidance in a calmer tone. This adjusted guidance method data is output.

[0979] Step 7:

[0980] Generate and send results

[0981] The server returns the organized route information and the adjusted guidance method to the terminal. The terminal receives them and uses a speech synthesis engine (e.g., speech synthesis software) to generate voice data for voice guidance. This voice data and visual guidance data are output.

[0982] Step 8:

[0983] User Information

[0984] The device uses voice data to announce, "The route to Tokyo Tower has been set." It also displays visual guidance on the screen, providing the user with a visual of the route.

[0985] (Application example 2)

[0986] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0987] While voice navigation is intuitive and convenient, it can be stressful depending on the user's emotional state, resulting in a poor user experience. For example, when the user is tired or excited, it can be difficult to receive appropriate guidance. Furthermore, many navigation systems provide uniform guidance without taking the user's emotional state into account, resulting in a lack of optimal guidance tailored to the situation. There is a need to solve these problems and provide optimal guidance tailored to the user's emotional state.

[0988] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0989] In this invention, the server includes means for acquiring voice input from the user, means for converting the voice input into text data, means for identifying the user's intention, means for recognizing the user's emotional state, means for acquiring destination information and calculating a route based on the user's intention and emotional state, means for providing the acquired information to the user by voice and visual means, and means for providing guidance in a tone corresponding to the emotional state, thereby enabling flexible and optimal navigation guidance according to the user's emotional state.

[0990] "Means for obtaining voice input" refers to a device or function that recognizes a user's voice commands and captures them as digital data.

[0991] "Means for converting voice input into text data" refers to software or hardware for analyzing acquired voice data and converting it into text data.

[0992] "Means for identifying user intent" refers to a device or function that uses natural language processing techniques to analyze the converted text data and understand what the user is requesting.

[0993] "Means for recognizing a user's emotional state" refers to technology for analyzing a user's voice characteristics and other input data to identify the user's emotions (e.g., tired, excited, etc.).

[0994] "Means for obtaining destination information and calculating a route" refers to a database and algorithm for obtaining destination information based on the user's intentions and calculating the optimal route to that destination in real time.

[0995] "Means for providing the user with audio and visual guidance of the acquired information" refers to a device or function that provides the user with calculated route information and other navigation information using voice synthesis technology and a display.

[0996] "Means for providing guidance in a tone appropriate to the emotional state" refers to techniques and algorithms for providing guidance in a tone or format appropriate to the emotional state of the user.

[0997] The present invention provides a system that allows a user to operate a car navigation function by voice and provides appropriate guidance based on the user's emotional state. Hereinafter, an embodiment of the present invention will be described in detail.

[0998] The overall system includes means for capturing voice input, analyzing the user's intent, recognizing the user's emotional state, and providing guidance based on that. First, the user issues a voice command in the car. The device captures the voice through a microphone and converts the voice data into text data using a speech recognition engine, which could be the Google Speech-to-Text API.

[0999] The acquired text data is then sent to a server, where natural language processing technology is used to analyze the user's intent. This part is implemented using the "Custom NLP API." The server then uses the "Custom Emotion Recognition API" to identify the user's emotional state from their voice characteristics.

[1000] Based on the analysis results, the server uses a location database and traffic data to obtain destination information and calculates the optimal route. Specifically, the process is as follows: The server obtains the destination coordinate data using the "Custom Location API," and then calculates the optimal route taking real-time traffic information into account using the "Custom Route API."

[1001] The server organizes the acquired and calculated route information and adjusts the guidance method based on the user's emotional state. For example, if the emotion engine recognizes that the user is tired, guidance will be provided in a calm tone. Finally, the resulting route information is sent back to the device, where it is converted into voice data by a speech synthesis engine and presented to the user. Visual guidance is also displayed on the device screen.

[1002] Specific examples

[1003] Destination settings:

[1004] The user says, "Navigate to Tokyo Station." The device receives the speech and converts it into text data, "Navigate to Tokyo Station," via a voice recognition engine. The server analyzes the text data and determines that the user's intention is to "set a destination," while the emotion engine simultaneously determines that the user is in a relaxed state. The server queries the location database and API to obtain the coordinate data of Tokyo Station, and calculates the optimal route taking into account real-time traffic information. The calculation results are sent back to the device, where a voice guide in a relaxed tone announces, "Route to Tokyo Station has been set," and the route is displayed on the screen.

[1005] Example prompt sentence:

[1006] "Taking voice data as input when a user says, 'Tell me the way to Tokyo Tower.' Analyze the user's intent, identify 'Tokyo Tower' as the destination preference, and check the user's emotional state. If the emotion engine determines that this user is tired, what are the steps to provide directions in a calmer tone?"

[1007] This system can provide flexible and optimal navigation guidance according to the user's emotional state, thereby improving the user's driving experience.

[1008] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1009] Step 1:

[1010] The user specifies the destination by voice.

[1011] Input: User's voice data

[1012] Specific operation: The user gives instructions in the car, such as "Navigate to Tokyo Tower."

[1013] Step 2:

[1014] The device receives the voice data and converts it into text data using a voice recognition engine.

[1015] Input: User's voice data

[1016] Data calculation: Converting voice data into text data using the Google Speech-to-Text API

[1017] Output: Text data "Navigate to Tokyo Tower"

[1018] Step 3:

[1019] The device sends text data to a server, which then uses natural language processing technology to analyze the user's intent.

[1020] Input: Text data

[1021] Data calculation: Using Custom NLP API, analyze the user's intent from the text data and identify "destination setting."

[1022] Output: User intent is "Set destination"

[1023] Step 4:

[1024] The server analyzes the user's emotional state from their voice characteristics.

[1025] Input: User's voice data

[1026] Data Computing: Identifying a user's emotional state from audio data using the Custom Emotion Recognition API

[1027] Output: User's emotional state (e.g., relaxed)

[1028] Step 5:

[1029] The server obtains the destination information and calculates the route.

[1030] Input: User intent (destination setting), user emotional state

[1031] Data calculation: Obtain destination coordinate data using the Custom Location API and calculate the optimal route using the Custom Route API

[1032] Output: Destination coordinates, optimal route

[1033] Step 6:

[1034] The server organizes the information it obtains and adjusts the guidance method according to the emotional state.

[1035] Input: Optimal route, user's emotional state

[1036] Data calculations: Adjusting the tone of guidance based on the user's emotional state (e.g., a relaxed tone)

[1037] Output: Adjusted guidance information

[1038] Step 7:

[1039] The server transmits the adjusted guidance information to the terminal.

[1040] Input: Adjusted guidance information

[1041] Data Computing: Packaging and transmitting information

[1042] Output: Guidance information sent to the terminal

[1043] Step 8:

[1044] The terminal uses a speech synthesis engine and a display to provide audio and visual guidance to the user.

[1045] Input: Guidance information sent to the terminal

[1046] Specific operation: The device generates a guidance message using a speech synthesis engine and announces in a relaxed tone, "A route to Tokyo Tower has been set." Visual information is also displayed on the device's display.

[1047] Output: Guidance information provided to the user

[1048] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1049] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1050] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1051] [Fourth embodiment]

[1052] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1053] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1054] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1055] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1056] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1057] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1058] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1059] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1060] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1061] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1062] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1063] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1064] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1065] This system allows a user to operate the functions of a car navigation system by voice. An embodiment of this system will be described below.

[1066] The system includes a means for capturing voice input and converting it into text using a speech recognition engine. The text data is then sent to a server, which uses natural language processing techniques to identify the user's intent, retrieves destination information, calculates routes, and finally provides the retrieved route information to the user via audio and visual guidance.

[1067] Program processing

[1068] 1. Acquiring voice input

[1069] The user issues a voice command such as "Tell me the way to Tokyo Tower." The device uses a microphone to capture the voice and converts it into text data using a built-in speech recognition engine. At this stage, the text data "Tell me the way to Tokyo Tower" is generated.

[1070] 2. Parsing the command

[1071] The device sends the text data to the server, which uses a natural language processing engine to analyze the text data and determine that the user's intent is to "set a destination."

[1072] 3. Database integration

[1073] The server queries the location database and API to obtain the coordinate data of Tokyo Tower, and calculates the optimal route using traffic condition data. This determines the optimal route taking real-time traffic information into account.

[1074] 4. Generate and send results

[1075] The server returns the route information it has acquired and calculated to the terminal. The terminal then uses a speech synthesis engine to convert the received route information into voice guidance data, and provides audible guidance to the user, saying, "The route to Tokyo Tower has been set." The route information is then displayed as visual guidance on the terminal screen.

[1076] Specific examples

[1077] Example 1: Setting a destination

[1078] The user says "Navigate to Tokyo Station" while in the car. The device receives the speech and uses a speech recognition engine to convert it into text data: "Navigate to Tokyo Station." The device then sends the text data to the server. The server analyzes the text data and determines that the user's intention is to "set a destination." The server queries a location database and API to obtain the coordinate data of Tokyo Station, and then calculates the optimal route taking into account real-time traffic information. The calculation results are sent back to the device, which then provides voice guidance to the user saying, "Route to Tokyo Station has been set," and displays the route on the screen.

[1079] Example 2: Search for nearby stores

[1080] The user says, "Find a nearby restaurant." The device receives the speech and converts it into text data saying, "Find a nearby restaurant" using a speech recognition engine. The text data is sent to the server, which uses a natural language processing engine to identify the user's intent as "Search for nearby stores." The server uses a location database and API to obtain information about restaurants near the current location and sends the results back to the device. The device then provides voice guidance saying, "There are three nearby restaurants: A, B, and C," and displays the locations of the restaurants on the screen.

[1081] summary

[1082] This system allows users to intuitively operate various car navigation functions using only their voice, improving safety while driving and increasing convenience of operation.

[1083] The processing flow will be explained below.

[1084] Step 1:

[1085] A user says, "Tell me the route to Tokyo Tower."

[1086] Step 2:

[1087] The device uses a built-in microphone to obtain the user's voice input.

[1088] Step 3:

[1089] The device calls a speech recognition engine and converts the acquired voice input into text data. For example, text data such as "Tell me the way to Tokyo Tower" is generated.

[1090] Step 4:

[1091] The device sends the generated text data to the server using the HTTP / HTTPS communication protocol.

[1092] Step 5:

[1093] The server receives the text data and invokes a natural language processing engine to analyze the text data.

[1094] Step 6:

[1095] The server identifies the user's intention from the analysis results. In this case, the user's intention is determined to be "setting a destination."

[1096] Step 7:

[1097] The server queries a location database or API to obtain the coordinate data of Tokyo Tower.

[1098] Step 8:

[1099] The server obtains traffic data and uses this data to calculate the optimal route from the current location to Tokyo Tower.

[1100] Step 9:

[1101] The server organizes the calculated route information and sends it back to the device, including route guidance steps and map data.

[1102] Step 10:

[1103] The route information received by the terminal is passed to a voice synthesis engine and converted into voice data for voice guidance.

[1104] Step 11:

[1105] The terminal uses the voice data to provide a voice message to the user saying, "A route to Tokyo Tower has been set."

[1106] Step 12:

[1107] The device displays a visual guide to Tokyo Tower on the screen, plotting the route on a map and indicating necessary guidance points.

[1108] Example 1

[1109] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1110] Conventional car navigation systems require users to manually input their destination or navigate complex menus, posing safety and operational challenges while driving. In particular, when voice control is inefficient, it can be difficult to correctly understand the user's intentions, leading to the inability to provide appropriate route guidance. Therefore, a system is needed that can more accurately and quickly analyze voice input and provide the optimal route based on the user's intentions.

[1111] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1112] In this invention, the server includes means for acquiring voice input from a user, means for converting the voice input into text data, means for analyzing the text data to identify the user's intention, means for acquiring destination information and calculating a route based on the user's intention, means for providing the acquired information to the user aurally and visually, means for analyzing the voice input converted into a digital signal using a voice recognition engine, means for analyzing the text data using natural language processing technology to identify the user's intention, and means for calculating an optimal route using a location information database and traffic condition data. This allows the user to intuitively operate the car navigation function using only voice, improving safety while driving and enabling faster and more accurate route guidance.

[1113] A "means for obtaining audio input" is a device or method that captures a user's voice and converts it into a digital signal.

[1114] The "means for converting voice input into text data" refers to the technology and engine that analyzes acquired voice data and converts it into text format data.

[1115] "Means for identifying user intent by analyzing text data" refers to a method for analyzing text data using natural language processing technology to clearly understand the content and purpose of the user's instructions.

[1116] The "means for acquiring destination information and calculating a route" refers to a database and analytical technology for acquiring the coordinates of a target point based on the user's intentions and calculating the optimal route.

[1117] The "means for providing audio and visual guidance to the user" refers to a method for visually and audibly informing the user of the generated route information using voice synthesis and display.

[1118] "Means for analyzing voice input converted into a digital signal using a voice recognition engine" refers to a technology that analyzes voice input with high accuracy and processes the voice data captured as a digital signal.

[1119] "Means for analyzing text data using natural language processing technology to identify user intent" refers to a method for analyzing text data using advanced natural language processing algorithms to accurately understand user intent.

[1120] "Means for calculating optimal routes using location information databases and traffic data" refers to systems and technologies that calculate the most efficient route for a user based on location information and real-time traffic information.

[1121] This invention is a system that allows users to operate car navigation functions by voice input. This system can acquire voice input and automatically perform recognition, analysis, destination setting, and route guidance.

[1122] Hardware and software used

[1123] To implement this system, the following hardware and software are used:

[1124] Hardware:

[1125] Microphone: Captures voice input from the user.

[1126] Display: Provides visual guidance to the user.

[1127] Processor: Performs speech recognition and data analysis.

[1128] Speaker: Provides audio guidance to the user.

[1129] software:

[1130] Speech recognition engine: For example, using a common speech recognition API (such as Google Speech-to-Text API), converts voice input into text data.

[1131] Natural language processing engine: For example, using a common natural language processing API (such as Google Natural Language API) to analyze text data and identify user intent.

[1132] Location database: For example, use a common geographic information API (such as Google Maps API) to obtain coordinate data of the destination.

[1133] Traffic information API: For example, use a common traffic information API (such as Waze API) to obtain real-time traffic conditions.

[1134] Speech synthesis engine: For example, use a common speech synthesis API (such as Google Text-to-Speech API) to convert text data into speech data.

[1135] Example

[1136] Example 1: Setting a destination

[1137] The user says "Navigate to Tokyo Station" while in the car. The device captures the speech and uses a speech recognition engine to convert it into text data saying "Navigate to Tokyo Station." The device then sends this text data to the server. The server uses a natural language processing engine to analyze the text data and identify the user's intent as "set destination." The server queries the location information database and traffic information API to obtain coordinate data for Tokyo Station and real-time traffic information. Based on this, the server calculates the optimal route and sends this information back to the device. The device then uses a speech synthesis engine to announce "Route to Tokyo Station has been set" and displays the route information on the display.

[1138] Example 2: Search for nearby stores

[1139] The user says, "Find a nearby restaurant." The device receives this speech and converts it into text data saying, "Find a nearby restaurant" using a speech recognition engine. The device then sends the text data to the server. The server uses a natural language processing engine to determine that the user's intent is to "search for nearby stores." The server then queries the location information database and traffic information API to obtain information about restaurants near the current location. The server sends the results back to the device, and the device announces, "Nearby restaurants are A, B, and C," and displays the restaurant's location information on the display.

[1140] Prompt Sentence Examples

[1141] "Navigate to Tokyo Station"

[1142] "Find a nearby restaurant"

[1143] This invention allows a user to intuitively operate car navigation functions using only voice, improving safety while driving and enabling faster and more accurate route guidance.

[1144] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1145] Step 1:

[1146] The user inputs voice. For example, "Tell me the way to Tokyo Tower." The device uses a microphone to capture this voice. The input is the user's voice signal, which is converted into a digital format. The output is digital voice data.

[1147] Step 2:

[1148] The device uses a speech recognition engine to convert digital voice data into text data. The input is digital voice data, and the engine performs phonetic analysis and parses the voice content into text format. The output is text data such as "Tell me the way to Tokyo Tower."

[1149] Step 3:

[1150] The terminal sends the generated text data to the server. The input is the text data, and a secure communication protocol (e.g., HTTPS) is used to securely transport this data to the server. The output is the server's confirmation of receipt.

[1151] Step 4:

[1152] The server receives the text data and analyzes it using a natural language processing engine. The input is text data, and the engine performs grammatical analysis and keyword extraction to identify the user's intent. The output is the intent information, "Set destination."

[1153] Step 5:

[1154] Based on the intent information, the server accesses the location information database to obtain the coordinate data of the destination. It also queries the traffic data API for real-time traffic conditions. The input is the destination name "Tokyo Tower" and the intent information "set destination." The database returns the geographic coordinates, and the traffic data API returns the latest traffic conditions. The output is the destination coordinate data and traffic condition data.

[1155] Step 6:

[1156] The server calculates the optimal route based on coordinate data and traffic condition data. The input is the destination coordinate data and traffic condition data, and the server uses a route calculation algorithm to derive the shortest and most efficient route. The output is the optimal route information.

[1157] Step 7:

[1158] The server sends the optimal route information to the terminal. The input is the optimal route information, which is sent to the terminal via a secure communication protocol. The output is the terminal's confirmation of receipt.

[1159] Step 8:

[1160] The terminal receives the optimal route information and generates voice guidance using a speech synthesis engine. The input is the optimal route information, and the speech synthesis engine converts the text-format route information into voice data. The output is voice data saying, "A route to Tokyo Tower has been set."

[1161] Step 9:

[1162] The terminal provides the generated voice data to the user through a speaker and simultaneously displays visual route guidance on the display. The input is voice data and visual guidance data, and the output is voice guidance and display.

[1163] (Application example 1)

[1164] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1165] Conventional car navigation systems often require manual operation by the user, resulting in issues of safety and convenience while driving. Furthermore, in autonomous vehicles, there is no easy way for users to give complex voice instructions, limiting operability. To solve this issue, there is a demand for a system that allows users to intuitively and efficiently navigate and operate autonomous vehicles and other vehicle operations using only voice input.

[1166] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1167] In this invention, the server includes means for acquiring voice input from a user, means for converting the voice input into text data, means for analyzing the text data to identify the user's intention, means for acquiring destination information and calculating a route based on the user's intention, means for providing the acquired route information to the user via voice and visual means, and means for operating the autonomous vehicle using voice, thereby minimizing manual operations while driving and enabling safe and efficient operation of the autonomous vehicle.

[1168] "Voice input" refers to a user providing words or instructions through a voice input device such as a microphone.

[1169] "Text data" refers to data that is expressed as text information after analyzing voice input using a voice recognition engine or the like.

[1170] "Means for identifying user intent" refers to a method for analyzing acquired text data using natural language processing technology, etc., to understand the user's requests and instructions.

[1171] A "location information database" is a database that stores location information such as geographic coordinates and addresses, and is used to locate specific locations and calculate routes.

[1172] "Traffic condition data" refers to data that indicates current road conditions and traffic information, and is used for route calculation and navigation.

[1173] "Route information" is information that includes specific directions and directional instructions for reaching the user's destination.

[1174] "Audio and visual guidance means" refers to a method of providing route information and instructions to the user both audibly and visually using a speech synthesis engine, a display, etc.

[1175] "Means for operating an autonomous vehicle using voice" refers to a method for controlling and adjusting various functions and settings within an autonomous vehicle in response to voice instructions from the user.

[1176] System Configuration

[1177] This system allows users to navigate and operate an autonomous vehicle through voice input. The specific configuration and processing steps of the system are shown below.

[1178] Hardware and Software

[1179] Smartphone: Equipped with a microphone and a display, it allows voice input and displays results.

[1180] Server: Maintains the voice recognition engine, natural language processing engine, location information database, and traffic situation database, and performs data analysis and calculations.

[1181] Specifically, the following technologies are used:

[1182] Speech recognition engine (e.g. Google Speech-to-Text API)

[1183] Natural language processing engine (e.g. Google Cloud Natural Language API)

[1184] Speech synthesis engine (e.g. Google Text-to-Speech API)

[1185] Data acquisition and processing

[1186] Acquiring voice input

[1187] The user inputs voice instructions through the smartphone's microphone, for example, "Turn right at the next traffic light."

[1188] Speech recognition and text conversion

[1189] The smartphone uses a speech recognition engine to convert the spoken input into text data in the form of "Turn right at the next traffic light."

[1190] Analysis using natural language processing

[1191] Sending to the server

[1192] The text data is sent to a server and analyzed by a natural language processing engine, which determines that the user's intent is to "turn right."

[1193] Route calculation and information acquisition

[1194] The server accesses the location information database and the traffic condition database to calculate the optimal right turn position based on the user's current location.

[1195] Audio and visual guidance

[1196] Server response and routing information transmission

[1197] The server sends the calculation results and route information to the smartphone, which then uses a speech synthesis engine to provide voice guidance such as "Turn right at the next traffic light." The smartphone also visually displays the right-turn location on the display.

[1198] Examples and prompts

[1199] Example 1: Reaching a highway entrance / exit

[1200] When a user says, "Take me to the next exit," the system analyzes their intent and guides them to the optimal route to the exit.

[1201] Example 2: Car charging station search

[1202] When a user says, "Tell me where the nearest charging station is," the system takes into account the current battery status and location information and provides directions to the best charging station.

[1203] Prompt Sentence Examples

[1204] The following prompts can be fed into the generative AI model to perform proper speech recognition and route calculation:

[1205] "Voice input: Turn right at the next traffic light."

[1206] "prompt:

[1207] 1. Convert speech to text.

[1208] 2. Identify the right turn indication from the text.

[1209] 3. Calculate the best right turn location from your current location.

[1210] 4. Please output visual and audio right turn instructions.

[1211] In this way, a system is realized that allows users to easily and safely navigate and operate an autonomous vehicle using only voice input.

[1212] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1213] Step 1:

[1214] The user performs voice input. The user speaks into the smartphone microphone, saying, "Turn right at the next traffic light." Voice data is generated as input.

[1215] Step 2:

[1216] The smartphone converts the voice into text data. It then uses a speech recognition engine (e.g., Google Speech-to-Text API) to analyze the voice data and convert it into text data such as "Turn right at the next traffic light." This generates text data as output.

[1217] Step 3:

[1218] The smartphone sends text data to the server. The sent text data arrives at the server, where it is analyzed to determine the user's intent using a natural language processing engine (e.g., Google Cloud Natural Language API). The input is the text data, and the output is the intent to "turn right."

[1219] Step 4:

[1220] The server obtains the current location information and calculates the optimal route. The server accesses the location information database and traffic condition database and calculates the route taking into account the user's current location and destination (right turn position at the next traffic light). The input is the user's current location and right turn instructions, and the output is right turn route information.

[1221] Step 5:

[1222] The server sends route information to the smartphone. The route information is sent from the server and reaches the smartphone. The route information is sent as input, and the information necessary for route guidance reaches the smartphone.

[1223] Step 6:

[1224] The smartphone provides route information through voice and visual guidance. The smartphone uses a speech synthesis engine (e.g., Google Text-to-Speech API) to provide voice guidance such as "Turn right at the next traffic light." The route information is also displayed visually on the display. Route information is input, and voice guidance and visual displays are generated as output.

[1225] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1226] The present invention allows a user to operate car navigation functions by voice, while at the same time recognizing the user's emotional state and providing an appropriate guidance method based on that emotion. The following describes an embodiment of the present invention.

[1227] The system includes a means for receiving voice input and converting it into text data using a voice recognition engine. The text data is then sent to a server, which uses natural language processing technology to identify the user's intent. The server also uses an emotion engine to analyze the user's emotional state and provides appropriate guidance based on this. Specifically, the system first converts the voice into text using a voice recognition engine, analyzes the user's intent using natural language processing technology, and then identifies the user's emotion using the emotion engine. Based on this information, the system obtains destination information, calculates a route, and provides audio and visual guidance to the user.

[1228] Program processing

[1229] 1. Acquiring voice input

[1230] The user issues a voice command such as "Tell me the way to Tokyo Tower." The device uses a microphone to capture the voice and converts it into text data using a built-in speech recognition engine. At this stage, the text data "Tell me the way to Tokyo Tower" is generated.

[1231] 2. Command analysis and emotion recognition

[1232] The device sends text data to the server. The server uses a natural language processing engine to analyze the text data and determine that the user's intent is to "set a destination." At the same time, it calls an emotion engine to determine the user's emotional state from their voice characteristics. For example, if the user is excited, the emotion engine will recognize this as "excited."

[1233] 3. Database integration

[1234] The server queries the location database and API to obtain the coordinate data of Tokyo Tower, and calculates the optimal route using traffic condition data. This determines the optimal route taking real-time traffic information into account.

[1235] 4. Emotion-based guidance adjustment

[1236] The server organizes the route information it obtains and calculates, and adjusts the guidance method based on the user's emotional state. For example, if the server detects that the user is tired, it will provide guidance in a calmer tone.

[1237] 5. Generate and send results

[1238] The server returns the organized route information to the terminal. The terminal uses a speech synthesis engine to convert the received route information into voice data for voice guidance, and provides audible guidance to the user, saying, "The route to Tokyo Tower has been set." The route information is then displayed as visual guidance on the terminal screen.

[1239] Specific examples

[1240] Example 1: Destination setting and emotion recognition

[1241] The user says "Navigate to Tokyo Station" while in the car. The device captures the speech and uses a speech recognition engine to convert it into text data: "Navigate to Tokyo Station." The device then sends the text data to the server. The server analyzes the text data and determines that the user's intention is to "set a destination." At the same time, the emotion engine analyzes the user's voice and determines, for example, that the user is in a relaxed state. The server queries a location database and API to obtain coordinate data for Tokyo Station, and then calculates the optimal route taking real-time traffic information into consideration. The calculation results are sent back to the device, which then provides voice guidance in a relaxed tone, saying "Route to Tokyo Station has been set," while displaying the route on the screen.

[1242] Example 2: Nearby store search and emotion recognition

[1243] The user says, "Find a nearby restaurant." The device captures the speech and converts it into text data saying, "Find a nearby restaurant" using a speech recognition engine. The text data is sent to the server, which uses a natural language processing engine to identify the user's intent as "search for nearby stores." At the same time, the emotion engine identifies the user's emotional state, recognizing, for example, that the user is excited. The server uses a location database and API to obtain information about restaurants near the current location and sends the results back to the device. The device then provides voice guidance in a soft tone, saying, "There are three nearby restaurants: A, B, and C," and displays the locations of the restaurants on the screen.

[1244] In this way, by combining the system with an emotion engine, it becomes possible to provide flexible guidance that corresponds to the user's emotional state, improving the convenience of operation and the user experience.

[1245] The processing flow will be explained below.

[1246] Step 1:

[1247] A user says, "Tell me the route to Tokyo Tower."

[1248] Step 2:

[1249] The device uses a built-in microphone to obtain the user's voice input.

[1250] Step 3:

[1251] The device calls a speech recognition engine and converts the acquired voice input into text data. For example, text data such as "Tell me the way to Tokyo Tower" is generated.

[1252] Step 4:

[1253] The device sends the generated text data to the server using the HTTP / HTTPS communication protocol.

[1254] Step 5:

[1255] The server receives the text data and calls an emotion engine to identify the user's emotional state from the user's voice characteristics. For example, emotions such as "excited" or "relaxed" can be recognized from the tone, speed, and rhythm of the user's voice.

[1256] Step 6:

[1257] The server calls a natural language processing engine to analyze the text data, and identifies the user's intent as "setting a destination" from the text data.

[1258] Step 7:

[1259] The server queries a location database or API to obtain the coordinate data of Tokyo Tower.

[1260] Step 8:

[1261] The server collects traffic data and uses it to calculate the optimal route from the user's current location to Tokyo Tower. Traffic information includes information on traffic congestion and accidents.

[1262] Step 9:

[1263] The server then organizes the calculated route information and adjusts the guidance based on the user's emotional state. For example, if the user is "excited," the server will set the guidance to be in a calm tone.

[1264] Step 10:

[1265] The server returns organized route information to the device, including each step of the route guidance and map data.

[1266] Step 11:

[1267] The route information received by the terminal is passed to a voice synthesis engine and converted into voice data for voice guidance.

[1268] Step 12:

[1269] The device uses the voice data to guide the user by voice, saying, "A route to Tokyo Tower has been set." The tone of the voice guidance is adjusted according to the user's emotional state.

[1270] Step 13:

[1271] The device displays a visual guide to Tokyo Tower on the screen, drawing the route on a map and highlighting the next guide point.

[1272] Step 14:

[1273] The user continues driving to the destination using the audio and visual guidance as a reference.

[1274] Example 2

[1275] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1276] Conventional car navigation systems set destinations and provide route guidance based on user voice instructions, but they are unable to consider the user's emotional state, resulting in issues with ease of operation and user experience.In addition, when the user's emotional state changes, the system may not provide an appropriate tone or guidance method, which can be stressful.

[1277] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1278] In this invention, the server includes a means for converting voice input into text data, a means for analyzing the user's intention and emotional state, and a means for determining a guidance method based on the analysis results, thereby making it possible to provide flexible and appropriate guidance according to the user's intention and emotional state.

[1279] "Means for obtaining voice input" refers to devices and programs for collecting the user's voice in digital form.

[1280] The "means for converting voice input into text data" refers to a device and program that has the function of analyzing collected voice and converting it into text information.

[1281] "Means for analyzing text data to identify user intent" refers to devices and programs that have the function of analyzing converted text information using natural language processing technology and understanding the user's requests and instructions.

[1282] The "means for analyzing the user's emotional state" refers to a device or program that has the function of detecting the user's emotions based on the characteristics of voice or text and identifying that state.

[1283] The "means for determining an appropriate guidance method" refers to a device and a program that have the function of selecting the most appropriate guidance method based on the analyzed user's intention and emotional state.

[1284] The "means for acquiring destination information and calculating a route" refers to a device and a program that has the function of acquiring information about a specified destination and calculating the optimal route to that point.

[1285] The "means for providing the user with the acquired information by voice and visually" refers to a device and a program that has the function of providing the user with calculated route information by voice and visually.

[1286] The present invention allows a user to operate car navigation functions by voice, while at the same time recognizing the emotional state of the user and providing an appropriate guidance method based on that emotion.

[1287] Acquiring voice input

[1288] When a user says, "Tell me the way to Tokyo Tower," the device picks up the voice using a microphone installed in the car. The acquired voice data is converted into text data via a built-in voice recognition engine (e.g., general-purpose voice recognition software). At this stage, the text data "Tell me the way to Tokyo Tower" is generated.

[1289] Command parsing and emotion recognition

[1290] The text data generated by the device is sent to the server. The server then uses a natural language processing engine (e.g., natural language processing software) to analyze the text data and determine that the user's intent is to "set a destination." At the same time, it invokes an emotion engine (e.g., emotion analysis software) to determine the user's emotional state from the user's voice characteristics. For example, if the user is excited, the emotion engine will recognize this as "excited."

[1291] Database integration

[1292] The server queries a location database (e.g., location API) or other APIs to obtain the coordinate data of Tokyo Tower. It also uses a traffic condition API to collect real-time traffic information and calculates the optimal route. This determines the optimal route taking real-time traffic information into account.

[1293] Emotion-based guidance adjustment

[1294] The server organizes the route information it obtains and calculates, and adjusts the guidance method based on the user's emotional state. For example, if the emotion engine determines that the user is tired, the guidance method will be adjusted to be more calming.

[1295] Generate and send results

[1296] The server returns the organized route information to the terminal. The terminal uses a speech synthesis engine (for example, speech synthesis software) to convert the received route information into audio data for voice guidance, and provides audible guidance to the user, such as "The route to Tokyo Tower has been set." The terminal also displays the route information on the screen as visual guidance.

[1297] Specific examples

[1298] Example 1: Destination setting and emotion recognition

[1299] The user says "Navigate to Tokyo Station" while in the car. The device uses a microphone to capture the speech and converts it into text data, "Navigate to Tokyo Station," using a speech recognition engine. The device then sends the text data to a server. The server analyzes the speech using natural language processing software and determines that the user's intent is "set destination." At the same time, emotion analysis software analyzes the user's voice and determines that the user is relaxed. The server queries location information APIs and other APIs to obtain coordinate data for Tokyo Station, and calculates the optimal route taking real-time traffic information into account. The calculation results are sent back to the device, which then provides voice guidance in a relaxed tone, saying "Route to Tokyo Station has been set," while displaying the route on the screen.

[1300] Example 2: Nearby store search and emotion recognition

[1301] The user says, "Find a nearby restaurant." The device captures the speech and converts it into text data, "Find a nearby restaurant," using a speech recognition engine. The text data is sent to the server. The server uses natural language processing software to identify the user's intent as "search for nearby restaurants." At the same time, emotion analysis software identifies the user's emotional state and recognizes that they are excited. The server uses location APIs and other APIs to obtain information about restaurants near the current location and sends the results back to the device. The device then provides voice guidance in a softer tone to calm the user's excitement, saying, "There are three restaurants nearby: A, B, and C," and displays the restaurant locations on the screen.

[1302] This system enables flexible and appropriate guidance to be provided according to the user's emotional state, improving operational convenience and user experience.

[1303] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1304] Step 1:

[1305] Acquiring voice input

[1306] The user says, "Tell me the way to Tokyo Tower." The device uses a microphone to capture the user's voice. The captured voice data is input and output.

[1307] Step 2:

[1308] Converting speech to text data

[1309] The acquired voice data is input into a voice recognition engine (for example, general-purpose voice recognition software). This engine converts the voice data into text data and outputs it. Specifically, the text data generated is "Tell me the way to Tokyo Tower."

[1310] Step 3:

[1311] Parsing commands

[1312] The device sends text data to the server. The text data is input into the server's natural language processing engine (e.g., natural language processing software). The server analyzes the text data, identifies the user's intention, and outputs the intention data "set destination."

[1313] Step 4:

[1314] Emotion recognition

[1315] The server invokes an emotion engine (e.g., emotion analysis software) to analyze the characteristics of the voice data, which determines the user's emotional state and outputs the emotion data (e.g., "excited").

[1316] Step 5:

[1317] Database integration

[1318] The server communicates with a location information database (e.g., location information API) to obtain coordinate data for a specific destination (e.g., Tokyo Tower). As a result of querying the API, the destination coordinate data is output. Furthermore, the server queries the traffic conditions API for real-time traffic information, calculates the optimal route, and outputs this route data.

[1319] Step 6:

[1320] Emotion-based guidance adjustment

[1321] The server determines an appropriate guidance method based on the user's emotional data and route data. For example, if the server determines that the user is tired, it adjusts the guidance method to provide guidance in a calmer tone. This adjusted guidance method data is output.

[1322] Step 7:

[1323] Generate and send results

[1324] The server returns the organized route information and the adjusted guidance method to the terminal. The terminal receives them and uses a speech synthesis engine (e.g., speech synthesis software) to generate voice data for voice guidance. This voice data and visual guidance data are output.

[1325] Step 8:

[1326] User Information

[1327] The device uses voice data to announce, "The route to Tokyo Tower has been set." It also displays visual guidance on the screen, providing the user with a visual of the route.

[1328] (Application example 2)

[1329] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1330] While voice navigation is intuitive and convenient, it can be stressful depending on the user's emotional state, resulting in a poor user experience. For example, when the user is tired or excited, it can be difficult to receive appropriate guidance. Furthermore, many navigation systems provide uniform guidance without taking the user's emotional state into account, resulting in a lack of optimal guidance tailored to the situation. There is a need to solve these problems and provide optimal guidance tailored to the user's emotional state.

[1331] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1332] In this invention, the server includes means for acquiring voice input from the user, means for converting the voice input into text data, means for identifying the user's intention, means for recognizing the user's emotional state, means for acquiring destination information and calculating a route based on the user's intention and emotional state, means for providing the acquired information to the user by voice and visual means, and means for providing guidance in a tone corresponding to the emotional state, thereby enabling flexible and optimal navigation guidance according to the user's emotional state.

[1333] "Means for obtaining voice input" refers to a device or function that recognizes a user's voice commands and captures them as digital data.

[1334] "Means for converting voice input into text data" refers to software or hardware for analyzing acquired voice data and converting it into text data.

[1335] "Means for identifying user intent" refers to a device or function that uses natural language processing techniques to analyze the converted text data and understand what the user is requesting.

[1336] "Means for recognizing a user's emotional state" refers to technology for analyzing a user's voice characteristics and other input data to identify the user's emotions (e.g., tired, excited, etc.).

[1337] "Means for obtaining destination information and calculating a route" refers to a database and algorithm for obtaining destination information based on the user's intentions and calculating the optimal route to that destination in real time.

[1338] "Means for providing the user with audio and visual guidance of the acquired information" refers to a device or function that provides the user with calculated route information and other navigation information using voice synthesis technology and a display.

[1339] "Means for providing guidance in a tone appropriate to the emotional state" refers to techniques and algorithms for providing guidance in a tone or format appropriate to the emotional state of the user.

[1340] The present invention provides a system that allows a user to operate a car navigation function by voice and provides appropriate guidance based on the user's emotional state. Hereinafter, an embodiment of the present invention will be described in detail.

[1341] The overall system includes means for capturing voice input, analyzing the user's intent, recognizing the user's emotional state, and providing guidance based on that. First, the user issues a voice command in the car. The device captures the voice through a microphone and converts the voice data into text data using a speech recognition engine, which could be the Google Speech-to-Text API.

[1342] The acquired text data is then sent to a server, where natural language processing technology is used to analyze the user's intent. This part is implemented using the "Custom NLP API." The server then uses the "Custom Emotion Recognition API" to identify the user's emotional state from their voice characteristics.

[1343] Based on the analysis results, the server uses a location database and traffic data to obtain destination information and calculates the optimal route. Specifically, the process is as follows: The server obtains the destination coordinate data using the "Custom Location API," and then calculates the optimal route taking real-time traffic information into account using the "Custom Route API."

[1344] The server organizes the acquired and calculated route information and adjusts the guidance method based on the user's emotional state. For example, if the emotion engine recognizes that the user is tired, guidance will be provided in a calm tone. Finally, the resulting route information is sent back to the device, where it is converted into voice data by a speech synthesis engine and presented to the user. Visual guidance is also displayed on the device screen.

[1345] Specific examples

[1346] Destination settings:

[1347] The user says, "Navigate to Tokyo Station." The device receives the speech and converts it into text data, "Navigate to Tokyo Station," via a voice recognition engine. The server analyzes the text data and determines that the user's intention is to "set a destination," while the emotion engine simultaneously determines that the user is in a relaxed state. The server queries the location database and API to obtain the coordinate data of Tokyo Station, and calculates the optimal route taking into account real-time traffic information. The calculation results are sent back to the device, where a voice guide in a relaxed tone announces, "Route to Tokyo Station has been set," and the route is displayed on the screen.

[1348] Example prompt sentence:

[1349] "Taking voice data as input when a user says, 'Tell me the way to Tokyo Tower.' Analyze the user's intent, identify 'Tokyo Tower' as the destination preference, and check the user's emotional state. If the emotion engine determines that this user is tired, what are the steps to provide directions in a calmer tone?"

[1350] This system can provide flexible and optimal navigation guidance according to the user's emotional state, thereby improving the user's driving experience.

[1351] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1352] Step 1:

[1353] The user specifies the destination by voice.

[1354] Input: User's voice data

[1355] Specific operation: The user gives instructions in the car, such as "Navigate to Tokyo Tower."

[1356] Step 2:

[1357] The device receives the voice data and converts it into text data using a voice recognition engine.

[1358] Input: User's voice data

[1359] Data calculation: Converting voice data into text data using the Google Speech-to-Text API

[1360] Output: Text data "Navigate to Tokyo Tower"

[1361] Step 3:

[1362] The device sends text data to a server, which then uses natural language processing technology to analyze the user's intent.

[1363] Input: Text data

[1364] Data calculation: Using Custom NLP API, analyze the user's intent from the text data and identify "destination setting."

[1365] Output: User intent is "Set destination"

[1366] Step 4:

[1367] The server analyzes the user's emotional state from their voice characteristics.

[1368] Input: User's voice data

[1369] Data Computing: Identifying a user's emotional state from audio data using the Custom Emotion Recognition API

[1370] Output: User's emotional state (e.g., relaxed)

[1371] Step 5:

[1372] The server obtains the destination information and calculates the route.

[1373] Input: User intent (destination setting), user emotional state

[1374] Data calculation: Obtain destination coordinate data using the Custom Location API and calculate the optimal route using the Custom Route API

[1375] Output: Destination coordinates, optimal route

[1376] Step 6:

[1377] The server organizes the information it obtains and adjusts the guidance method according to the emotional state.

[1378] Input: Optimal route, user's emotional state

[1379] Data calculations: Adjusting the tone of guidance based on the user's emotional state (e.g., a relaxed tone)

[1380] Output: Adjusted guidance information

[1381] Step 7:

[1382] The server transmits the adjusted guidance information to the terminal.

[1383] Input: Adjusted guidance information

[1384] Data Computing: Packaging and transmitting information

[1385] Output: Guidance information sent to the terminal

[1386] Step 8:

[1387] The terminal uses a speech synthesis engine and a display to provide audio and visual guidance to the user.

[1388] Input: Guidance information sent to the terminal

[1389] Specific operation: The device generates a guidance message using a speech synthesis engine and announces in a relaxed tone, "A route to Tokyo Tower has been set." Visual information is also displayed on the device's display.

[1390] Output: Guidance information provided to the user

[1391] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1392] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1393] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1394] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1395] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1396] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1397] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1398] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1399] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1400] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1401] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1402] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1403] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1404] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1405] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1406] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1407] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1408] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1409] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1410] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1411] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1412] The following is further disclosed regarding the above embodiment.

[1413] (Claim 1)

[1414] means for obtaining speech input from a user;

[1415] means for converting voice input into text data;

[1416] means for analyzing the text data to identify a user's intention;

[1417] means for acquiring destination information and calculating a route based on the user's intention;

[1418] means for providing the acquired information to the user by voice and visual means;

[1419] A system including:

[1420] (Claim 2)

[1421] 2. The system of claim 1, wherein the means for identifying the user's intent uses natural language processing techniques.

[1422] (Claim 3)

[1423] 2. The system of claim 1, wherein the means for calculating destination information and route uses a location information database and traffic condition data.

[1424] "Example 1"

[1425] (Claim 1)

[1426] means for obtaining speech input from a user;

[1427] means for converting voice input into text data;

[1428] means for analyzing the text data to identify a user's intention;

[1429] means for acquiring destination information and calculating a route based on the user's intention;

[1430] means for providing the acquired information to the user by voice and visual means;

[1431] means for analyzing the voice input converted into a digital signal using a voice recognition engine;

[1432] A means for analyzing text data using natural language processing technology and identifying user intent;

[1433] means for calculating an optimal route using a location database and traffic data;

[1434] A system including:

[1435] (Claim 2)

[1436] 10. The system of claim 1, wherein the system uses natural language processing techniques.

[1437] (Claim 3)

[1438] 10. The system of claim 1, wherein a location database and traffic data are used.

[1439] "Application Example 1"

[1440] (Claim 1)

[1441] means for obtaining speech input from a user;

[1442] means for converting voice input into text data;

[1443] means for analyzing the text data to identify a user's intention;

[1444] means for acquiring destination information and calculating a route based on the user's intention;

[1445] means for providing the acquired route information to the user by voice and visual means;

[1446] a means for controlling an autonomous vehicle using voice;

[1447] A system including:

[1448] (Claim 2)

[1449] 2. The system of claim 1, wherein the means for identifying the user's intent uses natural language processing techniques.

[1450] (Claim 3)

[1451] 2. The system of claim 1, wherein the means for calculating destination information and route uses a location information database and traffic condition data.

[1452] "Example 2: Combining Emotion Engines"

[1453] (Claim 1)

[1454] means for obtaining speech input from a user;

[1455] means for converting voice input into text data;

[1456] means for analyzing the text data to identify a user's intention;

[1457] means for analyzing the emotional state of a user;

[1458] means for determining an appropriate guidance method based on the user's intention and emotional state;

[1459] means for acquiring destination information and calculating a route based on the user's intention and emotional state;

[1460] means for providing the acquired information to the user by voice and visual means;

[1461] A system including:

[1462] (Claim 2)

[1463] 2. The system of claim 1, wherein the means for identifying the user's intent uses natural language processing techniques.

[1464] (Claim 3)

[1465] 2. The system of claim 1, wherein the means for calculating destination information and route uses a location information database and traffic condition data.

[1466] "Application example 2 when combining emotion engines"

[1467] (Claim 1)

[1468] means for obtaining speech input from a user;

[1469] means for converting voice input into text data;

[1470] means for analyzing the text data to identify a user's intention;

[1471] means for recognizing the emotional state of a user;

[1472] means for acquiring destination information and calculating a route based on the user's intention and emotional state;

[1473] means for providing the acquired information to the user by voice and visual means;

[1474] a means for providing guidance in a tone that corresponds to an emotional state;

[1475] A system including:

[1476] (Claim 2)

[1477] 2. The system of claim 1, wherein the means for identifying the user's intent uses natural language processing techniques.

[1478] (Claim 3)

[1479] 2. The system of claim 1, wherein the means for calculating destination information and route uses a location information database and traffic condition data. [Explanation of symbols]

[1480] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for obtaining speech input from a user; means for converting voice input into text data; means for analyzing the text data to identify a user's intention; means for acquiring destination information and calculating a route based on the user's intention; means for providing the acquired information to the user by voice and visual means; A system including:

2. 10. The system of claim 1, wherein the means for identifying user intent uses natural language processing techniques.

3. 2. The system of claim 1, wherein the means for calculating destination information and routes utilizes a location information database and traffic condition data.

Images