System

Abstract

A system is provided.SOLUTION: A system comprising: means for converting a voice input from a user into text data; means for analyzing the text data and extracting a destination and waypoints; means for calculating an optimal route based on the destination and waypoints; means for suggesting the calculated route and waypoint information to the user; means for setting a final route based on a response from the user; and means for monitoring a change in traffic conditions, recalculating a route according to the change in traffic conditions, and suggesting the route to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Existing navigation systems require users to manually set destinations and intermediate stops in advance, which is cumbersome and extremely dangerous to operate while driving. Furthermore, it is difficult to respond quickly and safely to changes in traffic conditions, such as when to take a break or when to adjust the route. The present invention aims to solve these problems and provide a safer and more convenient navigation experience. [Means for solving the problem]

[0005] The present invention is a system including: means for converting voice input from a user into text data; means for analyzing the text data and extracting a destination and intermediate points; means for calculating an optimal route based on the destination and intermediate points; means for suggesting information about the calculated route and intermediate points to the user; means for setting a final route based on responses from the user; and means for monitoring changes in traffic conditions, recalculating the route in response to the changes in traffic conditions, and suggesting the route to the user. Furthermore, the system includes means for adjusting the timing of rest stops and intermediate points in accordance with the acquired changes in traffic conditions and suggesting them to the user, and means for responding to voice input from the user in a natural conversational style and setting and correcting the destination and intermediate points, thereby enabling users to use navigation safely and conveniently.

[0006] "Voice input" refers to speech used by a user to provide instructions and information to a navigation system.

[0007] "Text data" refers to character string information obtained by analyzing voice input.

[0008] "Analysis" refers to the process of extracting meaning and important information from text data.

[0009] "Destination" refers to the location where the user ultimately wants to arrive.

[0010] "Stopover" refers to a place you plan to stop at on the way to your destination.

[0011] "Route" refers to the path or directions leading to the destination and intermediate points.

[0012] An "optimal route" refers to the most efficient route calculated based on predetermined conditions (e.g., time, distance, traffic conditions) for a specified destination and intermediate destinations.

[0013] "Suggestion" refers to the act of informing users of route and stopover information calculated by the system.

[0014] "Response" refers to the act of a user replying to a suggestion from the system.

[0015] "Traffic conditions" refers to information such as road congestion, traffic jams, and accidents.

[0016] "Monitoring" refers to the act of the system observing traffic conditions and other relevant information in real time.

[0017] "Recalculation" refers to the process of recalculating an existing route based on changing traffic conditions.

[0018] "Natural conversational style" refers to interactions between users and systems that are similar to human conversations. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] overview

[0041] The present invention relates to a navigation system that uses voice input. Users can set destinations and intermediate points through voice, and can smoothly adjust routes and rest areas according to changes in traffic conditions. This system supports users in traveling safely and conveniently.

[0042] Program processing

[0043] System configuration

[0044] The system mainly consists of the following elements:

[0045] 1. Voice input devices (smartphones and car navigation devices)

[0046] 2. Server (analysis and navigation data processing)

[0047] 3. Navigation API (e.g., map services)

[0048] 1. Acquiring voice input

[0049] User: First, the user speaks to the voice input device about their destination and stopovers. For example, they might say, "Set the destination to Hotel X as my destination. I'd like to eat ramen on the way around 1:00 PM."

[0050] Terminal: The terminal converts this voice input into text data and sends the text data to the server.

[0051] 2.Analysis of voice data and route calculation

[0052] Server: The server analyzes the received text data and extracts the destination and intermediate points. This analysis uses natural language processing technology. Specifically, it automatically extracts information about location and time from the text.

[0053] The server then uses a navigation API to calculate the optimal route based on the destination and intermediate stops, taking real-time traffic information into account.

[0054] Server: The server generates a message to suggest the calculated route and stopover information to the user and sends it to the terminal.

[0055] 3. Confirm the proposal and set the final route

[0056] Terminal: The terminal plays the received proposal message as audio and asks the user for confirmation.

[0057] User: The user responds to the proposed stopovers and route with a response such as "Yes, that's fine."

[0058] Terminal: Convert this response back into text and send it to the server.

[0059] Server: The server verifies the user's consent and sets the final route, which includes detailed navigation instructions.

[0060] 4. Traffic monitoring and route recalculation

[0061] Server: The server monitors traffic conditions in real time and calculates a new route if there are changes in traffic conditions along the specified route.

[0062] Server: Generates messages to propose new routes and stops and sends them to the device.

[0063] Terminal: The terminal will play the new suggestion aloud and ask the user for confirmation.

[0064] User: The user responds to the new suggestion, for example by giving instructions such as "Change to XX Restaurant."

[0065] Server: The server determines the new route and sends it to the device, which then continues navigation according to the new route.

[0066] Specific examples

[0067] Example 1: Setting a destination

[0068] 1. User: Says to the device, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1pm."

[0069] 2. Device: Converts voice input into text and sends it to the server.

[0070] 3. Server: Analyzes the text data, extracts the destination and intermediate points, and calculates the optimal route using the navigation API.

[0071] 4. Server: Generates a message proposing the calculation result and sends it to the terminal.

[0072] 5. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[0073] 6. User: Agrees with the proposal and responds, "Yes, I'd like that."

[0074] 7. Server: Determines the final route and sends navigation data to the device.

[0075] Example 2: Route adjustment due to changing traffic conditions

[0076] 1. Server: Monitors traffic conditions and obtains new information.

[0077] 2. Server: Calculates new routes and generates proposal messages.

[0078] 3. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[0079] 4. User: Agrees with the new suggestion and responds, "Change to XX Restaurant."

[0080] 5. Server: Determines the new route and sends navigation data to the device.

[0081] 6. Device: Continue navigation according to the new route.

[0082] As a result, the system of the present invention allows the user to set the destination and intermediate points by voice and responds to changes in traffic conditions, providing safe and convenient navigation.

[0083] The processing flow will be explained below.

[0084] Specific explanation of program processing

[0085] 1. Destination and intermediate destination setting process

[0086] Step 1:

[0087] User: Gives instructions to the voice input device, such as "Set Hotel X as my destination. I'd like to eat ramen on the way around 1pm."

[0088] Step 2:

[0089] Device: Calls the speech recognition API to convert voice input into text data. Retrieves the converted text data.

[0090] Step 3:

[0091] Terminal: Send text data to the server using an API endpoint.

[0092] Step 4:

[0093] Server: The received text data is input into ChatGPT, and the destination and intermediate points are extracted using natural language analysis.

[0094] Step 5:

[0095] Server: Uses a navigation API (e.g., Google Maps API) to calculate the optimal route based on the extracted destination and intermediate destinations.

[0096] Step 6:

[0097] Server: Based on the calculation results, a message is generated to be suggested to the user. It prepares a message such as "Suggested route to destination XX Hotel, stopping at XX ramen shop around 1 PM."

[0098] Step 7:

[0099] Server: Sends a proposal message to the terminal.

[0100] Step 8:

[0101] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[0102] Step 9:

[0103] User: Review the proposed route and respond "Yes, that's fine."

[0104] Step 10:

[0105] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[0106] Step 11:

[0107] Server: Confirms user consent and finalizes the route, which includes detailed navigation instructions.

[0108] Step 12:

[0109] Server: Sends the confirmed route data to the terminal.

[0110] Step 13:

[0111] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[0112] 2. Monitoring traffic conditions and handling route corrections

[0113] Step 1:

[0114] Server: Monitors traffic condition data in real time. Uses a dedicated API to periodically obtain information on congestion, accidents, etc.

[0115] Step 2:

[0116] Server: Determines whether there are any changes in traffic conditions that will affect the currently configured route.

[0117] Step 3:

[0118] Server: If a change in traffic conditions is detected, calculate a new route, resetting intermediate stops if necessary.

[0119] Step 4:

[0120] Server: Generate a message containing a new route or stopover suggestion. For example, "There is currently a traffic jam, so we suggest a revised route."

[0121] Step 5:

[0122] Server: Sends a proposal message to the terminal.

[0123] Step 6:

[0124] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[0125] Step 7:

[0126] User: Review the new proposal and respond, "Yes, I'd like the revised route."

[0127] Step 8:

[0128] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[0129] Step 9:

[0130] Server: Confirms user consent and confirms the new route, which also includes detailed navigation instructions.

[0131] Step 10:

[0132] Server: Sends new route data to the device.

[0133] Step 11:

[0134] Device: Continues navigation based on the new route, providing visual and audio guidance.

[0135] Example 1

[0136] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0137] Conventional navigation systems require a lot of manual input to set destinations and intermediate points, and it is also difficult to flexibly recalculate routes to accommodate real-time traffic conditions. This requires cumbersome operations for users, which reduces safety and convenience. Since manual input while driving is particularly dangerous, there is a demand for systems that utilize voice input.

[0138] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0139] In this invention, the server includes means for capturing a user's voice input and converting it into text data using voice recognition software, means for extracting a destination and intermediate points from the text using natural language processing technology, means for calculating a route taking real-time traffic information into consideration using a navigation API, means for playing a suggested message aloud using voice synthesis software and requesting confirmation from the user, and means for sending a final route instruction to the terminal. This allows the user to easily set a destination and intermediate points through voice input and enables flexible route changes according to real-time traffic information.

[0140] "Voice input" is a means of acquiring the words spoken by a user as digital signals.

[0141] The "means for converting into text data" refers to software or hardware for converting acquired voice input into text information.

[0142] "Means for analyzing text data" refers to technologies or systems for deciphering converted text data and extracting specific information.

[0143] "Destination and stopover points" refers to the final destination set by the user and points to be passed along the way.

[0144] The "means for calculating the optimal route" is a system that calculates the optimal route based on the destination and intermediate points, taking into account conditions such as time, distance, and traffic conditions.

[0145] "Means of providing suggestions to users" refers to the methods and techniques used to communicate calculated route information to users, and may involve voice or text.

[0146] The "means for setting the final route" is a system for determining the final route based on the user's response.

[0147] "Means for monitoring changes in traffic conditions" refers to a system for collecting traffic information in real time and monitoring the situation.

[0148] The "means for recalculating the route" is a system for reevaluating the originally calculated route in response to changes in traffic conditions and calculating a new optimal route.

[0149] "Speech recognition software" is software technology for converting voice input into text data.

[0150] "Natural language processing technology" is a technology for deciphering human language and extracting specific information.

[0151] A "navigation API" is an application program interface for calculating routes and obtaining traffic information using map services, etc.

[0152] "Speech synthesis software" is a technology for reproducing text data as speech.

[0153] System configuration

[0154] The navigation system consists of the following elements:

[0155] 1. Voice input devices (e.g. smartphones, car navigation devices)

[0156] 2. Server (processes analysis and navigation data)

[0157] 3. Navigation API (e.g., map service)

[0158] Program processing

[0159] Acquiring voice input

[0160] The user speaks to the voice input device about the destination and stopovers, for example, "Set the destination to XX Hotel. I'd like to eat ramen on the way around 1 PM."

[0161] Audio data conversion

[0162] The device captures the user's voice, converts it into text data using speech recognition software (e.g., Google Speech-to-Text API), and sends the text data to the server.

[0163] Analysis of audio data

[0164] The server analyzes the received text data and extracts the destination and intermediate points. This analysis uses natural language processing technology (e.g., SpaCy, NLTK). The server automatically extracts the location and time information specified by the user from the text.

[0165] Route calculation

[0166] The server uses a navigation API (e.g., Google Maps API) to calculate the optimal route based on the destination and intermediate stops, taking real-time traffic information into account.

[0167] Generate and send a proposal message

[0168] The server generates a proposal message based on the calculated route and stopover information and sends it to the terminal.

[0169] Playing the suggestion message

[0170] The device uses voice synthesis software (e.g., Google Text-to-Speech API) to play the received proposal message aloud and ask the user for confirmation.

[0171] User Response

[0172] The user responds to the proposed stopovers and route by saying something like "Yes, that's fine."

[0173] Sending a Response

[0174] The terminal converts the user's response back into text and sends it to the server.

[0175] Setting the final route

[0176] The server verifies the user's consent and sets the final route, which includes detailed navigation instructions.

[0177] Traffic monitoring and route recalculation

[0178] The server monitors traffic conditions in real time, calculates a new route if the conditions change, and generates a message proposing a new route and intermediate destinations, which is then sent to the device.

[0179] Specific examples

[0180] Setting a destination

[0181] 1. User: "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1pm."

[0182] 2. Device: Converts voice input into text and sends it to the server.

[0183] 3. Server: Analyzes the text data, extracts the destination and intermediate points, and calculates the optimal route using the navigation API.

[0184] 4. Server: Generates a message proposing the calculation result and sends it to the terminal.

[0185] 5. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[0186] 6. User: "Yes, that's fine."

[0187] 7. Server: Determines the final route and sends navigation data to the device.

[0188] Route adjustments due to changing traffic conditions

[0189] 1. Server: Monitors traffic conditions and obtains new information.

[0190] 2. Server: Calculates new routes and generates proposal messages.

[0191] 3. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[0192] 4. User: Responds, "Change to XX Restaurant."

[0193] 5. Server: Determines the new route and sends navigation data to the device.

[0194] 6. Device: Continue navigation according to the new route.

[0195] Example prompt sentence:

[0196] "Set the destination as Hotel X. I'd like to stop by for ramen around 1pm on the way."

[0197] "Change it to XX Restaurant."

[0198] This system allows users to set their destination and route by voice and responds to changes in traffic conditions, making navigation safer and more convenient.

[0199] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0200] Step 1: Getting voice input

[0201] The user speaks to the voice input device, saying, "Set the destination as Hotel X. I'd like to eat ramen on the way around 1pm." This acquires voice data.

[0202] Input: User's voice

[0203] Output: Audio data

[0204] Step 2: Convert the audio data

[0205] The device converts the acquired voice data into text data using voice recognition software (e.g., Google Speech-to-Text API), and then sends the converted text data to the server.

[0206] Input: Audio data

[0207] Output: Text data

[0208] Step 3: Analyzing the text data

[0209] The server analyzes the received text data and uses natural language processing technology (e.g., SpaCy, NLTK) to extract the destination "XX Hotel," the stopover "Ramen Shop," and the time "1:00 PM."

[0210] Input: Text data

[0211] Output: Analysis results (destination, intermediate points, time)

[0212] Step 4: Calculate the route

[0213] The server uses a navigation API (e.g., Google Maps API) to calculate the optimal route based on the destination and intermediate destinations, taking into account real-time traffic information.

[0214] Input: Analysis results (destination, transit points, time) and traffic information

[0215] Output: Calculated route information

[0216] Step 5: Generate and send a proposal message

[0217] The server generates a proposal message based on the calculated route and stopover information and sends it to the terminal.

[0218] Input: Calculated route information

[0219] Output: Proposal message

[0220] Step 6: Play the suggestion message

[0221] The device uses voice synthesis software (e.g., Google Text-to-Speech API) to play the received proposal message aloud and ask the user for confirmation.

[0222] Input: Proposal message

[0223] Output: Spoken suggestions

[0224] Step 7: User response

[0225] The user responds to the proposed route and stop by saying something like, "Yes, that's fine." This response indicates the user's intent.

[0226] Input: Voice response

[0227] Output: User response data

[0228] Step 8: Sending response data

[0229] The terminal converts the user's response back into text and sends it to the server.

[0230] Input: User response data (voice)

[0231] Output: User response data (text)

[0232] Step 9: Setting the Final Route

[0233] The server verifies the user's consent and sets the final route, which includes detailed navigation instructions.

[0234] Input: User response data

[0235] Output: Final route setting data

[0236] Step 10: Sending navigation data

[0237] The server sends the final route configuration to the device.

[0238] Input: Final route setting data

[0239] Output: Navigation data

[0240] Step 11: Traffic monitoring

[0241] The server monitors traffic conditions in real time and checks for changes in the situation, using traffic data providers (e.g., Waze Traffic API).

[0242] Input: Real-time traffic information

[0243] Output: Monitoring results

[0244] Step 12: Calculate the new route

[0245] The server calculates a new route if traffic conditions change along the specified route.

[0246] Input: Monitoring results

[0247] Output: New route information

[0248] Step 13: Generate and send a new proposal message

[0249] The server generates a proposal message based on the new route information and sends it to the terminal.

[0250] Input: New route information

[0251] Output: New proposal message

[0252] Step 14: Playing a new proposal

[0253] The terminal will play the new suggestion message aloud and ask the user for confirmation.

[0254] Input: New proposed message

[0255] Output: New audio suggestions

[0256] Step 15: User's new response

[0257] The user responds to the new suggestion by saying something like, "Change to XX Restaurant."

[0258] Input: New spoken response

[0259] Output: New response data

[0260] Step 16: Send new response data

[0261] The terminal converts the new response into text and sends it to the server.

[0262] Input: New response data (audio)

[0263] Output: New response data (text)

[0264] Step 17: Confirm the new route

[0265] The server determines a new route based on the user's new instructions and transmits navigation data to the terminal.

[0266] Input: New response data

[0267] Output: New final route data

[0268] Step 18: Continued Navigation

[0269] The device continues navigation according to the new final route data.

[0270] Input: New final route data

[0271] Output: Updated navigation information

[0272] In this way, the system allows for voice input, destination and intermediate destination settings, and even flexible route recalculation based on real-time traffic information.

[0273] (Application example 1)

[0274] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0275] Conventional navigation systems have the problem of being unable to quickly respond to changes in real-time facility conditions and checkpoints in security services and facility management. Particularly in large facilities or locations with complex layouts, manually understanding these changes and setting optimal patrol routes takes time and effort, so there is a need for improved efficiency and safety.

[0276] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0277] In this invention, the server includes means for converting voice input from the user into text data, means for analyzing the text data to extract a destination and intermediate points, and means for calculating an optimal tour route based on the destination and intermediate points, thereby enabling efficient and safe tour route planning in response to changes in facility conditions and checkpoints in real time.

[0278] "Means for converting voice input into text data" refers to a device or program that converts a user's voice instructions into text data using voice recognition technology.

[0279] "Means for analyzing text data and extracting destinations and intermediate points" refers to devices or programs that have the function of identifying the destination and intermediate points specified by the user from the converted text data using natural language processing technology.

[0280] "Means for calculating the optimal travel route based on the destination and intermediate points" refers to a device or program that calculates the optimal travel route for a specified destination and intermediate points, taking into account current traffic information and conditions within the facility.

[0281] "Means for proposing calculated route and route information to users" refers to devices or programs that have the function of presenting calculated route and route information to users in the form of confirmation and suggestions.

[0282] "Means for setting a final route based on a response from the user" refers to a device or program that finalizes a proposed route based on voice or input data from the user and reflects it in the navigation.

[0283] "Means for monitoring changes in facility conditions, recalculating routes in response to those changes, and proposing new routes to users" refers to devices or programs that have the function of detecting real-time changes in conditions inside and outside the facility, recalculating routes based on those changes, and proposing new routes to users.

[0284] "Means for adjusting the timing of checkpoints and intermediate destinations and suggesting them to users" refers to devices or programs that have the function of dynamically reconfiguring checkpoints and intermediate destinations to be reached during a patrol in response to changes in facility conditions and suggesting them to users.

[0285] As an embodiment of the present invention, a voice-input navigation system for security services is proposed. The system of the present invention realizes voice-input instructions and optimizes patrol routes in response to real-time changes in facility conditions.

[0286] Hardware and Software Configuration

[0287] Hardware:

[0288] Audio input device (e.g. smartphone, Bluetooth microphone)

[0289] Server (real-time processing and data analysis)

[0290] Network communication devices (Wi-Fi, cellular networks)

[0291] software:

[0292] Speech recognition software (e.g., Google Speech-to-Text API, Python's speech_recognition library)

[0293] Natural language processing software (e.g., NLTK, spaCy)

[0294] Navigation APIs (e.g. Google Maps API)

[0295] Processing method

[0296] Speech to text:

[0297] First, the user gives instructions for the patrol into the voice input device. For example, they might say, "I'd like to go to the administration building and then check the warehouse." The device picks up this voice and converts it into text data using voice recognition software.

[0298] Analyzing text data:

[0299] The server then analyzes the converted text data to extract destinations and stopovers using natural language processing software, which automatically identifies destinations and stopovers from specific keywords and phrases.

[0300] Route optimization and calculation:

[0301] The server uses a navigation API to calculate the optimal route based on the extracted destinations and intermediate stops, taking into account real-time traffic information and in-facility situation information.

[0302] Suggestions for users:

[0303] The calculated route and route information are sent from the server to the device, which then proposes it to the user via voice. The user can respond to the proposal by saying something like "Yes, that's fine." If the user responds in any other way, the device will propose and correct the route again.

[0304] Real-time monitoring:

[0305] The server constantly monitors the facility status and recalculates the route if there are any changes, and as a result, it re-proposes new routes and checkpoints to the user and modifies the tour route as necessary.

[0306] Specific examples

[0307] Example 1: Facility patrol during business hours

[0308] The user gives voice instructions for the patrol route. For example, "I want to go to the administration building, then move to the warehouse." The device converts the voice into text, which the server analyzes and calculates the route. The calculation results are then presented to the user, and the final route is set based on the user's response. The server monitors the facility situation in real time, and if the situation changes, it immediately suggests a new route.

[0309] Prompt Sentence Examples

[0310] User: "I want to go to the administration building and then to the warehouse."

[0311] Device: "Converting speech to text..."

[0312] Server: "Parsing destination and waypoints..."

[0313] This allows the system of the present invention to efficiently and safely patrol facilities using voice input, significantly improving the efficiency and effectiveness of security operations.

[0314] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0315] Step 1:

[0316] The user inputs the destination and stopover points into the voice input device, for example, by saying, "I want to go to the administration building and then check the warehouse." The input is the user's voice data, which the device receives and sends to the server.

[0317] Step 2:

[0318] The device converts the captured voice data into text data using speech recognition software such as the Google Speech-to-Text API or the Python speech_recognition library. The output is the converted text data, which is then sent to the server.

[0319] Step 3:

[0320] The server analyzes the received text data using natural language processing software to extract destinations and intermediate points. This analysis uses natural language processing libraries such as NLTK and spaCy. The input is text data, and the output is the extracted destinations and intermediate points.

[0321] Step 4:

[0322] The server sends the extracted destinations and intermediate points to a navigation API, which calculates the optimal route. The navigation API uses the Google Maps API, etc. The input is the destination and intermediate points, and the output is the calculated route.

[0323] Step 5:

[0324] The server generates a message to propose the calculated route and stop information to the user and sends it to the terminal. The input is the calculated tour route, and the output is the proposal message.

[0325] Step 6:

[0326] The device then plays the received suggestion message aloud and asks the user for confirmation. Specifically, it uses a generative AI model to read the suggestion aloud in a natural voice. The input is the suggestion message, and the output is the spoken suggestion.

[0327] Step 7:

[0328] The user responds to the proposed route by voice, for example, saying, "Yes, that's it." The input is the user's voice instruction, and the device converts the voice data back into text and sends it to the server.

[0329] Step 8:

[0330] The server sets the final route based on the user's response. The input is the user's text data, and the output is the final route.

[0331] Step 9:

[0332] The server monitors facility and traffic conditions in real time. If any changes occur, the server recalculates the route using the new information and sends a new proposal message to the terminal. The input is real-time situation data, and the output is a recalculated tour route and a new proposal message.

[0333] Step 10:

[0334] The terminal plays the new suggestion message aloud and asks the user for confirmation. Again, based on the user's response, the server adjusts the route and performs the optimal tour. The input is the suggestion message, and the output is the adjusted final route.

[0335] This enables real-time facility patrols via voice input, improving the efficiency and safety of security services.

[0336] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0337] overview

[0338] This invention relates to a voice-input navigation system that incorporates a function to recognize user emotions to provide more personalized route suggestions and user experience. Users can specify destinations and intermediate stops by voice, and the system can not only adjust the route based on changes in traffic conditions, but also adapt the suggestions based on the user's emotions.

[0339] Program processing

[0340] System configuration

[0341] The system mainly consists of the following elements:

[0342] 1. Voice input devices (smartphones and car navigation devices)

[0343] 2. Server (voice analysis, emotion recognition, and navigation data processing)

[0344] 3. Navigation API (e.g., map services)

[0345] 4. Emotion engine (recognizes user emotions from voice input)

[0346] 1. Voice input capture and emotion recognition

[0347] User: The user gives instructions to the voice input device, such as, "Set Hotel X as the destination. I'd like to eat ramen on the way around 1:00 p.m."

[0348] Device: Calls the speech recognition API to convert voice input into text data. Retrieves the converted text data.

[0349] Terminal: Sends text data and voice data itself to the server.

[0350] Server: The received text data is input into ChatGPT, and the destination and intermediate points are extracted using natural language analysis.

[0351] Server: Uses an emotion engine to recognize the user's emotions from the voice data. For example, it determines whether the user is anxious or relaxed based on the tone and speed of the voice.

[0352] 2. Route calculation and emotion-based suggestion adaptation

[0353] Server: Uses the navigation API to calculate the optimal route based on the extracted destination and intermediate destinations, taking real-time traffic information into account.

[0354] Server: Adjusts the recommendations based on the user's emotions as recognized by the emotion engine. For example, if the user is determined to be tired, the server will suggest more relaxing rest areas or stopovers.

[0355] Server: Generates a message to suggest to the user based on the calculation results and adjustments based on emotions. For example, it could say, "We suggest a route to the destination hotel, stopping at the ramen shop around 1:00 PM. We also suggest places to rest and refresh yourself."

[0356] Server: Sends a proposal message to the terminal.

[0357] 3. Confirm the proposal and set the final route

[0358] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[0359] User: Review the proposed route and respond with a response such as "Yes, that's fine."

[0360] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[0361] Server: Confirms user consent and finalizes the route, which includes detailed navigation instructions.

[0362] Server: Sends the confirmed route data to the terminal.

[0363] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[0364] 4. Traffic monitoring and route correction

[0365] Server: Monitors traffic condition data in real time. Uses a dedicated API to periodically obtain information on congestion, accidents, etc.

[0366] Server: Determines whether there are any changes in traffic conditions that will affect the currently configured route.

[0367] Server: If a change in traffic conditions is detected, calculate a new route, resetting intermediate stops if necessary.

[0368] Server: The emotion engine re-identifies the user's emotions and adjusts the suggestions for new routes and stops.

[0369] Server: Generate a message containing a new route or stopover suggestion, such as "We are currently experiencing traffic congestion, so we suggest a revised route. We also recommend changing the rest stop to ____."

[0370] Server: Sends a proposal message to the terminal.

[0371] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[0372] User: Review the new proposal and respond, "Yes, I'd like the revised route."

[0373] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[0374] Server: Confirms user consent and confirms the new route, which also includes detailed navigation instructions.

[0375] Server: Sends new route data to the device.

[0376] Device: Continues navigation based on the new route, providing visual and audio guidance.

[0377] Specific examples

[0378] Example 1: Destination setting and emotion recognition

[0379] 1. User: Says to the device, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1pm."

[0380] 2. Device: Converts voice input into text and sends it to the server.

[0381] 3. Server: Analyzes the text data and extracts the destination and intermediate points.

[0382] 4. Server: Using the emotion engine, recognize the user's emotion from the voice data. For example, it determines that the user is tired.

[0383] 5. Server: Calculates the optimal route using the navigation API.

[0384] 6. Server: Adjust the recommendations based on the user's emotions. Create recommendations that include relaxing stops and rest areas.

[0385] 7. Server: Generates a message proposing the calculation result and sends it to the terminal.

[0386] 8. Terminal: Play the proposed message aloud and ask the user for confirmation.

[0387] 9. User: Review the proposed route and respond, "Yes, that's fine."

[0388] 10. Server: Determines the final route and sends navigation data to the device.

[0389] Example 2: Route adjustments based on traffic changes and sentiment

[0390] 1. Server: Monitors traffic conditions and obtains new information.

[0391] 2. Server: Calculate a new route and recognize the user's emotions again using the emotion engine.

[0392] 3. Server: Generates messages with new suggestions and tailors them with sentiment-based content.

[0393] 4. Terminal: Play the proposed message aloud and ask the user for confirmation.

[0394] 5. User: Agrees with the new suggestion and responds, "Change to XX Restaurant."

[0395] 6. Server: Determines the new route and sends navigation data to the device.

[0396] 7. On the device: Continue navigation based on the new route.

[0397] As a result, the system of the present invention recognizes the user's emotions and suggests appropriate routes and stopovers accordingly, thereby providing safe and convenient navigation.

[0398] The processing flow will be explained below.

[0399] Processing steps of a navigation system including an emotion engine

[0400] 1. Destination and route setting and emotion recognition processing

[0401] Step 1:

[0402] User: Gives instructions to the voice input device, such as "Set Hotel X as my destination. I'd like to eat ramen on the way around 1pm."

[0403] Step 2:

[0404] Device: Calls the speech recognition API to convert voice input into text data. Retrieves the converted text data.

[0405] Step 3:

[0406] Terminal: Transmits the converted text data and the audio data itself to the server.

[0407] Step 4:

[0408] Server: The received text data is input to ChatGPT for analysis. ChatGPT extracts the destination and intermediate points from the text data.

[0409] Step 5:

[0410] Server: Uses an emotion engine to recognize the user's emotions from the voice data, for example, determining whether the user is tired or relaxed from the tone and speed of the voice.

[0411] Step 6:

[0412] Server: Uses the navigation API to calculate the optimal route based on the extracted destinations and intermediate destinations, including real-time traffic information.

[0413] Step 7:

[0414] Server: Adjusts the recommendations based on the user's perceived emotions. For example, if the user is tired, it suggests more relaxing rest stops or stopovers.

[0415] Step 8:

[0416] Server: Generates a message containing the calculated route and sentiment-based suggestions. For example, "We suggest a route with the destination being the XX hotel, stopping at the XX ramen shop around 1:00 PM. Considering recent fatigue, we also recommend rest stops along the way."

[0417] Step 9:

[0418] Server: Sends the generated proposal message to the terminal.

[0419] Step 10:

[0420] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[0421] Step 11:

[0422] User: Review the proposed route and respond "Yes, that's fine."

[0423] Step 12:

[0424] Terminal: The user's response is converted into text again using the speech recognition API and sent to the server.

[0425] Step 13:

[0426] Server: Confirms user consent and finalizes the route, including detailed navigation instructions.

[0427] Step 14:

[0428] Server: Sends the confirmed route data to the terminal.

[0429] Step 15:

[0430] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[0431] 2. Monitoring traffic conditions and handling route corrections

[0432] Step 1:

[0433] Server: Monitors traffic condition data in real time. Uses a dedicated API to periodically obtain information on congestion, accidents, etc.

[0434] Step 2:

[0435] Server: Determines whether there are any changes in traffic conditions that will affect the currently configured route.

[0436] Step 3:

[0437] Server: If a change in traffic conditions is detected, calculate a new route, resetting intermediate stops if necessary.

[0438] Step 4:

[0439] Server: Again, the emotion engine recognizes the user's emotions and adjusts the suggestions for new routes and stops.

[0440] Step 5:

[0441] Server: Generates a suggestion message that includes a new route and adjustments based on emotions. For example, "Currently, there is traffic congestion, so we suggest a revised route. Considering fatigue, we recommend changing the rest area to △△."

[0442] Step 6:

[0443] Server: Sends the generated proposal message to the terminal.

[0444] Step 7:

[0445] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[0446] Step 8:

[0447] User: Review the new proposal and respond, "Yes, I'd like the revised route."

[0448] Step 9:

[0449] Terminal: The user's response is converted into text again using the speech recognition API and sent to the server.

[0450] Step 10:

[0451] Server: Confirms user consent and confirms the new route, which also includes detailed navigation instructions.

[0452] Step 11:

[0453] Server: Sends new route data to the device.

[0454] Step 12:

[0455] Device: Continues navigation based on the new route, providing visual and audio guidance.

[0456] As a result, the system of the present invention recognizes the user's emotions, suggests appropriate routes and stopovers accordingly, and provides safe and convenient navigation.

[0457] Example 2

[0458] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0459] Conventional navigation systems are equipped with functions for recalculating routes and setting destinations in response to changes in traffic conditions, but they are unable to provide appropriate suggestions based on the user's emotional state, making it difficult to improve user satisfaction. Furthermore, even when setting a destination based on voice input, there is a lack of natural conversational responses, so there is a need to improve the user experience.

[0460] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for recognizing the user's emotion from voice data, means for adjusting the content of suggestions based on the recognized user's emotion, and means for monitoring changes in traffic conditions, recalculating the route in accordance with the changes in traffic conditions, and suggesting the route to the user. This enables personalized route suggestions that take the user's emotion into consideration and optimal route adjustments in real time.

[0461] "Means for converting voice input into text data" refers to technology or devices that process a user's voice and convert the voice data into text format data.

[0462] "Means for analyzing text data and voice data to extract destinations and intermediate points" refers to technology or devices that automatically identify specified destinations and intermediate points from voice-input text and voice data.

[0463] "Means for recognizing a user's emotions from voice data" refers to technology or devices that analyze characteristics of voice data such as tone, speed, and volume of the voice to determine the user's emotional state (for example, fatigue, tension, relaxation, etc.).

[0464] "Means for calculating the optimal route based on the destination and intermediate points" refers to technology or devices that calculate the optimal travel route based on the set destination and intermediate points, taking into account traffic information and map information.

[0465] "Means for adjusting the content of suggestions based on the recognized user emotions" refers to technology or devices that take into account the user's emotional state and adaptively change the suggested route and intermediate points.

[0466] "Means for suggesting calculated route and stopover information to users" refers to technology or devices that present information on optimal routes and stopovers to users.

[0467] "Means for setting a final route based on a user's response" refers to technology or devices that determine a final navigation route based on the user's confirmation or consent.

[0468] "Means for monitoring changes in traffic conditions, recalculating routes in response to the changes in traffic conditions, and proposing routes to users" refers to technologies and devices that monitor traffic information in real time, calculate new routes in response to changes in traffic conditions, and provide that information to users.

[0469] The navigation system of the present invention is composed of a voice input device, a server, a navigation API, and an emotion engine. This system converts the user's voice input into text data and extracts the destination and intermediate points from the text data. It also recognizes the user's emotions from the voice data and proposes the optimal route. This section specifies the hardware and software used and explains how the system specifically operates.

[0470] System configuration

[0471] 1. Voice input device: A voice input device such as a smartphone or car navigation terminal is used to obtain voice instructions from the user.

[0472] 2. Speech recognition API: Use Google Speech-to-Text or similar to convert voice data into text data.

[0473] 3. Text and voice data analysis: The server uses ChatGPT or a similar generative AI model to analyze the text data and extract destinations and intermediate stops.

[0474] 4. Emotion Recognition Engine: Uses IBM Watson and other emotion recognition technologies to recognize user emotions from voice data.

[0475] 5. Navigation API: Uses Google Maps API etc. to calculate the optimal route reflecting real-time traffic information.

[0476] Example of operation

[0477] Example 1: Destination setting and emotion recognition

[0478] 1. User: The user speaks into a voice input device, saying, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1:00 p.m."

[0479] 2. Device: Calls the speech recognition API to convert this voice input into text data.

[0480] 3. Terminal: The converted text data and audio data are sent to the server.

[0481] 4. Server: The server inputs the text data into ChatGPT and extracts the destination "XX Hotel" and the stopover "Ramen Shop" through natural language analysis.

[0482] 5. Server: Using the emotion engine, recognize the user's emotion from the voice data. For example, it determines that the user is tired.

[0483] 6. Server: Calculates the optimal route using the navigation API.

[0484] 7. Server: Adjust the recommendations based on the user's emotions and create recommendations that include relaxing stops and rest areas.

[0485] 8. Server: Sends the calculation result to the terminal as a proposal message.

[0486] 9. Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[0487] 10. User: Review the proposed route and respond, "Yes, that's fine."

[0488] 11. Terminal: The user's response is again converted into text using the speech recognition API and sent to the server.

[0489] 12. Server: Confirms user consent, finalizes the route, and sends the final route data to the device.

[0490] 13. Terminal: Starts navigation based on the received route data, and provides screen display and voice guidance.

[0491] Prompt Sentence Examples

[0492] Prompt example 1: Speech data text conversion and emotion recognition

[0493] "Voice data: 'I want to eat ramen at 1pm.' Convert this voice data into text and recognize the user's emotions."

[0494] Example prompt 2: Route calculation and adjustment

[0495] "Destination: 'X hotel, Shibuya-ku, Tokyo' Stop: 'X ramen shop' User's emotion: 'Tired' Calculate the optimal route based on this data and generate a suggested message including stopovers and rest areas where people can relax."

[0496] This allows the system of the present invention to propose a personalized route that takes into account the user's emotions and to adjust the route optimally in real time.

[0497] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0498] Step 1:

[0499] User: The user gives instructions to the voice input device, such as, "Set Hotel X as the destination. I'd like to eat ramen on the way around 1:00 p.m."

[0500] Specific behavior:

[0501] Users speak into the microphone on their smartphone or car navigation device.

[0502] Input: Audio data

[0503] Output: Audio data

[0504] Step 2:

[0505] On the device: Call a speech recognition API (such as Google Speech-to-Text) to convert the voice input into text data. Retrieve the converted text data.

[0506] Specific behavior:

[0507] Send the voice data to the voice recognition API and convert it into text data.

[0508] Keeps the converted text in memory.

[0509] Input: Audio data

[0510] Output: Text data

[0511] Step 3:

[0512] Terminal: Sends text data and voice data itself to the server.

[0513] Specific behavior:

[0514] Use an HTTP POST request to send text and audio data to a specified endpoint on the server.

[0515] Input: Text data, audio data

[0516] Output: Send data to the server

[0517] Step 4:

[0518] Server: The received text data is input into ChatGPT, and the destination and intermediate points are extracted using natural language analysis.

[0519] Specific behavior:

[0520] Call the ChatGPT API and send text data as input data.

[0521] Extract the destination "XX Hotel" and the transit point "△△ Ramen Shop" from the ChatGPT output.

[0522] Input: Text data

[0523] Output: Destination, intermediate destination

[0524] Step 5:

[0525] Server: Uses an emotion engine to recognize the user's emotions from the voice data, for example, determining whether the user is anxious or relaxed based on the tone and speed of the voice.

[0526] Specific behavior:

[0527] Extract features such as tone, speed, and volume of speech.

[0528] Feature data is input into the emotion engine to recognize emotions (e.g., "tired").

[0529] Input: Audio data

[0530] Output: Emotion data

[0531] Step 6:

[0532] Server: Uses the navigation API (Google Maps API) to calculate the optimal route based on the extracted destination and intermediate destinations, taking real-time traffic information into account.

[0533] Specific behavior:

[0534] Send a request to the Google Maps API to get the optimal route via "XX Hotel" and "△△ Ramen Shop."

[0535] Receive route data including real-time traffic information.

[0536] Input: Destination, stopover

[0537] Output: Optimal route data

[0538] Step 7:

[0539] Server: Adjusts the recommendations based on the user's emotions as recognized by the emotion engine. For example, if the user is determined to be tired, the server will suggest more relaxing rest areas or stopovers.

[0540] Specific behavior:

[0541] The output of the emotion engine will be analyzed, and an option will be added to suggest a place to relax if the user's emotion is recognized as "tired."

[0542] Input: Emotion data, optimal route data

[0543] Output: Adjusted proposal

[0544] Step 8:

[0545] Server: Generates a message to suggest to the user based on the calculation results and adjustments based on emotions.

[0546] Specific behavior:

[0547] By combining the navigation route and emotion data, a message such as "We suggest a route with the destination being the XX hotel, stopping at the △△ ramen shop around 1:00 p.m. We will also suggest a place to rest and refresh yourself" is generated.

[0548] Input: Adjusted proposal

[0549] Output: Proposal message

[0550] Step 9:

[0551] Server: Sends a proposal message to the terminal.

[0552] Specific behavior:

[0553] Send the generated message to the device using an HTTP POST request.

[0554] Input: Proposal message

[0555] Output: Sending a message to the terminal

[0556] Step 10:

[0557] Terminal: The proposed message is converted into speech using a speech synthesis API (e.g., Amazon Polly) and played back to the user.

[0558] Specific behavior:

[0559] The message text is sent to the speech synthesis API and converted into voice data.

[0560] The generated audio data is played back from the device's speaker.

[0561] Input: Proposal message

[0562] Output: Audio data

[0563] Step 11:

[0564] User: Review the proposed route and respond "Yes, that's fine."

[0565] Specific behavior:

[0566] The user responds to the system by saying, "Yes, that would be fine."

[0567] Input: Proposal message

[0568] Output: User's voice response

[0569] Step 12:

[0570] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[0571] Specific behavior:

[0572] Send the voice data to the voice recognition API and convert it into text data.

[0573] Sends text data to the server.

[0574] Input: User's voice response

[0575] Output: Text data

[0576] Step 13:

[0577] Server: Confirms user consent and finalizes the route, which includes detailed navigation instructions.

[0578] Specific behavior:

[0579] Receive the user's consent response and finalize the route.

[0580] Generate detailed route data including navigation instructions.

[0581] Input: User's text response

[0582] Output: Final route data

[0583] Step 14:

[0584] Server: Sends the confirmed route data to the terminal.

[0585] Specific behavior:

[0586] Send an HTTP POST request to send the confirmed route data to the device.

[0587] Input: Final route data

[0588] Output: Sending data to the terminal

[0589] Step 15:

[0590] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[0591] Specific behavior:

[0592] The received route data is analyzed and navigation begins.

[0593] Route instructions are provided on screen and via voice guidance.

[0594] Input: Final route data

[0595] Output: Navigation instructions (screen display, voice guide)

[0596] (Application example 2)

[0597] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0598] Conventional navigation systems only set destinations and intermediate stops based on user instructions and modify routes according to traffic conditions, so they were unable to provide personalized suggestions that took the user's emotional state into account.The present invention aims to provide more personalized route suggestions and user experiences by incorporating a function that recognizes the user's emotions into the navigation system, thereby reducing user stress and improving user satisfaction.

[0599] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for converting voice input from a user into text data, means for analyzing the text data to extract a destination and intermediate points, means for recognizing the user's emotions from the voice data, means for calculating an optimal route based on the destination and intermediate points, means for adapting the proposal content based on the user's emotions, means for proposing information on the calculated route and intermediate points to the user, means for setting a final route based on a response from the user, and means for monitoring changes in traffic conditions, recalculating a route in accordance with the changes in traffic conditions, and proposing the route to the user. This makes it possible to propose personalized routes and intermediate points according to the user's emotional state.

[0600] "Voice input data" refers to information provided by a user to give voice instructions to the navigation system.

[0601] "Text data" refers to character information converted from voice input data using voice recognition technology.

[0602] A "destination" is the final location that a user wants to reach using a navigation system.

[0603] A "waypoint" is a place where you plan to stop on the way to your destination.

[0604] "Emotion recognition method" is a technology that analyzes the user's emotional state from the tone, speed, and word choice of voice data, and identifies specific emotions.

[0605] An "optimal route" is the most efficient route to reach a destination calculated based on current traffic conditions and user settings.

[0606] "Suggested content" refers to information about routes, stops, rest spots, etc. that the navigation system presents to the user.

[0607] "Route recalculation" is the process of revising an existing route to accommodate new information based on changing traffic conditions and user responses.

[0608] The "final route" is the final route that reflects the user's responses and satisfies all conditions.

[0609] "Traffic condition monitoring" is a technology that monitors real-time data such as traffic congestion and accident information, and makes route adjustments as necessary.

[0610] "Rest Spots" are locations along the route that are suggested for users to take a rest along the way.

[0611] A "navigation system" is a system that integrates these elements and provides users with audio and visual guidance.

[0612] The present invention provides a more personalized route suggestion and user experience by incorporating a function that recognizes the user's emotions into a voice-based navigation system. The system of the present invention consists of the following main components:

[0613] System configuration

[0614] 1. Hardware:

[0615] Audio input devices (e.g. smartphones, microphones)

[0616] GPS sensor

[0617] Autonomous vehicle control computer

[0618] 2. Software:

[0619] Speech recognition API (e.g. Google API)

[0620] Emotion recognition engine (e.g. IBM Watson Tone Analyzer)

[0621] Navigation APIs (e.g. Google Maps API)

[0622] Server-side architecture (e.g., Node.js, Express server)

[0623] Database (e.g. MongoDB)

[0624] Speech synthesis API (e.g. Amazon Polly)

[0625] Generative AI API (e.g. OpenAI GPT)

[0626] Program processing

[0627] 1. Voice input capture and emotion recognition

[0628] The user gives instructions to the navigation system using a voice input device, for example, "Set the destination as Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1:00 PM."

[0629] The device converts the voice input into text data using a voice recognition API and sends it to the server.

[0630] The server receives the text and audio data itself, performs natural language analysis using ChatGPT, and extracts the destination and intermediate points.

[0631] Furthermore, an emotion recognition engine is used to recognize the user's emotions from the voice data, for example, determining whether the user is anxious or relaxed from the tone and speed of the voice.

[0632] 2. Route calculation and emotion-based suggestion adaptation

[0633] The server uses a navigation API to calculate the optimal route, taking real-time traffic information into account.

[0634] Data from the emotion recognition engine is used to tailor suggestions, for example, suggesting a route that includes relaxing rest stops for a tired user.

[0635] The server uses a generation AI to generate messages in natural language to suggest to users and sends them to the device.

[0636] 3. Confirm the proposal and set the final route

[0637] The server converts the generated proposal message into voice data using a voice synthesis API and sends it to the terminal.

[0638] The terminal plays the suggestion message aloud and asks the user for confirmation.

[0639] The user reviews the proposed route and responds with a response such as "Yes, that's fine."

[0640] The server receives the user's response and determines the final route.

[0641] The determined route data is sent to the terminal, providing route information to the autonomous vehicle's control computer.

[0642] 4. Traffic monitoring and route correction

[0643] The server monitors traffic data in real time and acquires new information.

[0644] It recalculates routes based on new traffic conditions and re-recognizes the user's emotional state.

[0645] The server generates and adjusts new proposals and sends them to the device.

[0646] The user reviews the new proposal and responds, and the server, upon approval, finalizes the new route and sends it back to the autonomous vehicle's control computer.

[0647] Specific examples

[0648] Navigation for users who feel fatigued

[0649] Example prompt: "I'm tired and would like to stop by a cafe or park where I can relax."

[0650] Server processing: The server determines that the user is tired and calculates a route that includes places where the user can refresh themselves.

[0651] Suggestion example: As a new route, suggest "We suggest stopping at the cafe in Shizuoka Parking Area on the way. You can take a break there."

[0652] The above is a specific embodiment for implementing the system of the present invention, which recognizes the user's emotions and enables personalized navigation routes and suggestions based on those emotions.

[0653] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0654] Step 1:

[0655] The user uses a voice input device to specify destinations and stopovers by voice. For example, the user might say, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1:00 PM."

[0656] Input: User's voice command

[0657] Output: Audio data

[0658] Specific operation: The voice input device acquires user instructions as voice data through a microphone.

[0659] Step 2:

[0660] The device converts the voice data into text data using a voice recognition API and sends it to the server.

[0661] Input: Audio data

[0662] Output: Text data

[0663] Specific operation: The device calls a speech recognition API (e.g., Google API) and converts the voice data into text data. The converted text data is then obtained and sent to the server via an HTTP request or similar.

[0664] Step 3:

[0665] The server receives the text data and voice data and extracts the destination and intermediate points using ChatGPT.

[0666] Input: Text data, audio data

[0667] Output: Destination and waypoint information

[0668] Specific operation: The server inputs the received text data into ChatGPT and performs natural language analysis, extracting the destination and intermediate points and storing them in a database.

[0669] Step 4:

[0670] The server uses an emotion recognition engine to recognize the user's emotion from the voice data.

[0671] Input: Audio data

[0672] Output: User's emotional state

[0673] Specific operation: The server inputs the voice data into an emotion recognition engine (e.g., IBM Watson Tone Analyzer) and analyzes the tone and speed of the voice to identify the user's emotion.

[0674] Step 5:

[0675] The server uses a navigation API to calculate the optimal route, taking real-time traffic information into account.

[0676] Input: Destination information, route information, traffic information

[0677] Output: Optimal route

[0678] Specific operation: The server calls a navigation API (e.g., Google Maps API) and calculates the optimal route based on the destination, intermediate destinations, and current traffic conditions.

[0679] Step 6:

[0680] The server adapts the suggestions based on the user's emotions.

[0681] Input: User's emotional state, optimal route

[0682] Output: Personalized recommendations

[0683] Specific operation: Based on the results of the emotion recognition engine, for example, the server sets a route including rest spots for a user who feels tired, and generates a suggested message using a generation AI.

[0684] Step 7:

[0685] The server sends a proposal message to the device, which converts the proposal message into voice data using a speech synthesis API and plays it back to the user.

[0686] Input: Proposal message

[0687] Output: Audio data, audio playback

[0688] Specific operation: The server sends the generated proposal message to the device via an HTTP request. The device converts the received message into voice data using a speech synthesis API (e.g., Amazon Polly) and plays it back to the user through the speaker.

[0689] Step 8:

[0690] The user confirms the proposed route and responds with a voice response such as "Yes, that's fine."

[0691] Input: Voice response

[0692] Output: Audio data

[0693] Specific actions: After listening to the proposed route, if the user agrees, they give instructions to the voice input device again.

[0694] Step 9:

[0695] The device converts the voice data into text data again using the voice recognition API and sends it to the server.

[0696] Input: Audio data

[0697] Output: Text data

[0698] Specific operation: The device calls the speech recognition API again, converts the voice data into text data, and sends the converted text data to the server.

[0699] Step 10:

[0700] The server receives the user's response, determines the final route, and sends the determined route data to the device to start navigation.

[0701] Input: User consent (text data)

[0702] Output: Final route data

[0703] Specific operation: The server confirms the user's consent, determines the final route using the navigation API, sends the determined route data to the device, and starts navigation.

[0704] Step 11:

[0705] The server monitors traffic data in real time, recalculates the route according to the new traffic conditions, and also recognizes the user's emotional state again.

[0706] Input: Real-time traffic data

[0707] Output: The modified route

[0708] Specific operation: The server periodically calls the traffic information API to obtain data, and if there is any impact on the existing route, it recalculates it. At the same time, it uses the emotion recognition engine to re-determine the user's emotional state.

[0709] Step 12:

[0710] The server generates a new proposal and sends it to the terminal, which plays it back to the user in the same way and asks for confirmation.

[0711] Input: New proposal

[0712] Output: Audio data, audio playback

[0713] How it works: The server uses the generation AI to generate new suggestions and sends them to the device via an HTTP request. The device then converts the received message into audio data using a speech synthesis API and plays it back to the user.

[0714] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0715] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0716] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0717] [Second embodiment]

[0718] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0719] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0720] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0721] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0722] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0723] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0724] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0725] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0726] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0727] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0728] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0729] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0730] overview

[0731] The present invention relates to a navigation system that uses voice input. Users can set destinations and intermediate points through voice, and can smoothly adjust routes and rest areas according to changes in traffic conditions. This system supports users in traveling safely and conveniently.

[0732] Program processing

[0733] System configuration

[0734] The system mainly consists of the following elements:

[0735] 1. Voice input devices (smartphones and car navigation devices)

[0736] 2. Server (analysis and navigation data processing)

[0737] 3. Navigation API (e.g., map services)

[0738] 1. Acquiring voice input

[0739] User: First, the user speaks to the voice input device about their destination and stopovers. For example, they might say, "Set the destination to Hotel X as my destination. I'd like to eat ramen on the way around 1:00 PM."

[0740] Terminal: The terminal converts this voice input into text data and sends the text data to the server.

[0741] 2.Analysis of voice data and route calculation

[0742] Server: The server analyzes the received text data and extracts the destination and intermediate points. This analysis uses natural language processing technology. Specifically, it automatically extracts information about location and time from the text.

[0743] The server then uses a navigation API to calculate the optimal route based on the destination and intermediate stops, taking real-time traffic information into account.

[0744] Server: The server generates a message to suggest the calculated route and stopover information to the user and sends it to the terminal.

[0745] 3. Confirm the proposal and set the final route

[0746] Terminal: The terminal plays the received proposal message as audio and asks the user for confirmation.

[0747] User: The user responds to the proposed stopovers and route with a response such as "Yes, that's fine."

[0748] Terminal: Convert this response back into text and send it to the server.

[0749] Server: The server verifies the user's consent and sets the final route, which includes detailed navigation instructions.

[0750] 4. Traffic monitoring and route recalculation

[0751] Server: The server monitors traffic conditions in real time and calculates a new route if there are changes in traffic conditions along the specified route.

[0752] Server: Generates messages to propose new routes and stops and sends them to the device.

[0753] Terminal: The terminal will play the new suggestion aloud and ask the user for confirmation.

[0754] User: The user responds to the new suggestion, for example by giving instructions such as "Change to XX Restaurant."

[0755] Server: The server determines the new route and sends it to the device, which then continues navigation according to the new route.

[0756] Specific examples

[0757] Example 1: Setting a destination

[0758] 1. User: Says to the device, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1pm."

[0759] 2. Device: Converts voice input into text and sends it to the server.

[0760] 3. Server: Analyzes the text data, extracts the destination and intermediate points, and calculates the optimal route using the navigation API.

[0761] 4. Server: Generates a message proposing the calculation result and sends it to the terminal.

[0762] 5. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[0763] 6. User: Agrees with the proposal and responds, "Yes, I'd like that."

[0764] 7. Server: Determines the final route and sends navigation data to the device.

[0765] Example 2: Route adjustment due to changing traffic conditions

[0766] 1. Server: Monitors traffic conditions and obtains new information.

[0767] 2. Server: Calculates new routes and generates proposal messages.

[0768] 3. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[0769] 4. User: Agrees with the new suggestion and responds, "Change to XX Restaurant."

[0770] 5. Server: Determines the new route and sends navigation data to the device.

[0771] 6. Device: Continue navigation according to the new route.

[0772] As a result, the system of the present invention allows the user to set the destination and intermediate points by voice and responds to changes in traffic conditions, providing safe and convenient navigation.

[0773] The processing flow will be explained below.

[0774] Specific explanation of program processing

[0775] 1. Destination and intermediate destination setting process

[0776] Step 1:

[0777] User: Gives instructions to the voice input device, such as "Set Hotel X as my destination. I'd like to eat ramen on the way around 1pm."

[0778] Step 2:

[0779] Device: Calls the speech recognition API to convert voice input into text data. Retrieves the converted text data.

[0780] Step 3:

[0781] Terminal: Send text data to the server using an API endpoint.

[0782] Step 4:

[0783] Server: The received text data is input into ChatGPT, and the destination and intermediate points are extracted using natural language analysis.

[0784] Step 5:

[0785] Server: Uses a navigation API (e.g., Google Maps API) to calculate the optimal route based on the extracted destination and intermediate destinations.

[0786] Step 6:

[0787] Server: Based on the calculation results, a message is generated to be suggested to the user. It prepares a message such as "Suggested route to destination XX Hotel, stopping at XX ramen shop around 1 PM."

[0788] Step 7:

[0789] Server: Sends a proposal message to the terminal.

[0790] Step 8:

[0791] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[0792] Step 9:

[0793] User: Review the proposed route and respond "Yes, that's fine."

[0794] Step 10:

[0795] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[0796] Step 11:

[0797] Server: Confirms user consent and finalizes the route, which includes detailed navigation instructions.

[0798] Step 12:

[0799] Server: Sends the confirmed route data to the terminal.

[0800] Step 13:

[0801] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[0802] 2. Monitoring traffic conditions and handling route corrections

[0803] Step 1:

[0804] Server: Monitors traffic condition data in real time. Uses a dedicated API to periodically obtain information on congestion, accidents, etc.

[0805] Step 2:

[0806] Server: Determines whether there are any changes in traffic conditions that will affect the currently configured route.

[0807] Step 3:

[0808] Server: If a change in traffic conditions is detected, calculate a new route, resetting intermediate stops if necessary.

[0809] Step 4:

[0810] Server: Generate a message containing a new route or stopover suggestion. For example, "There is currently a traffic jam, so we suggest a revised route."

[0811] Step 5:

[0812] Server: Sends a proposal message to the terminal.

[0813] Step 6:

[0814] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[0815] Step 7:

[0816] User: Review the new proposal and respond, "Yes, I'd like the revised route."

[0817] Step 8:

[0818] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[0819] Step 9:

[0820] Server: Confirms user consent and confirms the new route, which also includes detailed navigation instructions.

[0821] Step 10:

[0822] Server: Sends new route data to the device.

[0823] Step 11:

[0824] Device: Continues navigation based on the new route, providing visual and audio guidance.

[0825] Example 1

[0826] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0827] Conventional navigation systems require a lot of manual input to set destinations and intermediate points, and it is also difficult to flexibly recalculate routes to accommodate real-time traffic conditions. This requires cumbersome operations for users, which reduces safety and convenience. Since manual input while driving is particularly dangerous, there is a demand for systems that utilize voice input.

[0828] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0829] In this invention, the server includes means for capturing a user's voice input and converting it into text data using voice recognition software, means for extracting a destination and intermediate points from the text using natural language processing technology, means for calculating a route taking real-time traffic information into consideration using a navigation API, means for playing a suggested message aloud using voice synthesis software and requesting confirmation from the user, and means for sending a final route instruction to the terminal. This allows the user to easily set a destination and intermediate points through voice input and enables flexible route changes according to real-time traffic information.

[0830] "Voice input" is a means of acquiring the words spoken by a user as digital signals.

[0831] The "means for converting into text data" refers to software or hardware for converting acquired voice input into text information.

[0832] "Means for analyzing text data" refers to technologies or systems for deciphering converted text data and extracting specific information.

[0833] "Destination and stopover points" refers to the final destination set by the user and points to be passed along the way.

[0834] The "means for calculating the optimal route" is a system that calculates the optimal route based on the destination and intermediate points, taking into account conditions such as time, distance, and traffic conditions.

[0835] "Means of providing suggestions to users" refers to the methods and techniques used to communicate calculated route information to users, and may involve voice or text.

[0836] The "means for setting the final route" is a system for determining the final route based on the user's response.

[0837] "Means for monitoring changes in traffic conditions" refers to a system for collecting traffic information in real time and monitoring the situation.

[0838] The "means for recalculating the route" is a system for reevaluating the originally calculated route in response to changes in traffic conditions and calculating a new optimal route.

[0839] "Speech recognition software" is software technology for converting voice input into text data.

[0840] "Natural language processing technology" is a technology for deciphering human language and extracting specific information.

[0841] A "navigation API" is an application program interface for calculating routes and obtaining traffic information using map services, etc.

[0842] "Speech synthesis software" is a technology for reproducing text data as speech.

[0843] System configuration

[0844] The navigation system consists of the following elements:

[0845] 1. Voice input devices (e.g. smartphones, car navigation devices)

[0846] 2. Server (processes analysis and navigation data)

[0847] 3. Navigation API (e.g., map service)

[0848] Program processing

[0849] Acquiring voice input

[0850] The user speaks to the voice input device about the destination and stopovers, for example, "Set the destination to XX Hotel. I'd like to eat ramen on the way around 1 PM."

[0851] Audio data conversion

[0852] The device captures the user's voice, converts it into text data using speech recognition software (e.g., Google Speech-to-Text API), and sends the text data to the server.

[0853] Analysis of audio data

[0854] The server analyzes the received text data and extracts the destination and intermediate points. This analysis uses natural language processing technology (e.g., SpaCy, NLTK). The server automatically extracts the location and time information specified by the user from the text.

[0855] Route calculation

[0856] The server uses a navigation API (e.g., Google Maps API) to calculate the optimal route based on the destination and intermediate stops, taking real-time traffic information into account.

[0857] Generate and send a proposal message

[0858] The server generates a proposal message based on the calculated route and stopover information and sends it to the terminal.

[0859] Playing the suggestion message

[0860] The device uses voice synthesis software (e.g., Google Text-to-Speech API) to play the received proposal message aloud and ask the user for confirmation.

[0861] User Response

[0862] The user responds to the proposed stopovers and route by saying something like "Yes, that's fine."

[0863] Sending a Response

[0864] The terminal converts the user's response back into text and sends it to the server.

[0865] Setting the final route

[0866] The server verifies the user's consent and sets the final route, which includes detailed navigation instructions.

[0867] Traffic monitoring and route recalculation

[0868] The server monitors traffic conditions in real time, calculates a new route if the conditions change, and generates a message proposing a new route and intermediate destinations, which is then sent to the device.

[0869] Specific examples

[0870] Setting a destination

[0871] 1. User: "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1pm."

[0872] 2. Device: Converts voice input into text and sends it to the server.

[0873] 3. Server: Analyzes the text data, extracts the destination and intermediate points, and calculates the optimal route using the navigation API.

[0874] 4. Server: Generates a message proposing the calculation result and sends it to the terminal.

[0875] 5. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[0876] 6. User: "Yes, that's fine."

[0877] 7. Server: Determines the final route and sends navigation data to the device.

[0878] Route adjustments due to changing traffic conditions

[0879] 1. Server: Monitors traffic conditions and obtains new information.

[0880] 2. Server: Calculates new routes and generates proposal messages.

[0881] 3. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[0882] 4. User: Responds, "Change to XX Restaurant."

[0883] 5. Server: Determines the new route and sends navigation data to the device.

[0884] 6. Device: Continue navigation according to the new route.

[0885] Example prompt sentence:

[0886] "Set the destination as Hotel X. I'd like to stop by for ramen around 1pm on the way."

[0887] "Change it to XX Restaurant."

[0888] This system allows users to set their destination and route by voice and responds to changes in traffic conditions, making navigation safer and more convenient.

[0889] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0890] Step 1: Getting voice input

[0891] The user speaks to the voice input device, saying, "Set the destination as Hotel X. I'd like to eat ramen on the way around 1pm." This acquires voice data.

[0892] Input: User's voice

[0893] Output: Audio data

[0894] Step 2: Convert the audio data

[0895] The device converts the acquired voice data into text data using voice recognition software (e.g., Google Speech-to-Text API), and then sends the converted text data to the server.

[0896] Input: Audio data

[0897] Output: Text data

[0898] Step 3: Analyzing the text data

[0899] The server analyzes the received text data and uses natural language processing technology (e.g., SpaCy, NLTK) to extract the destination "XX Hotel," the stopover "Ramen Shop," and the time "1:00 PM."

[0900] Input: Text data

[0901] Output: Analysis results (destination, intermediate points, time)

[0902] Step 4: Calculate the route

[0903] The server uses a navigation API (e.g., Google Maps API) to calculate the optimal route based on the destination and intermediate destinations, taking into account real-time traffic information.

[0904] Input: Analysis results (destination, transit points, time) and traffic information

[0905] Output: Calculated route information

[0906] Step 5: Generate and send a proposal message

[0907] The server generates a proposal message based on the calculated route and stopover information and sends it to the terminal.

[0908] Input: Calculated route information

[0909] Output: Proposal message

[0910] Step 6: Play the suggestion message

[0911] The device uses voice synthesis software (e.g., Google Text-to-Speech API) to play the received proposal message aloud and ask the user for confirmation.

[0912] Input: Proposal message

[0913] Output: Spoken suggestions

[0914] Step 7: User response

[0915] The user responds to the proposed route and stop by saying something like, "Yes, that's fine." This response indicates the user's intent.

[0916] Input: Voice response

[0917] Output: User response data

[0918] Step 8: Sending response data

[0919] The terminal converts the user's response back into text and sends it to the server.

[0920] Input: User response data (voice)

[0921] Output: User response data (text)

[0922] Step 9: Setting the Final Route

[0923] The server verifies the user's consent and sets the final route, which includes detailed navigation instructions.

[0924] Input: User response data

[0925] Output: Final route setting data

[0926] Step 10: Sending navigation data

[0927] The server sends the final route configuration to the device.

[0928] Input: Final route setting data

[0929] Output: Navigation data

[0930] Step 11: Traffic monitoring

[0931] The server monitors traffic conditions in real time and checks for changes in the situation, using traffic data providers (e.g., Waze Traffic API).

[0932] Input: Real-time traffic information

[0933] Output: Monitoring results

[0934] Step 12: Calculate the new route

[0935] The server calculates a new route if traffic conditions change along the specified route.

[0936] Input: Monitoring results

[0937] Output: New route information

[0938] Step 13: Generate and send a new proposal message

[0939] The server generates a proposal message based on the new route information and sends it to the terminal.

[0940] Input: New route information

[0941] Output: New proposal message

[0942] Step 14: Playing a new proposal

[0943] The terminal will play the new suggestion message aloud and ask the user for confirmation.

[0944] Input: New proposed message

[0945] Output: New audio suggestions

[0946] Step 15: User's new response

[0947] The user responds to the new suggestion by saying something like, "Change to XX Restaurant."

[0948] Input: New spoken response

[0949] Output: New response data

[0950] Step 16: Send new response data

[0951] The terminal converts the new response into text and sends it to the server.

[0952] Input: New response data (audio)

[0953] Output: New response data (text)

[0954] Step 17: Confirm the new route

[0955] The server determines a new route based on the user's new instructions and transmits navigation data to the terminal.

[0956] Input: New response data

[0957] Output: New final route data

[0958] Step 18: Continued Navigation

[0959] The device continues navigation according to the new final route data.

[0960] Input: New final route data

[0961] Output: Updated navigation information

[0962] In this way, the system allows for voice input, destination and intermediate destination settings, and even flexible route recalculation based on real-time traffic information.

[0963] (Application example 1)

[0964] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0965] Conventional navigation systems have the problem of being unable to quickly respond to changes in real-time facility conditions and checkpoints in security services and facility management. Particularly in large facilities or locations with complex layouts, manually understanding these changes and setting optimal patrol routes takes time and effort, so there is a need for improved efficiency and safety.

[0966] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0967] In this invention, the server includes means for converting voice input from the user into text data, means for analyzing the text data to extract a destination and intermediate points, and means for calculating an optimal tour route based on the destination and intermediate points, thereby enabling efficient and safe tour route planning in response to changes in facility conditions and checkpoints in real time.

[0968] "Means for converting voice input into text data" refers to a device or program that converts a user's voice instructions into text data using voice recognition technology.

[0969] "Means for analyzing text data and extracting destinations and intermediate points" refers to devices or programs that have the function of identifying the destination and intermediate points specified by the user from the converted text data using natural language processing technology.

[0970] "Means for calculating the optimal travel route based on the destination and intermediate points" refers to a device or program that calculates the optimal travel route for a specified destination and intermediate points, taking into account current traffic information and conditions within the facility.

[0971] "Means for proposing calculated route and route information to users" refers to devices or programs that have the function of presenting calculated route and route information to users in the form of confirmation and suggestions.

[0972] "Means for setting a final route based on a response from the user" refers to a device or program that finalizes a proposed route based on voice or input data from the user and reflects it in the navigation.

[0973] "Means for monitoring changes in facility conditions, recalculating routes in response to those changes, and proposing new routes to users" refers to devices or programs that have the function of detecting real-time changes in conditions inside and outside the facility, recalculating routes based on those changes, and proposing new routes to users.

[0974] "Means for adjusting the timing of checkpoints and intermediate destinations and suggesting them to users" refers to devices or programs that have the function of dynamically reconfiguring checkpoints and intermediate destinations to be reached during a patrol in response to changes in facility conditions and suggesting them to users.

[0975] As an embodiment of the present invention, a voice-input navigation system for security services is proposed. The system of the present invention realizes voice-input instructions and optimizes patrol routes in response to real-time changes in facility conditions.

[0976] Hardware and Software Configuration

[0977] Hardware:

[0978] Audio input device (e.g. smartphone, Bluetooth microphone)

[0979] Server (real-time processing and data analysis)

[0980] Network communication devices (Wi-Fi, cellular networks)

[0981] software:

[0982] Speech recognition software (e.g., Google Speech-to-Text API, Python's speech_recognition library)

[0983] Natural language processing software (e.g., NLTK, spaCy)

[0984] Navigation APIs (e.g. Google Maps API)

[0985] Processing method

[0986] Speech to text:

[0987] First, the user gives instructions for the patrol into the voice input device. For example, they might say, "I'd like to go to the administration building and then check the warehouse." The device picks up this voice and converts it into text data using voice recognition software.

[0988] Analyzing text data:

[0989] The server then analyzes the converted text data to extract destinations and stopovers using natural language processing software, which automatically identifies destinations and stopovers from specific keywords and phrases.

[0990] Route optimization and calculation:

[0991] The server uses a navigation API to calculate the optimal route based on the extracted destinations and intermediate stops, taking into account real-time traffic information and in-facility situation information.

[0992] Suggestions for users:

[0993] The calculated route and route information are sent from the server to the device, which then proposes it to the user via voice. The user can respond to the proposal by saying something like "Yes, that's fine." If the user responds in any other way, the device will propose and correct the route again.

[0994] Real-time monitoring:

[0995] The server constantly monitors the facility status and recalculates the route if there are any changes, and as a result, it re-proposes new routes and checkpoints to the user and modifies the tour route as necessary.

[0996] Specific examples

[0997] Example 1: Facility patrol during business hours

[0998] The user gives voice instructions for the patrol route. For example, "I want to go to the administration building, then move to the warehouse." The device converts the voice into text, which the server analyzes and calculates the route. The calculation results are then presented to the user, and the final route is set based on the user's response. The server monitors the facility situation in real time, and if the situation changes, it immediately suggests a new route.

[0999] Prompt Sentence Examples

[1000] User: "I want to go to the administration building and then to the warehouse."

[1001] Device: "Converting speech to text..."

[1002] Server: "Parsing destination and waypoints..."

[1003] This allows the system of the present invention to efficiently and safely patrol facilities using voice input, significantly improving the efficiency and effectiveness of security operations.

[1004] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1005] Step 1:

[1006] The user inputs the destination and stopover points into the voice input device, for example, by saying, "I want to go to the administration building and then check the warehouse." The input is the user's voice data, which the device receives and sends to the server.

[1007] Step 2:

[1008] The device converts the captured voice data into text data using speech recognition software such as the Google Speech-to-Text API or the Python speech_recognition library. The output is the converted text data, which is then sent to the server.

[1009] Step 3:

[1010] The server analyzes the received text data using natural language processing software to extract destinations and intermediate points. This analysis uses natural language processing libraries such as NLTK and spaCy. The input is text data, and the output is the extracted destinations and intermediate points.

[1011] Step 4:

[1012] The server sends the extracted destinations and intermediate points to a navigation API, which calculates the optimal route. The navigation API uses the Google Maps API, etc. The input is the destination and intermediate points, and the output is the calculated route.

[1013] Step 5:

[1014] The server generates a message to propose the calculated route and stop information to the user and sends it to the terminal. The input is the calculated tour route, and the output is the proposal message.

[1015] Step 6:

[1016] The device then plays the received suggestion message aloud and asks the user for confirmation. Specifically, it uses a generative AI model to read the suggestion aloud in a natural voice. The input is the suggestion message, and the output is the spoken suggestion.

[1017] Step 7:

[1018] The user responds to the proposed route by voice, for example, saying, "Yes, that's it." The input is the user's voice instruction, and the device converts the voice data back into text and sends it to the server.

[1019] Step 8:

[1020] The server sets the final route based on the user's response. The input is the user's text data, and the output is the final route.

[1021] Step 9:

[1022] The server monitors facility and traffic conditions in real time. If any changes occur, the server recalculates the route using the new information and sends a new proposal message to the terminal. The input is real-time situation data, and the output is a recalculated tour route and a new proposal message.

[1023] Step 10:

[1024] The terminal plays the new suggestion message aloud and asks the user for confirmation. Again, based on the user's response, the server adjusts the route and performs the optimal tour. The input is the suggestion message, and the output is the adjusted final route.

[1025] This enables real-time facility patrols via voice input, improving the efficiency and safety of security services.

[1026] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1027] overview

[1028] This invention relates to a voice-input navigation system that incorporates a function to recognize user emotions to provide more personalized route suggestions and user experience. Users can specify destinations and intermediate stops by voice, and the system can not only adjust the route based on changes in traffic conditions, but also adapt the suggestions based on the user's emotions.

[1029] Program processing

[1030] System configuration

[1031] The system mainly consists of the following elements:

[1032] 1. Voice input devices (smartphones and car navigation devices)

[1033] 2. Server (voice analysis, emotion recognition, and navigation data processing)

[1034] 3. Navigation API (e.g., map services)

[1035] 4. Emotion engine (recognizes user emotions from voice input)

[1036] 1. Voice input capture and emotion recognition

[1037] User: The user gives instructions to the voice input device, such as, "Set Hotel X as the destination. I'd like to eat ramen on the way around 1:00 p.m."

[1038] Device: Calls the speech recognition API to convert voice input into text data. Retrieves the converted text data.

[1039] Terminal: Sends text data and voice data itself to the server.

[1040] Server: The received text data is input into ChatGPT, and the destination and intermediate points are extracted using natural language analysis.

[1041] Server: Uses an emotion engine to recognize the user's emotions from the voice data. For example, it determines whether the user is anxious or relaxed based on the tone and speed of the voice.

[1042] 2. Route calculation and emotion-based suggestion adaptation

[1043] Server: Uses the navigation API to calculate the optimal route based on the extracted destination and intermediate destinations, taking real-time traffic information into account.

[1044] Server: Adjusts the recommendations based on the user's emotions as recognized by the emotion engine. For example, if the user is determined to be tired, the server will suggest more relaxing rest areas or stopovers.

[1045] Server: Generates a message to suggest to the user based on the calculation results and adjustments based on emotions. For example, it could say, "We suggest a route to the destination hotel, stopping at the ramen shop around 1:00 PM. We also suggest places to rest and refresh yourself."

[1046] Server: Sends a proposal message to the terminal.

[1047] 3. Confirm the proposal and set the final route

[1048] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[1049] User: Review the proposed route and respond with a response such as "Yes, that's fine."

[1050] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[1051] Server: Confirms user consent and finalizes the route, which includes detailed navigation instructions.

[1052] Server: Sends the confirmed route data to the terminal.

[1053] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[1054] 4. Traffic monitoring and route correction

[1055] Server: Monitors traffic condition data in real time. Uses a dedicated API to periodically obtain information on congestion, accidents, etc.

[1056] Server: Determines whether there are any changes in traffic conditions that will affect the currently configured route.

[1057] Server: If a change in traffic conditions is detected, calculate a new route, resetting intermediate stops if necessary.

[1058] Server: The emotion engine re-identifies the user's emotions and adjusts the suggestions for new routes and stops.

[1059] Server: Generate a message containing a new route or stopover suggestion, such as "We are currently experiencing traffic congestion, so we suggest a revised route. We also recommend changing the rest stop to ____."

[1060] Server: Sends a proposal message to the terminal.

[1061] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[1062] User: Review the new proposal and respond, "Yes, I'd like the revised route."

[1063] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[1064] Server: Confirms user consent and confirms the new route, which also includes detailed navigation instructions.

[1065] Server: Sends new route data to the device.

[1066] Device: Continues navigation based on the new route, providing visual and audio guidance.

[1067] Specific examples

[1068] Example 1: Destination setting and emotion recognition

[1069] 1. User: Says to the device, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1pm."

[1070] 2. Device: Converts voice input into text and sends it to the server.

[1071] 3. Server: Analyzes the text data and extracts the destination and intermediate points.

[1072] 4. Server: Using the emotion engine, recognize the user's emotion from the voice data. For example, it determines that the user is tired.

[1073] 5. Server: Calculates the optimal route using the navigation API.

[1074] 6. Server: Adjust the recommendations based on the user's emotions. Create recommendations that include relaxing stops and rest areas.

[1075] 7. Server: Generates a message proposing the calculation result and sends it to the terminal.

[1076] 8. Terminal: Play the proposed message aloud and ask the user for confirmation.

[1077] 9. User: Review the proposed route and respond, "Yes, that's fine."

[1078] 10. Server: Determines the final route and sends navigation data to the device.

[1079] Example 2: Route adjustments based on traffic changes and sentiment

[1080] 1. Server: Monitors traffic conditions and obtains new information.

[1081] 2. Server: Calculate a new route and recognize the user's emotions again using the emotion engine.

[1082] 3. Server: Generates messages with new suggestions and tailors them with sentiment-based content.

[1083] 4. Terminal: Play the proposed message aloud and ask the user for confirmation.

[1084] 5. User: Agrees with the new suggestion and responds, "Change to XX Restaurant."

[1085] 6. Server: Determines the new route and sends navigation data to the device.

[1086] 7. On the device: Continue navigation based on the new route.

[1087] As a result, the system of the present invention recognizes the user's emotions and suggests appropriate routes and stopovers accordingly, thereby providing safe and convenient navigation.

[1088] The processing flow will be explained below.

[1089] Processing steps of a navigation system including an emotion engine

[1090] 1. Destination and route setting and emotion recognition processing

[1091] Step 1:

[1092] User: Gives instructions to the voice input device, such as "Set Hotel X as my destination. I'd like to eat ramen on the way around 1pm."

[1093] Step 2:

[1094] Device: Calls the speech recognition API to convert voice input into text data. Retrieves the converted text data.

[1095] Step 3:

[1096] Terminal: Transmits the converted text data and the audio data itself to the server.

[1097] Step 4:

[1098] Server: The received text data is input to ChatGPT for analysis. ChatGPT extracts the destination and intermediate points from the text data.

[1099] Step 5:

[1100] Server: Uses an emotion engine to recognize the user's emotions from the voice data, for example, determining whether the user is tired or relaxed from the tone and speed of the voice.

[1101] Step 6:

[1102] Server: Uses the navigation API to calculate the optimal route based on the extracted destinations and intermediate destinations, including real-time traffic information.

[1103] Step 7:

[1104] Server: Adjusts the recommendations based on the user's perceived emotions. For example, if the user is tired, it suggests more relaxing rest stops or stopovers.

[1105] Step 8:

[1106] Server: Generates a message containing the calculated route and sentiment-based suggestions. For example, "We suggest a route with the destination being the XX hotel, stopping at the XX ramen shop around 1:00 PM. Considering recent fatigue, we also recommend rest stops along the way."

[1107] Step 9:

[1108] Server: Sends the generated proposal message to the terminal.

[1109] Step 10:

[1110] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[1111] Step 11:

[1112] User: Review the proposed route and respond "Yes, that's fine."

[1113] Step 12:

[1114] Terminal: The user's response is converted into text again using the speech recognition API and sent to the server.

[1115] Step 13:

[1116] Server: Confirms user consent and finalizes the route, including detailed navigation instructions.

[1117] Step 14:

[1118] Server: Sends the confirmed route data to the terminal.

[1119] Step 15:

[1120] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[1121] 2. Monitoring traffic conditions and handling route corrections

[1122] Step 1:

[1123] Server: Monitors traffic condition data in real time. Uses a dedicated API to periodically obtain information on congestion, accidents, etc.

[1124] Step 2:

[1125] Server: Determines whether there are any changes in traffic conditions that will affect the currently configured route.

[1126] Step 3:

[1127] Server: If a change in traffic conditions is detected, calculate a new route, resetting intermediate stops if necessary.

[1128] Step 4:

[1129] Server: Again, the emotion engine recognizes the user's emotions and adjusts the suggestions for new routes and stops.

[1130] Step 5:

[1131] Server: Generates a suggestion message that includes a new route and adjustments based on emotions. For example, "Currently, there is traffic congestion, so we suggest a revised route. Considering fatigue, we recommend changing the rest area to △△."

[1132] Step 6:

[1133] Server: Sends the generated proposal message to the terminal.

[1134] Step 7:

[1135] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[1136] Step 8:

[1137] User: Review the new proposal and respond, "Yes, I'd like the revised route."

[1138] Step 9:

[1139] Terminal: The user's response is converted into text again using the speech recognition API and sent to the server.

[1140] Step 10:

[1141] Server: Confirms user consent and confirms the new route, which also includes detailed navigation instructions.

[1142] Step 11:

[1143] Server: Sends new route data to the device.

[1144] Step 12:

[1145] Device: Continues navigation based on the new route, providing visual and audio guidance.

[1146] As a result, the system of the present invention recognizes the user's emotions, suggests appropriate routes and stopovers accordingly, and provides safe and convenient navigation.

[1147] Example 2

[1148] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1149] Conventional navigation systems are equipped with functions for recalculating routes and setting destinations in response to changes in traffic conditions, but they are unable to provide appropriate suggestions based on the user's emotional state, making it difficult to improve user satisfaction. Furthermore, even when setting a destination based on voice input, there is a lack of natural conversational responses, so there is a need to improve the user experience.

[1150] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for recognizing the user's emotion from voice data, means for adjusting the content of suggestions based on the recognized user's emotion, and means for monitoring changes in traffic conditions, recalculating the route in accordance with the changes in traffic conditions, and suggesting the route to the user. This enables personalized route suggestions that take the user's emotion into consideration and optimal route adjustments in real time.

[1151] "Means for converting voice input into text data" refers to technology or devices that process a user's voice and convert the voice data into text format data.

[1152] "Means for analyzing text data and voice data to extract destinations and intermediate points" refers to technology or devices that automatically identify specified destinations and intermediate points from voice-input text and voice data.

[1153] "Means for recognizing a user's emotions from voice data" refers to technology or devices that analyze characteristics of voice data such as tone, speed, and volume of the voice to determine the user's emotional state (for example, fatigue, tension, relaxation, etc.).

[1154] "Means for calculating the optimal route based on the destination and intermediate points" refers to technology or devices that calculate the optimal travel route based on the set destination and intermediate points, taking into account traffic information and map information.

[1155] "Means for adjusting the content of suggestions based on the recognized user emotions" refers to technology or devices that take into account the user's emotional state and adaptively change the suggested route and intermediate points.

[1156] "Means for suggesting calculated route and stopover information to users" refers to technology or devices that present information on optimal routes and stopovers to users.

[1157] "Means for setting a final route based on a user's response" refers to technology or devices that determine a final navigation route based on the user's confirmation or consent.

[1158] "Means for monitoring changes in traffic conditions, recalculating routes in response to the changes in traffic conditions, and proposing routes to users" refers to technologies and devices that monitor traffic information in real time, calculate new routes in response to changes in traffic conditions, and provide that information to users.

[1159] The navigation system of the present invention is composed of a voice input device, a server, a navigation API, and an emotion engine. This system converts the user's voice input into text data and extracts the destination and intermediate points from the text data. It also recognizes the user's emotions from the voice data and proposes the optimal route. This section specifies the hardware and software used and explains how the system specifically operates.

[1160] System configuration

[1161] 1. Voice input device: A voice input device such as a smartphone or car navigation terminal is used to obtain voice instructions from the user.

[1162] 2. Speech recognition API: Use Google Speech-to-Text or similar to convert voice data into text data.

[1163] 3. Text and voice data analysis: The server uses ChatGPT or a similar generative AI model to analyze the text data and extract destinations and intermediate stops.

[1164] 4. Emotion Recognition Engine: Uses IBM Watson and other emotion recognition technologies to recognize user emotions from voice data.

[1165] 5. Navigation API: Uses Google Maps API etc. to calculate the optimal route reflecting real-time traffic information.

[1166] Example of operation

[1167] Example 1: Destination setting and emotion recognition

[1168] 1. User: The user speaks into a voice input device, saying, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1:00 p.m."

[1169] 2. Device: Calls the speech recognition API to convert this voice input into text data.

[1170] 3. Terminal: The converted text data and audio data are sent to the server.

[1171] 4. Server: The server inputs the text data into ChatGPT and extracts the destination "XX Hotel" and the stopover "Ramen Shop" through natural language analysis.

[1172] 5. Server: Using the emotion engine, recognize the user's emotion from the voice data. For example, it determines that the user is tired.

[1173] 6. Server: Calculates the optimal route using the navigation API.

[1174] 7. Server: Adjust the recommendations based on the user's emotions and create recommendations that include relaxing stops and rest areas.

[1175] 8. Server: Sends the calculation result to the terminal as a proposal message.

[1176] 9. Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[1177] 10. User: Review the proposed route and respond, "Yes, that's fine."

[1178] 11. Terminal: The user's response is again converted into text using the speech recognition API and sent to the server.

[1179] 12. Server: Confirms user consent, finalizes the route, and sends the final route data to the device.

[1180] 13. Terminal: Starts navigation based on the received route data, and provides screen display and voice guidance.

[1181] Prompt Sentence Examples

[1182] Prompt example 1: Speech data text conversion and emotion recognition

[1183] "Voice data: 'I want to eat ramen at 1pm.' Convert this voice data into text and recognize the user's emotions."

[1184] Example prompt 2: Route calculation and adjustment

[1185] "Destination: 'X hotel, Shibuya-ku, Tokyo' Stop: 'X ramen shop' User's emotion: 'Tired' Calculate the optimal route based on this data and generate a suggested message including stopovers and rest areas where people can relax."

[1186] This allows the system of the present invention to propose a personalized route that takes into account the user's emotions and to adjust the route optimally in real time.

[1187] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1188] Step 1:

[1189] User: The user gives instructions to the voice input device, such as, "Set Hotel X as the destination. I'd like to eat ramen on the way around 1:00 p.m."

[1190] Specific behavior:

[1191] Users speak into the microphone on their smartphone or car navigation device.

[1192] Input: Audio data

[1193] Output: Audio data

[1194] Step 2:

[1195] On the device: Call a speech recognition API (such as Google Speech-to-Text) to convert the voice input into text data. Retrieve the converted text data.

[1196] Specific behavior:

[1197] Send the voice data to the voice recognition API and convert it into text data.

[1198] Keeps the converted text in memory.

[1199] Input: Audio data

[1200] Output: Text data

[1201] Step 3:

[1202] Terminal: Sends text data and voice data itself to the server.

[1203] Specific behavior:

[1204] Use an HTTP POST request to send text and audio data to a specified endpoint on the server.

[1205] Input: Text data, audio data

[1206] Output: Send data to the server

[1207] Step 4:

[1208] Server: The received text data is input into ChatGPT, and the destination and intermediate points are extracted using natural language analysis.

[1209] Specific behavior:

[1210] Call the ChatGPT API and send text data as input data.

[1211] Extract the destination "XX Hotel" and the transit point "△△ Ramen Shop" from the ChatGPT output.

[1212] Input: Text data

[1213] Output: Destination, intermediate destination

[1214] Step 5:

[1215] Server: Uses an emotion engine to recognize the user's emotions from the voice data, for example, determining whether the user is anxious or relaxed based on the tone and speed of the voice.

[1216] Specific behavior:

[1217] Extract features such as tone, speed, and volume of speech.

[1218] Feature data is input into the emotion engine to recognize emotions (e.g., "tired").

[1219] Input: Audio data

[1220] Output: Emotion data

[1221] Step 6:

[1222] Server: Uses the navigation API (Google Maps API) to calculate the optimal route based on the extracted destination and intermediate destinations, taking real-time traffic information into account.

[1223] Specific behavior:

[1224] Send a request to the Google Maps API to get the optimal route via "XX Hotel" and "△△ Ramen Shop."

[1225] Receive route data including real-time traffic information.

[1226] Input: Destination, stopover

[1227] Output: Optimal route data

[1228] Step 7:

[1229] Server: Adjusts the recommendations based on the user's emotions as recognized by the emotion engine. For example, if the user is determined to be tired, the server will suggest more relaxing rest areas or stopovers.

[1230] Specific behavior:

[1231] The output of the emotion engine will be analyzed, and an option will be added to suggest a place to relax if the user's emotion is recognized as "tired."

[1232] Input: Emotion data, optimal route data

[1233] Output: Adjusted proposal

[1234] Step 8:

[1235] Server: Generates a message to suggest to the user based on the calculation results and adjustments based on emotions.

[1236] Specific behavior:

[1237] By combining the navigation route and emotion data, a message such as "We suggest a route with the destination being the XX hotel, stopping at the △△ ramen shop around 1:00 p.m. We will also suggest a place to rest and refresh yourself" is generated.

[1238] Input: Adjusted proposal

[1239] Output: Proposal message

[1240] Step 9:

[1241] Server: Sends a proposal message to the terminal.

[1242] Specific behavior:

[1243] Send the generated message to the device using an HTTP POST request.

[1244] Input: Proposal message

[1245] Output: Sending a message to the terminal

[1246] Step 10:

[1247] Terminal: The proposed message is converted into speech using a speech synthesis API (e.g., Amazon Polly) and played back to the user.

[1248] Specific behavior:

[1249] The message text is sent to the speech synthesis API and converted into voice data.

[1250] The generated audio data is played back from the device's speaker.

[1251] Input: Proposal message

[1252] Output: Audio data

[1253] Step 11:

[1254] User: Review the proposed route and respond "Yes, that's fine."

[1255] Specific behavior:

[1256] The user responds to the system by saying, "Yes, that would be fine."

[1257] Input: Proposal message

[1258] Output: User's voice response

[1259] Step 12:

[1260] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[1261] Specific behavior:

[1262] Send the voice data to the voice recognition API and convert it into text data.

[1263] Sends text data to the server.

[1264] Input: User's voice response

[1265] Output: Text data

[1266] Step 13:

[1267] Server: Confirms user consent and finalizes the route, which includes detailed navigation instructions.

[1268] Specific behavior:

[1269] Receive the user's consent response and finalize the route.

[1270] Generate detailed route data including navigation instructions.

[1271] Input: User's text response

[1272] Output: Final route data

[1273] Step 14:

[1274] Server: Sends the confirmed route data to the terminal.

[1275] Specific behavior:

[1276] Send an HTTP POST request to send the confirmed route data to the device.

[1277] Input: Final route data

[1278] Output: Sending data to the terminal

[1279] Step 15:

[1280] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[1281] Specific behavior:

[1282] The received route data is analyzed and navigation begins.

[1283] Route instructions are provided on screen and via voice guidance.

[1284] Input: Final route data

[1285] Output: Navigation instructions (screen display, voice guide)

[1286] (Application example 2)

[1287] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1288] Conventional navigation systems only set destinations and intermediate stops based on user instructions and modify routes according to traffic conditions, so they were unable to provide personalized suggestions that took the user's emotional state into account.The present invention aims to provide more personalized route suggestions and user experiences by incorporating a function that recognizes the user's emotions into the navigation system, thereby reducing user stress and improving user satisfaction.

[1289] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for converting voice input from a user into text data, means for analyzing the text data to extract a destination and intermediate points, means for recognizing the user's emotions from the voice data, means for calculating an optimal route based on the destination and intermediate points, means for adapting the proposal content based on the user's emotions, means for proposing information on the calculated route and intermediate points to the user, means for setting a final route based on a response from the user, and means for monitoring changes in traffic conditions, recalculating a route in accordance with the changes in traffic conditions, and proposing the route to the user. This makes it possible to propose personalized routes and intermediate points according to the user's emotional state.

[1290] "Voice input data" refers to information provided by a user to give voice instructions to the navigation system.

[1291] "Text data" refers to character information converted from voice input data using voice recognition technology.

[1292] A "destination" is the final location that a user wants to reach using a navigation system.

[1293] A "waypoint" is a place where you plan to stop on the way to your destination.

[1294] "Emotion recognition method" is a technology that analyzes the user's emotional state from the tone, speed, and word choice of voice data, and identifies specific emotions.

[1295] An "optimal route" is the most efficient route to reach a destination calculated based on current traffic conditions and user settings.

[1296] "Suggested content" refers to information about routes, stops, rest spots, etc. that the navigation system presents to the user.

[1297] "Route recalculation" is the process of revising an existing route to accommodate new information based on changing traffic conditions and user responses.

[1298] The "final route" is the final route that reflects the user's responses and satisfies all conditions.

[1299] "Traffic condition monitoring" is a technology that monitors real-time data such as traffic congestion and accident information, and makes route adjustments as necessary.

[1300] "Rest Spots" are locations along the route that are suggested for users to take a rest along the way.

[1301] A "navigation system" is a system that integrates these elements and provides users with audio and visual guidance.

[1302] The present invention provides a more personalized route suggestion and user experience by incorporating a function that recognizes the user's emotions into a voice-based navigation system. The system of the present invention consists of the following main components:

[1303] System configuration

[1304] 1. Hardware:

[1305] Audio input devices (e.g. smartphones, microphones)

[1306] GPS sensor

[1307] Autonomous vehicle control computer

[1308] 2. Software:

[1309] Speech recognition API (e.g. Google API)

[1310] Emotion recognition engine (e.g. IBM Watson Tone Analyzer)

[1311] Navigation APIs (e.g. Google Maps API)

[1312] Server-side architecture (e.g., Node.js, Express server)

[1313] Database (e.g. MongoDB)

[1314] Speech synthesis API (e.g. Amazon Polly)

[1315] Generative AI API (e.g. OpenAI GPT)

[1316] Program processing

[1317] 1. Voice input capture and emotion recognition

[1318] The user gives instructions to the navigation system using a voice input device, for example, "Set the destination as Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1:00 PM."

[1319] The device converts the voice input into text data using a voice recognition API and sends it to the server.

[1320] The server receives the text and audio data itself, performs natural language analysis using ChatGPT, and extracts the destination and intermediate points.

[1321] Furthermore, an emotion recognition engine is used to recognize the user's emotions from the voice data, for example, determining whether the user is anxious or relaxed from the tone and speed of the voice.

[1322] 2. Route calculation and emotion-based suggestion adaptation

[1323] The server uses a navigation API to calculate the optimal route, taking real-time traffic information into account.

[1324] Data from the emotion recognition engine is used to tailor suggestions, for example, suggesting a route that includes relaxing rest stops for a tired user.

[1325] The server uses a generation AI to generate messages in natural language to suggest to users and sends them to the device.

[1326] 3. Confirm the proposal and set the final route

[1327] The server converts the generated proposal message into voice data using a voice synthesis API and sends it to the terminal.

[1328] The terminal plays the suggestion message aloud and asks the user for confirmation.

[1329] The user reviews the proposed route and responds with a response such as "Yes, that's fine."

[1330] The server receives the user's response and determines the final route.

[1331] The determined route data is sent to the terminal, providing route information to the autonomous vehicle's control computer.

[1332] 4. Traffic monitoring and route correction

[1333] The server monitors traffic data in real time and acquires new information.

[1334] It recalculates routes based on new traffic conditions and re-recognizes the user's emotional state.

[1335] The server generates and adjusts new proposals and sends them to the device.

[1336] The user reviews the new proposal and responds, and the server, upon approval, finalizes the new route and sends it back to the autonomous vehicle's control computer.

[1337] Specific examples

[1338] Navigation for users who feel fatigued

[1339] Example prompt: "I'm tired and would like to stop by a cafe or park where I can relax."

[1340] Server processing: The server determines that the user is tired and calculates a route that includes places where the user can refresh themselves.

[1341] Suggestion example: As a new route, suggest "We suggest stopping at the cafe in Shizuoka Parking Area on the way. You can take a break there."

[1342] The above is a specific embodiment for implementing the system of the present invention, which recognizes the user's emotions and enables personalized navigation routes and suggestions based on those emotions.

[1343] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1344] Step 1:

[1345] The user uses a voice input device to specify destinations and stopovers by voice. For example, the user might say, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1:00 PM."

[1346] Input: User's voice command

[1347] Output: Audio data

[1348] Specific operation: The voice input device acquires user instructions as voice data through a microphone.

[1349] Step 2:

[1350] The device converts the voice data into text data using a voice recognition API and sends it to the server.

[1351] Input: Audio data

[1352] Output: Text data

[1353] Specific operation: The device calls a speech recognition API (e.g., Google API) and converts the voice data into text data. The converted text data is then obtained and sent to the server via an HTTP request or similar.

[1354] Step 3:

[1355] The server receives the text data and voice data and extracts the destination and intermediate points using ChatGPT.

[1356] Input: Text data, audio data

[1357] Output: Destination and waypoint information

[1358] Specific operation: The server inputs the received text data into ChatGPT and performs natural language analysis, extracting the destination and intermediate points and storing them in a database.

[1359] Step 4:

[1360] The server uses an emotion recognition engine to recognize the user's emotion from the voice data.

[1361] Input: Audio data

[1362] Output: User's emotional state

[1363] Specific operation: The server inputs the voice data into an emotion recognition engine (e.g., IBM Watson Tone Analyzer) and analyzes the tone and speed of the voice to identify the user's emotion.

[1364] Step 5:

[1365] The server uses a navigation API to calculate the optimal route, taking real-time traffic information into account.

[1366] Input: Destination information, route information, traffic information

[1367] Output: Optimal route

[1368] Specific operation: The server calls a navigation API (e.g., Google Maps API) and calculates the optimal route based on the destination, intermediate destinations, and current traffic conditions.

[1369] Step 6:

[1370] The server adapts the suggestions based on the user's emotions.

[1371] Input: User's emotional state, optimal route

[1372] Output: Personalized recommendations

[1373] Specific operation: Based on the results of the emotion recognition engine, for example, the server sets a route including rest spots for a user who feels tired, and generates a suggested message using a generation AI.

[1374] Step 7:

[1375] The server sends a proposal message to the device, which converts the proposal message into voice data using a speech synthesis API and plays it back to the user.

[1376] Input: Proposal message

[1377] Output: Audio data, audio playback

[1378] Specific operation: The server sends the generated proposal message to the device via an HTTP request. The device converts the received message into voice data using a speech synthesis API (e.g., Amazon Polly) and plays it back to the user through the speaker.

[1379] Step 8:

[1380] The user confirms the proposed route and responds with a voice response such as "Yes, that's fine."

[1381] Input: Voice response

[1382] Output: Audio data

[1383] Specific actions: After listening to the proposed route, if the user agrees, they give instructions to the voice input device again.

[1384] Step 9:

[1385] The device converts the voice data into text data again using the voice recognition API and sends it to the server.

[1386] Input: Audio data

[1387] Output: Text data

[1388] Specific operation: The device calls the speech recognition API again, converts the voice data into text data, and sends the converted text data to the server.

[1389] Step 10:

[1390] The server receives the user's response, determines the final route, and sends the determined route data to the device to start navigation.

[1391] Input: User consent (text data)

[1392] Output: Final route data

[1393] Specific operation: The server confirms the user's consent, determines the final route using the navigation API, sends the determined route data to the device, and starts navigation.

[1394] Step 11:

[1395] The server monitors traffic data in real time, recalculates the route according to the new traffic conditions, and also recognizes the user's emotional state again.

[1396] Input: Real-time traffic data

[1397] Output: The modified route

[1398] Specific operation: The server periodically calls the traffic information API to obtain data, and if there is any impact on the existing route, it recalculates it. At the same time, it uses the emotion recognition engine to re-determine the user's emotional state.

[1399] Step 12:

[1400] The server generates a new proposal and sends it to the terminal, which plays it back to the user in the same way and asks for confirmation.

[1401] Input: New proposal

[1402] Output: Audio data, audio playback

[1403] How it works: The server uses the generation AI to generate new suggestions and sends them to the device via an HTTP request. The device then converts the received message into audio data using a speech synthesis API and plays it back to the user.

[1404] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1405] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1406] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1407] [Third embodiment]

[1408] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1409] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1410] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1411] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1412] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1413] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1414] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1415] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1416] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1417] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1418] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1419] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1420] overview

[1421] The present invention relates to a navigation system that uses voice input. Users can set destinations and intermediate points through voice, and can smoothly adjust routes and rest areas according to changes in traffic conditions. This system supports users in traveling safely and conveniently.

[1422] Program processing

[1423] System configuration

[1424] The system mainly consists of the following elements:

[1425] 1. Voice input devices (smartphones and car navigation devices)

[1426] 2. Server (analysis and navigation data processing)

[1427] 3. Navigation API (e.g., map services)

[1428] 1. Acquiring voice input

[1429] User: First, the user speaks to the voice input device about their destination and stopovers. For example, they might say, "Set the destination to Hotel X as my destination. I'd like to eat ramen on the way around 1:00 PM."

[1430] Terminal: The terminal converts this voice input into text data and sends the text data to the server.

[1431] 2.Analysis of voice data and route calculation

[1432] Server: The server analyzes the received text data and extracts the destination and intermediate points. This analysis uses natural language processing technology. Specifically, it automatically extracts information about location and time from the text.

[1433] The server then uses a navigation API to calculate the optimal route based on the destination and intermediate stops, taking real-time traffic information into account.

[1434] Server: The server generates a message to suggest the calculated route and stopover information to the user and sends it to the terminal.

[1435] 3. Confirm the proposal and set the final route

[1436] Terminal: The terminal plays the received proposal message as audio and asks the user for confirmation.

[1437] User: The user responds to the proposed stopovers and route with a response such as "Yes, that's fine."

[1438] Terminal: Convert this response back into text and send it to the server.

[1439] Server: The server verifies the user's consent and sets the final route, which includes detailed navigation instructions.

[1440] 4. Traffic monitoring and route recalculation

[1441] Server: The server monitors traffic conditions in real time and calculates a new route if there are changes in traffic conditions along the specified route.

[1442] Server: Generates messages to propose new routes and stops and sends them to the device.

[1443] Terminal: The terminal will play the new suggestion aloud and ask the user for confirmation.

[1444] User: The user responds to the new suggestion, for example by giving instructions such as "Change to XX Restaurant."

[1445] Server: The server determines the new route and sends it to the device, which then continues navigation according to the new route.

[1446] Specific examples

[1447] Example 1: Setting a destination

[1448] 1. User: Says to the device, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1pm."

[1449] 2. Device: Converts voice input into text and sends it to the server.

[1450] 3. Server: Analyzes the text data, extracts the destination and intermediate points, and calculates the optimal route using the navigation API.

[1451] 4. Server: Generates a message proposing the calculation result and sends it to the terminal.

[1452] 5. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[1453] 6. User: Agrees with the proposal and responds, "Yes, I'd like that."

[1454] 7. Server: Determines the final route and sends navigation data to the device.

[1455] Example 2: Route adjustment due to changing traffic conditions

[1456] 1. Server: Monitors traffic conditions and obtains new information.

[1457] 2. Server: Calculates new routes and generates proposal messages.

[1458] 3. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[1459] 4. User: Agrees with the new suggestion and responds, "Change to XX Restaurant."

[1460] 5. Server: Determines the new route and sends navigation data to the device.

[1461] 6. Device: Continue navigation according to the new route.

[1462] As a result, the system of the present invention allows the user to set the destination and intermediate points by voice and responds to changes in traffic conditions, providing safe and convenient navigation.

[1463] The processing flow will be explained below.

[1464] Specific explanation of program processing

[1465] 1. Destination and intermediate destination setting process

[1466] Step 1:

[1467] User: Gives instructions to the voice input device, such as "Set Hotel X as my destination. I'd like to eat ramen on the way around 1pm."

[1468] Step 2:

[1469] Device: Calls the speech recognition API to convert voice input into text data. Retrieves the converted text data.

[1470] Step 3:

[1471] Terminal: Send text data to the server using an API endpoint.

[1472] Step 4:

[1473] Server: The received text data is input into ChatGPT, and the destination and intermediate points are extracted using natural language analysis.

[1474] Step 5:

[1475] Server: Uses a navigation API (e.g., Google Maps API) to calculate the optimal route based on the extracted destination and intermediate destinations.

[1476] Step 6:

[1477] Server: Based on the calculation results, a message is generated to be suggested to the user. It prepares a message such as "Suggested route to destination XX Hotel, stopping at XX ramen shop around 1 PM."

[1478] Step 7:

[1479] Server: Sends a proposal message to the terminal.

[1480] Step 8:

[1481] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[1482] Step 9:

[1483] User: Review the proposed route and respond "Yes, that's fine."

[1484] Step 10:

[1485] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[1486] Step 11:

[1487] Server: Confirms user consent and finalizes the route, which includes detailed navigation instructions.

[1488] Step 12:

[1489] Server: Sends the confirmed route data to the terminal.

[1490] Step 13:

[1491] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[1492] 2. Monitoring traffic conditions and handling route corrections

[1493] Step 1:

[1494] Server: Monitors traffic condition data in real time. Uses a dedicated API to periodically obtain information on congestion, accidents, etc.

[1495] Step 2:

[1496] Server: Determines whether there are any changes in traffic conditions that will affect the currently configured route.

[1497] Step 3:

[1498] Server: If a change in traffic conditions is detected, calculate a new route, resetting intermediate stops if necessary.

[1499] Step 4:

[1500] Server: Generate a message containing a new route or stopover suggestion. For example, "There is currently a traffic jam, so we suggest a revised route."

[1501] Step 5:

[1502] Server: Sends a proposal message to the terminal.

[1503] Step 6:

[1504] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[1505] Step 7:

[1506] User: Review the new proposal and respond, "Yes, I'd like the revised route."

[1507] Step 8:

[1508] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[1509] Step 9:

[1510] Server: Confirms user consent and confirms the new route, which also includes detailed navigation instructions.

[1511] Step 10:

[1512] Server: Sends new route data to the device.

[1513] Step 11:

[1514] Device: Continues navigation based on the new route, providing visual and audio guidance.

[1515] Example 1

[1516] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1517] Conventional navigation systems require a lot of manual input to set destinations and intermediate points, and it is also difficult to flexibly recalculate routes to accommodate real-time traffic conditions. This requires cumbersome operations for users, which reduces safety and convenience. Since manual input while driving is particularly dangerous, there is a demand for systems that utilize voice input.

[1518] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1519] In this invention, the server includes means for capturing a user's voice input and converting it into text data using voice recognition software, means for extracting a destination and intermediate points from the text using natural language processing technology, means for calculating a route taking real-time traffic information into consideration using a navigation API, means for playing a suggested message aloud using voice synthesis software and requesting confirmation from the user, and means for sending a final route instruction to the terminal. This allows the user to easily set a destination and intermediate points through voice input and enables flexible route changes according to real-time traffic information.

[1520] "Voice input" is a means of acquiring the words spoken by a user as digital signals.

[1521] The "means for converting into text data" refers to software or hardware for converting acquired voice input into text information.

[1522] "Means for analyzing text data" refers to technologies or systems for deciphering converted text data and extracting specific information.

[1523] "Destination and stopover points" refers to the final destination set by the user and points to be passed along the way.

[1524] The "means for calculating the optimal route" is a system that calculates the optimal route based on the destination and intermediate points, taking into account conditions such as time, distance, and traffic conditions.

[1525] "Means of providing suggestions to users" refers to the methods and techniques used to communicate calculated route information to users, and may involve voice or text.

[1526] The "means for setting the final route" is a system for determining the final route based on the user's response.

[1527] "Means for monitoring changes in traffic conditions" refers to a system for collecting traffic information in real time and monitoring the situation.

[1528] The "means for recalculating the route" is a system for reevaluating the originally calculated route in response to changes in traffic conditions and calculating a new optimal route.

[1529] "Speech recognition software" is software technology for converting voice input into text data.

[1530] "Natural language processing technology" is a technology for deciphering human language and extracting specific information.

[1531] A "navigation API" is an application program interface for calculating routes and obtaining traffic information using map services, etc.

[1532] "Speech synthesis software" is a technology for reproducing text data as speech.

[1533] System configuration

[1534] The navigation system consists of the following elements:

[1535] 1. Voice input devices (e.g. smartphones, car navigation devices)

[1536] 2. Server (processes analysis and navigation data)

[1537] 3. Navigation API (e.g., map service)

[1538] Program processing

[1539] Acquiring voice input

[1540] The user speaks to the voice input device about the destination and stopovers, for example, "Set the destination to XX Hotel. I'd like to eat ramen on the way around 1 PM."

[1541] Audio data conversion

[1542] The device captures the user's voice, converts it into text data using speech recognition software (e.g., Google Speech-to-Text API), and sends the text data to the server.

[1543] Analysis of audio data

[1544] The server analyzes the received text data and extracts the destination and intermediate points. This analysis uses natural language processing technology (e.g., SpaCy, NLTK). The server automatically extracts the location and time information specified by the user from the text.

[1545] Route calculation

[1546] The server uses a navigation API (e.g., Google Maps API) to calculate the optimal route based on the destination and intermediate stops, taking real-time traffic information into account.

[1547] Generate and send a proposal message

[1548] The server generates a proposal message based on the calculated route and stopover information and sends it to the terminal.

[1549] Playing the suggestion message

[1550] The device uses voice synthesis software (e.g., Google Text-to-Speech API) to play the received proposal message aloud and ask the user for confirmation.

[1551] User Response

[1552] The user responds to the proposed stopovers and route by saying something like "Yes, that's fine."

[1553] Sending a Response

[1554] The terminal converts the user's response back into text and sends it to the server.

[1555] Setting the final route

[1556] The server verifies the user's consent and sets the final route, which includes detailed navigation instructions.

[1557] Traffic monitoring and route recalculation

[1558] The server monitors traffic conditions in real time, calculates a new route if the conditions change, and generates a message proposing a new route and intermediate destinations, which is then sent to the device.

[1559] Specific examples

[1560] Setting a destination

[1561] 1. User: "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1pm."

[1562] 2. Device: Converts voice input into text and sends it to the server.

[1563] 3. Server: Analyzes the text data, extracts the destination and intermediate points, and calculates the optimal route using the navigation API.

[1564] 4. Server: Generates a message proposing the calculation result and sends it to the terminal.

[1565] 5. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[1566] 6. User: "Yes, that's fine."

[1567] 7. Server: Determines the final route and sends navigation data to the device.

[1568] Route adjustments due to changing traffic conditions

[1569] 1. Server: Monitors traffic conditions and obtains new information.

[1570] 2. Server: Calculates new routes and generates proposal messages.

[1571] 3. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[1572] 4. User: Responds, "Change to XX Restaurant."

[1573] 5. Server: Determines the new route and sends navigation data to the device.

[1574] 6. Device: Continue navigation according to the new route.

[1575] Example prompt sentence:

[1576] "Set the destination as Hotel X. I'd like to stop by for ramen around 1pm on the way."

[1577] "Change it to XX Restaurant."

[1578] This system allows users to set their destination and route by voice and responds to changes in traffic conditions, making navigation safer and more convenient.

[1579] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1580] Step 1: Getting voice input

[1581] The user speaks to the voice input device, saying, "Set the destination as Hotel X. I'd like to eat ramen on the way around 1pm." This acquires voice data.

[1582] Input: User's voice

[1583] Output: Audio data

[1584] Step 2: Convert the audio data

[1585] The device converts the acquired voice data into text data using voice recognition software (e.g., Google Speech-to-Text API), and then sends the converted text data to the server.

[1586] Input: Audio data

[1587] Output: Text data

[1588] Step 3: Analyzing the text data

[1589] The server analyzes the received text data and uses natural language processing technology (e.g., SpaCy, NLTK) to extract the destination "XX Hotel," the stopover "Ramen Shop," and the time "1:00 PM."

[1590] Input: Text data

[1591] Output: Analysis results (destination, intermediate points, time)

[1592] Step 4: Calculate the route

[1593] The server uses a navigation API (e.g., Google Maps API) to calculate the optimal route based on the destination and intermediate destinations, taking into account real-time traffic information.

[1594] Input: Analysis results (destination, transit points, time) and traffic information

[1595] Output: Calculated route information

[1596] Step 5: Generate and send a proposal message

[1597] The server generates a proposal message based on the calculated route and stopover information and sends it to the terminal.

[1598] Input: Calculated route information

[1599] Output: Proposal message

[1600] Step 6: Play the suggestion message

[1601] The device uses voice synthesis software (e.g., Google Text-to-Speech API) to play the received proposal message aloud and ask the user for confirmation.

[1602] Input: Proposal message

[1603] Output: Spoken suggestions

[1604] Step 7: User response

[1605] The user responds to the proposed route and stop by saying something like, "Yes, that's fine." This response indicates the user's intent.

[1606] Input: Voice response

[1607] Output: User response data

[1608] Step 8: Sending response data

[1609] The terminal converts the user's response back into text and sends it to the server.

[1610] Input: User response data (voice)

[1611] Output: User response data (text)

[1612] Step 9: Setting the Final Route

[1613] The server verifies the user's consent and sets the final route, which includes detailed navigation instructions.

[1614] Input: User response data

[1615] Output: Final route setting data

[1616] Step 10: Sending navigation data

[1617] The server sends the final route configuration to the device.

[1618] Input: Final route setting data

[1619] Output: Navigation data

[1620] Step 11: Traffic monitoring

[1621] The server monitors traffic conditions in real time and checks for changes in the situation, using traffic data providers (e.g., Waze Traffic API).

[1622] Input: Real-time traffic information

[1623] Output: Monitoring results

[1624] Step 12: Calculate the new route

[1625] The server calculates a new route if traffic conditions change along the specified route.

[1626] Input: Monitoring results

[1627] Output: New route information

[1628] Step 13: Generate and send a new proposal message

[1629] The server generates a proposal message based on the new route information and sends it to the terminal.

[1630] Input: New route information

[1631] Output: New proposal message

[1632] Step 14: Playing a new proposal

[1633] The terminal will play the new suggestion message aloud and ask the user for confirmation.

[1634] Input: New proposed message

[1635] Output: New audio suggestions

[1636] Step 15: User's new response

[1637] The user responds to the new suggestion by saying something like, "Change to XX Restaurant."

[1638] Input: New spoken response

[1639] Output: New response data

[1640] Step 16: Send new response data

[1641] The terminal converts the new response into text and sends it to the server.

[1642] Input: New response data (audio)

[1643] Output: New response data (text)

[1644] Step 17: Confirm the new route

[1645] The server determines a new route based on the user's new instructions and transmits navigation data to the terminal.

[1646] Input: New response data

[1647] Output: New final route data

[1648] Step 18: Continued Navigation

[1649] The device continues navigation according to the new final route data.

[1650] Input: New final route data

[1651] Output: Updated navigation information

[1652] In this way, the system allows for voice input, destination and intermediate destination settings, and even flexible route recalculation based on real-time traffic information.

[1653] (Application example 1)

[1654] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1655] Conventional navigation systems have the problem of being unable to quickly respond to changes in real-time facility conditions and checkpoints in security services and facility management. Particularly in large facilities or locations with complex layouts, manually understanding these changes and setting optimal patrol routes takes time and effort, so there is a need for improved efficiency and safety.

[1656] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1657] In this invention, the server includes means for converting voice input from the user into text data, means for analyzing the text data to extract a destination and intermediate points, and means for calculating an optimal tour route based on the destination and intermediate points, thereby enabling efficient and safe tour route planning in response to changes in facility conditions and checkpoints in real time.

[1658] "Means for converting voice input into text data" refers to a device or program that converts a user's voice instructions into text data using voice recognition technology.

[1659] "Means for analyzing text data and extracting destinations and intermediate points" refers to devices or programs that have the function of identifying the destination and intermediate points specified by the user from the converted text data using natural language processing technology.

[1660] "Means for calculating the optimal travel route based on the destination and intermediate points" refers to a device or program that calculates the optimal travel route for a specified destination and intermediate points, taking into account current traffic information and conditions within the facility.

[1661] "Means for proposing calculated route and route information to users" refers to devices or programs that have the function of presenting calculated route and route information to users in the form of confirmation and suggestions.

[1662] "Means for setting a final route based on a response from the user" refers to a device or program that finalizes a proposed route based on voice or input data from the user and reflects it in the navigation.

[1663] "Means for monitoring changes in facility conditions, recalculating routes in response to those changes, and proposing new routes to users" refers to devices or programs that have the function of detecting real-time changes in conditions inside and outside the facility, recalculating routes based on those changes, and proposing new routes to users.

[1664] "Means for adjusting the timing of checkpoints and intermediate destinations and suggesting them to users" refers to devices or programs that have the function of dynamically reconfiguring checkpoints and intermediate destinations to be reached during a patrol in response to changes in facility conditions and suggesting them to users.

[1665] As an embodiment of the present invention, a voice-input navigation system for security services is proposed. The system of the present invention realizes voice-input instructions and optimizes patrol routes in response to real-time changes in facility conditions.

[1666] Hardware and Software Configuration

[1667] Hardware:

[1668] Audio input device (e.g. smartphone, Bluetooth microphone)

[1669] Server (real-time processing and data analysis)

[1670] Network communication devices (Wi-Fi, cellular networks)

[1671] software:

[1672] Speech recognition software (e.g., Google Speech-to-Text API, Python's speech_recognition library)

[1673] Natural language processing software (e.g., NLTK, spaCy)

[1674] Navigation APIs (e.g. Google Maps API)

[1675] Processing method

[1676] Speech to text:

[1677] First, the user gives instructions for the patrol into the voice input device. For example, they might say, "I'd like to go to the administration building and then check the warehouse." The device picks up this voice and converts it into text data using voice recognition software.

[1678] Analyzing text data:

[1679] The server then analyzes the converted text data to extract destinations and stopovers using natural language processing software, which automatically identifies destinations and stopovers from specific keywords and phrases.

[1680] Route optimization and calculation:

[1681] The server uses a navigation API to calculate the optimal route based on the extracted destinations and intermediate stops, taking into account real-time traffic information and in-facility situation information.

[1682] Suggestions for users:

[1683] The calculated route and route information are sent from the server to the device, which then proposes it to the user via voice. The user can respond to the proposal by saying something like "Yes, that's fine." If the user responds in any other way, the device will propose and correct the route again.

[1684] Real-time monitoring:

[1685] The server constantly monitors the facility status and recalculates the route if there are any changes, and as a result, it re-proposes new routes and checkpoints to the user and modifies the tour route as necessary.

[1686] Specific examples

[1687] Example 1: Facility patrol during business hours

[1688] The user gives voice instructions for the patrol route. For example, "I want to go to the administration building, then move to the warehouse." The device converts the voice into text, which the server analyzes and calculates the route. The calculation results are then presented to the user, and the final route is set based on the user's response. The server monitors the facility situation in real time, and if the situation changes, it immediately suggests a new route.

[1689] Prompt Sentence Examples

[1690] User: "I want to go to the administration building and then to the warehouse."

[1691] Device: "Converting speech to text..."

[1692] Server: "Parsing destination and waypoints..."

[1693] This allows the system of the present invention to efficiently and safely patrol facilities using voice input, significantly improving the efficiency and effectiveness of security operations.

[1694] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1695] Step 1:

[1696] The user inputs the destination and stopover points into the voice input device, for example, by saying, "I want to go to the administration building and then check the warehouse." The input is the user's voice data, which the device receives and sends to the server.

[1697] Step 2:

[1698] The device converts the captured voice data into text data using speech recognition software such as the Google Speech-to-Text API or the Python speech_recognition library. The output is the converted text data, which is then sent to the server.

[1699] Step 3:

[1700] The server analyzes the received text data using natural language processing software to extract destinations and intermediate points. This analysis uses natural language processing libraries such as NLTK and spaCy. The input is text data, and the output is the extracted destinations and intermediate points.

[1701] Step 4:

[1702] The server sends the extracted destinations and intermediate points to a navigation API, which calculates the optimal route. The navigation API uses the Google Maps API, etc. The input is the destination and intermediate points, and the output is the calculated route.

[1703] Step 5:

[1704] The server generates a message to propose the calculated route and stop information to the user and sends it to the terminal. The input is the calculated tour route, and the output is the proposal message.

[1705] Step 6:

[1706] The device then plays the received suggestion message aloud and asks the user for confirmation. Specifically, it uses a generative AI model to read the suggestion aloud in a natural voice. The input is the suggestion message, and the output is the spoken suggestion.

[1707] Step 7:

[1708] The user responds to the proposed route by voice, for example, saying, "Yes, that's it." The input is the user's voice instruction, and the device converts the voice data back into text and sends it to the server.

[1709] Step 8:

[1710] The server sets the final route based on the user's response. The input is the user's text data, and the output is the final route.

[1711] Step 9:

[1712] The server monitors facility and traffic conditions in real time. If any changes occur, the server recalculates the route using the new information and sends a new proposal message to the terminal. The input is real-time situation data, and the output is a recalculated tour route and a new proposal message.

[1713] Step 10:

[1714] The terminal plays the new suggestion message aloud and asks the user for confirmation. Again, based on the user's response, the server adjusts the route and performs the optimal tour. The input is the suggestion message, and the output is the adjusted final route.

[1715] This enables real-time facility patrols via voice input, improving the efficiency and safety of security services.

[1716] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1717] overview

[1718] This invention relates to a voice-input navigation system that incorporates a function to recognize user emotions to provide more personalized route suggestions and user experience. Users can specify destinations and intermediate stops by voice, and the system can not only adjust the route based on changes in traffic conditions, but also adapt the suggestions based on the user's emotions.

[1719] Program processing

[1720] System configuration

[1721] The system mainly consists of the following elements:

[1722] 1. Voice input devices (smartphones and car navigation devices)

[1723] 2. Server (voice analysis, emotion recognition, and navigation data processing)

[1724] 3. Navigation API (e.g., map services)

[1725] 4. Emotion engine (recognizes user emotions from voice input)

[1726] 1. Voice input capture and emotion recognition

[1727] User: The user gives instructions to the voice input device, such as, "Set Hotel X as the destination. I'd like to eat ramen on the way around 1:00 p.m."

[1728] Device: Calls the speech recognition API to convert voice input into text data. Retrieves the converted text data.

[1729] Terminal: Sends text data and voice data itself to the server.

[1730] Server: The received text data is input into ChatGPT, and the destination and intermediate points are extracted using natural language analysis.

[1731] Server: Uses an emotion engine to recognize the user's emotions from the voice data. For example, it determines whether the user is anxious or relaxed based on the tone and speed of the voice.

[1732] 2. Route calculation and emotion-based suggestion adaptation

[1733] Server: Uses the navigation API to calculate the optimal route based on the extracted destination and intermediate destinations, taking real-time traffic information into account.

[1734] Server: Adjusts the recommendations based on the user's emotions as recognized by the emotion engine. For example, if the user is determined to be tired, the server will suggest more relaxing rest areas or stopovers.

[1735] Server: Generates a message to suggest to the user based on the calculation results and adjustments based on emotions. For example, it could say, "We suggest a route to the destination hotel, stopping at the ramen shop around 1:00 PM. We also suggest places to rest and refresh yourself."

[1736] Server: Sends a proposal message to the terminal.

[1737] 3. Confirm the proposal and set the final route

[1738] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[1739] User: Review the proposed route and respond with a response such as "Yes, that's fine."

[1740] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[1741] Server: Confirms user consent and finalizes the route, which includes detailed navigation instructions.

[1742] Server: Sends the confirmed route data to the terminal.

[1743] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[1744] 4. Traffic monitoring and route correction

[1745] Server: Monitors traffic condition data in real time. Uses a dedicated API to periodically obtain information on congestion, accidents, etc.

[1746] Server: Determines whether there are any changes in traffic conditions that will affect the currently configured route.

[1747] Server: If a change in traffic conditions is detected, calculate a new route, resetting intermediate stops if necessary.

[1748] Server: The emotion engine re-identifies the user's emotions and adjusts the suggestions for new routes and stops.

[1749] Server: Generate a message containing a new route or stopover suggestion, such as "We are currently experiencing traffic congestion, so we suggest a revised route. We also recommend changing the rest stop to ____."

[1750] Server: Sends a proposal message to the terminal.

[1751] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[1752] User: Review the new proposal and respond, "Yes, I'd like the revised route."

[1753] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[1754] Server: Confirms user consent and confirms the new route, which also includes detailed navigation instructions.

[1755] Server: Sends new route data to the device.

[1756] Device: Continues navigation based on the new route, providing visual and audio guidance.

[1757] Specific examples

[1758] Example 1: Destination setting and emotion recognition

[1759] 1. User: Says to the device, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1pm."

[1760] 2. Device: Converts voice input into text and sends it to the server.

[1761] 3. Server: Analyzes the text data and extracts the destination and intermediate points.

[1762] 4. Server: Using the emotion engine, recognize the user's emotion from the voice data. For example, it determines that the user is tired.

[1763] 5. Server: Calculates the optimal route using the navigation API.

[1764] 6. Server: Adjust the recommendations based on the user's emotions. Create recommendations that include relaxing stops and rest areas.

[1765] 7. Server: Generates a message proposing the calculation result and sends it to the terminal.

[1766] 8. Terminal: Play the proposed message aloud and ask the user for confirmation.

[1767] 9. User: Review the proposed route and respond, "Yes, that's fine."

[1768] 10. Server: Determines the final route and sends navigation data to the device.

[1769] Example 2: Route adjustments based on traffic changes and sentiment

[1770] 1. Server: Monitors traffic conditions and obtains new information.

[1771] 2. Server: Calculate a new route and recognize the user's emotions again using the emotion engine.

[1772] 3. Server: Generates messages with new suggestions and tailors them with sentiment-based content.

[1773] 4. Terminal: Play the proposed message aloud and ask the user for confirmation.

[1774] 5. User: Agrees with the new suggestion and responds, "Change to XX Restaurant."

[1775] 6. Server: Determines the new route and sends navigation data to the device.

[1776] 7. On the device: Continue navigation based on the new route.

[1777] As a result, the system of the present invention recognizes the user's emotions and suggests appropriate routes and stopovers accordingly, thereby providing safe and convenient navigation.

[1778] The processing flow will be explained below.

[1779] Processing steps of a navigation system including an emotion engine

[1780] 1. Destination and route setting and emotion recognition processing

[1781] Step 1:

[1782] User: Gives instructions to the voice input device, such as "Set Hotel X as my destination. I'd like to eat ramen on the way around 1pm."

[1783] Step 2:

[1784] Device: Calls the speech recognition API to convert voice input into text data. Retrieves the converted text data.

[1785] Step 3:

[1786] Terminal: Transmits the converted text data and the audio data itself to the server.

[1787] Step 4:

[1788] Server: The received text data is input to ChatGPT for analysis. ChatGPT extracts the destination and intermediate points from the text data.

[1789] Step 5:

[1790] Server: Uses an emotion engine to recognize the user's emotions from the voice data, for example, determining whether the user is tired or relaxed from the tone and speed of the voice.

[1791] Step 6:

[1792] Server: Uses the navigation API to calculate the optimal route based on the extracted destinations and intermediate destinations, including real-time traffic information.

[1793] Step 7:

[1794] Server: Adjusts the recommendations based on the user's perceived emotions. For example, if the user is tired, it suggests more relaxing rest stops or stopovers.

[1795] Step 8:

[1796] Server: Generates a message containing the calculated route and sentiment-based suggestions. For example, "We suggest a route with the destination being the XX hotel, stopping at the XX ramen shop around 1:00 PM. Considering recent fatigue, we also recommend rest stops along the way."

[1797] Step 9:

[1798] Server: Sends the generated proposal message to the terminal.

[1799] Step 10:

[1800] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[1801] Step 11:

[1802] User: Review the proposed route and respond "Yes, that's fine."

[1803] Step 12:

[1804] Terminal: The user's response is converted into text again using the speech recognition API and sent to the server.

[1805] Step 13:

[1806] Server: Confirms user consent and finalizes the route, including detailed navigation instructions.

[1807] Step 14:

[1808] Server: Sends the confirmed route data to the terminal.

[1809] Step 15:

[1810] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[1811] 2. Monitoring traffic conditions and handling route corrections

[1812] Step 1:

[1813] Server: Monitors traffic condition data in real time. Uses a dedicated API to periodically obtain information on congestion, accidents, etc.

[1814] Step 2:

[1815] Server: Determines whether there are any changes in traffic conditions that will affect the currently configured route.

[1816] Step 3:

[1817] Server: If a change in traffic conditions is detected, calculate a new route, resetting intermediate stops if necessary.

[1818] Step 4:

[1819] Server: Again, the emotion engine recognizes the user's emotions and adjusts the suggestions for new routes and stops.

[1820] Step 5:

[1821] Server: Generates a suggestion message that includes a new route and adjustments based on emotions. For example, "Currently, there is traffic congestion, so we suggest a revised route. Considering fatigue, we recommend changing the rest area to △△."

[1822] Step 6:

[1823] Server: Sends the generated proposal message to the terminal.

[1824] Step 7:

[1825] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[1826] Step 8:

[1827] User: Review the new proposal and respond, "Yes, I'd like the revised route."

[1828] Step 9:

[1829] Terminal: The user's response is converted into text again using the speech recognition API and sent to the server.

[1830] Step 10:

[1831] Server: Confirms user consent and confirms the new route, which also includes detailed navigation instructions.

[1832] Step 11:

[1833] Server: Sends new route data to the device.

[1834] Step 12:

[1835] Device: Continues navigation based on the new route, providing visual and audio guidance.

[1836] As a result, the system of the present invention recognizes the user's emotions, suggests appropriate routes and stopovers accordingly, and provides safe and convenient navigation.

[1837] Example 2

[1838] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1839] Conventional navigation systems are equipped with functions for recalculating routes and setting destinations in response to changes in traffic conditions, but they are unable to provide appropriate suggestions based on the user's emotional state, making it difficult to improve user satisfaction. Furthermore, even when setting a destination based on voice input, there is a lack of natural conversational responses, so there is a need to improve the user experience.

[1840] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for recognizing the user's emotion from voice data, means for adjusting the content of suggestions based on the recognized user's emotion, and means for monitoring changes in traffic conditions, recalculating the route in accordance with the changes in traffic conditions, and suggesting the route to the user. This enables personalized route suggestions that take the user's emotion into consideration and optimal route adjustments in real time.

[1841] "Means for converting voice input into text data" refers to technology or devices that process a user's voice and convert the voice data into text format data.

[1842] "Means for analyzing text data and voice data to extract destinations and intermediate points" refers to technology or devices that automatically identify specified destinations and intermediate points from voice-input text and voice data.

[1843] "Means for recognizing a user's emotions from voice data" refers to technology or devices that analyze characteristics of voice data such as tone, speed, and volume of the voice to determine the user's emotional state (for example, fatigue, tension, relaxation, etc.).

[1844] "Means for calculating the optimal route based on the destination and intermediate points" refers to technology or devices that calculate the optimal travel route based on the set destination and intermediate points, taking into account traffic information and map information.

[1845] "Means for adjusting the content of suggestions based on the recognized user emotions" refers to technology or devices that take into account the user's emotional state and adaptively change the suggested route and intermediate points.

[1846] "Means for suggesting calculated route and stopover information to users" refers to technology or devices that present information on optimal routes and stopovers to users.

[1847] "Means for setting a final route based on a user's response" refers to technology or devices that determine a final navigation route based on the user's confirmation or consent.

[1848] "Means for monitoring changes in traffic conditions, recalculating routes in response to the changes in traffic conditions, and proposing routes to users" refers to technologies and devices that monitor traffic information in real time, calculate new routes in response to changes in traffic conditions, and provide that information to users.

[1849] The navigation system of the present invention is composed of a voice input device, a server, a navigation API, and an emotion engine. This system converts the user's voice input into text data and extracts the destination and intermediate points from the text data. It also recognizes the user's emotions from the voice data and proposes the optimal route. This section specifies the hardware and software used and explains how the system specifically operates.

[1850] System configuration

[1851] 1. Voice input device: A voice input device such as a smartphone or car navigation terminal is used to obtain voice instructions from the user.

[1852] 2. Speech recognition API: Use Google Speech-to-Text or similar to convert voice data into text data.

[1853] 3. Text and voice data analysis: The server uses ChatGPT or a similar generative AI model to analyze the text data and extract destinations and intermediate stops.

[1854] 4. Emotion Recognition Engine: Uses IBM Watson and other emotion recognition technologies to recognize user emotions from voice data.

[1855] 5. Navigation API: Uses Google Maps API etc. to calculate the optimal route reflecting real-time traffic information.

[1856] Example of operation

[1857] Example 1: Destination setting and emotion recognition

[1858] 1. User: The user speaks into a voice input device, saying, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1:00 p.m."

[1859] 2. Device: Calls the speech recognition API to convert this voice input into text data.

[1860] 3. Terminal: The converted text data and audio data are sent to the server.

[1861] 4. Server: The server inputs the text data into ChatGPT and extracts the destination "XX Hotel" and the stopover "Ramen Shop" through natural language analysis.

[1862] 5. Server: Using the emotion engine, recognize the user's emotion from the voice data. For example, it determines that the user is tired.

[1863] 6. Server: Calculates the optimal route using the navigation API.

[1864] 7. Server: Adjust the recommendations based on the user's emotions and create recommendations that include relaxing stops and rest areas.

[1865] 8. Server: Sends the calculation result to the terminal as a proposal message.

[1866] 9. Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[1867] 10. User: Review the proposed route and respond, "Yes, that's fine."

[1868] 11. Terminal: The user's response is again converted into text using the speech recognition API and sent to the server.

[1869] 12. Server: Confirms user consent, finalizes the route, and sends the final route data to the device.

[1870] 13. Terminal: Starts navigation based on the received route data, and provides screen display and voice guidance.

[1871] Prompt Sentence Examples

[1872] Prompt example 1: Speech data text conversion and emotion recognition

[1873] "Voice data: 'I want to eat ramen at 1pm.' Convert this voice data into text and recognize the user's emotions."

[1874] Example prompt 2: Route calculation and adjustment

[1875] "Destination: 'X hotel, Shibuya-ku, Tokyo' Stop: 'X ramen shop' User's emotion: 'Tired' Calculate the optimal route based on this data and generate a suggested message including stopovers and rest areas where people can relax."

[1876] This allows the system of the present invention to propose a personalized route that takes into account the user's emotions and to adjust the route optimally in real time.

[1877] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1878] Step 1:

[1879] User: The user gives instructions to the voice input device, such as, "Set Hotel X as the destination. I'd like to eat ramen on the way around 1:00 p.m."

[1880] Specific behavior:

[1881] Users speak into the microphone on their smartphone or car navigation device.

[1882] Input: Audio data

[1883] Output: Audio data

[1884] Step 2:

[1885] On the device: Call a speech recognition API (such as Google Speech-to-Text) to convert the voice input into text data. Retrieve the converted text data.

[1886] Specific behavior:

[1887] Send the voice data to the voice recognition API and convert it into text data.

[1888] Keeps the converted text in memory.

[1889] Input: Audio data

[1890] Output: Text data

[1891] Step 3:

[1892] Terminal: Sends text data and voice data itself to the server.

[1893] Specific behavior:

[1894] Use an HTTP POST request to send text and audio data to a specified endpoint on the server.

[1895] Input: Text data, audio data

[1896] Output: Send data to the server

[1897] Step 4:

[1898] Server: The received text data is input into ChatGPT, and the destination and intermediate points are extracted using natural language analysis.

[1899] Specific behavior:

[1900] Call the ChatGPT API and send text data as input data.

[1901] Extract the destination "XX Hotel" and the transit point "△△ Ramen Shop" from the ChatGPT output.

[1902] Input: Text data

[1903] Output: Destination, intermediate destination

[1904] Step 5:

[1905] Server: Uses an emotion engine to recognize the user's emotions from the voice data, for example, determining whether the user is anxious or relaxed based on the tone and speed of the voice.

[1906] Specific behavior:

[1907] Extract features such as tone, speed, and volume of speech.

[1908] Feature data is input into the emotion engine to recognize emotions (e.g., "tired").

[1909] Input: Audio data

[1910] Output: Emotion data

[1911] Step 6:

[1912] Server: Uses the navigation API (Google Maps API) to calculate the optimal route based on the extracted destination and intermediate destinations, taking real-time traffic information into account.

[1913] Specific behavior:

[1914] Send a request to the Google Maps API to get the optimal route via "XX Hotel" and "△△ Ramen Shop."

[1915] Receive route data including real-time traffic information.

[1916] Input: Destination, stopover

[1917] Output: Optimal route data

[1918] Step 7:

[1919] Server: Adjusts the recommendations based on the user's emotions as recognized by the emotion engine. For example, if the user is determined to be tired, the server will suggest more relaxing rest areas or stopovers.

[1920] Specific behavior:

[1921] The output of the emotion engine will be analyzed, and an option will be added to suggest a place to relax if the user's emotion is recognized as "tired."

[1922] Input: Emotion data, optimal route data

[1923] Output: Adjusted proposal

[1924] Step 8:

[1925] Server: Generates a message to suggest to the user based on the calculation results and adjustments based on emotions.

[1926] Specific behavior:

[1927] By combining the navigation route and emotion data, a message such as "We suggest a route with the destination being the XX hotel, stopping at the △△ ramen shop around 1:00 p.m. We will also suggest a place to rest and refresh yourself" is generated.

[1928] Input: Adjusted proposal

[1929] Output: Proposal message

[1930] Step 9:

[1931] Server: Sends a proposal message to the terminal.

[1932] Specific behavior:

[1933] Send the generated message to the device using an HTTP POST request.

[1934] Input: Proposal message

[1935] Output: Sending a message to the terminal

[1936] Step 10:

[1937] Terminal: The proposed message is converted into speech using a speech synthesis API (e.g., Amazon Polly) and played back to the user.

[1938] Specific behavior:

[1939] The message text is sent to the speech synthesis API and converted into voice data.

[1940] The generated audio data is played back from the device's speaker.

[1941] Input: Proposal message

[1942] Output: Audio data

[1943] Step 11:

[1944] User: Review the proposed route and respond "Yes, that's fine."

[1945] Specific behavior:

[1946] The user responds to the system by saying, "Yes, that would be fine."

[1947] Input: Proposal message

[1948] Output: User's voice response

[1949] Step 12:

[1950] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[1951] Specific behavior:

[1952] Send the voice data to the voice recognition API and convert it into text data.

[1953] Sends text data to the server.

[1954] Input: User's voice response

[1955] Output: Text data

[1956] Step 13:

[1957] Server: Confirms user consent and finalizes the route, which includes detailed navigation instructions.

[1958] Specific behavior:

[1959] Receive the user's consent response and finalize the route.

[1960] Generate detailed route data including navigation instructions.

[1961] Input: User's text response

[1962] Output: Final route data

[1963] Step 14:

[1964] Server: Sends the confirmed route data to the terminal.

[1965] Specific behavior:

[1966] Send an HTTP POST request to send the confirmed route data to the device.

[1967] Input: Final route data

[1968] Output: Sending data to the terminal

[1969] Step 15:

[1970] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[1971] Specific behavior:

[1972] The received route data is analyzed and navigation begins.

[1973] Route instructions are provided on screen and via voice guidance.

[1974] Input: Final route data

[1975] Output: Navigation instructions (screen display, voice guide)

[1976] (Application example 2)

[1977] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1978] Conventional navigation systems only set destinations and intermediate stops based on user instructions and modify routes according to traffic conditions, so they were unable to provide personalized suggestions that took the user's emotional state into account.The present invention aims to provide more personalized route suggestions and user experiences by incorporating a function that recognizes the user's emotions into the navigation system, thereby reducing user stress and improving user satisfaction.

[1979] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for converting voice input from a user into text data, means for analyzing the text data to extract a destination and intermediate points, means for recognizing the user's emotions from the voice data, means for calculating an optimal route based on the destination and intermediate points, means for adapting the proposal content based on the user's emotions, means for proposing information on the calculated route and intermediate points to the user, means for setting a final route based on a response from the user, and means for monitoring changes in traffic conditions, recalculating a route in accordance with the changes in traffic conditions, and proposing the route to the user. This makes it possible to propose personalized routes and intermediate points according to the user's emotional state.

[1980] "Voice input data" refers to information provided by a user to give voice instructions to the navigation system.

[1981] "Text data" refers to character information converted from voice input data using voice recognition technology.

[1982] A "destination" is the final location that a user wants to reach using a navigation system.

[1983] A "waypoint" is a place where you plan to stop on the way to your destination.

[1984] "Emotion recognition method" is a technology that analyzes the user's emotional state from the tone, speed, and word choice of voice data, and identifies specific emotions.

[1985] An "optimal route" is the most efficient route to reach a destination calculated based on current traffic conditions and user settings.

[1986] "Suggested content" refers to information about routes, stops, rest spots, etc. that the navigation system presents to the user.

[1987] "Route recalculation" is the process of revising an existing route to accommodate new information based on changing traffic conditions and user responses.

[1988] The "final route" is the final route that reflects the user's responses and satisfies all conditions.

[1989] "Traffic condition monitoring" is a technology that monitors real-time data such as traffic congestion and accident information, and makes route adjustments as necessary.

[1990] "Rest Spots" are locations along the route that are suggested for users to take a rest along the way.

[1991] A "navigation system" is a system that integrates these elements and provides users with audio and visual guidance.

[1992] The present invention provides a more personalized route suggestion and user experience by incorporating a function that recognizes the user's emotions into a voice-based navigation system. The system of the present invention consists of the following main components:

[1993] System configuration

[1994] 1. Hardware:

[1995] Audio input devices (e.g. smartphones, microphones)

[1996] GPS sensor

[1997] Autonomous vehicle control computer

[1998] 2. Software:

[1999] Speech recognition API (e.g. Google API)

[2000] Emotion recognition engine (e.g. IBM Watson Tone Analyzer)

[2001] Navigation APIs (e.g. Google Maps API)

[2002] Server-side architecture (e.g., Node.js, Express server)

[2003] Database (e.g. MongoDB)

[2004] Speech synthesis API (e.g. Amazon Polly)

[2005] Generative AI API (e.g. OpenAI GPT)

[2006] Program processing

[2007] 1. Voice input capture and emotion recognition

[2008] The user gives instructions to the navigation system using a voice input device, for example, "Set the destination as Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1:00 PM."

[2009] The device converts the voice input into text data using a voice recognition API and sends it to the server.

[2010] The server receives the text and audio data itself, performs natural language analysis using ChatGPT, and extracts the destination and intermediate points.

[2011] Furthermore, an emotion recognition engine is used to recognize the user's emotions from the voice data, for example, determining whether the user is anxious or relaxed from the tone and speed of the voice.

[2012] 2. Route calculation and emotion-based suggestion adaptation

[2013] The server uses a navigation API to calculate the optimal route, taking real-time traffic information into account.

[2014] Data from the emotion recognition engine is used to tailor suggestions, for example, suggesting a route that includes relaxing rest stops for a tired user.

[2015] The server uses a generation AI to generate messages in natural language to suggest to users and sends them to the device.

[2016] 3. Confirm the proposal and set the final route

[2017] The server converts the generated proposal message into voice data using a voice synthesis API and sends it to the terminal.

[2018] The terminal plays the suggestion message aloud and asks the user for confirmation.

[2019] The user reviews the proposed route and responds with a response such as "Yes, that's fine."

[2020] The server receives the user's response and determines the final route.

[2021] The determined route data is sent to the terminal, providing route information to the autonomous vehicle's control computer.

[2022] 4. Traffic monitoring and route correction

[2023] The server monitors traffic data in real time and acquires new information.

[2024] It recalculates routes based on new traffic conditions and re-recognizes the user's emotional state.

[2025] The server generates and adjusts new proposals and sends them to the device.

[2026] The user reviews the new proposal and responds, and the server, upon approval, finalizes the new route and sends it back to the autonomous vehicle's control computer.

[2027] Specific examples

[2028] Navigation for users who feel fatigued

[2029] Example prompt: "I'm tired and would like to stop by a cafe or park where I can relax."

[2030] Server processing: The server determines that the user is tired and calculates a route that includes places where the user can refresh themselves.

[2031] Suggestion example: As a new route, suggest "We suggest stopping at the cafe in Shizuoka Parking Area on the way. You can take a break there."

[2032] The above is a specific embodiment for implementing the system of the present invention, which recognizes the user's emotions and enables personalized navigation routes and suggestions based on those emotions.

[2033] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2034] Step 1:

[2035] The user uses a voice input device to specify destinations and stopovers by voice. For example, the user might say, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1:00 PM."

[2036] Input: User's voice command

[2037] Output: Audio data

[2038] Specific operation: The voice input device acquires user instructions as voice data through a microphone.

[2039] Step 2:

[2040] The device converts the voice data into text data using a voice recognition API and sends it to the server.

[2041] Input: Audio data

[2042] Output: Text data

[2043] Specific operation: The device calls a speech recognition API (e.g., Google API) and converts the voice data into text data. The converted text data is then obtained and sent to the server via an HTTP request or similar.

[2044] Step 3:

[2045] The server receives the text data and voice data and extracts the destination and intermediate points using ChatGPT.

[2046] Input: Text data, audio data

[2047] Output: Destination and waypoint information

[2048] Specific operation: The server inputs the received text data into ChatGPT and performs natural language analysis, extracting the destination and intermediate points and storing them in a database.

[2049] Step 4:

[2050] The server uses an emotion recognition engine to recognize the user's emotion from the voice data.

[2051] Input: Audio data

[2052] Output: User's emotional state

[2053] Specific operation: The server inputs the voice data into an emotion recognition engine (e.g., IBM Watson Tone Analyzer) and analyzes the tone and speed of the voice to identify the user's emotion.

[2054] Step 5:

[2055] The server uses a navigation API to calculate the optimal route, taking real-time traffic information into account.

[2056] Input: Destination information, route information, traffic information

[2057] Output: Optimal route

[2058] Specific operation: The server calls a navigation API (e.g., Google Maps API) and calculates the optimal route based on the destination, intermediate destinations, and current traffic conditions.

[2059] Step 6:

[2060] The server adapts the suggestions based on the user's emotions.

[2061] Input: User's emotional state, optimal route

[2062] Output: Personalized recommendations

[2063] Specific operation: Based on the results of the emotion recognition engine, for example, the server sets a route including rest spots for a user who feels tired, and generates a suggested message using a generation AI.

[2064] Step 7:

[2065] The server sends a proposal message to the device, which converts the proposal message into voice data using a speech synthesis API and plays it back to the user.

[2066] Input: Proposal message

[2067] Output: Audio data, audio playback

[2068] Specific operation: The server sends the generated proposal message to the device via an HTTP request. The device converts the received message into voice data using a speech synthesis API (e.g., Amazon Polly) and plays it back to the user through the speaker.

[2069] Step 8:

[2070] The user confirms the proposed route and responds with a voice response such as "Yes, that's fine."

[2071] Input: Voice response

[2072] Output: Audio data

[2073] Specific actions: After listening to the proposed route, if the user agrees, they give instructions to the voice input device again.

[2074] Step 9:

[2075] The device converts the voice data into text data again using the voice recognition API and sends it to the server.

[2076] Input: Audio data

[2077] Output: Text data

[2078] Specific operation: The device calls the speech recognition API again, converts the voice data into text data, and sends the converted text data to the server.

[2079] Step 10:

[2080] The server receives the user's response, determines the final route, and sends the determined route data to the device to start navigation.

[2081] Input: User consent (text data)

[2082] Output: Final route data

[2083] Specific operation: The server confirms the user's consent, determines the final route using the navigation API, sends the determined route data to the device, and starts navigation.

[2084] Step 11:

[2085] The server monitors traffic data in real time, recalculates the route according to the new traffic conditions, and also recognizes the user's emotional state again.

[2086] Input: Real-time traffic data

[2087] Output: The modified route

[2088] Specific operation: The server periodically calls the traffic information API to obtain data, and if there is any impact on the existing route, it recalculates it. At the same time, it uses the emotion recognition engine to re-determine the user's emotional state.

[2089] Step 12:

[2090] The server generates a new proposal and sends it to the terminal, which plays it back to the user in the same way and asks for confirmation.

[2091] Input: New proposal

[2092] Output: Audio data, audio playback

[2093] How it works: The server uses the generation AI to generate new suggestions and sends them to the device via an HTTP request. The device then converts the received message into audio data using a speech synthesis API and plays it back to the user.

[2094] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2095] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2096] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2097] [Fourth embodiment]

[2098] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2099] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2100] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2101] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2102] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2103] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2104] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2105] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2106] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2107] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2108] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2109] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2110] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2111] overview

[2112] The present invention relates to a navigation system that uses voice input. Users can set destinations and intermediate points through voice, and can smoothly adjust routes and rest areas according to changes in traffic conditions. This system supports users in traveling safely and conveniently.

[2113] Program processing

[2114] System configuration

[2115] The system mainly consists of the following elements:

[2116] 1. Voice input devices (smartphones and car navigation devices)

[2117] 2. Server (analysis and navigation data processing)

[2118] 3. Navigation API (e.g., map services)

[2119] 1. Acquiring voice input

[2120] User: First, the user speaks to the voice input device about their destination and stopovers. For example, they might say, "Set the destination to Hotel X as my destination. I'd like to eat ramen on the way around 1:00 PM."

[2121] Terminal: The terminal converts this voice input into text data and sends the text data to the server.

[2122] 2.Analysis of voice data and route calculation

[2123] Server: The server analyzes the received text data and extracts the destination and intermediate points. This analysis uses natural language processing technology. Specifically, it automatically extracts information about location and time from the text.

[2124] The server then uses a navigation API to calculate the optimal route based on the destination and intermediate stops, taking real-time traffic information into account.

[2125] Server: The server generates a message to suggest the calculated route and stopover information to the user and sends it to the terminal.

[2126] 3. Confirm the proposal and set the final route

[2127] Terminal: The terminal plays the received proposal message as audio and asks the user for confirmation.

[2128] User: The user responds to the proposed stopovers and route with a response such as "Yes, that's fine."

[2129] Terminal: Convert this response back into text and send it to the server.

[2130] Server: The server verifies the user's consent and sets the final route, which includes detailed navigation instructions.

[2131] 4. Traffic monitoring and route recalculation

[2132] Server: The server monitors traffic conditions in real time and calculates a new route if there are changes in traffic conditions along the specified route.

[2133] Server: Generates messages to propose new routes and stops and sends them to the device.

[2134] Terminal: The terminal will play the new suggestion aloud and ask the user for confirmation.

[2135] User: The user responds to the new suggestion, for example by giving instructions such as "Change to XX Restaurant."

[2136] Server: The server determines the new route and sends it to the device, which then continues navigation according to the new route.

[2137] Specific examples

[2138] Example 1: Setting a destination

[2139] 1. User: Says to the device, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1pm."

[2140] 2. Device: Converts voice input into text and sends it to the server.

[2141] 3. Server: Analyzes the text data, extracts the destination and intermediate points, and calculates the optimal route using the navigation API.

[2142] 4. Server: Generates a message proposing the calculation result and sends it to the terminal.

[2143] 5. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[2144] 6. User: Agrees with the proposal and responds, "Yes, I'd like that."

[2145] 7. Server: Determines the final route and sends navigation data to the device.

[2146] Example 2: Route adjustment due to changing traffic conditions

[2147] 1. Server: Monitors traffic conditions and obtains new information.

[2148] 2. Server: Calculates new routes and generates proposal messages.

[2149] 3. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[2150] 4. User: Agrees with the new suggestion and responds, "Change to XX Restaurant."

[2151] 5. Server: Determines the new route and sends navigation data to the device.

[2152] 6. Device: Continue navigation according to the new route.

[2153] As a result, the system of the present invention allows the user to set the destination and intermediate points by voice and responds to changes in traffic conditions, providing safe and convenient navigation.

[2154] The processing flow will be explained below.

[2155] Specific explanation of program processing

[2156] 1. Destination and intermediate destination setting process

[2157] Step 1:

[2158] User: Gives instructions to the voice input device, such as "Set Hotel X as my destination. I'd like to eat ramen on the way around 1pm."

[2159] Step 2:

[2160] Device: Calls the speech recognition API to convert voice input into text data. Retrieves the converted text data.

[2161] Step 3:

[2162] Terminal: Send text data to the server using an API endpoint.

[2163] Step 4:

[2164] Server: The received text data is input into ChatGPT, and the destination and intermediate points are extracted using natural language analysis.

[2165] Step 5:

[2166] Server: Uses a navigation API (e.g., Google Maps API) to calculate the optimal route based on the extracted destination and intermediate destinations.

[2167] Step 6:

[2168] Server: Based on the calculation results, a message is generated to be suggested to the user. It prepares a message such as "Suggested route to destination XX Hotel, stopping at XX ramen shop around 1 PM."

[2169] Step 7:

[2170] Server: Sends a proposal message to the terminal.

[2171] Step 8:

[2172] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[2173] Step 9:

[2174] User: Review the proposed route and respond "Yes, that's fine."

[2175] Step 10:

[2176] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[2177] Step 11:

[2178] Server: Confirms user consent and finalizes the route, which includes detailed navigation instructions.

[2179] Step 12:

[2180] Server: Sends the confirmed route data to the terminal.

[2181] Step 13:

[2182] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[2183] 2. Monitoring traffic conditions and handling route corrections

[2184] Step 1:

[2185] Server: Monitors traffic condition data in real time. Uses a dedicated API to periodically obtain information on congestion, accidents, etc.

[2186] Step 2:

[2187] Server: Determines whether there are any changes in traffic conditions that will affect the currently configured route.

[2188] Step 3:

[2189] Server: If a change in traffic conditions is detected, calculate a new route, resetting intermediate stops if necessary.

[2190] Step 4:

[2191] Server: Generate a message containing a new route or stopover suggestion. For example, "There is currently a traffic jam, so we suggest a revised route."

[2192] Step 5:

[2193] Server: Sends a proposal message to the terminal.

[2194] Step 6:

[2195] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[2196] Step 7:

[2197] User: Review the new proposal and respond, "Yes, I'd like the revised route."

[2198] Step 8:

[2199] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[2200] Step 9:

[2201] Server: Confirms user consent and confirms the new route, which also includes detailed navigation instructions.

[2202] Step 10:

[2203] Server: Sends new route data to the device.

[2204] Step 11:

[2205] Device: Continues navigation based on the new route, providing visual and audio guidance.

[2206] Example 1

[2207] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2208] Conventional navigation systems require a lot of manual input to set destinations and intermediate points, and it is also difficult to flexibly recalculate routes to accommodate real-time traffic conditions. This requires cumbersome operations for users, which reduces safety and convenience. Since manual input while driving is particularly dangerous, there is a demand for systems that utilize voice input.

[2209] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2210] In this invention, the server includes means for capturing a user's voice input and converting it into text data using voice recognition software, means for extracting a destination and intermediate points from the text using natural language processing technology, means for calculating a route taking real-time traffic information into consideration using a navigation API, means for playing a suggested message aloud using voice synthesis software and requesting confirmation from the user, and means for sending a final route instruction to the terminal. This allows the user to easily set a destination and intermediate points through voice input and enables flexible route changes according to real-time traffic information.

[2211] "Voice input" is a means of acquiring the words spoken by a user as digital signals.

[2212] The "means for converting into text data" refers to software or hardware for converting acquired voice input into text information.

[2213] "Means for analyzing text data" refers to technologies or systems for deciphering converted text data and extracting specific information.

[2214] "Destination and stopover points" refers to the final destination set by the user and points to be passed along the way.

[2215] The "means for calculating the optimal route" is a system that calculates the optimal route based on the destination and intermediate points, taking into account conditions such as time, distance, and traffic conditions.

[2216] "Means of providing suggestions to users" refers to the methods and techniques used to communicate calculated route information to users, and may involve voice or text.

[2217] The "means for setting the final route" is a system for determining the final route based on the user's response.

[2218] "Means for monitoring changes in traffic conditions" refers to a system for collecting traffic information in real time and monitoring the situation.

[2219] The "means for recalculating the route" is a system for reevaluating the originally calculated route in response to changes in traffic conditions and calculating a new optimal route.

[2220] "Speech recognition software" is software technology for converting voice input into text data.

[2221] "Natural language processing technology" is a technology for deciphering human language and extracting specific information.

[2222] A "navigation API" is an application program interface for calculating routes and obtaining traffic information using map services, etc.

[2223] "Speech synthesis software" is a technology for reproducing text data as speech.

[2224] System configuration

[2225] The navigation system consists of the following elements:

[2226] 1. Voice input devices (e.g. smartphones, car navigation devices)

[2227] 2. Server (processes analysis and navigation data)

[2228] 3. Navigation API (e.g., map service)

[2229] Program processing

[2230] Acquiring voice input

[2231] The user speaks to the voice input device about the destination and stopovers, for example, "Set the destination to XX Hotel. I'd like to eat ramen on the way around 1 PM."

[2232] Audio data conversion

[2233] The device captures the user's voice, converts it into text data using speech recognition software (e.g., Google Speech-to-Text API), and sends the text data to the server.

[2234] Analysis of audio data

[2235] The server analyzes the received text data and extracts the destination and intermediate points. This analysis uses natural language processing technology (e.g., SpaCy, NLTK). The server automatically extracts the location and time information specified by the user from the text.

[2236] Route calculation

[2237] The server uses a navigation API (e.g., Google Maps API) to calculate the optimal route based on the destination and intermediate stops, taking real-time traffic information into account.

[2238] Generate and send a proposal message

[2239] The server generates a proposal message based on the calculated route and stopover information and sends it to the terminal.

[2240] Playing the suggestion message

[2241] The device uses voice synthesis software (e.g., Google Text-to-Speech API) to play the received proposal message aloud and ask the user for confirmation.

[2242] User Response

[2243] The user responds to the proposed stopovers and route by saying something like "Yes, that's fine."

[2244] Sending a Response

[2245] The terminal converts the user's response back into text and sends it to the server.

[2246] Setting the final route

[2247] The server verifies the user's consent and sets the final route, which includes detailed navigation instructions.

[2248] Traffic monitoring and route recalculation

[2249] The server monitors traffic conditions in real time, calculates a new route if the conditions change, and generates a message proposing a new route and intermediate destinations, which is then sent to the device.

[2250] Specific examples

[2251] Setting a destination

[2252] 1. User: "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1pm."

[2253] 2. Device: Converts voice input into text and sends it to the server.

[2254] 3. Server: Analyzes the text data, extracts the destination and intermediate points, and calculates the optimal route using the navigation API.

[2255] 4. Server: Generates a message proposing the calculation result and sends it to the terminal.

[2256] 5. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[2257] 6. User: "Yes, that's fine."

[2258] 7. Server: Determines the final route and sends navigation data to the device.

[2259] Route adjustments due to changing traffic conditions

[2260] 1. Server: Monitors traffic conditions and obtains new information.

[2261] 2. Server: Calculates new routes and generates proposal messages.

[2262] 3. Terminal: Play the suggestion message aloud and ask the user for confirmation.

[2263] 4. User: Responds, "Change to XX Restaurant."

[2264] 5. Server: Determines the new route and sends navigation data to the device.

[2265] 6. Device: Continue navigation according to the new route.

[2266] Example prompt sentence:

[2267] "Set the destination as Hotel X. I'd like to stop by for ramen around 1pm on the way."

[2268] "Change it to XX Restaurant."

[2269] This system allows users to set their destination and route by voice and responds to changes in traffic conditions, making navigation safer and more convenient.

[2270] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2271] Step 1: Getting voice input

[2272] The user speaks to the voice input device, saying, "Set the destination as Hotel X. I'd like to eat ramen on the way around 1pm." This acquires voice data.

[2273] Input: User's voice

[2274] Output: Audio data

[2275] Step 2: Convert the audio data

[2276] The device converts the acquired voice data into text data using voice recognition software (e.g., Google Speech-to-Text API), and then sends the converted text data to the server.

[2277] Input: Audio data

[2278] Output: Text data

[2279] Step 3: Analyzing the text data

[2280] The server analyzes the received text data and uses natural language processing technology (e.g., SpaCy, NLTK) to extract the destination "XX Hotel," the stopover "Ramen Shop," and the time "1:00 PM."

[2281] Input: Text data

[2282] Output: Analysis results (destination, intermediate points, time)

[2283] Step 4: Calculate the route

[2284] The server uses a navigation API (e.g., Google Maps API) to calculate the optimal route based on the destination and intermediate destinations, taking into account real-time traffic information.

[2285] Input: Analysis results (destination, transit points, time) and traffic information

[2286] Output: Calculated route information

[2287] Step 5: Generate and send a proposal message

[2288] The server generates a proposal message based on the calculated route and stopover information and sends it to the terminal.

[2289] Input: Calculated route information

[2290] Output: Proposal message

[2291] Step 6: Play the suggestion message

[2292] The device uses voice synthesis software (e.g., Google Text-to-Speech API) to play the received proposal message aloud and ask the user for confirmation.

[2293] Input: Proposal message

[2294] Output: Spoken suggestions

[2295] Step 7: User response

[2296] The user responds to the proposed route and stop by saying something like, "Yes, that's fine." This response indicates the user's intent.

[2297] Input: Voice response

[2298] Output: User response data

[2299] Step 8: Sending response data

[2300] The terminal converts the user's response back into text and sends it to the server.

[2301] Input: User response data (voice)

[2302] Output: User response data (text)

[2303] Step 9: Setting the Final Route

[2304] The server verifies the user's consent and sets the final route, which includes detailed navigation instructions.

[2305] Input: User response data

[2306] Output: Final route setting data

[2307] Step 10: Sending navigation data

[2308] The server sends the final route configuration to the device.

[2309] Input: Final route setting data

[2310] Output: Navigation data

[2311] Step 11: Traffic monitoring

[2312] The server monitors traffic conditions in real time and checks for changes in the situation, using traffic data providers (e.g., Waze Traffic API).

[2313] Input: Real-time traffic information

[2314] Output: Monitoring results

[2315] Step 12: Calculate the new route

[2316] The server calculates a new route if traffic conditions change along the specified route.

[2317] Input: Monitoring results

[2318] Output: New route information

[2319] Step 13: Generate and send a new proposal message

[2320] The server generates a proposal message based on the new route information and sends it to the terminal.

[2321] Input: New route information

[2322] Output: New proposal message

[2323] Step 14: Playing a new proposal

[2324] The terminal will play the new suggestion message aloud and ask the user for confirmation.

[2325] Input: New proposed message

[2326] Output: New audio suggestions

[2327] Step 15: User's new response

[2328] The user responds to the new suggestion by saying something like, "Change to XX Restaurant."

[2329] Input: New spoken response

[2330] Output: New response data

[2331] Step 16: Send new response data

[2332] The terminal converts the new response into text and sends it to the server.

[2333] Input: New response data (audio)

[2334] Output: New response data (text)

[2335] Step 17: Confirm the new route

[2336] The server determines a new route based on the user's new instructions and transmits navigation data to the terminal.

[2337] Input: New response data

[2338] Output: New final route data

[2339] Step 18: Continued Navigation

[2340] The device continues navigation according to the new final route data.

[2341] Input: New final route data

[2342] Output: Updated navigation information

[2343] In this way, the system allows for voice input, destination and intermediate destination settings, and even flexible route recalculation based on real-time traffic information.

[2344] (Application example 1)

[2345] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2346] Conventional navigation systems have the problem of being unable to quickly respond to changes in real-time facility conditions and checkpoints in security services and facility management. Particularly in large facilities or locations with complex layouts, manually understanding these changes and setting optimal patrol routes takes time and effort, so there is a need for improved efficiency and safety.

[2347] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2348] In this invention, the server includes means for converting voice input from the user into text data, means for analyzing the text data to extract a destination and intermediate points, and means for calculating an optimal tour route based on the destination and intermediate points, thereby enabling efficient and safe tour route planning in response to changes in facility conditions and checkpoints in real time.

[2349] "Means for converting voice input into text data" refers to a device or program that converts a user's voice instructions into text data using voice recognition technology.

[2350] "Means for analyzing text data and extracting destinations and intermediate points" refers to devices or programs that have the function of identifying the destination and intermediate points specified by the user from the converted text data using natural language processing technology.

[2351] "Means for calculating the optimal travel route based on the destination and intermediate points" refers to a device or program that calculates the optimal travel route for a specified destination and intermediate points, taking into account current traffic information and conditions within the facility.

[2352] "Means for proposing calculated route and route information to users" refers to devices or programs that have the function of presenting calculated route and route information to users in the form of confirmation and suggestions.

[2353] "Means for setting a final route based on a response from the user" refers to a device or program that finalizes a proposed route based on voice or input data from the user and reflects it in the navigation.

[2354] "Means for monitoring changes in facility conditions, recalculating routes in response to those changes, and proposing new routes to users" refers to devices or programs that have the function of detecting real-time changes in conditions inside and outside the facility, recalculating routes based on those changes, and proposing new routes to users.

[2355] "Means for adjusting the timing of checkpoints and intermediate destinations and suggesting them to users" refers to devices or programs that have the function of dynamically reconfiguring checkpoints and intermediate destinations to be reached during a patrol in response to changes in facility conditions and suggesting them to users.

[2356] As an embodiment of the present invention, a voice-input navigation system for security services is proposed. The system of the present invention realizes voice-input instructions and optimizes patrol routes in response to real-time changes in facility conditions.

[2357] Hardware and Software Configuration

[2358] Hardware:

[2359] Audio input device (e.g. smartphone, Bluetooth microphone)

[2360] Server (real-time processing and data analysis)

[2361] Network communication devices (Wi-Fi, cellular networks)

[2362] software:

[2363] Speech recognition software (e.g., Google Speech-to-Text API, Python's speech_recognition library)

[2364] Natural language processing software (e.g., NLTK, spaCy)

[2365] Navigation APIs (e.g. Google Maps API)

[2366] Processing method

[2367] Speech to text:

[2368] First, the user gives instructions for the patrol into the voice input device. For example, they might say, "I'd like to go to the administration building and then check the warehouse." The device picks up this voice and converts it into text data using voice recognition software.

[2369] Analyzing text data:

[2370] The server then analyzes the converted text data to extract destinations and stopovers using natural language processing software, which automatically identifies destinations and stopovers from specific keywords and phrases.

[2371] Route optimization and calculation:

[2372] The server uses a navigation API to calculate the optimal route based on the extracted destinations and intermediate stops, taking into account real-time traffic information and in-facility situation information.

[2373] Suggestions for users:

[2374] The calculated route and route information are sent from the server to the device, which then proposes it to the user via voice. The user can respond to the proposal by saying something like "Yes, that's fine." If the user responds in any other way, the device will propose and correct the route again.

[2375] Real-time monitoring:

[2376] The server constantly monitors the facility status and recalculates the route if there are any changes, and as a result, it re-proposes new routes and checkpoints to the user and modifies the tour route as necessary.

[2377] Specific examples

[2378] Example 1: Facility patrol during business hours

[2379] The user gives voice instructions for the patrol route. For example, "I want to go to the administration building, then move to the warehouse." The device converts the voice into text, which the server analyzes and calculates the route. The calculation results are then presented to the user, and the final route is set based on the user's response. The server monitors the facility situation in real time, and if the situation changes, it immediately suggests a new route.

[2380] Prompt Sentence Examples

[2381] User: "I want to go to the administration building and then to the warehouse."

[2382] Device: "Converting speech to text..."

[2383] Server: "Parsing destination and waypoints..."

[2384] This allows the system of the present invention to efficiently and safely patrol facilities using voice input, significantly improving the efficiency and effectiveness of security operations.

[2385] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2386] Step 1:

[2387] The user inputs the destination and stopover points into the voice input device, for example, by saying, "I want to go to the administration building and then check the warehouse." The input is the user's voice data, which the device receives and sends to the server.

[2388] Step 2:

[2389] The device converts the captured voice data into text data using speech recognition software such as the Google Speech-to-Text API or the Python speech_recognition library. The output is the converted text data, which is then sent to the server.

[2390] Step 3:

[2391] The server analyzes the received text data using natural language processing software to extract destinations and intermediate points. This analysis uses natural language processing libraries such as NLTK and spaCy. The input is text data, and the output is the extracted destinations and intermediate points.

[2392] Step 4:

[2393] The server sends the extracted destinations and intermediate points to a navigation API, which calculates the optimal route. The navigation API uses the Google Maps API, etc. The input is the destination and intermediate points, and the output is the calculated route.

[2394] Step 5:

[2395] The server generates a message to propose the calculated route and stop information to the user and sends it to the terminal. The input is the calculated tour route, and the output is the proposal message.

[2396] Step 6:

[2397] The device then plays the received suggestion message aloud and asks the user for confirmation. Specifically, it uses a generative AI model to read the suggestion aloud in a natural voice. The input is the suggestion message, and the output is the spoken suggestion.

[2398] Step 7:

[2399] The user responds to the proposed route by voice, for example, saying, "Yes, that's it." The input is the user's voice instruction, and the device converts the voice data back into text and sends it to the server.

[2400] Step 8:

[2401] The server sets the final route based on the user's response. The input is the user's text data, and the output is the final route.

[2402] Step 9:

[2403] The server monitors facility and traffic conditions in real time. If any changes occur, the server recalculates the route using the new information and sends a new proposal message to the terminal. The input is real-time situation data, and the output is a recalculated tour route and a new proposal message.

[2404] Step 10:

[2405] The terminal plays the new suggestion message aloud and asks the user for confirmation. Again, based on the user's response, the server adjusts the route and performs the optimal tour. The input is the suggestion message, and the output is the adjusted final route.

[2406] This enables real-time facility patrols via voice input, improving the efficiency and safety of security services.

[2407] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2408] overview

[2409] This invention relates to a voice-input navigation system that incorporates a function to recognize user emotions to provide more personalized route suggestions and user experience. Users can specify destinations and intermediate stops by voice, and the system can not only adjust the route based on changes in traffic conditions, but also adapt the suggestions based on the user's emotions.

[2410] Program processing

[2411] System configuration

[2412] The system mainly consists of the following elements:

[2413] 1. Voice input devices (smartphones and car navigation devices)

[2414] 2. Server (voice analysis, emotion recognition, and navigation data processing)

[2415] 3. Navigation API (e.g., map services)

[2416] 4. Emotion engine (recognizes user emotions from voice input)

[2417] 1. Voice input capture and emotion recognition

[2418] User: The user gives instructions to the voice input device, such as, "Set Hotel X as the destination. I'd like to eat ramen on the way around 1:00 p.m."

[2419] Device: Calls the speech recognition API to convert voice input into text data. Retrieves the converted text data.

[2420] Terminal: Sends text data and voice data itself to the server.

[2421] Server: The received text data is input into ChatGPT, and the destination and intermediate points are extracted using natural language analysis.

[2422] Server: Uses an emotion engine to recognize the user's emotions from the voice data. For example, it determines whether the user is anxious or relaxed based on the tone and speed of the voice.

[2423] 2. Route calculation and emotion-based suggestion adaptation

[2424] Server: Uses the navigation API to calculate the optimal route based on the extracted destination and intermediate destinations, taking real-time traffic information into account.

[2425] Server: Adjusts the recommendations based on the user's emotions as recognized by the emotion engine. For example, if the user is determined to be tired, the server will suggest more relaxing rest areas or stopovers.

[2426] Server: Generates a message to suggest to the user based on the calculation results and adjustments based on emotions. For example, it could say, "We suggest a route to the destination hotel, stopping at the ramen shop around 1:00 PM. We also suggest places to rest and refresh yourself."

[2427] Server: Sends a proposal message to the terminal.

[2428] 3. Confirm the proposal and set the final route

[2429] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[2430] User: Review the proposed route and respond with a response such as "Yes, that's fine."

[2431] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[2432] Server: Confirms user consent and finalizes the route, which includes detailed navigation instructions.

[2433] Server: Sends the confirmed route data to the terminal.

[2434] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[2435] 4. Traffic monitoring and route correction

[2436] Server: Monitors traffic condition data in real time. Uses a dedicated API to periodically obtain information on congestion, accidents, etc.

[2437] Server: Determines whether there are any changes in traffic conditions that will affect the currently configured route.

[2438] Server: If a change in traffic conditions is detected, calculate a new route, resetting intermediate stops if necessary.

[2439] Server: The emotion engine re-identifies the user's emotions and adjusts the suggestions for new routes and stops.

[2440] Server: Generate a message containing a new route or stopover suggestion, such as "We are currently experiencing traffic congestion, so we suggest a revised route. We also recommend changing the rest stop to ____."

[2441] Server: Sends a proposal message to the terminal.

[2442] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[2443] User: Review the new proposal and respond, "Yes, I'd like the revised route."

[2444] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[2445] Server: Confirms user consent and confirms the new route, which also includes detailed navigation instructions.

[2446] Server: Sends new route data to the device.

[2447] Device: Continues navigation based on the new route, providing visual and audio guidance.

[2448] Specific examples

[2449] Example 1: Destination setting and emotion recognition

[2450] 1. User: Says to the device, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1pm."

[2451] 2. Device: Converts voice input into text and sends it to the server.

[2452] 3. Server: Analyzes the text data and extracts the destination and intermediate points.

[2453] 4. Server: Using the emotion engine, recognize the user's emotion from the voice data. For example, it determines that the user is tired.

[2454] 5. Server: Calculates the optimal route using the navigation API.

[2455] 6. Server: Adjust the recommendations based on the user's emotions. Create recommendations that include relaxing stops and rest areas.

[2456] 7. Server: Generates a message proposing the calculation result and sends it to the terminal.

[2457] 8. Terminal: Play the proposed message aloud and ask the user for confirmation.

[2458] 9. User: Review the proposed route and respond, "Yes, that's fine."

[2459] 10. Server: Determines the final route and sends navigation data to the device.

[2460] Example 2: Route adjustments based on traffic changes and sentiment

[2461] 1. Server: Monitors traffic conditions and obtains new information.

[2462] 2. Server: Calculate a new route and recognize the user's emotions again using the emotion engine.

[2463] 3. Server: Generates messages with new suggestions and tailors them with sentiment-based content.

[2464] 4. Terminal: Play the proposed message aloud and ask the user for confirmation.

[2465] 5. User: Agrees with the new suggestion and responds, "Change to XX Restaurant."

[2466] 6. Server: Determines the new route and sends navigation data to the device.

[2467] 7. On the device: Continue navigation based on the new route.

[2468] As a result, the system of the present invention recognizes the user's emotions and suggests appropriate routes and stopovers accordingly, thereby providing safe and convenient navigation.

[2469] The processing flow will be explained below.

[2470] Processing steps of a navigation system including an emotion engine

[2471] 1. Destination and route setting and emotion recognition processing

[2472] Step 1:

[2473] User: Gives instructions to the voice input device, such as "Set Hotel X as my destination. I'd like to eat ramen on the way around 1pm."

[2474] Step 2:

[2475] Device: Calls the speech recognition API to convert voice input into text data. Retrieves the converted text data.

[2476] Step 3:

[2477] Terminal: Transmits the converted text data and the audio data itself to the server.

[2478] Step 4:

[2479] Server: The received text data is input to ChatGPT for analysis. ChatGPT extracts the destination and intermediate points from the text data.

[2480] Step 5:

[2481] Server: Uses an emotion engine to recognize the user's emotions from the voice data, for example, determining whether the user is tired or relaxed from the tone and speed of the voice.

[2482] Step 6:

[2483] Server: Uses the navigation API to calculate the optimal route based on the extracted destinations and intermediate destinations, including real-time traffic information.

[2484] Step 7:

[2485] Server: Adjusts the recommendations based on the user's perceived emotions. For example, if the user is tired, it suggests more relaxing rest stops or stopovers.

[2486] Step 8:

[2487] Server: Generates a message containing the calculated route and sentiment-based suggestions. For example, "We suggest a route with the destination being the XX hotel, stopping at the XX ramen shop around 1:00 PM. Considering recent fatigue, we also recommend rest stops along the way."

[2488] Step 9:

[2489] Server: Sends the generated proposal message to the terminal.

[2490] Step 10:

[2491] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[2492] Step 11:

[2493] User: Review the proposed route and respond "Yes, that's fine."

[2494] Step 12:

[2495] Terminal: The user's response is converted into text again using the speech recognition API and sent to the server.

[2496] Step 13:

[2497] Server: Confirms user consent and finalizes the route, including detailed navigation instructions.

[2498] Step 14:

[2499] Server: Sends the confirmed route data to the terminal.

[2500] Step 15:

[2501] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[2502] 2. Monitoring traffic conditions and handling route corrections

[2503] Step 1:

[2504] Server: Monitors traffic condition data in real time. Uses a dedicated API to periodically obtain information on congestion, accidents, etc.

[2505] Step 2:

[2506] Server: Determines whether there are any changes in traffic conditions that will affect the currently configured route.

[2507] Step 3:

[2508] Server: If a change in traffic conditions is detected, calculate a new route, resetting intermediate stops if necessary.

[2509] Step 4:

[2510] Server: Again, the emotion engine recognizes the user's emotions and adjusts the suggestions for new routes and stops.

[2511] Step 5:

[2512] Server: Generates a suggestion message that includes a new route and adjustments based on emotions. For example, "Currently, there is traffic congestion, so we suggest a revised route. Considering fatigue, we recommend changing the rest area to △△."

[2513] Step 6:

[2514] Server: Sends the generated proposal message to the terminal.

[2515] Step 7:

[2516] Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[2517] Step 8:

[2518] User: Review the new proposal and respond, "Yes, I'd like the revised route."

[2519] Step 9:

[2520] Terminal: The user's response is converted into text again using the speech recognition API and sent to the server.

[2521] Step 10:

[2522] Server: Confirms user consent and confirms the new route, which also includes detailed navigation instructions.

[2523] Step 11:

[2524] Server: Sends new route data to the device.

[2525] Step 12:

[2526] Device: Continues navigation based on the new route, providing visual and audio guidance.

[2527] As a result, the system of the present invention recognizes the user's emotions, suggests appropriate routes and stopovers accordingly, and provides safe and convenient navigation.

[2528] Example 2

[2529] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2530] Conventional navigation systems are equipped with functions for recalculating routes and setting destinations in response to changes in traffic conditions, but they are unable to provide appropriate suggestions based on the user's emotional state, making it difficult to improve user satisfaction. Furthermore, even when setting a destination based on voice input, there is a lack of natural conversational responses, so there is a need to improve the user experience.

[2531] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for recognizing the user's emotion from voice data, means for adjusting the content of suggestions based on the recognized user's emotion, and means for monitoring changes in traffic conditions, recalculating the route in accordance with the changes in traffic conditions, and suggesting the route to the user. This enables personalized route suggestions that take the user's emotion into consideration and optimal route adjustments in real time.

[2532] "Means for converting voice input into text data" refers to technology or devices that process a user's voice and convert the voice data into text format data.

[2533] "Means for analyzing text data and voice data to extract destinations and intermediate points" refers to technology or devices that automatically identify specified destinations and intermediate points from voice-input text and voice data.

[2534] "Means for recognizing a user's emotions from voice data" refers to technology or devices that analyze characteristics of voice data such as tone, speed, and volume of the voice to determine the user's emotional state (for example, fatigue, tension, relaxation, etc.).

[2535] "Means for calculating the optimal route based on the destination and intermediate points" refers to technology or devices that calculate the optimal travel route based on the set destination and intermediate points, taking into account traffic information and map information.

[2536] "Means for adjusting the content of suggestions based on the recognized user emotions" refers to technology or devices that take into account the user's emotional state and adaptively change the suggested route and intermediate points.

[2537] "Means for suggesting calculated route and stopover information to users" refers to technology or devices that present information on optimal routes and stopovers to users.

[2538] "Means for setting a final route based on a user's response" refers to technology or devices that determine a final navigation route based on the user's confirmation or consent.

[2539] "Means for monitoring changes in traffic conditions, recalculating routes in response to the changes in traffic conditions, and proposing routes to users" refers to technologies and devices that monitor traffic information in real time, calculate new routes in response to changes in traffic conditions, and provide that information to users.

[2540] The navigation system of the present invention is composed of a voice input device, a server, a navigation API, and an emotion engine. This system converts the user's voice input into text data and extracts the destination and intermediate points from the text data. It also recognizes the user's emotions from the voice data and proposes the optimal route. This section specifies the hardware and software used and explains how the system specifically operates.

[2541] System configuration

[2542] 1. Voice input device: A voice input device such as a smartphone or car navigation terminal is used to obtain voice instructions from the user.

[2543] 2. Speech recognition API: Use Google Speech-to-Text or similar to convert voice data into text data.

[2544] 3. Text and voice data analysis: The server uses ChatGPT or a similar generative AI model to analyze the text data and extract destinations and intermediate stops.

[2545] 4. Emotion Recognition Engine: Uses IBM Watson and other emotion recognition technologies to recognize user emotions from voice data.

[2546] 5. Navigation API: Uses Google Maps API etc. to calculate the optimal route reflecting real-time traffic information.

[2547] Example of operation

[2548] Example 1: Destination setting and emotion recognition

[2549] 1. User: The user speaks into a voice input device, saying, "Set the destination to Hotel X in Shibuya, Tokyo. I'd like to eat ramen on the way around 1:00 p.m."

[2550] 2. Device: Calls the speech recognition API to convert this voice input into text data.

[2551] 3. Terminal: The converted text data and audio data are sent to the server.

[2552] 4. Server: The server inputs the text data into ChatGPT and extracts the destination "XX Hotel" and the stopover "Ramen Shop" through natural language analysis.

[2553] 5. Server: Using the emotion engine, recognize the user's emotion from the voice data. For example, it determines that the user is tired.

[2554] 6. Server: Calculates the optimal route using the navigation API.

[2555] 7. Server: Adjust the recommendations based on the user's emotions and create recommendations that include relaxing stops and rest areas.

[2556] 8. Server: Sends the calculation result to the terminal as a proposal message.

[2557] 9. Terminal: The proposed message is converted into speech using a speech synthesis API and played back to the user.

[2558] 10. User: Review the proposed route and respond, "Yes, that's fine."

[2559] 11. Terminal: The user's response is again converted into text using the speech recognition API and sent to the server.

[2560] 12. Server: Confirms user consent, finalizes the route, and sends the final route data to the device.

[2561] 13. Terminal: Starts navigation based on the received route data, and provides screen display and voice guidance.

[2562] Prompt Sentence Examples

[2563] Prompt example 1: Speech data text conversion and emotion recognition

[2564] "Voice data: 'I want to eat ramen at 1pm.' Convert this voice data into text and recognize the user's emotions."

[2565] Example prompt 2: Route calculation and adjustment

[2566] "Destination: 'X hotel, Shibuya-ku, Tokyo' Stop: 'X ramen shop' User's emotion: 'Tired' Calculate the optimal route based on this data and generate a suggested message including stopovers and rest areas where people can relax."

[2567] This allows the system of the present invention to propose a personalized route that takes into account the user's emotions and to adjust the route optimally in real time.

[2568] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2569] Step 1:

[2570] User: The user gives instructions to the voice input device, such as, "Set Hotel X as the destination. I'd like to eat ramen on the way around 1:00 p.m."

[2571] Specific behavior:

[2572] Users speak into the microphone on their smartphone or car navigation device.

[2573] Input: Audio data

[2574] Output: Audio data

[2575] Step 2:

[2576] On the device: Call a speech recognition API (such as Google Speech-to-Text) to convert the voice input into text data. Retrieve the converted text data.

[2577] Specific behavior:

[2578] Send the voice data to the voice recognition API and convert it into text data.

[2579] Keeps the converted text in memory.

[2580] Input: Audio data

[2581] Output: Text data

[2582] Step 3:

[2583] Terminal: Sends text data and voice data itself to the server.

[2584] Specific behavior:

[2585] Use an HTTP POST request to send text and audio data to a specified endpoint on the server.

[2586] Input: Text data, audio data

[2587] Output: Send data to the server

[2588] Step 4:

[2589] Server: The received text data is input into ChatGPT, and the destination and intermediate points are extracted using natural language analysis.

[2590] Specific behavior:

[2591] Call the ChatGPT API and send text data as input data.

[2592] Extract the destination "XX Hotel" and the transit point "△△ Ramen Shop" from the ChatGPT output.

[2593] Input: Text data

[2594] Output: Destination, intermediate destination

[2595] Step 5:

[2596] Server: Uses an emotion engine to recognize the user's emotions from the voice data, for example, determining whether the user is anxious or relaxed based on the tone and speed of the voice.

[2597] Specific behavior:

[2598] Extract features such as tone, speed, and volume of speech.

[2599] Feature data is input into the emotion engine to recognize emotions (e.g., "tired").

[2600] Input: Audio data

[2601] Output: Emotion data

[2602] Step 6:

[2603] Server: Uses the navigation API (Google Maps API) to calculate the optimal route based on the extracted destination and intermediate destinations, taking real-time traffic information into account.

[2604] Specific behavior:

[2605] Send a request to the Google Maps API to get the optimal route via "XX Hotel" and "△△ Ramen Shop."

[2606] Receive route data including real-time traffic information.

[2607] Input: Destination, stopover

[2608] Output: Optimal route data

[2609] Step 7:

[2610] Server: Adjusts the recommendations based on the user's emotions as recognized by the emotion engine. For example, if the user is determined to be tired, the server will suggest more relaxing rest areas or stopovers.

[2611] Specific behavior:

[2612] The output of the emotion engine will be analyzed, and an option will be added to suggest a place to relax if the user's emotion is recognized as "tired."

[2613] Input: Emotion data, optimal route data

[2614] Output: Adjusted proposal

[2615] Step 8:

[2616] Server: Generates a message to suggest to the user based on the calculation results and adjustments based on emotions.

[2617] Specific behavior:

[2618] By combining the navigation route and emotion data, a message such as "We suggest a route with the destination being the XX hotel, stopping at the △△ ramen shop around 1:00 p.m. We will also suggest a place to rest and refresh yourself" is generated.

[2619] Input: Adjusted proposal

[2620] Output: Proposal message

[2621] Step 9:

[2622] Server: Sends a proposal message to the terminal.

[2623] Specific behavior:

[2624] Send the generated message to the device using an HTTP POST request.

[2625] Input: Proposal message

[2626] Output: Sending a message to the terminal

[2627] Step 10:

[2628] Terminal: The proposed message is converted into speech using a speech synthesis API (e.g., Amazon Polly) and played back to the user.

[2629] Specific behavior:

[2630] The message text is sent to the speech synthesis API and converted into voice data.

[2631] The generated audio data is played back from the device's speaker.

[2632] Input: Proposal message

[2633] Output: Audio data

[2634] Step 11:

[2635] User: Review the proposed route and respond "Yes, that's fine."

[2636] Specific behavior:

[2637] The user responds to the system by saying, "Yes, that would be fine."

[2638] Input: Proposal message

[2639] Output: User's voice response

[2640] Step 12:

[2641] Terminal: The user's response is converted into text again using the speech recognition API, and the data is sent to the server.

[2642] Specific behavior:

[2643] Send the voice data to the voice recognition API and convert it into text data.

[2644] Sends text data to the server.

[2645] Input: User's voice response

[2646] Output: Text data

[2647] Step 13:

[2648] Server: Confirms user consent and finalizes the route, which includes detailed navigation instructions.

[2649] Specific behavior:

[2650] Receive the user's consent response and finalize the route.

[2651] Generate detailed route data including navigation instructions.

[2652] Input: User's text response

[2653] Output: Final route data

[2654] Step 14:

[2655] Server: Sends the confirmed route data to the terminal.

[2656] Specific behavior:

[2657] Send an HTTP POST request to send the confirmed route data to the device.

[2658] Input: Final route data

[2659] Output: Sending data to the terminal

[2660] Step 15:

[2661] Terminal: Starts navigation based on received route data, provides on-screen display and audio guidance.

[2662] Specific behavior:

[2663] The received route data is analyzed and navigation begins.

[2664] Route instructions are provided on screen and via voice guidance.

[2665] Input: Final route data

[2666] Output: Navigation instructions (screen display, voice guide)

[2667] (Application example 2)

[2668] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2669] Conventional navigation systems only set destinations and intermediate stops based on user instructions and modify routes according to traffic conditions, so they were unable to provide personalized suggestions that took the user's emotional state into account.The present invention aims to provide more personalized route suggestions and user experiences by incorporating a function that recognizes the user's emotions into the navigation system, thereby reducing user stress and improving user satisfaction.

[2670] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for converting voice input from a user into text data, means for analyzing the text data to extract a destination and intermediate points, means for recognizing the user's emotions from the voice data, means for calculating an optimal route based on the destination and intermediate points, means for adapting the proposal content based on the user's emotions, means for proposing information on the calculated route and intermediate points to the user, means for setting a final route based on a response from the user, and means for monitoring changes in traffic conditions, recalculating a route in accordance with the changes in traffic conditions, and proposing the route to the user. This makes it possible to propose personalized routes and intermediate points according to the user's emotional state.

[2671] "Voice input data" refers to information provided by a user to give voice instructions to the navigation system.

[2672] "Text data" refers to character information converted from voice input data using voice recognition technology.

[2673] A "destination" is the final location that a user wants to reach using a navigation system.

[2674] A "waypoint" is a place where you plan to stop on the way to your destination.

[2675] "Emotion recognition method" is a technology that analyzes the user...

Claims

1. means for converting voice input from a user into text data; means for analyzing the text data and extracting destinations and intermediate points; means for calculating an optimal route based on the destination and intermediate points; means for suggesting the calculated route and route information to a user; a means for setting a final route based on the response from the user; a means for monitoring changes in traffic conditions, recalculating routes in response to the changes in traffic conditions, and suggesting the routes to the user; A system including:

2. 2. The system according to claim 1, further comprising means for adjusting timings and route points for rest breaks in accordance with changes in the acquired traffic conditions and suggesting these to the user.

3. 2. The system according to claim 1, further comprising means for responding to voice input from a user in a natural conversational manner and for setting and correcting a destination and intermediate points.

Images