System

Abstract

To provide a system capable of route guidance that corresponds to a user's emotional state.SOLUTION: The present system comprises: means for receiving a voice indication by a user; means for acquiring the current location of the user; means for searching for at least one of a fuel supply facility and a parking lot facility which is currently unoccupied; means for acquiring a route from the current location to the searched fuel supply facility or parking lot facility; means for identifying the emotional state of the user by using an emotion engine; means for generating a prompt sentence that directs generation of a corrected route on the basis of the identified emotion of the user and the route; means for generating a corrected route by using the generated prompt sentence and a generative AI model; and means for presenting, to the user route, guidance information along the generated route after correction.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, when users want to find the nearest, cheapest gas station or the cheapest available parking space, they have to use multiple applications and websites to compare information, which is time-consuming, laborious, and inefficient. [Means for solving the problem]

[0005] The present invention provides a system that accepts voice instructions through natural dialogue, searches for the nearest inexpensive gas station or the cheapest parking lot that is currently available from the current location, and provides route guidance to the searched gas station or parking lot. This allows the user to efficiently search for the optimal gas station or parking lot and receive route guidance by simply using one system. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The system of the present invention includes a voice recognition module that accepts voice instructions from a user, a search module that searches for information on gas stations and parking lots, and a navigation module that provides route guidance. The voice recognition module accepts voice instructions from a user through natural dialogue and converts them into text data. The search module searches for the nearest and cheapest gas station from the current location or the cheapest parking lot that is currently available based on the converted text data. The navigation module provides route guidance to the searched gas station or parking lot.

[0029] "Example 2"

[0030] In a specific embodiment, when a user issues a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts the command into text data. Then, the search module searches for the nearest cheap gas station from the current location. The search results are sent to the navigation module, which provides the user with optimal route guidance.

[0031] The process flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: The user gives a voice command, for example, "What is the nearest cheap gas station?"

[0034] Step 2: The voice recognition module accepts the user's voice commands and converts them into text data.

[0035] Step 3: The search module uses the converted text data to search for the closest, cheapest gas station from the current location.

[0036] Step 4: The navigation module provides directions to the searched gas station.

[0037] "Example 2"

[0038] Step 1: The user issues a voice command, for example, "Tell me where the nearest available parking space is."

[0039] Step 2: The voice recognition module accepts the user's voice commands and converts them into text data.

[0040] Step 3: The search module uses the converted text data to search for the closest, most available and cheapest parking lot from the current location.

[0041] Step 4: The navigation module provides route guidance to the searched parking lot.

[0042] Example 1

[0043] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0044] Conventional navigation systems have the problem that even if a user issues voice instructions, the accuracy of voice recognition is low, making it difficult to obtain accurate search results. In addition, when searching for gas stations or parking lots, it is often impossible to quickly find the nearest and cheapest facility. Furthermore, route guidance based on search results is insufficient, and users may not be able to reach their destination efficiently. To solve these problems, there is a demand for a system that improves the accuracy of voice recognition, provides quick and accurate search results, and provides efficient route guidance.

[0045] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0046] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for converting the voice instructions into text data, a means for searching for the cheapest fuel supply facility closest to the current location, a means for searching for the cheapest parking facility closest to the current location that is currently available, a means for providing route guidance to the searched fuel supply facility or parking facility, and a means for transmitting and receiving data including the voice instructions, search results, and route guidance. This allows the user to give voice instructions in a natural dialogue format, improves the accuracy of voice recognition, and enables quick and accurate search results to be obtained. Furthermore, efficient route guidance allows the user to quickly reach the destination.

[0047] "Natural dialogue voice instructions" refers to voice input given by a user to the system using everyday words and phrases.

[0048] "Means for converting voice instructions into text data" refers to technology or devices for analyzing a user's voice input and converting it into text information.

[0049] "Means for searching for the nearest cheapest fuel supply facility from the current location" refers to technology or devices for identifying the nearest gas station or other facility with the cheapest fuel price based on the user's current location.

[0050] "Means for searching for the closest, most available, and least expensive parking facility from the current location" refers to technology or devices for identifying the closest parking facility that is available and has the lowest parking fees based on the user's current location.

[0051] "Means for providing route guidance to the searched fuel supply facility or parking facility" refers to technology or devices for providing the user with the optimal route to reach their destination based on the search results.

[0052] "Means for transmitting and receiving data including voice instructions, search results, and route guidance" refers to technology or devices for transmitting and receiving information regarding a user's voice instructions, search results, and route guidance via a communications network.

[0053] The present invention is a system that receives voice instructions from a user, searches for the nearest and cheapest fuel supply facility or parking facility, and provides route guidance based on the results. A specific embodiment of this system will be described below.

[0054] System Configuration

[0055] Subject: User

[0056] A user issues voice instructions to a terminal while in a car or on the move, for example, using prompt sentences such as "Tell me where the nearest cheap gas station is" or "Find the nearest cheap parking lot."

[0057] Subject: Terminal

[0058] The terminal uses a microphone to receive voice instructions from the user. The voice data is temporarily stored in the terminal and transmitted to a voice recognition module. A general voice recognition API is used for the voice recognition. For example, Google (registered trademark) Cloud Speech-to-Text API is used.

[0059] The voice recognition module converts the voice data into text data that accurately reflects the user's intent, and the converted text data is sent from the terminal to the server.

[0060] Subject: Server

[0061] The server starts a search module based on the received text data. The search module searches for the nearest and cheapest fuel supply station or the cheapest parking facility that is currently available based on the user's current location information. A general map API is used for the search. For example, Google Maps API is used.

[0062] The search module retrieves search results and returns them to the server, which may include the names, addresses, price information, etc. of establishments. The server then transmits these search results to the terminal.

[0063] The terminal starts a navigation module based on the received search results. The navigation module provides route guidance to the user. For the route guidance, a general navigation API is used. For example, the Google Maps Directions API is used.

[0064] Examples

[0065] Example 1: Searching for gas stations

[0066] When a user says, "Tell me where the nearest cheap gas station is," the device's microphone captures the speech. The speech recognition module converts the speech into text data and sends it to the server. The server uses the Google Maps API to find the nearest cheap gas station and returns the results to the device. The device then launches the navigation module and provides route guidance to the user.

[0067] Example 2: Searching for parking spaces

[0068] When a user says, "Find the nearest cheap parking," the device's microphone captures the voice. The speech recognition module converts the voice into text data and sends it to the server. The server uses the Google Maps API to find the nearest, available, and cheapest parking spots and returns the results to the device. The device then launches the navigation module and provides route guidance to the user.

[0069] In this way, the system starts with the user's voice command, and each step of the voice recognition, search, and navigation work together. This allows the user to give voice commands in a natural dialogue format, improving the accuracy of voice recognition and enabling quick and accurate search results. In addition, efficient route guidance allows the user to reach their destination quickly.

[0070] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0071] Step 1:

[0072] The user issues a voice command to the terminal saying, "Tell me the nearest cheap gas station."

[0073] Input: User's voice command

[0074] Output: Audio data is input to the device.

[0075] Specific operation: When a user issues a voice command inside the car, the device's microphone captures the voice.

[0076] Step 2:

[0077] The terminal uses a built-in microphone to collect the user's voice instructions.

[0078] Input: User's voice data

[0079] Output: Collected audio data

[0080] Specific operation: The device temporarily stores the voice data and prepares it to be sent to the voice recognition module.

[0081] Step 3:

[0082] The terminal starts the voice recognition module and sends the collected voice data to the voice recognition API.

[0083] Input: Collected audio data

[0084] Output: Audio data sent to the speech recognition API

[0085] What happens: The device calls the Google Cloud Speech-to-Text API to convert the voice data into text.

[0086] Step 4:

[0087] The voice recognition module converts the voice data into text data.

[0088] Input: Voice data sent to the speech recognition API

[0089] Output: Converted text data

[0090] What happens: The speech recognition module converts the speech "What's the nearest cheap gas station?" into text.

[0091] Step 5:

[0092] The terminal transmits the converted text data to the server.

[0093] Input: Converted text data

[0094] Output: Text data sent to the server

[0095] Specific operation: The device sends text data to the server using the HTTPS protocol.

[0096] Step 6:

[0097] The server launches a search module based on the received text data.

[0098] Input: Text data sent to the server

[0099] Output: Search module launch

[0100] Specific operations: The server analyzes the text data and launches the search module.

[0101] Step 7:

[0102] The search module searches for the nearest, least expensive fuel supply facility based on the user's current location.

[0103] Input: User's current location and text data

[0104] Output: Search results (fuel supply facility information)

[0105] What it does: The search module uses the Google Maps API to find the nearest and cheapest gas stations.

[0106] Step 8:

[0107] The search module retrieves the search results and returns them to the server.

[0108] Input: Search results (fuel supply facility information)

[0109] Output: Search results returned to the server

[0110] Specific Operation: The search module returns search results to the server, including the business's name, address, price information, etc.

[0111] Step 9:

[0112] The server transmits the obtained search results to the terminal.

[0113] Input: Search results returned by the server

[0114] Output: Search results sent to the terminal

[0115] Specific operation: The server sends the search results to the terminal using the HTTPS protocol.

[0116] Step 10:

[0117] The terminal launches a navigation module based on the received search results.

[0118] Input: Search results sent to the device

[0119] Output: Launch of navigation module

[0120] Specific operations: The device analyzes the search results and launches the navigation module.

[0121] Step 11:

[0122] The navigation module provides route guidance to the user.

[0123] Input: Launch of navigation module and search results

[0124] Output: Directions provided to the user

[0125] What it does: The navigation module uses the Google Maps Directions API to provide the user with specific route directions via voice.

[0126] (Application example 1)

[0127] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0128] Conventional navigation systems require users to manually search for gas stations and parking lots, which may compromise safety while driving. It is also difficult to efficiently find the nearest and cheapest gas stations and parking lots. In addition, route guidance based on search results may not be intuitive, which may reduce user convenience.

[0129] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0130] In this invention, the server includes a means for receiving voice instructions through natural dialogue, a means for searching for the cheapest gas station closest to the current location, a means for searching for the cheapest parking lot closest to the current location that is currently available, a means for providing route guidance to the searched gas station or parking lot, a means for converting the voice instructions into text data, and a means for displaying a route on a map based on the search results. This enables a user to safely and efficiently search for a gas station or parking lot using voice instructions and receive intuitive route guidance even while driving.

[0131] "Natural dialogue voice instructions" refers to voice input given to the system by a user using everyday language.

[0132] "Means for accepting voice instructions" refers to devices or software that have the function of recognizing voice input from a user and incorporating it into the system.

[0133] "Means for searching for the closest, least expensive gas station from a user's current location" means a device or software capable of identifying the closest, least expensive gas station based on a user's current location.

[0134] "Means for searching for the closest, most available, and least expensive parking space from the current location" refers to a device or software that has the functionality to identify the closest, most available, and least expensive parking space based on the user's current location.

[0135] "Means for providing route guidance to the searched gas station or parking lot" refers to devices or software that have the function of guiding the user to their destination based on the search results.

[0136] "Means for converting voice instructions into text data" refers to a device or software that has the function of converting a user's voice input into text information.

[0137] "Means for displaying a route on a map based on the search results" refers to a device or software having a function for visually displaying a route to a searched destination on a map.

[0138] The following system configuration is used as an embodiment of the present invention. The system includes a voice recognition module, a search module, and a navigation module. These modules work together to accept voice instructions from a user, search for information on gas stations and parking lots, and provide route guidance.

[0139] Speech Recognition Module

[0140] The speech recognition module accepts the user's voice instructions and converts them into text data. Specifically, it uses a microphone to capture the user's voice and converts the voice into text using the SpeechRecognition library. This allows the user to give instructions to the system in a natural dialogue format.

[0141] Search Module

[0142] The search module searches for the nearest and cheapest gas station or the cheapest parking space that is currently available based on the text data received from the voice recognition module. Specifically, it uses the Google Places API to perform searches based on the user's current location. The search results are selected based on the most convenient results for the user.

[0143] Navigation Module

[0144] The navigation module provides route guidance to the destination based on the results obtained from the search module. Specifically, it uses the Folium library to display the route based on the search results on a map. This allows the user to visually check the route to the destination.

[0145] Hardware and Software Used

[0146] Hardware: Microphone, GPS module

[0147] Software: SpeechRecognition library, requests library, Google Places API, Folium library

[0148] Examples

[0149] When a user says, "Tell me where the nearest gas station is," the system searches for the nearest gas station from the user's current location and displays the route. The following is an example of a prompt:

[0150] Examples of prompt statements

[0151] User: "What's the closest gas station?"

[0152] System: "Voice commands please."

[0153] User: "What's the closest gas station?"

[0154] System: "Recognized text: Where is the nearest gas station?"

[0155] System: "Searching for gas stations..."

[0156] System: "Route to gas station x has been generated."

[0157] In this way, users can safely and efficiently search for gas stations and parking lots and receive intuitive route guidance while driving.

[0158] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0159] Step 1:

[0160] The user issues a voice command.

[0161] The user issues a voice command into the microphone, such as "Tell me where the nearest gas station is." The input is the user's voice, and the output is voice data.

[0162] Step 2:

[0163] The terminal captures the voice data and sends it to the voice recognition module.

[0164] The device captures the user's voice using a microphone and converts the voice data into text data using the SpeechRecognition library. The input is voice data and the output is text data.

[0165] Step 3:

[0166] The server receives the text data and sends it to the search module.

[0167] The server analyzes the text data received from the speech recognition module to understand the user's intent. The input is the text data, and the output is the search query.

[0168] Step 4:

[0169] Based on the search query, the server searches for the closest, cheapest gas stations and parking lots from the current location.

[0170] The server uses the Google Places API to search for gas stations and parking lots based on the user's current location. The input is the search query and the current location data, and the output is the search results.

[0171] Step 5:

[0172] The server sends the search results to the navigation module.

[0173] The server analyzes the search results and selects the most suitable gas station or parking lot. The input is the search results, and the output is the selected destination data.

[0174] Step 6:

[0175] The terminal receives the selected destination data and provides route guidance.

[0176] The device uses the Folium library to display routes to selected destinations on a map. The input is destination data, and the output is a map with route directions.

[0177] Step 7:

[0178] The user follows the route guidance to the destination.

[0179] The user follows the route guidance displayed on the terminal to head to a gas station or parking lot. The input is a map of the route guidance, and the output is reaching the destination.

[0180] Example 2

[0181] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0182] In conventional systems, when a user gives voice instructions, it is difficult to accurately understand the instructions and provide appropriate information. In addition, route guidance based on search results for gas stations and parking lots is insufficient, resulting in low user convenience. Furthermore, the accuracy and speed of the process of converting voice instructions into text data are also issues. There is a demand for a system that can solve these problems and provide users with quick and accurate information and route guidance.

[0183] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0184] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for converting the voice instructions into text data, a means for searching for the cheapest gas station closest to the current location, a means for transmitting the search results to the navigation module, and a means for providing route guidance to the searched gas station. This makes it possible to accurately understand voice instructions given by the user and provide appropriate information promptly. In addition, by providing optimal route guidance based on the search results, it is possible to improve user convenience.

[0185] "Natural dialogue voice instructions" refers to voice input by the user in everyday language, with the goal of the system understanding the speech and responding appropriately.

[0186] "Means for converting voice instructions into text data" refers to technology that analyzes a user's voice input and converts it into a corresponding text format.

[0187] "Current location" refers to where a user is currently located, typically determined based on location information such as GPS data.

[0188] "Method of finding nearest cheapest gas stations" refers to a search algorithm or database used to identify the nearest cheapest gas stations based on a user's current location.

[0189] "Means for transmitting search results to a navigation module" refers to a technique for transmitting information on the searched gas stations to a navigation system and generating route guidance.

[0190] "Means of route guidance" refers to technology for providing users with the optimal route to their destination, and includes map displays and voice guidance.

[0191] This invention is a system that searches for the nearest inexpensive gas station from the user's current location in response to voice instructions, and provides route guidance based on the search results. A specific embodiment of this system will be described below.

[0192] Receiving and converting voice commands

[0193] The user issues a voice command such as "Tell me where the cheapest gas station is nearby." A voice recognition module (e.g., a voice recognition API) installed in the device converts this voice command into text data. The voice recognition module analyzes the voice signal and generates the corresponding text.

[0194] Gas station search

[0195] The server receives the text data and searches for the nearest and cheapest gas station from the user's current location using a search module (e.g., map API). The search module obtains the user's current location information from GPS data and gas station price information from a database.

[0196] Providing navigation

[0197] The server sends the search results to a navigation module (e.g., navigation API) to generate optimal route guidance. The device provides this route guidance to the user. Specifically, the map and route are displayed on the smartphone screen, and voice guidance is also provided.

[0198] Examples

[0199] For example, if a user is near Tokyo Station, they may give a voice command such as, "Tell me the nearest cheap gas station." The voice recognition module converts this command into text data, and the server uses a map API to search for gas stations near Tokyo Station. The search results may reveal that "Gas Station A" is the closest and cheapest. The server uses the navigation API to generate the optimal route to "Gas Station A," and the device provides route guidance to the user. Specifically, a map and route to "Gas Station A" are displayed on the smartphone screen, and voice guidance such as, "Turn right 500 meters ahead" is given.

[0200] Examples of prompt statements

[0201] "If a user says, 'Tell me where the cheapest gas station is nearby,' how would you proceed?"

[0202] This invention allows users to quickly find the nearest, cheapest gas station by simply giving voice instructions, and furthermore, to receive optimal route guidance to that location, which greatly improves user convenience.

[0203] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0204] Step 1:

[0205] The user gives a voice command.

[0206] A user issues a voice command such as, "Tell me the nearest cheap gas station." The input is the user's voice, and the output is a voice signal.

[0207] Step 2:

[0208] The device converts the voice into text data.

[0209] A voice recognition module (e.g., a voice recognition API) installed in the device receives the user's voice instructions and converts them into text data. Specifically, it analyzes the voice signal and generates the corresponding text. The input is the voice signal, and the output is the text data.

[0210] Step 3:

[0211] The server receives the text data.

[0212] The terminal generates text data and sends it to the server. The server receives the text data and proceeds to the next processing step. The input is the text data, and the output is a receipt confirmation.

[0213] Step 4:

[0214] The server searches for gas stations.

[0215] The server analyzes the text data and obtains the user's current location information. It then uses a search module (e.g., map API) to search for the nearest, least expensive gas station from the user's current location. Specifically, it uses GPS data to identify the current location and obtains gas station price information from a database. The input is the text data and GPS data, and the output is the search results.

[0216] Step 5:

[0217] The server sends the search results to the navigation module.

[0218] The server sends the search results to a navigation module (e.g., navigation API) and generates optimal route guidance. Specifically, it calculates the optimal route from the user's current location based on the location information of gas stations obtained as search results. The input is the search results, and the output is route guidance data.

[0219] Step 6:

[0220] The terminal provides route guidance to the user.

[0221] The terminal provides the route guidance received from the navigation module to the user. Specifically, the map and route are displayed on the smartphone screen, and voice guidance is also provided. For example, specific instructions such as "Turn right 500 meters ahead" are provided by voice. The input is route guidance data, and the output is route guidance to the user.

[0222] (Application example 2)

[0223] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0224] Conventional navigation systems have difficulty efficiently searching for the nearest and cheapest gas stations and parking lots and providing real-time updated information, even when users give voice instructions. In addition, they lack sufficient coordination with autonomous vehicles and are unable to provide optimal route guidance. This can lead to users spending a lot of time and effort to reach their destination.

[0225] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0226] In this invention, the server includes a means for receiving voice instructions through natural dialogue, a means for searching for the nearest and cheapest gas station from the current location, a means for searching for the nearest and cheapest parking lot from the current location that is currently available, a means for providing route guidance to the searched gas station or parking lot, a means for updating price and location information in real time, a means for providing search results based on the voice instructions, and a means for providing optimal route guidance in cooperation with the navigation system of the autonomous vehicle. This allows the user to efficiently search for the nearest and cheapest gas station or parking lot through voice instructions, and to receive optimal route guidance based on information updated in real time.

[0227] "Voice instructions through natural dialogue" refers to voice input made by the user in natural language, similar to everyday conversation.

[0228] "Means for searching for the closest, least expensive gas station from a current location" refers to a method or device for identifying the closest, least expensive gas station based on a user's current location.

[0229] "Means for searching for the closest, currently available, and cheapest parking lot from the current location" refers to a method or device for identifying the closest, available, and cheapest parking lot based on the user's current location.

[0230] "Means for providing route guidance to a searched gas station or parking lot" refers to a method or device for providing a user with the optimal route to a gas station or parking lot that the user has searched for.

[0231] "Means for updating price and location information in real time" refers to a method or apparatus for obtaining and updating price and location information for gas stations and parking lots in real time.

[0232] "Means for providing search results based on voice instructions" refers to a method or device for generating and providing search results based on a user's voice instructions.

[0233] "Means for providing optimal route guidance in cooperation with the navigation system of an autonomous vehicle" refers to a method or device for providing optimal routes in cooperation with the navigation system of an autonomous vehicle.

[0234] The following system configuration will be described as an embodiment of the present invention.

[0235] System Configuration

[0236] The system includes a voice recognition module, a search module, a navigation module, a real-time update module, and a module that interfaces with the autonomous vehicle's navigation system.

[0237] Speech Recognition Module

[0238] The voice recognition module converts the voice into text data when the user gives a voice command through natural dialogue. Specifically, it inputs the voice using a microphone and converts the voice into text using the speech_recognition library.

[0239] Search Module

[0240] The search module searches for the nearest and cheapest gas station and the cheapest parking lot that is currently available based on the user's current location. To do this, it uses the GPS module to get the current location and the requests library to get information about gas stations and parking lots from external APIs.

[0241] Navigation Module

[0242] The navigation module provides optimal route guidance based on the search results. It uses the navigation API to calculate the shortest route from the user's current location to the destination and provides guidance.

[0243] Real-time update module

[0244] The real-time update module updates prices and location information for gas stations and parking lots in real time, so users can always get the most up-to-date search results.

[0245] Linking with autonomous vehicle navigation systems

[0246] The module, which works with the autonomous vehicle's navigation system, sends the search results to the autonomous vehicle's navigation system and provides optimal route guidance, allowing users to reach their destination efficiently through voice instructions.

[0247] Examples

[0248] For example, when a user gives a voice command such as "Tell me the nearest cheapest gas station," the voice recognition module converts the command into text data. The search module then searches for the nearest cheapest gas station from the current location. The search results are sent to the navigation module, which provides the optimal route guidance for the user. The real-time update module keeps the gas station prices and location information always up to date. By linking with the autonomous vehicle's navigation system, users can use the autonomous vehicle to reach their destination efficiently.

[0249] Examples of prompt statements

[0250] When a user gives a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts this command into text data, and the search module searches for the nearest cheap gas station from the current location. The search results are sent to the navigation module, and the system is designed to provide the optimal route guidance for the user.

[0251] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0252] Step 1:

[0253] The user gives a voice command.

[0254] The user gives voice instructions in a natural dialogue, such as "Tell me where the nearest cheap gas station is." The input is the user's voice, and the output is voice data.

[0255] Step 2:

[0256] The terminal converts the voice data into text data.

[0257] The speech recognition module uses a microphone to capture voice data and converts the voice to text using the speech_recognition library. The input is the voice data and the output is the text data.

[0258] Step 3:

[0259] The server gets the current location.

[0260] Use the device's GPS module to obtain the user's current location. The input is the GPS signal, and the output is the coordinate data of the current location.

[0261] Step 4:

[0262] The server obtains gas station information.

[0263] Use the requests library to get gas station location and price information from an external API. The input is the API request, and the output is data including gas station location and price information.

[0264] Step 5:

[0265] The server searches for the nearest cheapest gas station.

[0266] Based on the acquired gas station information, the distance from the current location and the price are calculated to identify the nearest and cheapest gas station. The input is the gas station location and price information, and the current location coordinate data, and the output is the optimal gas station information.

[0267] Step 6:

[0268] The server generates the navigation information.

[0269] Use the navigation API to generate route guidance from the current location to the best gas station. The input is the coordinate data of the current location and the location information of the best gas station, and the output is the route guidance information.

[0270] Step 7:

[0271] The terminal provides route guidance to the user.

[0272] The navigation module guides the user on the optimal route by voice or visually based on the generated route guidance information. The input is the route guidance information, and the output is navigation instructions to the user.

[0273] Step 8:

[0274] The server updates the information in real time.

[0275] The real-time update module periodically retrieves gas station prices and location information from an external API and updates the data in the system. The input is the API request and the output is the updated gas station information.

[0276] Step 9:

[0277] The server works in conjunction with the autonomous vehicle's navigation system.

[0278] The search results and route guidance information are sent to the navigation system of the autonomous vehicle, and the system controls the vehicle so that it automatically drives along the optimal route. The input is route guidance information, and the output is control instructions for the autonomous vehicle.

[0279] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0280] "Example 1"

[0281] As an embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotion of a user. The system includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station from the current location, a means for searching for the cheapest parking lot that is currently available from the current location, and a means for providing route guidance to the searched gas station or parking lot. The system further includes an emotion engine that recognizes the emotion of the user. The emotion engine analyzes the emotion from the user's voice and provides the result to other parts of the system. For example, if the user feels that he or she is in a hurry, the emotion engine provides the information to the search means, and the search means searches for the nearest gas station or parking lot with priority. Also, if the user feels that he or she is relaxed, the emotion engine provides the information to the route guidance means, and the route guidance means can provide services according to the user's emotion, such as suggesting a route with a better view.

[0282] "Example 2"

[0283] As an embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotion of a user. The system includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station from the current location, a means for searching for the cheapest parking lot that is currently available from the current location, and a means for providing route guidance to the searched gas station or parking lot. The system further includes an emotion engine that recognizes the emotion of the user. The emotion engine analyzes the emotion from the user's voice and provides the result to other parts of the system. For example, if the user feels that he or she is in a hurry, the emotion engine provides the information to the search means, and the search means searches for the nearest gas station or parking lot with priority. Also, if the user feels that he or she is relaxed, the emotion engine provides the information to the route guidance means, and the route guidance means can provide services according to the user's emotion, such as suggesting a route with a better view.

[0284] The process flow of each embodiment will be described below.

[0285] "Example 1"

[0286] Step 1: The user issues a voice command to the system, such as "Find the nearest cheap gas station."

[0287] Step 2: The system's voice recognition module converts the voice instructions into text data, while the emotion engine analyzes the emotions from the user's voice.

[0288] Step 3: Based on the analysis result of the sentiment engine, the system's search module searches for the nearest cheap gas station. For example, if the user feels that he is in a hurry, it will prioritize searching for the nearest gas station.

[0289] Step 4: Based on the search results, the system's navigation module will provide route guidance to the searched gas station. For example, if the user feels relaxed, it will suggest a more scenic route.

[0290] Example 1

[0291] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0292] Conventional navigation systems can accept voice instructions from users, but they are unable to provide search results or route guidance that take the user's emotions into account. This means that there is an issue with them being unable to provide optimal guidance according to the situation, such as when the user is in a hurry or relaxed.

[0293] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0294] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for converting the voice instructions into text data, a means for searching for the cheapest fuel supply facility closest to the current location, a means for searching for the cheapest parking facility closest to the current location that is currently available, a means for analyzing the user's emotions, a means for providing route guidance to the searched fuel supply facility or parking facility, and a means for adjusting the search results and route guidance based on the user's emotions, thereby enabling optimal search results and route guidance according to the user's emotions and circumstances.

[0295] "Natural dialogue voice instruction" refers to voice input by the user in everyday language, and is voice instruction that does not rely on specific commands or keywords.

[0296] "Means for converting voice instructions into text data" refers to a function that uses voice recognition technology to convert a user's voice input into text information.

[0297] A "fuel distribution facility" refers to a facility that supplies fuel or energy to vehicles, such as a gas station or an electric vehicle charging station.

[0298] A "parking facility" is a place for temporarily parking a vehicle, and refers to facilities including parking lots and parking meters.

[0299] "Means for analyzing user emotions" refers to technologies used to determine a user's emotional state from voice or other input data.

[0300] "Means for adjusting search results and route guidance" refers to the ability to change the content of search results and route guidance provided depending on the user's emotions and situation.

[0301] The present invention is a system that receives voice instructions from a user, analyzes emotions, searches for optimal fuel supply facilities and parking facilities, and provides appropriate route guidance. A specific embodiment of this system will be described below.

[0302] Hardware and Software Used

[0303] 1. Speech Recognition Module: Uses speech recognition technology to convert voice instructions into text data. This module uses a common speech recognition API.

[0304] Example: Speech Recognition API

[0305] 2. Search module: Uses the map API to search for the nearest cheapest fuel and parking facilities from your current location.

[0306] Example: Map Information API

[0307] 3. Navigation module: Uses the navigation API to provide route guidance to the searched facilities.

[0308] Example: Navigation API

[0309] 4. Emotion Engine: Uses emotion analysis technology to analyze emotions from the user's voice.

[0310] Example: Sentiment analysis API

[0311] Specific operation of the system

[0312] 1. Accepting user voice commands

[0313] The user issues a voice command in the car saying, "Tell me the nearest cheap gas station."

[0314] The device's microphone captures the user's voice and transmits it to the system as voice data.

[0315] 2. Voice-to-text conversion

[0316] The terminal transmits the voice data to a voice recognition module.

[0317] The speech recognition module converts voice data into text data using a speech recognition API. For example, a voice saying "Tell me where the nearest cheap gas station is" is converted into text saying "Tell me where the nearest cheap gas station is."

[0318] 3. Analysis and Search of Text Data

[0319] The server receives the converted text data and sends it to a search module.

[0320] The search module uses the map information API to search for the nearest and cheapest fuel supply facility from the current location. For example, if the current location is Shibuya-ku, Tokyo, the search module retrieves information on the nearest and cheapest gas station.

[0321] 4. Emotion Analysis

[0322] The terminal transmits the user's voice data to the emotion engine.

[0323] The emotion engine uses the emotion analysis API to analyze the user's emotions. For example, if it determines that the user's voice sounds urgent, it will send that information to the server.

[0324] 5. Getting search results

[0325] The server receives the results from the search module and the sentiment analysis results from the sentiment engine.

[0326] The server integrates the information of the nearest and cheapest fuel station with the emotion information that the user is in a hurry. For example, if the search result is "Gas Station A in Shibuya Ward" and it is determined that the user is in a hurry, the server sends the emotion information to the navigation module.

[0327] 6. Providing route guidance

[0328] The server sends the search results to the navigation module.

[0329] The navigation module uses the navigation API to provide route guidance to the searched fuel supply facility. For example, it guides the user on the shortest route to "Gas Station A in Shibuya Ward" by voice and map.

[0330] Examples and prompts

[0331] Examples

[0332] While driving, a user issues a voice command such as "Tell me the nearest cheap gas station." The device captures the voice and converts it into text using a voice recognition module. The server analyzes the text data and searches for the nearest cheapest gas station using a map information API. The emotion engine determines from the user's voice that the user is in a hurry and provides this information to the server. The server integrates the search results with the emotion information and sends it to the navigation module. The navigation module uses the navigation API to guide the user along the shortest route.

[0333] Examples of prompt statements

[0334] "Tell me where the nearest cheap gas station is."

[0335] "Find a parking spot that's currently available"

[0336] "Tell me the scenic route"

[0337] The system accepts the user's voice commands, analyzes emotions, searches for optimal fuel and parking facilities, and provides appropriate route guidance.

[0338] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0339] Step 1:

[0340] Accepting user voice commands

[0341] Input: User says, "What is the nearest cheap gas station?"

[0342] Specific operation: The device's microphone captures the user's voice and sends it to the system as voice data.

[0343] Output: Audio data is generated and sent to the speech recognition module.

[0344] Step 2:

[0345] Transcription of voice commands

[0346] Input: Audio data sent from the device.

[0347] Specific operation: The device sends voice data to the voice recognition module, which uses the voice recognition API to convert the voice data into text data.

[0348] Output: The text data "Please tell me the nearest cheap gas station" is generated and sent to the server.

[0349] Step 3:

[0350] Text data analysis and search

[0351] Input: Text data sent from the speech recognition module.

[0352] Specific operation: The server receives the converted text data and sends it to the search module, which uses the map information API to search for the nearest cheapest fuel supply station from the current location.

[0353] Output: The closest cheapest fuel station to your current location is generated.

[0354] Step 4:

[0355] Sentiment Analysis

[0356] Input: Audio data sent from the device.

[0357] Specific operation: The device sends the user's voice data to the emotion engine, which then uses the emotion analysis API to analyze the user's emotions.

[0358] Output: User emotion information (e.g., in a hurry) is generated and sent to the server.

[0359] Step 5:

[0360] Getting search results

[0361] Input: Search results from the search module and sentiment analysis results from the sentiment engine.

[0362] What it does: The server combines information about the nearest and cheapest fuel station with the user's emotional information that he or she is in a hurry.

[0363] Output: The integrated search results and sentiment information are generated and sent to the navigation module.

[0364] Step 6:

[0365] Providing directions

[0366] Input: The integrated search results and sentiment information sent from the server.

[0367] What happens next: The navigation module uses the navigation API to provide directions to the searched fueling facilities.

[0368] Output: Guidance information for the shortest route is generated and provided to the user via voice and map.

[0369] (Application example 1)

[0370] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0371] Conventional navigation systems can search for gas stations and parking lots and provide route guidance based on the user's voice commands, but they have a problem in that they cannot provide services that take the user's emotions into account. In particular, there is a demand for optimal route guidance depending on whether the user is in a hurry or relaxed.

[0372] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0373] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station nearest to the current location, a means for searching for the cheapest parking lot nearest to the current location that is currently available, a means for recognizing the user's emotions and providing services according to the emotions, and a means for preferentially searching for the nearest gas station or parking lot when the user is in a hurry and for suggesting a route with a nice view when the user is relaxed. This makes it possible to provide optimal route guidance according to the user's emotions.

[0374] "Natural dialogue voice instructions" refers to voice input given by a user to the system in everyday language.

[0375] A "gas station" is a facility for supplying fuel to vehicles.

[0376] A "parking lot" is a place for temporarily parking vehicles.

[0377] "Route guidance" refers to providing information showing directions to a destination.

[0378] "Means of recognizing emotions" refers to technology that analyzes emotions from the user's voice, facial expressions, etc., and provides the results to the system.

[0379] "Means of providing services" refers to technology that provides appropriate information and functions according to user requests and circumstances.

[0380] The "means for preferentially searching for the nearest gas station or parking lot" is a technology that preferentially searches for the nearest gas station or parking lot from the user's current location when the user is in a hurry.

[0381] The "means for suggesting scenic routes" is a technology that provides guidance on routes that are considered to have beautiful scenery when the user is relaxed.

[0382] The system for implementing the present invention includes a voice recognition module, a search module, a navigation module, and an emotion engine. The details of each module and its operation are described below.

[0383] Speech Recognition Module

[0384] The speech recognition module accepts the user's voice instructions and converts them into text data. Specifically, it uses a microphone to input the user's voice and uses the speech_recognition library to convert the voice into text. This text data is used for subsequent processing.

[0385] Search Module

[0386] The search module searches for the nearest and cheapest gas station and the cheapest parking lot that is currently available based on the text data obtained from the voice recognition module. Specifically, it uses the requests library to send requests to external APIs to obtain information on gas stations and parking lots.

[0387] Navigation Module

[0388] The navigation module provides route guidance to the gas station or parking lot obtained from the search module. Specifically, it uses the navigation_system module to calculate the optimal route to the destination and guides the user along it.

[0389] Emotion Engine

[0390] The emotion engine analyzes emotions from the user's voice and provides the results to the rest of the system. Specifically, it uses the emotion_recognition library to recognize emotions from voice data, and prioritizes finding the nearest gas station or parking lot if the user is in a hurry, or suggests scenic routes if the user is relaxed.

[0391] Examples

[0392] For example, if a user says, "Tell me where the nearest gas station is," the voice recognition module converts the voice to text, and the search module searches for the nearest gas station. The emotion engine analyzes the user's emotions, and if the user is in a hurry, it will prioritize the nearest gas station, and if the user is relaxed, it will suggest a scenic route.

[0393] Examples of prompt statements

[0394] When a user says "Tell me where the nearest gas station is," the speech recognition module converts the speech to text, and the search module searches for the nearest gas station. The emotion engine analyzes the user's emotions and prioritizes the nearest gas station if the user is in a hurry. The navigation module provides route guidance.

[0395] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0396] Step 1:

[0397] The user issues a voice command.

[0398] The user issues a voice command into the microphone, such as "Tell me where the nearest gas station is." The input is the user's voice, and the output is voice data.

[0399] Step 2:

[0400] The terminal converts the voice data into text data.

[0401] The speech recognition module uses the speech_recognition library to convert voice data into text data. The input is voice data and the output is text data.

[0402] Step 3:

[0403] The server searches for information on gas stations and parking lots based on the text data.

[0404] The search module uses the requests library to send requests to external APIs to get the closest and cheapest gas station from the current location, the cheapest parking lot that is currently open, etc. The input is text data, and the output is a list of gas stations and parking lots.

[0405] Step 4:

[0406] The server analyzes the user's emotions.

[0407] The emotion engine uses the emotion_recognition library to recognize user emotions from voice data. The input is voice data and the output is emotion data.

[0408] Step 5:

[0409] The server selects the best search results based on the emotion data.

[0410] Based on the user's emotional data, the emotion engine prioritizes the nearest gas station or parking lot if the user is in a hurry, and suggests a scenic route if the user is relaxed. The input is the emotional data and a list of gas stations and parking lots, and the output is the information on the optimal gas station or parking lot.

[0411] Step 6:

[0412] The server calculates the optimal route and starts navigation.

[0413] The navigation module uses the navigation_system module to calculate the route to the best gas station or parking lot and guide the user. The input is the best gas station or parking lot information, and the output is the route guidance information.

[0414] Example 2

[0415] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0416] Conventional navigation systems can search for optimal gas stations and parking lots and provide route guidance based on the user's voice commands, but they have a problem in that they cannot provide services that take the user's emotions into account. For example, they cannot provide search results or route guidance that correspond to whether the user is in a hurry or relaxed.

[0417] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0418] In this invention, the server includes means for accepting voice instructions through natural dialogue, means for converting the voice instructions into text data, means for acquiring a current location, means for searching for the cheapest fuel supply facility closest to the current location, means for searching for the cheapest parking facility closest to the current location that is currently available, means for providing route guidance to the searched fuel supply facility or parking facility, means for analyzing emotions from the user's voice, and means for providing search results and route guidance according to the user's emotions, thereby enabling optimal search results and route guidance according to the user's emotions.

[0419] "Natural dialogue voice instructions" refers to voice input given by a user to the system in everyday language.

[0420] "Means for converting voice instructions into text data" refers to a function that uses voice recognition technology to convert a user's voice into text information.

[0421] "Location capabilities" refers to the ability to determine a user's current location using GPS or other location technologies.

[0422] "Fuel distribution facility" means a gasoline station or other place that dispenses fuel.

[0423] "Parking facility" means a place for the temporary parking of vehicles.

[0424] "Means of providing route guidance" refers to a function that provides the user with the optimal route to reach their destination.

[0425] "Means for analyzing emotions from a user's voice" refers to a function that uses voice analysis technology to determine the user's emotional state.

[0426] "Means of providing search results and route guidance according to the user's emotions" refers to a function that suggests optimal search results and routes based on the user's emotional state.

[0427] This invention is a system that searches for the nearest inexpensive fuel supply facility or parking facility from the current location and provides optimal route guidance by using voice instructions from the user. It can also analyze the user's emotions and provide services according to those emotions.

[0428] Hardware and Software Used

[0429] Speech Recognition Module

[0430] The device uses a commonly used speech recognition API (e.g., Google Cloud Speech-to-Text API) as a speech recognition module that converts the user's voice instructions into text data.

[0431] GPS Features

[0432] The device uses its built-in GPS to obtain the user's current location, which can then be used to locate fuel and parking facilities.

[0433] Search Module

[0434] The server uses a map API (e.g., Google Maps API) as a search module that searches for the nearest, cheapest fuel and parking facilities from the user's current location.

[0435] Navigation Module

[0436] The terminal uses a navigation API (e.g., Google Maps Navigation) as a navigation module, which provides optimal route guidance to the user based on the search results.

[0437] Emotion Engine

[0438] The terminal uses an emotion analysis API (e.g., IBM Watson® Tone Analyzer) as an emotion engine that analyzes emotions from the user's voice and provides the results to other parts of the system.

[0439] Examples

[0440] Example 1: Searching for gas stations

[0441] The user gives a voice command such as "Tell me where the nearest cheapest gas station is." The device converts the voice to text using the Google Cloud Speech-to-Text API and sends it to the server. The server uses the Google Maps API to find the nearest cheapest fuel station and sends the results to the device. The device uses Google Maps Navigation to provide route guidance to the user.

[0442] Example 2: Emotion-based route guidance

[0443] The user gives a voice command such as "Tell me the nearest cheap parking lot." The device converts the voice into text and analyzes the user's emotions using an emotion engine (IBM Watson Tone Analyzer). If it is determined that the user is in a hurry, the server prioritizes searching for the nearest parking facility and sends it to the device. The device uses a navigation module to provide the optimal route.

[0444] Examples of prompt statements

[0445] "If a user says, 'Where is the nearest cheapest gas station?' Describe a system that uses a speech recognition module to convert the speech to text, finds the nearest cheapest gas station from the user's current location, and uses a navigation module to provide route guidance."

[0446] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0447] Step 1:

[0448] The user gives a voice command such as "Tell me the nearest cheap gas station."

[0449] Input: User's voice command

[0450] Output: Audio data

[0451] Specific operation: The user gives voice instructions into the microphone of the terminal.

[0452] Step 2:

[0453] The terminal uses a voice recognition module to convert voice instructions into text data.

[0454] Input: Audio data

[0455] Output: Text data

[0456] Specific operation: The device calls the Google Cloud Speech-to-Text API and converts the voice data into text data such as "What is the nearest cheapest gas station?"

[0457] Step 3:

[0458] The device uses the GPS function to obtain the user's current location.

[0459] Input: None (internal processing of the terminal)

[0460] Output: Current location data

[0461] Specific operation: The device uses its built-in GPS function to obtain the user's current location as "1-1-1 Marunouchi, Chiyoda-ku, Tokyo."

[0462] Step 4:

[0463] The server uses a search module to search for the closest, least expensive fuel supply facility from the current location.

[0464] Input: current location data, text data

[0465] Output: Search result data

[0466] Specific operation: The server uses the Google Maps API to search for the nearest gas station from "1-1-1 Marunouchi, Chiyoda-ku, Tokyo," and obtains the result as "Gas Station A."

[0467] Step 5:

[0468] The server transmits the search results to the terminal.

[0469] Input: Search result data

[0470] Output: Search result data (terminal side)

[0471] Specific operation: The server sends information about "Gas Station A" to the terminal.

[0472] Step 6:

[0473] The device uses the navigation module to provide the user with optimal route guidance based on the search results.

[0474] Input: Search result data

[0475] Output: Route directions data

[0476] Specific operation: The device uses Google Maps Navigation to guide the user to the optimal route to "Gas Station A." For example, the device may provide guidance such as "Go straight, turn right, then turn left to reach your destination."

[0477] Step 7:

[0478] The terminal uses an emotion engine to analyze emotions from the user's voice.

[0479] Input: Audio data

[0480] Output: Emotion data

[0481] Specific operation: The device uses IBM Watson Tone Analyzer to analyze the emotion of "hurry" from the user's voice.

[0482] Step 8:

[0483] Based on the analysis results of the emotion engine, the server provides search results and route guidance that correspond to the user's emotions.

[0484] Input: emotion data, search result data

[0485] Output: Search results data according to emotions, route guidance data

[0486] Specific operation: If the user is determined to be in a hurry, the server will search for the nearest gas station first and send it to the device. The device will then use the navigation module to provide the optimal route.

[0487] (Application example 2)

[0488] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0489] Conventional navigation systems were able to search for the nearest and cheapest gas stations and parking lots and provide route guidance based on the user's voice instructions, but they did not provide services that took the user's emotions into consideration. As a result, it was not possible to provide optimal services according to the user's situation, such as when the user was in a hurry or relaxed. The present invention aims to provide a navigation system that is more comfortable and efficient for users by recognizing the user's emotions and providing search results and route guidance according to those emotions.

[0490] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0491] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station closest to the current location, a means for searching for the cheapest parking lot that is closest to the current location and currently available, a means for providing route guidance to the searched gas station or parking lot, a means for recognizing the user's emotions, and a means for adjusting the search results and route guidance in accordance with the user's emotions, thereby enabling optimal search results and route guidance in accordance with the user's emotions.

[0492] "Means for accepting voice instructions through natural dialogue" refers to means for recognizing voice instructions given by a user in natural language and inputting them into the system.

[0493] "A means for searching for the nearest and cheapest gas station from the current location" is a means for identifying the nearest and cheapest gas station based on the user's current location information.

[0494] "Means for searching for the nearest parking lot that is available and has the lowest price from the current location" refers to a means for identifying the nearest parking lot that is available and has the lowest price based on the user's current location information.

[0495] "Means for providing route guidance to the searched gas station or parking lot" refers to means for providing a route for guiding the user to the destination based on the search results.

[0496] "Means for recognizing a user's emotions" means means for analyzing emotions from a user's voice or other input and identifying the user's emotional state.

[0497] "Means for adjusting search results and route guidance according to user emotions" refers to means for changing and optimizing the content of search results and route guidance based on recognized user emotions.

[0498] The following system configuration is used as an embodiment of the present invention. The system is composed of a voice recognition module, an emotion recognition module, a search module, and a navigation module. These modules are installed on a server and a user terminal (e.g., a navigation system for an autonomous vehicle).

[0499] Speech Recognition Module

[0500] The speech recognition module accepts the user's voice instructions and converts them into text data. Specifically, it uses a microphone to input voice and uses the speech_recognition library to convert the voice data into text data. This text data is passed to the next processing step.

[0501] Emotion Recognition Module

[0502] The emotion recognition module analyzes emotions from the user's voice. The voice data is analyzed with the emotion_recognition library to identify the user's emotional state. This emotion information is provided to the search and navigation modules.

[0503] Search Module

[0504] The search module searches for the nearest and cheapest gas stations and parking lots based on the user's current location. Specifically, it makes an API request to identify the nearest gas stations and parking lots with the cheapest prices from the user's current location. The search results are passed to the navigation module.

[0505] Navigation Module

[0506] The navigation module guides the user to the destination based on the search results. The navigation system uses the navigation_system module to calculate the optimal route and provide real-time guidance to the user. It is also possible to adjust the content of the route guidance based on the user's emotion information.

[0507] Examples

[0508] For example, if a user gives a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts the command into text data. Next, the emotion recognition module analyzes the emotion from the user's voice and, if it determines that the user is in a hurry, the search module will prioritize searching for the nearest gas station. The search results are sent to the navigation module, which provides the user with optimal route guidance.

[0509] Examples of prompt statements

[0510] When a user gives a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts this command into text data. Next, the search module searches for the nearest cheap gas station from the current location. The search results are sent to the navigation module, which provides optimal route guidance for the user. In addition, an emotion engine that recognizes the user's emotions is combined, and if the user feels that they are in a hurry, the nearest gas station will be searched for first.

[0511] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0512] Step 1:

[0513] The user gives a voice command. For example, the user speaks a voice command into the microphone, such as "Tell me where the nearest cheap gas station is." The input is the user's voice data, and the output is the voice data that is passed to the voice recognition module.

[0514] Step 2:

[0515] The terminal uses a voice recognition module to convert voice data into text data. Specifically, it uses the speech_recognition library to analyze the voice data and generate corresponding text data. The input is voice data, and the output is text data.

[0516] Step 3:

[0517] The device uses the emotion recognition module to analyze the user's emotions. The voice data is analyzed with the emotion_recognition library to identify the user's emotional state. The input is the voice data, and the output is the user's emotional information.

[0518] Step 4:

[0519] The server uses a search module to search for the nearest and cheapest gas station based on the user's current location. It makes an API request to identify the nearest gas station with the cheapest price from the current location. The input is the current location and the user's sentiment, and the output is the search result.

[0520] Step 5:

[0521] The server uses a search module to search for the nearest, available, and cheapest parking lot based on the user's current location. It makes an API request to identify the nearest and cheapest parking lot from the current location. The input is the current location and the user's sentiment information, and the output is the search result.

[0522] Step 6:

[0523] The server uses the navigation module to guide the user to the destination based on the search results. It uses the navigation_system module to calculate the optimal route and provide real-time guidance to the user. The input is the search results and the user's emotion information, and the output is the route guidance information.

[0524] Step 7:

[0525] The terminal provides the user with route guidance information from the navigation module. Specifically, the terminal provides route guidance to the user through voice and display. The input is route guidance information, and the output is guidance information for the user.

[0526] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0527] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0528] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0529] In the above embodiment, an example has been given in which the specific process is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0530] [Second embodiment]

[0531] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0532] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0533] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0534] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0535] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0536] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0537] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0538] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0539] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0540] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0541] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0542] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0543] "Example 1"

[0544] The system of the present invention includes a voice recognition module that accepts voice instructions from a user, a search module that searches for information on gas stations and parking lots, and a navigation module that provides route guidance. The voice recognition module accepts voice instructions from a user through natural dialogue and converts them into text data. The search module searches for the nearest and cheapest gas station from the current location or the cheapest parking lot that is currently available based on the converted text data. The navigation module provides route guidance to the searched gas station or parking lot.

[0545] "Example 2"

[0546] In a specific embodiment, when a user issues a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts the command into text data. Then, the search module searches for the nearest cheap gas station from the current location. The search results are sent to the navigation module, which provides the user with optimal route guidance.

[0547] The process flow of each embodiment will be described below.

[0548] "Example 1"

[0549] Step 1: The user gives a voice command, for example, "What is the nearest cheap gas station?"

[0550] Step 2: The voice recognition module accepts the user's voice commands and converts them into text data.

[0551] Step 3: The search module uses the converted text data to search for the closest, cheapest gas station from the current location.

[0552] Step 4: The navigation module provides directions to the searched gas station.

[0553] "Example 2"

[0554] Step 1: The user issues a voice command, for example, "Tell me where the nearest available parking space is."

[0555] Step 2: The voice recognition module accepts the user's voice commands and converts them into text data.

[0556] Step 3: The search module uses the converted text data to search for the closest, most available and cheapest parking lot from the current location.

[0557] Step 4: The navigation module provides route guidance to the searched parking lot.

[0558] Example 1

[0559] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0560] Conventional navigation systems have the problem that even if a user issues voice instructions, the accuracy of voice recognition is low, making it difficult to obtain accurate search results. In addition, when searching for gas stations or parking lots, it is often impossible to quickly find the nearest and cheapest facility. Furthermore, route guidance based on search results is insufficient, and users may not be able to reach their destination efficiently. To solve these problems, there is a demand for a system that improves the accuracy of voice recognition, provides quick and accurate search results, and provides efficient route guidance.

[0561] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0562] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for converting the voice instructions into text data, a means for searching for the cheapest fuel supply facility closest to the current location, a means for searching for the cheapest parking facility closest to the current location that is currently available, a means for providing route guidance to the searched fuel supply facility or parking facility, and a means for transmitting and receiving data including the voice instructions, search results, and route guidance. This allows the user to give voice instructions in a natural dialogue format, improves the accuracy of voice recognition, and enables quick and accurate search results to be obtained. Furthermore, efficient route guidance allows the user to quickly reach the destination.

[0563] "Natural dialogue voice instructions" refers to voice input given by a user to the system using everyday words and phrases.

[0564] "Means for converting voice instructions into text data" refers to technology or devices for analyzing a user's voice input and converting it into text information.

[0565] "Means for searching for the nearest cheapest fuel supply facility from the current location" refers to technology or devices for identifying the nearest gas station or other facility with the cheapest fuel price based on the user's current location.

[0566] "Means for searching for the closest, most available, and least expensive parking facility from the current location" refers to technology or devices for identifying the closest parking facility that is available and has the lowest parking fees based on the user's current location.

[0567] "Means for providing route guidance to the searched fuel supply facility or parking facility" refers to technology or devices for providing the user with the optimal route to reach their destination based on the search results.

[0568] "Means for transmitting and receiving data including voice instructions, search results, and route guidance" refers to technology or devices for transmitting and receiving information regarding a user's voice instructions, search results, and route guidance via a communications network.

[0569] The present invention is a system that receives voice instructions from a user, searches for the nearest and cheapest fuel supply facility or parking facility, and provides route guidance based on the results. A specific embodiment of this system will be described below.

[0570] System Configuration

[0571] Subject: User

[0572] A user issues voice instructions to a terminal while in a car or on the move, for example, using prompt sentences such as "Tell me where the nearest cheap gas station is" or "Find the nearest cheap parking lot."

[0573] Subject: Terminal

[0574] The device uses a microphone to receive voice instructions from the user. The voice data is temporarily stored in the device and then sent to a voice recognition module. A common voice recognition API is used for voice recognition. For example, Google Cloud Speech-to-Text API is used.

[0575] The voice recognition module converts the voice data into text data that accurately reflects the user's intent, and the converted text data is sent from the terminal to the server.

[0576] Subject: Server

[0577] The server starts a search module based on the received text data. The search module searches for the nearest and cheapest fuel supply station or the cheapest parking facility that is currently available based on the user's current location information. A general map API is used for the search. For example, Google Maps API is used.

[0578] The search module retrieves search results and returns them to the server, which may include the names, addresses, price information, etc. of establishments. The server then transmits these search results to the terminal.

[0579] The terminal starts a navigation module based on the received search results. The navigation module provides route guidance to the user. For the route guidance, a general navigation API is used. For example, the Google Maps Directions API is used.

[0580] Examples

[0581] Example 1: Searching for gas stations

[0582] When a user says, "Tell me where the nearest cheap gas station is," the device's microphone captures the speech. The speech recognition module converts the speech into text data and sends it to the server. The server uses the Google Maps API to find the nearest cheap gas station and returns the results to the device. The device then launches the navigation module and provides route guidance to the user.

[0583] Example 2: Searching for parking spaces

[0584] When a user says, "Find the nearest cheap parking," the device's microphone captures the voice. The speech recognition module converts the voice into text data and sends it to the server. The server uses the Google Maps API to find the nearest, available, and cheapest parking spots and returns the results to the device. The device then launches the navigation module and provides route guidance to the user.

[0585] In this way, the system starts with the user's voice command, and each step of the voice recognition, search, and navigation work together. This allows the user to give voice commands in a natural dialogue format, improving the accuracy of voice recognition and enabling quick and accurate search results. In addition, efficient route guidance allows the user to reach their destination quickly.

[0586] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0587] Step 1:

[0588] The user issues a voice command to the terminal saying, "Tell me the nearest cheap gas station."

[0589] Input: User's voice command

[0590] Output: Audio data is input to the device.

[0591] Specific operation: When a user issues a voice command inside the car, the device's microphone captures the voice.

[0592] Step 2:

[0593] The terminal uses a built-in microphone to collect the user's voice instructions.

[0594] Input: User's voice data

[0595] Output: Collected audio data

[0596] Specific operation: The device temporarily stores the voice data and prepares it to be sent to the voice recognition module.

[0597] Step 3:

[0598] The terminal starts the voice recognition module and sends the collected voice data to the voice recognition API.

[0599] Input: Collected audio data

[0600] Output: Audio data sent to the speech recognition API

[0601] What happens: The device calls the Google Cloud Speech-to-Text API to convert the voice data into text.

[0602] Step 4:

[0603] The voice recognition module converts the voice data into text data.

[0604] Input: Voice data sent to the speech recognition API

[0605] Output: Converted text data

[0606] What happens: The speech recognition module converts the speech "What's the nearest cheap gas station?" into text.

[0607] Step 5:

[0608] The terminal transmits the converted text data to the server.

[0609] Input: Converted text data

[0610] Output: Text data sent to the server

[0611] Specific operation: The device sends text data to the server using the HTTPS protocol.

[0612] Step 6:

[0613] The server launches a search module based on the received text data.

[0614] Input: Text data sent to the server

[0615] Output: Search module launch

[0616] Specific operations: The server analyzes the text data and launches the search module.

[0617] Step 7:

[0618] The search module searches for the nearest, least expensive fuel supply facility based on the user's current location.

[0619] Input: User's current location and text data

[0620] Output: Search results (fuel supply facility information)

[0621] What it does: The search module uses the Google Maps API to find the nearest and cheapest gas stations.

[0622] Step 8:

[0623] The search module retrieves the search results and returns them to the server.

[0624] Input: Search results (fuel supply facility information)

[0625] Output: Search results returned to the server

[0626] Specific Operation: The search module returns search results to the server, including the business's name, address, price information, etc.

[0627] Step 9:

[0628] The server transmits the obtained search results to the terminal.

[0629] Input: Search results returned by the server

[0630] Output: Search results sent to the terminal

[0631] Specific operation: The server sends the search results to the terminal using the HTTPS protocol.

[0632] Step 10:

[0633] The terminal launches a navigation module based on the received search results.

[0634] Input: Search results sent to the device

[0635] Output: Launch of navigation module

[0636] Specific operations: The device analyzes the search results and launches the navigation module.

[0637] Step 11:

[0638] The navigation module provides route guidance to the user.

[0639] Input: Launch of navigation module and search results

[0640] Output: Directions provided to the user

[0641] What it does: The navigation module uses the Google Maps Directions API to provide the user with specific route directions via voice.

[0642] (Application example 1)

[0643] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0644] Conventional navigation systems require users to manually search for gas stations and parking lots, which may compromise safety while driving. It is also difficult to efficiently find the nearest and cheapest gas stations and parking lots. In addition, route guidance based on search results may not be intuitive, which may reduce user convenience.

[0645] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0646] In this invention, the server includes a means for receiving voice instructions through natural dialogue, a means for searching for the cheapest gas station closest to the current location, a means for searching for the cheapest parking lot closest to the current location that is currently available, a means for providing route guidance to the searched gas station or parking lot, a means for converting the voice instructions into text data, and a means for displaying a route on a map based on the search results. This enables a user to safely and efficiently search for a gas station or parking lot using voice instructions and receive intuitive route guidance even while driving.

[0647] "Natural dialogue voice instructions" refers to voice input given to the system by a user using everyday language.

[0648] "Means for accepting voice instructions" refers to devices or software that have the function of recognizing voice input from a user and incorporating it into the system.

[0649] "Means for searching for the closest, least expensive gas station from a user's current location" means a device or software capable of identifying the closest, least expensive gas station based on a user's current location.

[0650] "Means for searching for the closest, most available, and least expensive parking space from the current location" refers to a device or software that has the functionality to identify the closest, most available, and least expensive parking space based on the user's current location.

[0651] "Means for providing route guidance to the searched gas station or parking lot" refers to devices or software that have the function of guiding the user to their destination based on the search results.

[0652] "Means for converting voice instructions into text data" refers to a device or software that has the function of converting a user's voice input into text information.

[0653] "Means for displaying a route on a map based on the search results" refers to a device or software having a function for visually displaying a route to a searched destination on a map.

[0654] The following system configuration is used as an embodiment of the present invention. The system includes a voice recognition module, a search module, and a navigation module. These modules work together to accept voice instructions from a user, search for information on gas stations and parking lots, and provide route guidance.

[0655] Speech Recognition Module

[0656] The speech recognition module accepts the user's voice instructions and converts them into text data. Specifically, it uses a microphone to capture the user's voice and converts the voice into text using the SpeechRecognition library. This allows the user to give instructions to the system in a natural dialogue format.

[0657] Search Module

[0658] The search module searches for the nearest and cheapest gas station or the cheapest parking space that is currently available based on the text data received from the voice recognition module. Specifically, it uses the Google Places API to perform searches based on the user's current location. The search results are selected based on the most convenient results for the user.

[0659] Navigation Module

[0660] The navigation module provides route guidance to the destination based on the results obtained from the search module. Specifically, it uses the Folium library to display the route based on the search results on a map. This allows the user to visually check the route to the destination.

[0661] Hardware and Software Used

[0662] Hardware: Microphone, GPS module

[0663] Software: SpeechRecognition library, requests library, Google Places API, Folium library

[0664] Examples

[0665] When a user says, "Tell me where the nearest gas station is," the system searches for the nearest gas station from the user's current location and displays the route. The following is an example of a prompt:

[0666] Examples of prompt statements

[0667] User: "What's the closest gas station?"

[0668] System: "Voice commands please."

[0669] User: "What's the closest gas station?"

[0670] System: "Recognized text: Where is the nearest gas station?"

[0671] System: "Searching for gas stations..."

[0672] System: "Route to gas station x has been generated."

[0673] In this way, users can safely and efficiently search for gas stations and parking lots and receive intuitive route guidance while driving.

[0674] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0675] Step 1:

[0676] The user issues a voice command.

[0677] The user issues a voice command into the microphone, such as "Tell me where the nearest gas station is." The input is the user's voice, and the output is voice data.

[0678] Step 2:

[0679] The terminal captures the voice data and sends it to the voice recognition module.

[0680] The device captures the user's voice using a microphone and converts the voice data into text data using the SpeechRecognition library. The input is voice data and the output is text data.

[0681] Step 3:

[0682] The server receives the text data and sends it to the search module.

[0683] The server analyzes the text data received from the speech recognition module to understand the user's intent. The input is the text data, and the output is the search query.

[0684] Step 4:

[0685] Based on the search query, the server searches for the closest, cheapest gas stations and parking lots from the current location.

[0686] The server uses the Google Places API to search for gas stations and parking lots based on the user's current location. The input is the search query and the current location data, and the output is the search results.

[0687] Step 5:

[0688] The server sends the search results to the navigation module.

[0689] The server analyzes the search results and selects the most suitable gas station or parking lot. The input is the search results, and the output is the selected destination data.

[0690] Step 6:

[0691] The terminal receives the selected destination data and provides route guidance.

[0692] The device uses the Folium library to display routes to selected destinations on a map. The input is destination data, and the output is a map with route directions.

[0693] Step 7:

[0694] The user follows the route guidance to the destination.

[0695] The user follows the route guidance displayed on the terminal to head to a gas station or parking lot. The input is a map of the route guidance, and the output is reaching the destination.

[0696] Example 2

[0697] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0698] In conventional systems, when a user gives voice instructions, it is difficult to accurately understand the instructions and provide appropriate information. In addition, route guidance based on search results for gas stations and parking lots is insufficient, resulting in low user convenience. Furthermore, the accuracy and speed of the process of converting voice instructions into text data are also issues. There is a demand for a system that can solve these problems and provide users with quick and accurate information and route guidance.

[0699] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0700] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for converting the voice instructions into text data, a means for searching for the cheapest gas station closest to the current location, a means for transmitting the search results to the navigation module, and a means for providing route guidance to the searched gas station. This makes it possible to accurately understand voice instructions given by the user and provide appropriate information promptly. In addition, by providing optimal route guidance based on the search results, it is possible to improve user convenience.

[0701] "Natural dialogue voice instructions" refers to voice input by the user in everyday language, with the goal of the system understanding the speech and responding appropriately.

[0702] "Means for converting voice instructions into text data" refers to technology that analyzes a user's voice input and converts it into a corresponding text format.

[0703] "Current location" refers to where a user is currently located, typically determined based on location information such as GPS data.

[0704] "Method of finding nearest cheapest gas stations" refers to a search algorithm or database used to identify the nearest cheapest gas stations based on a user's current location.

[0705] "Means for transmitting search results to a navigation module" refers to a technique for transmitting information on the searched gas stations to a navigation system and generating route guidance.

[0706] "Means of route guidance" refers to technology for providing users with the optimal route to their destination, and includes map displays and voice guidance.

[0707] This invention is a system that searches for the nearest inexpensive gas station from the user's current location in response to voice instructions, and provides route guidance based on the search results. A specific embodiment of this system will be described below.

[0708] Receiving and converting voice commands

[0709] The user issues a voice command such as "Tell me where the cheapest gas station is nearby." A voice recognition module (e.g., a voice recognition API) installed in the device converts this voice command into text data. The voice recognition module analyzes the voice signal and generates the corresponding text.

[0710] Gas station search

[0711] The server receives the text data and searches for the nearest and cheapest gas station from the user's current location using a search module (e.g., map API). The search module obtains the user's current location information from GPS data and gas station price information from a database.

[0712] Providing navigation

[0713] The server sends the search results to a navigation module (e.g., navigation API) to generate optimal route guidance. The device provides this route guidance to the user. Specifically, the map and route are displayed on the smartphone screen, and voice guidance is also provided.

[0714] Examples

[0715] For example, if a user is near Tokyo Station, they may give a voice command such as, "Tell me the nearest cheap gas station." The voice recognition module converts this command into text data, and the server uses a map API to search for gas stations near Tokyo Station. The search results may reveal that "Gas Station A" is the closest and cheapest. The server uses the navigation API to generate the optimal route to "Gas Station A," and the device provides route guidance to the user. Specifically, a map and route to "Gas Station A" are displayed on the smartphone screen, and voice guidance such as, "Turn right 500 meters ahead" is given.

[0716] Examples of prompt statements

[0717] "If a user says, 'Tell me where the cheapest gas station is nearby,' how would you proceed?"

[0718] This invention allows users to quickly find the nearest, cheapest gas station by simply giving voice instructions, and furthermore, to receive optimal route guidance to that location, which greatly improves user convenience.

[0719] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0720] Step 1:

[0721] The user gives a voice command.

[0722] A user issues a voice command such as, "Tell me the nearest cheap gas station." The input is the user's voice, and the output is a voice signal.

[0723] Step 2:

[0724] The device converts the voice into text data.

[0725] A voice recognition module (e.g., a voice recognition API) installed in the device receives the user's voice instructions and converts them into text data. Specifically, it analyzes the voice signal and generates the corresponding text. The input is the voice signal, and the output is the text data.

[0726] Step 3:

[0727] The server receives the text data.

[0728] The terminal generates text data and sends it to the server. The server receives the text data and proceeds to the next processing step. The input is the text data, and the output is a receipt confirmation.

[0729] Step 4:

[0730] The server searches for gas stations.

[0731] The server analyzes the text data and obtains the user's current location information. It then uses a search module (e.g., map API) to search for the nearest, least expensive gas station from the user's current location. Specifically, it uses GPS data to identify the current location and obtains gas station price information from a database. The input is the text data and GPS data, and the output is the search results.

[0732] Step 5:

[0733] The server sends the search results to the navigation module.

[0734] The server sends the search results to a navigation module (e.g., navigation API) and generates optimal route guidance. Specifically, it calculates the optimal route from the user's current location based on the location information of gas stations obtained as search results. The input is the search results, and the output is route guidance data.

[0735] Step 6:

[0736] The terminal provides route guidance to the user.

[0737] The terminal provides the route guidance received from the navigation module to the user. Specifically, the map and route are displayed on the smartphone screen, and voice guidance is also provided. For example, specific instructions such as "Turn right 500 meters ahead" are provided by voice. The input is route guidance data, and the output is route guidance to the user.

[0738] (Application example 2)

[0739] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0740] Conventional navigation systems have difficulty efficiently searching for the nearest and cheapest gas stations and parking lots and providing real-time updated information, even when users give voice instructions. In addition, they lack sufficient coordination with autonomous vehicles and are unable to provide optimal route guidance. This can lead to users spending a lot of time and effort to reach their destination.

[0741] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0742] In this invention, the server includes a means for receiving voice instructions through natural dialogue, a means for searching for the nearest and cheapest gas station from the current location, a means for searching for the nearest and cheapest parking lot from the current location that is currently available, a means for providing route guidance to the searched gas station or parking lot, a means for updating price and location information in real time, a means for providing search results based on the voice instructions, and a means for providing optimal route guidance in cooperation with the navigation system of the autonomous vehicle. This allows the user to efficiently search for the nearest and cheapest gas station or parking lot through voice instructions, and to receive optimal route guidance based on information updated in real time.

[0743] "Voice instructions through natural dialogue" refers to voice input made by the user in natural language, similar to everyday conversation.

[0744] "Means for searching for the closest, least expensive gas station from a current location" refers to a method or device for identifying the closest, least expensive gas station based on a user's current location.

[0745] "Means for searching for the closest, currently available, and cheapest parking lot from the current location" refers to a method or device for identifying the closest, available, and cheapest parking lot based on the user's current location.

[0746] "Means for providing route guidance to a searched gas station or parking lot" refers to a method or device for providing a user with the optimal route to a gas station or parking lot that the user has searched for.

[0747] "Means for updating price and location information in real time" refers to a method or apparatus for obtaining and updating price and location information for gas stations and parking lots in real time.

[0748] "Means for providing search results based on voice instructions" refers to a method or device for generating and providing search results based on a user's voice instructions.

[0749] "Means for providing optimal route guidance in cooperation with the navigation system of an autonomous vehicle" refers to a method or device for providing optimal routes in cooperation with the navigation system of an autonomous vehicle.

[0750] The following system configuration will be described as an embodiment of the present invention.

[0751] System Configuration

[0752] The system includes a voice recognition module, a search module, a navigation module, a real-time update module, and a module that interfaces with the autonomous vehicle's navigation system.

[0753] Speech Recognition Module

[0754] The voice recognition module converts the voice into text data when the user gives a voice command through natural dialogue. Specifically, it inputs the voice using a microphone and converts the voice into text using the speech_recognition library.

[0755] Search Module

[0756] The search module searches for the nearest and cheapest gas station and the cheapest parking lot that is currently available based on the user's current location. To do this, it uses the GPS module to get the current location and the requests library to get information about gas stations and parking lots from external APIs.

[0757] Navigation Module

[0758] The navigation module provides optimal route guidance based on the search results. It uses the navigation API to calculate the shortest route from the user's current location to the destination and provides guidance.

[0759] Real-time update module

[0760] The real-time update module updates prices and location information for gas stations and parking lots in real time, so users can always get the most up-to-date search results.

[0761] Linking with autonomous vehicle navigation systems

[0762] The module, which works with the autonomous vehicle's navigation system, sends the search results to the autonomous vehicle's navigation system and provides optimal route guidance, allowing users to reach their destination efficiently through voice instructions.

[0763] Examples

[0764] For example, when a user gives a voice command such as "Tell me the nearest cheapest gas station," the voice recognition module converts the command into text data. The search module then searches for the nearest cheapest gas station from the current location. The search results are sent to the navigation module, which provides the optimal route guidance for the user. The real-time update module keeps the gas station prices and location information always up to date. By linking with the autonomous vehicle's navigation system, users can use the autonomous vehicle to reach their destination efficiently.

[0765] Examples of prompt statements

[0766] When a user gives a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts this command into text data, and the search module searches for the nearest cheap gas station from the current location. The search results are sent to the navigation module, and the system is designed to provide the optimal route guidance for the user.

[0767] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0768] Step 1:

[0769] The user gives a voice command.

[0770] The user gives voice instructions in a natural dialogue, such as "Tell me where the nearest cheap gas station is." The input is the user's voice, and the output is voice data.

[0771] Step 2:

[0772] The terminal converts the voice data into text data.

[0773] The speech recognition module uses a microphone to capture voice data and converts the voice to text using the speech_recognition library. The input is the voice data and the output is the text data.

[0774] Step 3:

[0775] The server gets the current location.

[0776] Use the device's GPS module to obtain the user's current location. The input is the GPS signal, and the output is the coordinate data of the current location.

[0777] Step 4:

[0778] The server obtains gas station information.

[0779] Use the requests library to get gas station location and price information from an external API. The input is the API request, and the output is data including gas station location and price information.

[0780] Step 5:

[0781] The server searches for the nearest cheapest gas station.

[0782] Based on the acquired gas station information, the distance from the current location and the price are calculated to identify the nearest and cheapest gas station. The input is the gas station location and price information, and the current location coordinate data, and the output is the optimal gas station information.

[0783] Step 6:

[0784] The server generates the navigation information.

[0785] Use the navigation API to generate route guidance from the current location to the best gas station. The input is the coordinate data of the current location and the location information of the best gas station, and the output is the route guidance information.

[0786] Step 7:

[0787] The terminal provides route guidance to the user.

[0788] The navigation module guides the user on the optimal route by voice or visually based on the generated route guidance information. The input is the route guidance information, and the output is navigation instructions to the user.

[0789] Step 8:

[0790] The server updates the information in real time.

[0791] The real-time update module periodically retrieves gas station prices and location information from an external API and updates the data in the system. The input is the API request and the output is the updated gas station information.

[0792] Step 9:

[0793] The server works in conjunction with the autonomous vehicle's navigation system.

[0794] The search results and route guidance information are sent to the navigation system of the autonomous vehicle, and the system controls the vehicle so that it automatically drives along the optimal route. The input is route guidance information, and the output is control instructions for the autonomous vehicle.

[0795] In addition, an emotion engine that estimates the emotion of the user may be further combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0796] "Example 1"

[0797] As an embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotion of a user. The system includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station from the current location, a means for searching for the cheapest parking lot that is currently available from the current location, and a means for providing route guidance to the searched gas station or parking lot. The system further includes an emotion engine that recognizes the emotion of the user. The emotion engine analyzes the emotion from the user's voice and provides the result to other parts of the system. For example, if the user feels that he or she is in a hurry, the emotion engine provides the information to the search means, and the search means searches for the nearest gas station or parking lot with priority. Also, if the user feels that he or she is relaxed, the emotion engine provides the information to the route guidance means, and the route guidance means can provide services according to the user's emotion, such as suggesting a route with a better view.

[0798] "Example 2"

[0799] As an embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotion of a user. The system includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station from the current location, a means for searching for the cheapest parking lot that is currently available from the current location, and a means for providing route guidance to the searched gas station or parking lot. The system further includes an emotion engine that recognizes the emotion of the user. The emotion engine analyzes the emotion from the user's voice and provides the result to other parts of the system. For example, if the user feels that he or she is in a hurry, the emotion engine provides the information to the search means, and the search means searches for the nearest gas station or parking lot with priority. Also, if the user feels that he or she is relaxed, the emotion engine provides the information to the route guidance means, and the route guidance means can provide services according to the user's emotion, such as suggesting a route with a better view.

[0800] The process flow of each embodiment will be described below.

[0801] "Example 1"

[0802] Step 1: The user issues a voice command to the system, such as "Find the nearest cheap gas station."

[0803] Step 2: The system's voice recognition module converts the voice instructions into text data, while the emotion engine analyzes the emotions from the user's voice.

[0804] Step 3: Based on the analysis result of the sentiment engine, the system's search module searches for the nearest cheap gas station. For example, if the user feels that he is in a hurry, it will prioritize searching for the nearest gas station.

[0805] Step 4: Based on the search results, the system's navigation module will provide route guidance to the searched gas station. For example, if the user feels relaxed, it will suggest a more scenic route.

[0806] Example 1

[0807] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0808] Conventional navigation systems can accept voice instructions from users, but they are unable to provide search results or route guidance that take the user's emotions into account. This means that there is an issue with them being unable to provide optimal guidance according to the situation, such as when the user is in a hurry or relaxed.

[0809] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0810] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for converting the voice instructions into text data, a means for searching for the cheapest fuel supply facility closest to the current location, a means for searching for the cheapest parking facility closest to the current location that is currently available, a means for analyzing the user's emotions, a means for providing route guidance to the searched fuel supply facility or parking facility, and a means for adjusting the search results and route guidance based on the user's emotions, thereby enabling optimal search results and route guidance according to the user's emotions and circumstances.

[0811] "Natural dialogue voice instruction" refers to voice input by the user in everyday language, and is voice instruction that does not rely on specific commands or keywords.

[0812] "Means for converting voice instructions into text data" refers to a function that uses voice recognition technology to convert a user's voice input into text information.

[0813] A "fuel distribution facility" refers to a facility that supplies fuel or energy to vehicles, such as a gas station or an electric vehicle charging station.

[0814] A "parking facility" is a place for temporarily parking a vehicle, and refers to facilities including parking lots and parking meters.

[0815] "Means for analyzing user emotions" refers to technologies used to determine a user's emotional state from voice or other input data.

[0816] "Means for adjusting search results and route guidance" refers to the ability to change the content of search results and route guidance provided depending on the user's emotions and situation.

[0817] The present invention is a system that receives voice instructions from a user, analyzes emotions, searches for optimal fuel supply facilities and parking facilities, and provides appropriate route guidance. A specific embodiment of this system will be described below.

[0818] Hardware and Software Used

[0819] 1. Speech Recognition Module: Uses speech recognition technology to convert voice instructions into text data. This module uses a common speech recognition API.

[0820] Example: Speech Recognition API

[0821] 2. Search module: Uses the map API to search for the nearest cheapest fuel and parking facilities from your current location.

[0822] Example: Map Information API

[0823] 3. Navigation module: Uses the navigation API to provide route guidance to the searched facilities.

[0824] Example: Navigation API

[0825] 4. Emotion Engine: Uses emotion analysis technology to analyze emotions from the user's voice.

[0826] Example: Sentiment analysis API

[0827] Specific operation of the system

[0828] 1. Accepting user voice commands

[0829] The user issues a voice command in the car saying, "Tell me the nearest cheap gas station."

[0830] The device's microphone captures the user's voice and transmits it to the system as voice data.

[0831] 2. Voice-to-text conversion

[0832] The terminal transmits the voice data to a voice recognition module.

[0833] The speech recognition module converts voice data into text data using a speech recognition API. For example, a voice saying "Tell me where the nearest cheap gas station is" is converted into text saying "Tell me where the nearest cheap gas station is."

[0834] 3. Analysis and Search of Text Data

[0835] The server receives the converted text data and sends it to a search module.

[0836] The search module uses the map information API to search for the nearest and cheapest fuel supply facility from the current location. For example, if the current location is Shibuya-ku, Tokyo, the search module retrieves information on the nearest and cheapest gas station.

[0837] 4. Emotion Analysis

[0838] The terminal transmits the user's voice data to the emotion engine.

[0839] The emotion engine uses the emotion analysis API to analyze the user's emotions. For example, if it determines that the user's voice sounds urgent, it will send that information to the server.

[0840] 5. Getting search results

[0841] The server receives the results from the search module and the sentiment analysis results from the sentiment engine.

[0842] The server integrates the information of the nearest and cheapest fuel station with the emotion information that the user is in a hurry. For example, if the search result is "Gas Station A in Shibuya Ward" and it is determined that the user is in a hurry, the server sends the emotion information to the navigation module.

[0843] 6. Providing route guidance

[0844] The server sends the search results to the navigation module.

[0845] The navigation module uses the navigation API to provide route guidance to the searched fuel supply facility. For example, it guides the user on the shortest route to "Gas Station A in Shibuya Ward" by voice and map.

[0846] Examples and prompts

[0847] Examples

[0848] While driving, a user issues a voice command such as "Tell me the nearest cheap gas station." The device captures the voice and converts it into text using a voice recognition module. The server analyzes the text data and searches for the nearest cheapest gas station using a map information API. The emotion engine determines from the user's voice that the user is in a hurry and provides this information to the server. The server integrates the search results with the emotion information and sends it to the navigation module. The navigation module uses the navigation API to guide the user along the shortest route.

[0849] Examples of prompt statements

[0850] "Tell me where the nearest cheap gas station is."

[0851] "Find a parking spot that's currently available"

[0852] "Tell me the scenic route"

[0853] The system accepts the user's voice commands, analyzes emotions, searches for optimal fuel and parking facilities, and provides appropriate route guidance.

[0854] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0855] Step 1:

[0856] Accepting user voice commands

[0857] Input: User says, "What is the nearest cheap gas station?"

[0858] Specific operation: The device's microphone captures the user's voice and sends it to the system as voice data.

[0859] Output: Audio data is generated and sent to the speech recognition module.

[0860] Step 2:

[0861] Transcription of voice commands

[0862] Input: Audio data sent from the device.

[0863] Specific operation: The device sends voice data to the voice recognition module, which uses the voice recognition API to convert the voice data into text data.

[0864] Output: The text data "Please tell me the nearest cheap gas station" is generated and sent to the server.

[0865] Step 3:

[0866] Text data analysis and search

[0867] Input: Text data sent from the speech recognition module.

[0868] Specific operation: The server receives the converted text data and sends it to the search module, which uses the map information API to search for the nearest cheapest fuel supply station from the current location.

[0869] Output: The closest cheapest fuel station to your current location is generated.

[0870] Step 4:

[0871] Sentiment Analysis

[0872] Input: Audio data sent from the device.

[0873] Specific operation: The device sends the user's voice data to the emotion engine, which then uses the emotion analysis API to analyze the user's emotions.

[0874] Output: User emotion information (e.g., in a hurry) is generated and sent to the server.

[0875] Step 5:

[0876] Getting search results

[0877] Input: Search results from the search module and sentiment analysis results from the sentiment engine.

[0878] What it does: The server combines information about the nearest and cheapest fuel station with the user's emotional information that he or she is in a hurry.

[0879] Output: The integrated search results and sentiment information are generated and sent to the navigation module.

[0880] Step 6:

[0881] Providing directions

[0882] Input: The integrated search results and sentiment information sent from the server.

[0883] What happens next: The navigation module uses the navigation API to provide directions to the searched fueling facilities.

[0884] Output: Guidance information for the shortest route is generated and provided to the user via voice and map.

[0885] (Application example 1)

[0886] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0887] Conventional navigation systems can search for gas stations and parking lots and provide route guidance based on the user's voice commands, but they have a problem in that they cannot provide services that take the user's emotions into account. In particular, there is a demand for optimal route guidance depending on whether the user is in a hurry or relaxed.

[0888] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0889] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station nearest to the current location, a means for searching for the cheapest parking lot nearest to the current location that is currently available, a means for recognizing the user's emotions and providing services according to the emotions, and a means for preferentially searching for the nearest gas station or parking lot when the user is in a hurry and for suggesting a route with a nice view when the user is relaxed. This makes it possible to provide optimal route guidance according to the user's emotions.

[0890] "Natural dialogue voice instructions" refers to voice input given by a user to the system in everyday language.

[0891] A "gas station" is a facility for supplying fuel to vehicles.

[0892] A "parking lot" is a place for temporarily parking vehicles.

[0893] "Route guidance" refers to providing information showing directions to a destination.

[0894] "Means of recognizing emotions" refers to technology that analyzes emotions from the user's voice, facial expressions, etc., and provides the results to the system.

[0895] "Means of providing services" refers to technology that provides appropriate information and functions according to user requests and circumstances.

[0896] The "means for preferentially searching for the nearest gas station or parking lot" is a technology that preferentially searches for the nearest gas station or parking lot from the user's current location when the user is in a hurry.

[0897] The "means for suggesting scenic routes" is a technology that provides guidance on routes that are considered to have beautiful scenery when the user is relaxed.

[0898] The system for implementing the present invention includes a voice recognition module, a search module, a navigation module, and an emotion engine. The details of each module and its operation are described below.

[0899] Speech Recognition Module

[0900] The speech recognition module accepts the user's voice instructions and converts them into text data. Specifically, it uses a microphone to input the user's voice and uses the speech_recognition library to convert the voice into text. This text data is used for subsequent processing.

[0901] Search Module

[0902] The search module searches for the nearest and cheapest gas station and the cheapest parking lot that is currently available based on the text data obtained from the voice recognition module. Specifically, it uses the requests library to send requests to external APIs to obtain information on gas stations and parking lots.

[0903] Navigation Module

[0904] The navigation module provides route guidance to the gas station or parking lot obtained from the search module. Specifically, it uses the navigation_system module to calculate the optimal route to the destination and guides the user along it.

[0905] Emotion Engine

[0906] The emotion engine analyzes emotions from the user's voice and provides the results to the rest of the system. Specifically, it uses the emotion_recognition library to recognize emotions from voice data, and prioritizes finding the nearest gas station or parking lot if the user is in a hurry, or suggests scenic routes if the user is relaxed.

[0907] Examples

[0908] For example, if a user says, "Tell me where the nearest gas station is," the voice recognition module converts the voice to text, and the search module searches for the nearest gas station. The emotion engine analyzes the user's emotions, and if the user is in a hurry, it will prioritize the nearest gas station, and if the user is relaxed, it will suggest a scenic route.

[0909] Examples of prompt statements

[0910] When a user says "Tell me where the nearest gas station is," the speech recognition module converts the speech to text, and the search module searches for the nearest gas station. The emotion engine analyzes the user's emotions and prioritizes the nearest gas station if the user is in a hurry. The navigation module provides route guidance.

[0911] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0912] Step 1:

[0913] The user issues a voice command.

[0914] The user issues a voice command into the microphone, such as "Tell me where the nearest gas station is." The input is the user's voice, and the output is voice data.

[0915] Step 2:

[0916] The terminal converts the voice data into text data.

[0917] The speech recognition module uses the speech_recognition library to convert voice data into text data. The input is voice data and the output is text data.

[0918] Step 3:

[0919] The server searches for information on gas stations and parking lots based on the text data.

[0920] The search module uses the requests library to send requests to external APIs to get the closest and cheapest gas station from the current location, the cheapest parking lot that is currently open, etc. The input is text data, and the output is a list of gas stations and parking lots.

[0921] Step 4:

[0922] The server analyzes the user's emotions.

[0923] The emotion engine uses the emotion_recognition library to recognize user emotions from voice data. The input is voice data and the output is emotion data.

[0924] Step 5:

[0925] The server selects the best search results based on the emotion data.

[0926] Based on the user's emotional data, the emotion engine prioritizes the nearest gas station or parking lot if the user is in a hurry, and suggests a scenic route if the user is relaxed. The input is the emotional data and a list of gas stations and parking lots, and the output is the information on the optimal gas station or parking lot.

[0927] Step 6:

[0928] The server calculates the optimal route and starts navigation.

[0929] The navigation module uses the navigation_system module to calculate the route to the best gas station or parking lot and guide the user. The input is the best gas station or parking lot information, and the output is the route guidance information.

[0930] Example 2

[0931] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0932] Conventional navigation systems can search for optimal gas stations and parking lots and provide route guidance based on the user's voice commands, but they have a problem in that they cannot provide services that take the user's emotions into account. For example, they cannot provide search results or route guidance that correspond to whether the user is in a hurry or relaxed.

[0933] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0934] In this invention, the server includes means for accepting voice instructions through natural dialogue, means for converting the voice instructions into text data, means for acquiring a current location, means for searching for the cheapest fuel supply facility closest to the current location, means for searching for the cheapest parking facility closest to the current location that is currently available, means for providing route guidance to the searched fuel supply facility or parking facility, means for analyzing emotions from the user's voice, and means for providing search results and route guidance according to the user's emotions, thereby enabling optimal search results and route guidance according to the user's emotions.

[0935] "Natural dialogue voice instructions" refers to voice input given by a user to the system in everyday language.

[0936] "Means for converting voice instructions into text data" refers to a function that uses voice recognition technology to convert a user's voice into text information.

[0937] "Location capabilities" refers to the ability to determine a user's current location using GPS or other location technologies.

[0938] "Fuel distribution facility" means a gasoline station or other place that dispenses fuel.

[0939] "Parking facility" means a place for the temporary parking of vehicles.

[0940] "Means of providing route guidance" refers to a function that provides the user with the optimal route to reach their destination.

[0941] "Means for analyzing emotions from a user's voice" refers to a function that uses voice analysis technology to determine the user's emotional state.

[0942] "Means of providing search results and route guidance according to the user's emotions" refers to a function that suggests optimal search results and routes based on the user's emotional state.

[0943] This invention is a system that searches for the nearest inexpensive fuel supply facility or parking facility from the current location and provides optimal route guidance by using voice instructions from the user. It can also analyze the user's emotions and provide services according to those emotions.

[0944] Hardware and Software Used

[0945] Speech Recognition Module

[0946] The device uses a commonly used speech recognition API (e.g., Google Cloud Speech-to-Text API) as a speech recognition module that converts the user's voice instructions into text data.

[0947] GPS Features

[0948] The device uses its built-in GPS to obtain the user's current location, which can then be used to locate fuel and parking facilities.

[0949] Search Module

[0950] The server uses a map API (e.g., Google Maps API) as a search module that searches for the nearest, cheapest fuel and parking facilities from the user's current location.

[0951] Navigation Module

[0952] The terminal uses a navigation API (e.g., Google Maps Navigation) as a navigation module, which provides optimal route guidance to the user based on the search results.

[0953] Emotion Engine

[0954] The terminal uses an emotion analysis API (e.g., IBM Watson Tone Analyzer) as an emotion engine that analyzes emotions from the user's voice and provides the results to other parts of the system.

[0955] Examples

[0956] Example 1: Searching for gas stations

[0957] The user gives a voice command such as "Tell me where the nearest cheapest gas station is." The device converts the voice to text using the Google Cloud Speech-to-Text API and sends it to the server. The server uses the Google Maps API to find the nearest cheapest fuel station and sends the results to the device. The device uses Google Maps Navigation to provide route guidance to the user.

[0958] Example 2: Emotion-based route guidance

[0959] The user gives a voice command such as "Tell me the nearest cheap parking lot." The device converts the voice into text and analyzes the user's emotions using an emotion engine (IBM Watson Tone Analyzer). If it is determined that the user is in a hurry, the server prioritizes searching for the nearest parking facility and sends it to the device. The device uses a navigation module to provide the optimal route.

[0960] Examples of prompt statements

[0961] "If a user says, 'Where is the nearest cheapest gas station?' Describe a system that uses a speech recognition module to convert the speech to text, finds the nearest cheapest gas station from the user's current location, and uses a navigation module to provide route guidance."

[0962] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0963] Step 1:

[0964] The user gives a voice command such as "Tell me the nearest cheap gas station."

[0965] Input: User's voice command

[0966] Output: Audio data

[0967] Specific operation: The user gives voice instructions into the microphone of the terminal.

[0968] Step 2:

[0969] The terminal uses a voice recognition module to convert voice instructions into text data.

[0970] Input: Audio data

[0971] Output: Text data

[0972] Specific operation: The device calls the Google Cloud Speech-to-Text API and converts the voice data into text data such as "What is the nearest cheapest gas station?"

[0973] Step 3:

[0974] The device uses the GPS function to obtain the user's current location.

[0975] Input: None (internal processing of the terminal)

[0976] Output: Current location data

[0977] Specific operation: The device uses its built-in GPS function to obtain the user's current location as "1-1-1 Marunouchi, Chiyoda-ku, Tokyo."

[0978] Step 4:

[0979] The server uses a search module to search for the closest, least expensive fuel supply facility from the current location.

[0980] Input: current location data, text data

[0981] Output: Search result data

[0982] Specific operation: The server uses the Google Maps API to search for the nearest gas station from "1-1-1 Marunouchi, Chiyoda-ku, Tokyo," and obtains the result as "Gas Station A."

[0983] Step 5:

[0984] The server transmits the search results to the terminal.

[0985] Input: Search result data

[0986] Output: Search result data (terminal side)

[0987] Specific operation: The server sends information about "Gas Station A" to the terminal.

[0988] Step 6:

[0989] The device uses the navigation module to provide the user with optimal route guidance based on the search results.

[0990] Input: Search result data

[0991] Output: Route directions data

[0992] Specific operation: The device uses Google Maps Navigation to guide the user to the optimal route to "Gas Station A." For example, the device may provide guidance such as "Go straight, turn right, then turn left to reach your destination."

[0993] Step 7:

[0994] The terminal uses an emotion engine to analyze emotions from the user's voice.

[0995] Input: Audio data

[0996] Output: Emotion data

[0997] Specific operation: The device uses IBM Watson Tone Analyzer to analyze the emotion of "hurry" from the user's voice.

[0998] Step 8:

[0999] Based on the analysis results of the emotion engine, the server provides search results and route guidance that correspond to the user's emotions.

[1000] Input: emotion data, search result data

[1001] Output: Search results data according to emotions, route guidance data

[1002] Specific operation: If the user is determined to be in a hurry, the server will search for the nearest gas station first and send it to the device. The device will then use the navigation module to provide the optimal route.

[1003] (Application example 2)

[1004] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1005] Conventional navigation systems were able to search for the nearest and cheapest gas stations and parking lots and provide route guidance based on the user's voice instructions, but they did not provide services that took the user's emotions into consideration. As a result, it was not possible to provide optimal services according to the user's situation, such as when the user was in a hurry or relaxed. The present invention aims to provide a navigation system that is more comfortable and efficient for users by recognizing the user's emotions and providing search results and route guidance according to those emotions.

[1006] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1007] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station closest to the current location, a means for searching for the cheapest parking lot that is closest to the current location and currently available, a means for providing route guidance to the searched gas station or parking lot, a means for recognizing the user's emotions, and a means for adjusting the search results and route guidance in accordance with the user's emotions, thereby enabling optimal search results and route guidance in accordance with the user's emotions.

[1008] "Means for accepting voice instructions through natural dialogue" refers to means for recognizing voice instructions given by a user in natural language and inputting them into the system.

[1009] "A means for searching for the nearest and cheapest gas station from the current location" is a means for identifying the nearest and cheapest gas station based on the user's current location information.

[1010] "Means for searching for the nearest parking lot that is available and has the lowest price from the current location" refers to a means for identifying the nearest parking lot that is available and has the lowest price based on the user's current location information.

[1011] "Means for providing route guidance to the searched gas station or parking lot" refers to means for providing a route for guiding the user to the destination based on the search results.

[1012] "Means for recognizing a user's emotions" means means for analyzing emotions from a user's voice or other input and identifying the user's emotional state.

[1013] "Means for adjusting search results and route guidance according to user emotions" refers to means for changing and optimizing the content of search results and route guidance based on recognized user emotions.

[1014] The following system configuration is used as an embodiment of the present invention. The system is composed of a voice recognition module, an emotion recognition module, a search module, and a navigation module. These modules are installed on a server and a user terminal (e.g., a navigation system for an autonomous vehicle).

[1015] Speech Recognition Module

[1016] The speech recognition module accepts the user's voice instructions and converts them into text data. Specifically, it uses a microphone to input voice and uses the speech_recognition library to convert the voice data into text data. This text data is passed to the next processing step.

[1017] Emotion Recognition Module

[1018] The emotion recognition module analyzes emotions from the user's voice. The voice data is analyzed with the emotion_recognition library to identify the user's emotional state. This emotion information is provided to the search and navigation modules.

[1019] Search Module

[1020] The search module searches for the nearest and cheapest gas stations and parking lots based on the user's current location. Specifically, it makes an API request to identify the nearest gas stations and parking lots with the cheapest prices from the user's current location. The search results are passed to the navigation module.

[1021] Navigation Module

[1022] The navigation module guides the user to the destination based on the search results. The navigation system uses the navigation_system module to calculate the optimal route and provide real-time guidance to the user. It is also possible to adjust the content of the route guidance based on the user's emotion information.

[1023] Examples

[1024] For example, if a user gives a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts the command into text data. Next, the emotion recognition module analyzes the emotion from the user's voice and, if it determines that the user is in a hurry, the search module will prioritize searching for the nearest gas station. The search results are sent to the navigation module, which provides the user with optimal route guidance.

[1025] Examples of prompt statements

[1026] When a user gives a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts this command into text data. Next, the search module searches for the nearest cheap gas station from the current location. The search results are sent to the navigation module, which provides optimal route guidance for the user. In addition, an emotion engine that recognizes the user's emotions is combined, and if the user feels that they are in a hurry, the nearest gas station will be searched for first.

[1027] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1028] Step 1:

[1029] The user gives a voice command. For example, the user speaks a voice command into the microphone, such as "Tell me where the nearest cheap gas station is." The input is the user's voice data, and the output is the voice data that is passed to the voice recognition module.

[1030] Step 2:

[1031] The terminal uses a voice recognition module to convert voice data into text data. Specifically, it uses the speech_recognition library to analyze the voice data and generate corresponding text data. The input is voice data, and the output is text data.

[1032] Step 3:

[1033] The device uses the emotion recognition module to analyze the user's emotions. The voice data is analyzed with the emotion_recognition library to identify the user's emotional state. The input is the voice data, and the output is the user's emotional information.

[1034] Step 4:

[1035] The server uses a search module to search for the nearest and cheapest gas station based on the user's current location. It makes an API request to identify the nearest gas station with the cheapest price from the current location. The input is the current location and the user's sentiment, and the output is the search result.

[1036] Step 5:

[1037] The server uses a search module to search for the nearest, available, and cheapest parking lot based on the user's current location. It makes an API request to identify the nearest and cheapest parking lot from the current location. The input is the current location and the user's sentiment information, and the output is the search result.

[1038] Step 6:

[1039] The server uses the navigation module to guide the user to the destination based on the search results. It uses the navigation_system module to calculate the optimal route and provide real-time guidance to the user. The input is the search results and the user's emotion information, and the output is the route guidance information.

[1040] Step 7:

[1041] The terminal provides the user with route guidance information from the navigation module. Specifically, the terminal provides route guidance to the user through voice and display. The input is route guidance information, and the output is guidance information for the user.

[1042] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1043] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1044] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1045] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1046] [Third embodiment]

[1047] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1048] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1049] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1050] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1051] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1052] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1053] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1054] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1055] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1056] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1057] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1058] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1059] "Example 1"

[1060] The system of the present invention includes a voice recognition module that accepts voice instructions from a user, a search module that searches for information on gas stations and parking lots, and a navigation module that provides route guidance. The voice recognition module accepts voice instructions from a user through natural dialogue and converts them into text data. The search module searches for the nearest and cheapest gas station from the current location or the cheapest parking lot that is currently available based on the converted text data. The navigation module provides route guidance to the searched gas station or parking lot.

[1061] "Example 2"

[1062] In a specific embodiment, when a user issues a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts the command into text data. Then, the search module searches for the nearest cheap gas station from the current location. The search results are sent to the navigation module, which provides the user with optimal route guidance.

[1063] The process flow of each embodiment will be described below.

[1064] "Example 1"

[1065] Step 1: The user gives a voice command, for example, "What is the nearest cheap gas station?"

[1066] Step 2: The voice recognition module accepts the user's voice commands and converts them into text data.

[1067] Step 3: The search module uses the converted text data to search for the closest, cheapest gas station from the current location.

[1068] Step 4: The navigation module provides directions to the searched gas station.

[1069] "Example 2"

[1070] Step 1: The user issues a voice command, for example, "Tell me where the nearest available parking space is."

[1071] Step 2: The voice recognition module accepts the user's voice commands and converts them into text data.

[1072] Step 3: The search module uses the converted text data to search for the closest, most available and cheapest parking lot from the current location.

[1073] Step 4: The navigation module provides route guidance to the searched parking lot.

[1074] Example 1

[1075] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1076] Conventional navigation systems have the problem that even if a user issues voice instructions, the accuracy of voice recognition is low, making it difficult to obtain accurate search results. In addition, when searching for gas stations or parking lots, it is often impossible to quickly find the nearest and cheapest facility. Furthermore, route guidance based on search results is insufficient, and users may not be able to reach their destination efficiently. To solve these problems, there is a demand for a system that improves the accuracy of voice recognition, provides quick and accurate search results, and provides efficient route guidance.

[1077] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1078] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for converting the voice instructions into text data, a means for searching for the cheapest fuel supply facility closest to the current location, a means for searching for the cheapest parking facility closest to the current location that is currently available, a means for providing route guidance to the searched fuel supply facility or parking facility, and a means for transmitting and receiving data including the voice instructions, search results, and route guidance. This allows the user to give voice instructions in a natural dialogue format, improves the accuracy of voice recognition, and enables quick and accurate search results to be obtained. Furthermore, efficient route guidance allows the user to quickly reach the destination.

[1079] "Natural dialogue voice instructions" refers to voice input given by a user to the system using everyday words and phrases.

[1080] "Means for converting voice instructions into text data" refers to technology or devices for analyzing a user's voice input and converting it into text information.

[1081] "Means for searching for the nearest cheapest fuel supply facility from the current location" refers to technology or devices for identifying the nearest gas station or other facility with the cheapest fuel price based on the user's current location.

[1082] "Means for searching for the closest, most available, and least expensive parking facility from the current location" refers to technology or devices for identifying the closest parking facility that is available and has the lowest parking fees based on the user's current location.

[1083] "Means for providing route guidance to the searched fuel supply facility or parking facility" refers to technology or devices for providing the user with the optimal route to reach their destination based on the search results.

[1084] "Means for transmitting and receiving data including voice instructions, search results, and route guidance" refers to technology or devices for transmitting and receiving information regarding a user's voice instructions, search results, and route guidance via a communications network.

[1085] The present invention is a system that receives voice instructions from a user, searches for the nearest and cheapest fuel supply facility or parking facility, and provides route guidance based on the results. A specific embodiment of this system will be described below.

[1086] System Configuration

[1087] Subject: User

[1088] A user issues voice instructions to a terminal while in a car or on the move, for example, using prompt sentences such as "Tell me where the nearest cheap gas station is" or "Find the nearest cheap parking lot."

[1089] Subject: Terminal

[1090] The device uses a microphone to receive voice instructions from the user. The voice data is temporarily stored in the device and then sent to a voice recognition module. A common voice recognition API is used for voice recognition. For example, Google Cloud Speech-to-Text API is used.

[1091] The voice recognition module converts the voice data into text data that accurately reflects the user's intent, and the converted text data is sent from the terminal to the server.

[1092] Subject: Server

[1093] The server starts a search module based on the received text data. The search module searches for the nearest and cheapest fuel supply station or the cheapest parking facility that is currently available based on the user's current location information. A general map API is used for the search. For example, Google Maps API is used.

[1094] The search module retrieves search results and returns them to the server, which may include the names, addresses, price information, etc. of establishments. The server then transmits these search results to the terminal.

[1095] The terminal starts a navigation module based on the received search results. The navigation module provides route guidance to the user. For the route guidance, a general navigation API is used. For example, the Google Maps Directions API is used.

[1096] Examples

[1097] Example 1: Searching for gas stations

[1098] When a user says, "Tell me where the nearest cheap gas station is," the device's microphone captures the speech. The speech recognition module converts the speech into text data and sends it to the server. The server uses the Google Maps API to find the nearest cheap gas station and returns the results to the device. The device then launches the navigation module and provides route guidance to the user.

[1099] Example 2: Searching for parking spaces

[1100] When a user says, "Find the nearest cheap parking," the device's microphone captures the voice. The speech recognition module converts the voice into text data and sends it to the server. The server uses the Google Maps API to find the nearest, available, and cheapest parking spots and returns the results to the device. The device then launches the navigation module and provides route guidance to the user.

[1101] In this way, the system starts with the user's voice command, and each step of the voice recognition, search, and navigation work together. This allows the user to give voice commands in a natural dialogue format, improving the accuracy of voice recognition and enabling quick and accurate search results. In addition, efficient route guidance allows the user to reach their destination quickly.

[1102] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1103] Step 1:

[1104] The user issues a voice command to the terminal saying, "Tell me the nearest cheap gas station."

[1105] Input: User's voice command

[1106] Output: Audio data is input to the device.

[1107] Specific operation: When a user issues a voice command inside the car, the device's microphone captures the voice.

[1108] Step 2:

[1109] The terminal uses a built-in microphone to collect the user's voice instructions.

[1110] Input: User's voice data

[1111] Output: Collected audio data

[1112] Specific operation: The device temporarily stores the voice data and prepares it to be sent to the voice recognition module.

[1113] Step 3:

[1114] The terminal starts the voice recognition module and sends the collected voice data to the voice recognition API.

[1115] Input: Collected audio data

[1116] Output: Audio data sent to the speech recognition API

[1117] What happens: The device calls the Google Cloud Speech-to-Text API to convert the voice data into text.

[1118] Step 4:

[1119] The voice recognition module converts the voice data into text data.

[1120] Input: Voice data sent to the speech recognition API

[1121] Output: Converted text data

[1122] What happens: The speech recognition module converts the speech "What's the nearest cheap gas station?" into text.

[1123] Step 5:

[1124] The terminal transmits the converted text data to the server.

[1125] Input: Converted text data

[1126] Output: Text data sent to the server

[1127] Specific operation: The device sends text data to the server using the HTTPS protocol.

[1128] Step 6:

[1129] The server launches a search module based on the received text data.

[1130] Input: Text data sent to the server

[1131] Output: Search module launch

[1132] Specific operations: The server analyzes the text data and launches the search module.

[1133] Step 7:

[1134] The search module searches for the nearest, least expensive fuel supply facility based on the user's current location.

[1135] Input: User's current location and text data

[1136] Output: Search results (fuel supply facility information)

[1137] What it does: The search module uses the Google Maps API to find the nearest and cheapest gas stations.

[1138] Step 8:

[1139] The search module retrieves the search results and returns them to the server.

[1140] Input: Search results (fuel supply facility information)

[1141] Output: Search results returned to the server

[1142] Specific Operation: The search module returns search results to the server, including the business's name, address, price information, etc.

[1143] Step 9:

[1144] The server transmits the obtained search results to the terminal.

[1145] Input: Search results returned by the server

[1146] Output: Search results sent to the terminal

[1147] Specific operation: The server sends the search results to the terminal using the HTTPS protocol.

[1148] Step 10:

[1149] The terminal launches a navigation module based on the received search results.

[1150] Input: Search results sent to the device

[1151] Output: Launch of navigation module

[1152] Specific operations: The device analyzes the search results and launches the navigation module.

[1153] Step 11:

[1154] The navigation module provides route guidance to the user.

[1155] Input: Launch of navigation module and search results

[1156] Output: Directions provided to the user

[1157] What it does: The navigation module uses the Google Maps Directions API to provide the user with specific route directions via voice.

[1158] (Application example 1)

[1159] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1160] Conventional navigation systems require users to manually search for gas stations and parking lots, which may compromise safety while driving. It is also difficult to efficiently find the nearest and cheapest gas stations and parking lots. In addition, route guidance based on search results may not be intuitive, which may reduce user convenience.

[1161] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1162] In this invention, the server includes a means for receiving voice instructions through natural dialogue, a means for searching for the cheapest gas station closest to the current location, a means for searching for the cheapest parking lot closest to the current location that is currently available, a means for providing route guidance to the searched gas station or parking lot, a means for converting the voice instructions into text data, and a means for displaying a route on a map based on the search results. This enables a user to safely and efficiently search for a gas station or parking lot using voice instructions and receive intuitive route guidance even while driving.

[1163] "Natural dialogue voice instructions" refers to voice input given to the system by a user using everyday language.

[1164] "Means for accepting voice instructions" refers to devices or software that have the function of recognizing voice input from a user and incorporating it into the system.

[1165] "Means for searching for the closest, least expensive gas station from a user's current location" means a device or software capable of identifying the closest, least expensive gas station based on a user's current location.

[1166] "Means for searching for the closest, most available, and least expensive parking space from the current location" refers to a device or software that has the functionality to identify the closest, most available, and least expensive parking space based on the user's current location.

[1167] "Means for providing route guidance to the searched gas station or parking lot" refers to devices or software that have the function of guiding the user to their destination based on the search results.

[1168] "Means for converting voice instructions into text data" refers to a device or software that has the function of converting a user's voice input into text information.

[1169] "Means for displaying a route on a map based on the search results" refers to a device or software having a function for visually displaying a route to a searched destination on a map.

[1170] The following system configuration is used as an embodiment of the present invention. The system includes a voice recognition module, a search module, and a navigation module. These modules work together to accept voice instructions from a user, search for information on gas stations and parking lots, and provide route guidance.

[1171] Speech Recognition Module

[1172] The speech recognition module accepts the user's voice instructions and converts them into text data. Specifically, it uses a microphone to capture the user's voice and converts the voice into text using the SpeechRecognition library. This allows the user to give instructions to the system in a natural dialogue format.

[1173] Search Module

[1174] The search module searches for the nearest and cheapest gas station or the cheapest parking space that is currently available based on the text data received from the voice recognition module. Specifically, it uses the Google Places API to perform searches based on the user's current location. The search results are selected based on the most convenient results for the user.

[1175] Navigation Module

[1176] The navigation module provides route guidance to the destination based on the results obtained from the search module. Specifically, it uses the Folium library to display the route based on the search results on a map. This allows the user to visually check the route to the destination.

[1177] Hardware and Software Used

[1178] Hardware: Microphone, GPS module

[1179] Software: SpeechRecognition library, requests library, Google Places API, Folium library

[1180] Examples

[1181] When a user says, "Tell me where the nearest gas station is," the system searches for the nearest gas station from the user's current location and displays the route. The following is an example of a prompt:

[1182] Examples of prompt statements

[1183] User: "What's the closest gas station?"

[1184] System: "Voice commands please."

[1185] User: "What's the closest gas station?"

[1186] System: "Recognized text: Where is the nearest gas station?"

[1187] System: "Searching for gas stations..."

[1188] System: "Route to gas station x has been generated."

[1189] In this way, users can safely and efficiently search for gas stations and parking lots and receive intuitive route guidance while driving.

[1190] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1191] Step 1:

[1192] The user issues a voice command.

[1193] The user issues a voice command into the microphone, such as "Tell me where the nearest gas station is." The input is the user's voice, and the output is voice data.

[1194] Step 2:

[1195] The terminal captures the voice data and sends it to the voice recognition module.

[1196] The device captures the user's voice using a microphone and converts the voice data into text data using the SpeechRecognition library. The input is voice data and the output is text data.

[1197] Step 3:

[1198] The server receives the text data and sends it to the search module.

[1199] The server analyzes the text data received from the speech recognition module to understand the user's intent. The input is the text data, and the output is the search query.

[1200] Step 4:

[1201] Based on the search query, the server searches for the closest, cheapest gas stations and parking lots from the current location.

[1202] The server uses the Google Places API to search for gas stations and parking lots based on the user's current location. The input is the search query and the current location data, and the output is the search results.

[1203] Step 5:

[1204] The server sends the search results to the navigation module.

[1205] The server analyzes the search results and selects the most suitable gas station or parking lot. The input is the search results, and the output is the selected destination data.

[1206] Step 6:

[1207] The terminal receives the selected destination data and provides route guidance.

[1208] The device uses the Folium library to display routes to selected destinations on a map. The input is destination data, and the output is a map with route directions.

[1209] Step 7:

[1210] The user follows the route guidance to the destination.

[1211] The user follows the route guidance displayed on the terminal to head to a gas station or parking lot. The input is a map of the route guidance, and the output is reaching the destination.

[1212] Example 2

[1213] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1214] In conventional systems, when a user gives voice instructions, it is difficult to accurately understand the instructions and provide appropriate information. In addition, route guidance based on search results for gas stations and parking lots is insufficient, resulting in low user convenience. Furthermore, the accuracy and speed of the process of converting voice instructions into text data are also issues. There is a demand for a system that can solve these problems and provide users with quick and accurate information and route guidance.

[1215] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1216] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for converting the voice instructions into text data, a means for searching for the cheapest gas station closest to the current location, a means for transmitting the search results to the navigation module, and a means for providing route guidance to the searched gas station. This makes it possible to accurately understand voice instructions given by the user and provide appropriate information promptly. In addition, by providing optimal route guidance based on the search results, it is possible to improve user convenience.

[1217] "Natural dialogue voice instructions" refers to voice input by the user in everyday language, with the goal of the system understanding the speech and responding appropriately.

[1218] "Means for converting voice instructions into text data" refers to technology that analyzes a user's voice input and converts it into a corresponding text format.

[1219] "Current location" refers to where a user is currently located, typically determined based on location information such as GPS data.

[1220] "Method of finding nearest cheapest gas stations" refers to a search algorithm or database used to identify the nearest cheapest gas stations based on a user's current location.

[1221] "Means for transmitting search results to a navigation module" refers to a technique for transmitting information on the searched gas stations to a navigation system and generating route guidance.

[1222] "Means of route guidance" refers to technology for providing users with the optimal route to their destination, and includes map displays and voice guidance.

[1223] This invention is a system that searches for the nearest inexpensive gas station from the user's current location in response to voice instructions, and provides route guidance based on the search results. A specific embodiment of this system will be described below.

[1224] Receiving and converting voice commands

[1225] The user issues a voice command such as "Tell me where the cheapest gas station is nearby." A voice recognition module (e.g., a voice recognition API) installed in the device converts this voice command into text data. The voice recognition module analyzes the voice signal and generates the corresponding text.

[1226] Gas station search

[1227] The server receives the text data and searches for the nearest and cheapest gas station from the user's current location using a search module (e.g., map API). The search module obtains the user's current location information from GPS data and gas station price information from a database.

[1228] Providing navigation

[1229] The server sends the search results to a navigation module (e.g., navigation API) to generate optimal route guidance. The device provides this route guidance to the user. Specifically, the map and route are displayed on the smartphone screen, and voice guidance is also provided.

[1230] Examples

[1231] For example, if a user is near Tokyo Station, they may give a voice command such as, "Tell me the nearest cheap gas station." The voice recognition module converts this command into text data, and the server uses a map API to search for gas stations near Tokyo Station. The search results may reveal that "Gas Station A" is the closest and cheapest. The server uses the navigation API to generate the optimal route to "Gas Station A," and the device provides route guidance to the user. Specifically, a map and route to "Gas Station A" are displayed on the smartphone screen, and voice guidance such as, "Turn right 500 meters ahead" is given.

[1232] Examples of prompt statements

[1233] "If a user says, 'Tell me where the cheapest gas station is nearby,' how would you proceed?"

[1234] This invention allows users to quickly find the nearest, cheapest gas station by simply giving voice instructions, and furthermore, to receive optimal route guidance to that location, which greatly improves user convenience.

[1235] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1236] Step 1:

[1237] The user gives a voice command.

[1238] A user issues a voice command such as, "Tell me the nearest cheap gas station." The input is the user's voice, and the output is a voice signal.

[1239] Step 2:

[1240] The device converts the voice into text data.

[1241] A voice recognition module (e.g., a voice recognition API) installed in the device receives the user's voice instructions and converts them into text data. Specifically, it analyzes the voice signal and generates the corresponding text. The input is the voice signal, and the output is the text data.

[1242] Step 3:

[1243] The server receives the text data.

[1244] The terminal generates text data and sends it to the server. The server receives the text data and proceeds to the next processing step. The input is the text data, and the output is a receipt confirmation.

[1245] Step 4:

[1246] The server searches for gas stations.

[1247] The server analyzes the text data and obtains the user's current location information. It then uses a search module (e.g., map API) to search for the nearest, least expensive gas station from the user's current location. Specifically, it uses GPS data to identify the current location and obtains gas station price information from a database. The input is the text data and GPS data, and the output is the search results.

[1248] Step 5:

[1249] The server sends the search results to the navigation module.

[1250] The server sends the search results to a navigation module (e.g., navigation API) and generates optimal route guidance. Specifically, it calculates the optimal route from the user's current location based on the location information of gas stations obtained as search results. The input is the search results, and the output is route guidance data.

[1251] Step 6:

[1252] The terminal provides route guidance to the user.

[1253] The terminal provides the route guidance received from the navigation module to the user. Specifically, the map and route are displayed on the smartphone screen, and voice guidance is also provided. For example, specific instructions such as "Turn right 500 meters ahead" are provided by voice. The input is route guidance data, and the output is route guidance to the user.

[1254] (Application example 2)

[1255] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1256] Conventional navigation systems have difficulty efficiently searching for the nearest and cheapest gas stations and parking lots and providing real-time updated information, even when users give voice instructions. In addition, they lack sufficient coordination with autonomous vehicles and are unable to provide optimal route guidance. This can lead to users spending a lot of time and effort to reach their destination.

[1257] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1258] In this invention, the server includes a means for receiving voice instructions through natural dialogue, a means for searching for the nearest and cheapest gas station from the current location, a means for searching for the nearest and cheapest parking lot from the current location that is currently available, a means for providing route guidance to the searched gas station or parking lot, a means for updating price and location information in real time, a means for providing search results based on the voice instructions, and a means for providing optimal route guidance in cooperation with the navigation system of the autonomous vehicle. This allows the user to efficiently search for the nearest and cheapest gas station or parking lot through voice instructions, and to receive optimal route guidance based on information updated in real time.

[1259] "Voice instructions through natural dialogue" refers to voice input made by the user in natural language, similar to everyday conversation.

[1260] "Means for searching for the closest, least expensive gas station from a current location" refers to a method or device for identifying the closest, least expensive gas station based on a user's current location.

[1261] "Means for searching for the closest, currently available, and cheapest parking lot from the current location" refers to a method or device for identifying the closest, available, and cheapest parking lot based on the user's current location.

[1262] "Means for providing route guidance to a searched gas station or parking lot" refers to a method or device for providing a user with the optimal route to a gas station or parking lot that the user has searched for.

[1263] "Means for updating price and location information in real time" refers to a method or apparatus for obtaining and updating price and location information for gas stations and parking lots in real time.

[1264] "Means for providing search results based on voice instructions" refers to a method or device for generating and providing search results based on a user's voice instructions.

[1265] "Means for providing optimal route guidance in cooperation with the navigation system of an autonomous vehicle" refers to a method or device for providing optimal routes in cooperation with the navigation system of an autonomous vehicle.

[1266] The following system configuration will be described as an embodiment of the present invention.

[1267] System Configuration

[1268] The system includes a voice recognition module, a search module, a navigation module, a real-time update module, and a module that interfaces with the autonomous vehicle's navigation system.

[1269] Speech Recognition Module

[1270] The voice recognition module converts the voice into text data when the user gives a voice command through natural dialogue. Specifically, it inputs the voice using a microphone and converts the voice into text using the speech_recognition library.

[1271] Search Module

[1272] The search module searches for the nearest and cheapest gas station and the cheapest parking lot that is currently available based on the user's current location. To do this, it uses the GPS module to get the current location and the requests library to get information about gas stations and parking lots from external APIs.

[1273] Navigation Module

[1274] The navigation module provides optimal route guidance based on the search results. It uses the navigation API to calculate the shortest route from the user's current location to the destination and provides guidance.

[1275] Real-time update module

[1276] The real-time update module updates prices and location information for gas stations and parking lots in real time, so users can always get the most up-to-date search results.

[1277] Linking with autonomous vehicle navigation systems

[1278] The module, which works with the autonomous vehicle's navigation system, sends the search results to the autonomous vehicle's navigation system and provides optimal route guidance, allowing users to reach their destination efficiently through voice instructions.

[1279] Examples

[1280] For example, when a user gives a voice command such as "Tell me the nearest cheapest gas station," the voice recognition module converts the command into text data. The search module then searches for the nearest cheapest gas station from the current location. The search results are sent to the navigation module, which provides the optimal route guidance for the user. The real-time update module keeps the gas station prices and location information always up to date. By linking with the autonomous vehicle's navigation system, users can use the autonomous vehicle to reach their destination efficiently.

[1281] Examples of prompt statements

[1282] When a user gives a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts this command into text data, and the search module searches for the nearest cheap gas station from the current location. The search results are sent to the navigation module, and the system is designed to provide the optimal route guidance for the user.

[1283] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1284] Step 1:

[1285] The user gives a voice command.

[1286] The user gives voice instructions in a natural dialogue, such as "Tell me where the nearest cheap gas station is." The input is the user's voice, and the output is voice data.

[1287] Step 2:

[1288] The terminal converts the voice data into text data.

[1289] The speech recognition module uses a microphone to capture voice data and converts the voice to text using the speech_recognition library. The input is the voice data and the output is the text data.

[1290] Step 3:

[1291] The server gets the current location.

[1292] Use the device's GPS module to obtain the user's current location. The input is the GPS signal, and the output is the coordinate data of the current location.

[1293] Step 4:

[1294] The server obtains gas station information.

[1295] Use the requests library to get gas station location and price information from an external API. The input is the API request, and the output is data including gas station location and price information.

[1296] Step 5:

[1297] The server searches for the nearest cheapest gas station.

[1298] Based on the acquired gas station information, the distance from the current location and the price are calculated to identify the nearest and cheapest gas station. The input is the gas station location and price information, and the current location coordinate data, and the output is the optimal gas station information.

[1299] Step 6:

[1300] The server generates the navigation information.

[1301] Use the navigation API to generate route guidance from the current location to the best gas station. The input is the coordinate data of the current location and the location information of the best gas station, and the output is the route guidance information.

[1302] Step 7:

[1303] The terminal provides route guidance to the user.

[1304] The navigation module guides the user on the optimal route by voice or visually based on the generated route guidance information. The input is the route guidance information, and the output is navigation instructions to the user.

[1305] Step 8:

[1306] The server updates the information in real time.

[1307] The real-time update module periodically retrieves gas station prices and location information from an external API and updates the data in the system. The input is the API request and the output is the updated gas station information.

[1308] Step 9:

[1309] The server works in conjunction with the autonomous vehicle's navigation system.

[1310] The search results and route guidance information are sent to the navigation system of the autonomous vehicle, and the system controls the vehicle so that it automatically drives along the optimal route. The input is route guidance information, and the output is control instructions for the autonomous vehicle.

[1311] In addition, an emotion engine that estimates the emotion of the user may be further combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[1312] "Example 1"

[1313] As an embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotion of a user. The system includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station from the current location, a means for searching for the cheapest parking lot that is currently available from the current location, and a means for providing route guidance to the searched gas station or parking lot. The system further includes an emotion engine that recognizes the emotion of the user. The emotion engine analyzes the emotion from the user's voice and provides the result to other parts of the system. For example, if the user feels that he or she is in a hurry, the emotion engine provides the information to the search means, and the search means searches for the nearest gas station or parking lot with priority. Also, if the user feels that he or she is relaxed, the emotion engine provides the information to the route guidance means, and the route guidance means can provide services according to the user's emotion, such as suggesting a route with a better view.

[1314] "Example 2"

[1315] As an embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotion of a user. The system includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station from the current location, a means for searching for the cheapest parking lot that is currently available from the current location, and a means for providing route guidance to the searched gas station or parking lot. The system further includes an emotion engine that recognizes the emotion of the user. The emotion engine analyzes the emotion from the user's voice and provides the result to other parts of the system. For example, if the user feels that he or she is in a hurry, the emotion engine provides the information to the search means, and the search means searches for the nearest gas station or parking lot with priority. Also, if the user feels that he or she is relaxed, the emotion engine provides the information to the route guidance means, and the route guidance means can provide services according to the user's emotion, such as suggesting a route with a better view.

[1316] The process flow of each embodiment will be described below.

[1317] "Example 1"

[1318] Step 1: The user issues a voice command to the system, such as "Find the nearest cheap gas station."

[1319] Step 2: The system's voice recognition module converts the voice instructions into text data, while the emotion engine analyzes the emotions from the user's voice.

[1320] Step 3: Based on the analysis result of the sentiment engine, the system's search module searches for the nearest cheap gas station. For example, if the user feels that he is in a hurry, it will prioritize searching for the nearest gas station.

[1321] Step 4: Based on the search results, the system's navigation module will provide route guidance to the searched gas station. For example, if the user feels relaxed, it will suggest a more scenic route.

[1322] Example 1

[1323] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1324] Conventional navigation systems can accept voice instructions from users, but they cannot provide search results or route guidance that take the user's emotions into account. This means that there is an issue with the system being unable to provide optimal guidance according to the situation, such as when the user is in a hurry or relaxed.

[1325] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1326] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for converting the voice instructions into text data, a means for searching for the cheapest fuel supply facility closest to the current location, a means for searching for the cheapest parking facility closest to the current location that is currently available, a means for analyzing the user's emotions, a means for providing route guidance to the searched fuel supply facility or parking facility, and a means for adjusting the search results and route guidance based on the user's emotions, thereby enabling optimal search results and route guidance according to the user's emotions and circumstances.

[1327] "Natural dialogue voice instruction" refers to voice input by the user in everyday language, and is voice instruction that does not rely on specific commands or keywords.

[1328] "Means for converting voice instructions into text data" refers to a function that uses voice recognition technology to convert a user's voice input into text information.

[1329] A "fuel distribution facility" refers to a facility that supplies fuel or energy to vehicles, such as a gas station or an electric vehicle charging station.

[1330] A "parking facility" is a place for temporarily parking a vehicle, and refers to facilities including parking lots and parking meters.

[1331] "Means for analyzing user emotions" refers to technologies used to determine a user's emotional state from voice or other input data.

[1332] "Means for adjusting search results and route guidance" refers to the ability to change the content of search results and route guidance provided depending on the user's emotions and situation.

[1333] The present invention is a system that receives voice instructions from a user, analyzes emotions, searches for optimal fuel supply facilities and parking facilities, and provides appropriate route guidance. A specific embodiment of this system will be described below.

[1334] Hardware and Software Used

[1335] 1. Speech Recognition Module: Uses speech recognition technology to convert voice instructions into text data. This module uses a common speech recognition API.

[1336] Example: Speech Recognition API

[1337] 2. Search module: Uses the map API to search for the nearest cheapest fuel and parking facilities from your current location.

[1338] Example: Map Information API

[1339] 3. Navigation module: Uses the navigation API to provide route guidance to the searched facilities.

[1340] Example: Navigation API

[1341] 4. Emotion Engine: Uses emotion analysis technology to analyze emotions from the user's voice.

[1342] Example: Sentiment analysis API

[1343] Specific operation of the system

[1344] 1. Accepting user voice commands

[1345] The user issues a voice command in the car saying, "Tell me the nearest cheap gas station."

[1346] The device's microphone captures the user's voice and transmits it to the system as voice data.

[1347] 2. Voice-to-text conversion

[1348] The terminal transmits the voice data to a voice recognition module.

[1349] The speech recognition module converts voice data into text data using a speech recognition API. For example, a voice saying "Tell me where the nearest cheap gas station is" is converted into text saying "Tell me where the nearest cheap gas station is."

[1350] 3. Analysis and Search of Text Data

[1351] The server receives the converted text data and sends it to a search module.

[1352] The search module uses the map information API to search for the nearest and cheapest fuel supply facility from the current location. For example, if the current location is Shibuya-ku, Tokyo, the search module retrieves information on the nearest and cheapest gas station.

[1353] 4. Emotion Analysis

[1354] The terminal transmits the user's voice data to the emotion engine.

[1355] The emotion engine uses the emotion analysis API to analyze the user's emotions. For example, if it determines that the user's voice sounds urgent, it will send that information to the server.

[1356] 5. Getting search results

[1357] The server receives the results from the search module and the sentiment analysis results from the sentiment engine.

[1358] The server integrates the information of the nearest and cheapest fuel station with the emotion information that the user is in a hurry. For example, if the search result is "Gas Station A in Shibuya Ward" and it is determined that the user is in a hurry, the server sends the emotion information to the navigation module.

[1359] 6. Providing route guidance

[1360] The server sends the search results to the navigation module.

[1361] The navigation module uses the navigation API to provide route guidance to the searched fuel supply facility. For example, it guides the user on the shortest route to "Gas Station A in Shibuya Ward" by voice and map.

[1362] Examples and prompts

[1363] Examples

[1364] While driving, a user issues a voice command such as "Tell me the nearest cheap gas station." The device captures the voice and converts it into text using a voice recognition module. The server analyzes the text data and searches for the nearest cheapest gas station using a map information API. The emotion engine determines from the user's voice that the user is in a hurry and provides this information to the server. The server integrates the search results with the emotion information and sends it to the navigation module. The navigation module uses the navigation API to guide the user along the shortest route.

[1365] Examples of prompt statements

[1366] "Tell me where the nearest cheap gas station is."

[1367] "Find a parking spot that's currently available"

[1368] "Tell me the scenic route"

[1369] The system accepts the user's voice commands, analyzes emotions, searches for optimal fuel and parking facilities, and provides appropriate route guidance.

[1370] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1371] Step 1:

[1372] Accepting user voice commands

[1373] Input: User says, "What is the nearest cheap gas station?"

[1374] Specific operation: The device's microphone captures the user's voice and sends it to the system as voice data.

[1375] Output: Audio data is generated and sent to the speech recognition module.

[1376] Step 2:

[1377] Transcription of voice commands

[1378] Input: Audio data sent from the device.

[1379] Specific operation: The device sends voice data to the voice recognition module, which uses the voice recognition API to convert the voice data into text data.

[1380] Output: The text data "Please tell me the nearest cheap gas station" is generated and sent to the server.

[1381] Step 3:

[1382] Text data analysis and search

[1383] Input: Text data sent from the speech recognition module.

[1384] Specific operation: The server receives the converted text data and sends it to the search module, which uses the map information API to search for the nearest cheapest fuel supply station from the current location.

[1385] Output: The closest cheapest fuel station to your current location is generated.

[1386] Step 4:

[1387] Sentiment Analysis

[1388] Input: Audio data sent from the device.

[1389] Specific operation: The device sends the user's voice data to the emotion engine, which then uses the emotion analysis API to analyze the user's emotions.

[1390] Output: User emotion information (e.g., in a hurry) is generated and sent to the server.

[1391] Step 5:

[1392] Getting search results

[1393] Input: Search results from the search module and sentiment analysis results from the sentiment engine.

[1394] What it does: The server combines information about the nearest and cheapest fuel station with the user's emotional information that he or she is in a hurry.

[1395] Output: The integrated search results and sentiment information are generated and sent to the navigation module.

[1396] Step 6:

[1397] Providing directions

[1398] Input: The integrated search results and sentiment information sent from the server.

[1399] What happens next: The navigation module uses the navigation API to provide directions to the searched fueling facilities.

[1400] Output: Guidance information for the shortest route is generated and provided to the user via voice and map.

[1401] (Application example 1)

[1402] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1403] Conventional navigation systems can search for gas stations and parking lots and provide route guidance based on the user's voice commands, but they have a problem in that they cannot provide services that take the user's emotions into account. In particular, there is a demand for optimal route guidance depending on whether the user is in a hurry or relaxed.

[1404] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1405] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station nearest to the current location, a means for searching for the cheapest parking lot nearest to the current location that is currently available, a means for recognizing the user's emotions and providing services according to the emotions, and a means for preferentially searching for the nearest gas station or parking lot when the user is in a hurry and for suggesting a route with a nice view when the user is relaxed. This makes it possible to provide optimal route guidance according to the user's emotions.

[1406] "Natural dialogue voice instructions" refers to voice input given by a user to the system in everyday language.

[1407] A "gas station" is a facility for supplying fuel to vehicles.

[1408] A "parking lot" is a place for temporarily parking vehicles.

[1409] "Route guidance" refers to providing information showing directions to a destination.

[1410] "Means of recognizing emotions" refers to technology that analyzes emotions from the user's voice, facial expressions, etc., and provides the results to the system.

[1411] "Means of providing services" refers to technology that provides appropriate information and functions according to user requests and circumstances.

[1412] The "means for preferentially searching for the nearest gas station or parking lot" is a technology that preferentially searches for the nearest gas station or parking lot from the user's current location when the user is in a hurry.

[1413] The "means for suggesting scenic routes" is a technology that provides guidance on routes that are considered to have beautiful scenery when the user is relaxed.

[1414] The system for implementing the present invention includes a voice recognition module, a search module, a navigation module, and an emotion engine. The details of each module and its operation are described below.

[1415] Speech Recognition Module

[1416] The speech recognition module accepts the user's voice instructions and converts them into text data. Specifically, it uses a microphone to input the user's voice and uses the speech_recognition library to convert the voice into text. This text data is used for subsequent processing.

[1417] Search Module

[1418] The search module searches for the nearest and cheapest gas station and the cheapest parking lot that is currently available based on the text data obtained from the voice recognition module. Specifically, it uses the requests library to send requests to external APIs to obtain information on gas stations and parking lots.

[1419] Navigation Module

[1420] The navigation module provides route guidance to the gas station or parking lot obtained from the search module. Specifically, it uses the navigation_system module to calculate the optimal route to the destination and guides the user along it.

[1421] Emotion Engine

[1422] The emotion engine analyzes emotions from the user's voice and provides the results to the rest of the system. Specifically, it uses the emotion_recognition library to recognize emotions from voice data, and prioritizes finding the nearest gas station or parking lot if the user is in a hurry, or suggests scenic routes if the user is relaxed.

[1423] Examples

[1424] For example, if a user says, "Tell me where the nearest gas station is," the voice recognition module converts the voice to text, and the search module searches for the nearest gas station. The emotion engine analyzes the user's emotions, and if the user is in a hurry, it will prioritize the nearest gas station, and if the user is relaxed, it will suggest a scenic route.

[1425] Examples of prompt statements

[1426] When a user says "Tell me where the nearest gas station is," the speech recognition module converts the speech to text, and the search module searches for the nearest gas station. The emotion engine analyzes the user's emotions and prioritizes the nearest gas station if the user is in a hurry. The navigation module provides route guidance.

[1427] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1428] Step 1:

[1429] The user issues a voice command.

[1430] The user issues a voice command into the microphone, such as "Tell me where the nearest gas station is." The input is the user's voice, and the output is voice data.

[1431] Step 2:

[1432] The terminal converts the voice data into text data.

[1433] The speech recognition module uses the speech_recognition library to convert voice data into text data. The input is voice data and the output is text data.

[1434] Step 3:

[1435] The server searches for information on gas stations and parking lots based on the text data.

[1436] The search module uses the requests library to send requests to external APIs to get the closest and cheapest gas station from the current location, the cheapest parking lot that is currently open, etc. The input is text data, and the output is a list of gas stations and parking lots.

[1437] Step 4:

[1438] The server analyzes the user's emotions.

[1439] The emotion engine uses the emotion_recognition library to recognize user emotions from voice data. The input is voice data and the output is emotion data.

[1440] Step 5:

[1441] The server selects the best search results based on the emotion data.

[1442] Based on the user's emotional data, the emotion engine prioritizes the nearest gas station or parking lot if the user is in a hurry, and suggests a scenic route if the user is relaxed. The input is the emotional data and a list of gas stations and parking lots, and the output is the information on the optimal gas station or parking lot.

[1443] Step 6:

[1444] The server calculates the optimal route and starts navigation.

[1445] The navigation module uses the navigation_system module to calculate the route to the best gas station or parking lot and guide the user. The input is the best gas station or parking lot information, and the output is the route guidance information.

[1446] Example 2

[1447] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1448] Conventional navigation systems can search for optimal gas stations and parking lots and provide route guidance based on the user's voice commands, but they have a problem in that they cannot provide services that take the user's emotions into account. For example, they cannot provide search results or route guidance that correspond to whether the user is in a hurry or relaxed.

[1449] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1450] In this invention, the server includes means for accepting voice instructions through natural dialogue, means for converting the voice instructions into text data, means for acquiring a current location, means for searching for the cheapest fuel supply facility closest to the current location, means for searching for the cheapest parking facility closest to the current location that is currently available, means for providing route guidance to the searched fuel supply facility or parking facility, means for analyzing emotions from the user's voice, and means for providing search results and route guidance according to the user's emotions, thereby enabling optimal search results and route guidance according to the user's emotions.

[1451] "Natural dialogue voice instructions" refers to voice input given by a user to the system in everyday language.

[1452] "Means for converting voice instructions into text data" refers to a function that uses voice recognition technology to convert a user's voice into text information.

[1453] "Location capabilities" refers to the ability to determine a user's current location using GPS or other location technologies.

[1454] "Fuel distribution facility" means a gasoline station or other place that dispenses fuel.

[1455] "Parking facility" means a place for the temporary parking of vehicles.

[1456] "Means of providing route guidance" refers to a function that provides the user with the optimal route to reach their destination.

[1457] "Means for analyzing emotions from a user's voice" refers to a function that uses voice analysis technology to determine the user's emotional state.

[1458] "Means of providing search results and route guidance according to the user's emotions" refers to a function that suggests optimal search results and routes based on the user's emotional state.

[1459] This invention is a system that searches for the nearest inexpensive fuel supply facility or parking facility from the current location and provides optimal route guidance by using voice instructions from the user. It can also analyze the user's emotions and provide services according to those emotions.

[1460] Hardware and Software Used

[1461] Speech Recognition Module

[1462] The device uses a commonly used speech recognition API (e.g., Google Cloud Speech-to-Text API) as a speech recognition module that converts the user's voice instructions into text data.

[1463] GPS Features

[1464] The device uses its built-in GPS to obtain the user's current location, which can then be used to locate fuel and parking facilities.

[1465] Search Module

[1466] The server uses a map API (e.g., Google Maps API) as a search module that searches for the nearest, cheapest fuel and parking facilities from the user's current location.

[1467] Navigation Module

[1468] The terminal uses a navigation API (e.g., Google Maps Navigation) as a navigation module, which provides optimal route guidance to the user based on the search results.

[1469] Emotion Engine

[1470] The terminal uses an emotion analysis API (e.g., IBM Watson Tone Analyzer) as an emotion engine that analyzes emotions from the user's voice and provides the results to other parts of the system.

[1471] Examples

[1472] Example 1: Searching for gas stations

[1473] The user gives a voice command such as "Tell me where the nearest cheapest gas station is." The device converts the voice to text using the Google Cloud Speech-to-Text API and sends it to the server. The server uses the Google Maps API to find the nearest cheapest fuel station and sends the results to the device. The device uses Google Maps Navigation to provide route guidance to the user.

[1474] Example 2: Emotion-based route guidance

[1475] The user gives a voice command such as "Tell me the nearest cheap parking lot." The device converts the voice into text and analyzes the user's emotions using an emotion engine (IBM Watson Tone Analyzer). If it is determined that the user is in a hurry, the server prioritizes searching for the nearest parking facility and sends it to the device. The device uses a navigation module to provide the optimal route.

[1476] Examples of prompt statements

[1477] "If a user says, 'Where is the nearest cheapest gas station?' Describe a system that uses a speech recognition module to convert the speech to text, finds the nearest cheapest gas station from the user's current location, and uses a navigation module to provide route guidance."

[1478] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1479] Step 1:

[1480] The user gives a voice command such as "Tell me the nearest cheap gas station."

[1481] Input: User's voice command

[1482] Output: Audio data

[1483] Specific operation: The user gives voice instructions into the microphone of the terminal.

[1484] Step 2:

[1485] The terminal uses a voice recognition module to convert voice instructions into text data.

[1486] Input: Audio data

[1487] Output: Text data

[1488] Specific operation: The device calls the Google Cloud Speech-to-Text API and converts the voice data into text data such as "What is the nearest cheapest gas station?"

[1489] Step 3:

[1490] The device uses the GPS function to obtain the user's current location.

[1491] Input: None (internal processing of the terminal)

[1492] Output: Current location data

[1493] Specific operation: The device uses its built-in GPS function to obtain the user's current location as "1-1-1 Marunouchi, Chiyoda-ku, Tokyo."

[1494] Step 4:

[1495] The server uses a search module to search for the closest, least expensive fuel supply facility from the current location.

[1496] Input: current location data, text data

[1497] Output: Search result data

[1498] Specific operation: The server uses the Google Maps API to search for the nearest gas station from "1-1-1 Marunouchi, Chiyoda-ku, Tokyo," and obtains the result as "Gas Station A."

[1499] Step 5:

[1500] The server transmits the search results to the terminal.

[1501] Input: Search result data

[1502] Output: Search result data (terminal side)

[1503] Specific operation: The server sends information about "Gas Station A" to the terminal.

[1504] Step 6:

[1505] The device uses the navigation module to provide the user with optimal route guidance based on the search results.

[1506] Input: Search result data

[1507] Output: Route directions data

[1508] Specific operation: The device uses Google Maps Navigation to guide the user to the optimal route to "Gas Station A." For example, the device may provide guidance such as "Go straight, turn right, then turn left to reach your destination."

[1509] Step 7:

[1510] The terminal uses an emotion engine to analyze emotions from the user's voice.

[1511] Input: Audio data

[1512] Output: Emotion data

[1513] Specific operation: The device uses IBM Watson Tone Analyzer to analyze the emotion of "hurry" from the user's voice.

[1514] Step 8:

[1515] Based on the analysis results of the emotion engine, the server provides search results and route guidance that correspond to the user's emotions.

[1516] Input: emotion data, search result data

[1517] Output: Search results data according to emotions, route guidance data

[1518] Specific operation: If the user is determined to be in a hurry, the server will search for the nearest gas station first and send it to the device. The device will then use the navigation module to provide the optimal route.

[1519] (Application example 2)

[1520] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1521] Conventional navigation systems were able to search for the nearest and cheapest gas stations and parking lots and provide route guidance based on the user's voice instructions, but they did not provide services that took the user's emotions into consideration. As a result, it was not possible to provide optimal services according to the user's situation, such as when the user was in a hurry or relaxed. The present invention aims to provide a navigation system that is more comfortable and efficient for users by recognizing the user's emotions and providing search results and route guidance according to those emotions.

[1522] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1523] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station closest to the current location, a means for searching for the cheapest parking lot that is closest to the current location and currently available, a means for providing route guidance to the searched gas station or parking lot, a means for recognizing the user's emotions, and a means for adjusting the search results and route guidance in accordance with the user's emotions, thereby enabling optimal search results and route guidance in accordance with the user's emotions.

[1524] "Means for accepting voice instructions through natural dialogue" refers to means for recognizing voice instructions given by a user in natural language and inputting them into the system.

[1525] "A means for searching for the nearest and cheapest gas station from the current location" is a means for identifying the nearest and cheapest gas station based on the user's current location information.

[1526] "Means for searching for the nearest parking lot that is available and has the lowest price from the current location" refers to a means for identifying the nearest parking lot that is available and has the lowest price based on the user's current location information.

[1527] "Means for providing route guidance to the searched gas station or parking lot" refers to means for providing a route for guiding the user to the destination based on the search results.

[1528] "Means for recognizing a user's emotions" means means for analyzing emotions from a user's voice or other input and identifying the user's emotional state.

[1529] "Means for adjusting search results and route guidance according to user emotions" refers to means for changing and optimizing the content of search results and route guidance based on recognized user emotions.

[1530] The following system configuration is used as an embodiment of the present invention. The system is composed of a voice recognition module, an emotion recognition module, a search module, and a navigation module. These modules are installed on a server and a user terminal (e.g., a navigation system for an autonomous vehicle).

[1531] Speech Recognition Module

[1532] The speech recognition module accepts the user's voice instructions and converts them into text data. Specifically, it uses a microphone to input voice and uses the speech_recognition library to convert the voice data into text data. This text data is passed to the next processing step.

[1533] Emotion Recognition Module

[1534] The emotion recognition module analyzes emotions from the user's voice. The voice data is analyzed with the emotion_recognition library to identify the user's emotional state. This emotion information is provided to the search and navigation modules.

[1535] Search Module

[1536] The search module searches for the nearest and cheapest gas stations and parking lots based on the user's current location. Specifically, it makes an API request to identify the nearest gas stations and parking lots with the cheapest prices from the user's current location. The search results are passed to the navigation module.

[1537] Navigation Module

[1538] The navigation module guides the user to the destination based on the search results. The navigation system uses the navigation_system module to calculate the optimal route and provide real-time guidance to the user. It is also possible to adjust the content of the route guidance based on the user's emotion information.

[1539] Examples

[1540] For example, if a user gives a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts the command into text data. Next, the emotion recognition module analyzes the emotion from the user's voice and, if it determines that the user is in a hurry, the search module will prioritize searching for the nearest gas station. The search results are sent to the navigation module, which provides the user with optimal route guidance.

[1541] Examples of prompt statements

[1542] When a user gives a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts this command into text data. Next, the search module searches for the nearest cheap gas station from the current location. The search results are sent to the navigation module, which provides optimal route guidance for the user. In addition, an emotion engine that recognizes the user's emotions is combined, and if the user feels that they are in a hurry, the nearest gas station will be searched for first.

[1543] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1544] Step 1:

[1545] The user gives a voice command. For example, the user speaks a voice command into the microphone, such as "Tell me where the nearest cheap gas station is." The input is the user's voice data, and the output is the voice data that is passed to the voice recognition module.

[1546] Step 2:

[1547] The terminal uses a voice recognition module to convert voice data into text data. Specifically, it uses the speech_recognition library to analyze the voice data and generate corresponding text data. The input is voice data, and the output is text data.

[1548] Step 3:

[1549] The device uses the emotion recognition module to analyze the user's emotions. The voice data is analyzed with the emotion_recognition library to identify the user's emotional state. The input is the voice data, and the output is the user's emotional information.

[1550] Step 4:

[1551] The server uses a search module to search for the nearest and cheapest gas station based on the user's current location. It makes an API request to identify the nearest gas station with the cheapest price from the current location. The input is the current location and the user's sentiment, and the output is the search result.

[1552] Step 5:

[1553] The server uses a search module to search for the nearest, available, and cheapest parking lot based on the user's current location. It makes an API request to identify the nearest and cheapest parking lot from the current location. The input is the current location and the user's sentiment information, and the output is the search result.

[1554] Step 6:

[1555] The server uses the navigation module to guide the user to the destination based on the search results. It uses the navigation_system module to calculate the optimal route and provide real-time guidance to the user. The input is the search results and the user's emotion information, and the output is the route guidance information.

[1556] Step 7:

[1557] The terminal provides the user with route guidance information from the navigation module. Specifically, the terminal provides route guidance to the user through voice and display. The input is route guidance information, and the output is guidance information for the user.

[1558] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1559] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1560] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1561] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1562] [Fourth embodiment]

[1563] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1564] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1565] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1566] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1567] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1568] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1569] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1570] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1571] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1572] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1573] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1574] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1575] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1576] "Example 1"

[1577] The system of the present invention includes a voice recognition module that accepts voice instructions from a user, a search module that searches for information on gas stations and parking lots, and a navigation module that provides route guidance. The voice recognition module accepts voice instructions from a user through natural dialogue and converts them into text data. The search module searches for the nearest and cheapest gas station from the current location or the cheapest parking lot that is currently available based on the converted text data. The navigation module provides route guidance to the searched gas station or parking lot.

[1578] "Example 2"

[1579] In a specific embodiment, when a user issues a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts the command into text data. Then, the search module searches for the nearest cheap gas station from the current location. The search results are sent to the navigation module, which provides the user with optimal route guidance.

[1580] The process flow of each embodiment will be described below.

[1581] "Example 1"

[1582] Step 1: The user gives a voice command, for example, "What is the nearest cheap gas station?"

[1583] Step 2: The voice recognition module accepts the user's voice commands and converts them into text data.

[1584] Step 3: The search module uses the converted text data to search for the closest, cheapest gas station from the current location.

[1585] Step 4: The navigation module provides directions to the searched gas station.

[1586] "Example 2"

[1587] Step 1: The user issues a voice command, for example, "Tell me where the nearest available parking space is."

[1588] Step 2: The voice recognition module accepts the user's voice commands and converts them into text data.

[1589] Step 3: The search module uses the converted text data to search for the closest, most available and cheapest parking lot from the current location.

[1590] Step 4: The navigation module provides route guidance to the searched parking lot.

[1591] Example 1

[1592] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1593] Conventional navigation systems have the problem that even if a user issues voice instructions, the accuracy of voice recognition is low, making it difficult to obtain accurate search results. In addition, when searching for gas stations or parking lots, it is often impossible to quickly find the nearest and cheapest facility. Furthermore, route guidance based on search results is insufficient, and users may not be able to reach their destination efficiently. To solve these problems, there is a demand for a system that improves the accuracy of voice recognition, provides quick and accurate search results, and provides efficient route guidance.

[1594] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1595] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for converting the voice instructions into text data, a means for searching for the cheapest fuel supply facility closest to the current location, a means for searching for the cheapest parking facility closest to the current location that is currently available, a means for providing route guidance to the searched fuel supply facility or parking facility, and a means for transmitting and receiving data including the voice instructions, search results, and route guidance. This allows the user to give voice instructions in a natural dialogue format, improves the accuracy of voice recognition, and enables quick and accurate search results to be obtained. Furthermore, efficient route guidance allows the user to quickly reach the destination.

[1596] "Natural dialogue voice instructions" refers to voice input given by a user to the system using everyday words and phrases.

[1597] "Means for converting voice instructions into text data" refers to technology or devices for analyzing a user's voice input and converting it into text information.

[1598] "Means for searching for the nearest cheapest fuel supply facility from the current location" refers to technology or devices for identifying the nearest gas station or other facility with the cheapest fuel price based on the user's current location.

[1599] "Means for searching for the closest, most available, and least expensive parking facility from the current location" refers to technology or devices for identifying the closest parking facility that is available and has the lowest parking fees based on the user's current location.

[1600] "Means for providing route guidance to the searched fuel supply facility or parking facility" refers to technology or devices for providing the user with the optimal route to reach their destination based on the search results.

[1601] "Means for transmitting and receiving data including voice instructions, search results, and route guidance" refers to technology or devices for transmitting and receiving information regarding a user's voice instructions, search results, and route guidance via a communications network.

[1602] The present invention is a system that receives voice instructions from a user, searches for the nearest and cheapest fuel supply facility or parking facility, and provides route guidance based on the results. A specific embodiment of this system will be described below.

[1603] System Configuration

[1604] Subject: User

[1605] A user issues voice instructions to a terminal while in a car or on the move, for example, using prompt sentences such as "Tell me where the nearest cheap gas station is" or "Find the nearest cheap parking lot."

[1606] Subject: Terminal

[1607] The device uses a microphone to receive voice instructions from the user. The voice data is temporarily stored in the device and then sent to a voice recognition module. A common voice recognition API is used for voice recognition. For example, Google Cloud Speech-to-Text API is used.

[1608] The voice recognition module converts the voice data into text data that accurately reflects the user's intent, and the converted text data is sent from the terminal to the server.

[1609] Subject: Server

[1610] The server starts a search module based on the received text data. The search module searches for the nearest and cheapest fuel supply station or the cheapest parking facility that is currently available based on the user's current location information. A general map API is used for the search. For example, Google Maps API is used.

[1611] The search module retrieves search results and returns them to the server, which may include the names, addresses, price information, etc. of establishments. The server then transmits these search results to the terminal.

[1612] The terminal starts a navigation module based on the received search results. The navigation module provides route guidance to the user. For the route guidance, a general navigation API is used. For example, the Google Maps Directions API is used.

[1613] Examples

[1614] Example 1: Searching for gas stations

[1615] When a user says, "Tell me where the nearest cheap gas station is," the device's microphone captures the speech. The speech recognition module converts the speech into text data and sends it to the server. The server uses the Google Maps API to find the nearest cheap gas station and returns the results to the device. The device then launches the navigation module and provides route guidance to the user.

[1616] Example 2: Searching for parking spaces

[1617] When a user says, "Find the nearest cheap parking," the device's microphone captures the voice. The speech recognition module converts the voice into text data and sends it to the server. The server uses the Google Maps API to find the nearest, available, and cheapest parking spots and returns the results to the device. The device then launches the navigation module and provides route guidance to the user.

[1618] In this way, the system starts with the user's voice command, and each step of the voice recognition, search, and navigation work together. This allows the user to give voice commands in a natural dialogue format, improving the accuracy of voice recognition and enabling quick and accurate search results. In addition, efficient route guidance allows the user to reach their destination quickly.

[1619] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1620] Step 1:

[1621] The user issues a voice command to the terminal saying, "Tell me the nearest cheap gas station."

[1622] Input: User's voice command

[1623] Output: Audio data is input to the device.

[1624] Specific operation: When a user issues a voice command inside the car, the device's microphone captures the voice.

[1625] Step 2:

[1626] The terminal uses a built-in microphone to collect the user's voice instructions.

[1627] Input: User's voice data

[1628] Output: Collected audio data

[1629] Specific operation: The device temporarily stores the voice data and prepares it to be sent to the voice recognition module.

[1630] Step 3:

[1631] The terminal starts the voice recognition module and sends the collected voice data to the voice recognition API.

[1632] Input: Collected audio data

[1633] Output: Audio data sent to the speech recognition API

[1634] What happens: The device calls the Google Cloud Speech-to-Text API to convert the voice data into text.

[1635] Step 4:

[1636] The voice recognition module converts the voice data into text data.

[1637] Input: Voice data sent to the speech recognition API

[1638] Output: Converted text data

[1639] What happens: The speech recognition module converts the speech "What's the nearest cheap gas station?" into text.

[1640] Step 5:

[1641] The terminal transmits the converted text data to the server.

[1642] Input: Converted text data

[1643] Output: Text data sent to the server

[1644] Specific operation: The device sends text data to the server using the HTTPS protocol.

[1645] Step 6:

[1646] The server launches a search module based on the received text data.

[1647] Input: Text data sent to the server

[1648] Output: Search module launch

[1649] Specific operations: The server analyzes the text data and launches the search module.

[1650] Step 7:

[1651] The search module searches for the nearest, least expensive fuel supply facility based on the user's current location.

[1652] Input: User's current location and text data

[1653] Output: Search results (fuel supply facility information)

[1654] What it does: The search module uses the Google Maps API to find the nearest and cheapest gas stations.

[1655] Step 8:

[1656] The search module retrieves the search results and returns them to the server.

[1657] Input: Search results (fuel supply facility information)

[1658] Output: Search results returned to the server

[1659] Specific Operation: The search module returns search results to the server, including the business's name, address, price information, etc.

[1660] Step 9:

[1661] The server transmits the obtained search results to the terminal.

[1662] Input: Search results returned by the server

[1663] Output: Search results sent to the terminal

[1664] Specific operation: The server sends the search results to the terminal using the HTTPS protocol.

[1665] Step 10:

[1666] The terminal launches a navigation module based on the received search results.

[1667] Input: Search results sent to the device

[1668] Output: Launch of navigation module

[1669] Specific operations: The device analyzes the search results and launches the navigation module.

[1670] Step 11:

[1671] The navigation module provides route guidance to the user.

[1672] Input: Launch of navigation module and search results

[1673] Output: Directions provided to the user

[1674] What it does: The navigation module uses the Google Maps Directions API to provide the user with specific route directions via voice.

[1675] (Application example 1)

[1676] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1677] Conventional navigation systems require users to manually search for gas stations and parking lots, which may compromise safety while driving. It is also difficult to efficiently find the nearest and cheapest gas stations and parking lots. In addition, route guidance based on search results may not be intuitive, which may reduce user convenience.

[1678] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1679] In this invention, the server includes a means for receiving voice instructions through natural dialogue, a means for searching for the cheapest gas station closest to the current location, a means for searching for the cheapest parking lot closest to the current location that is currently available, a means for providing route guidance to the searched gas station or parking lot, a means for converting the voice instructions into text data, and a means for displaying a route on a map based on the search results. This enables a user to safely and efficiently search for a gas station or parking lot using voice instructions and receive intuitive route guidance even while driving.

[1680] "Natural dialogue voice instructions" refers to voice input given to the system by a user using everyday language.

[1681] "Means for accepting voice instructions" refers to devices or software that have the function of recognizing voice input from a user and incorporating it into the system.

[1682] "Means for searching for the closest, least expensive gas station from a user's current location" means a device or software capable of identifying the closest, least expensive gas station based on a user's current location.

[1683] "Means for searching for the closest, most available, and least expensive parking space from the current location" refers to a device or software that has the functionality to identify the closest, most available, and least expensive parking space based on the user's current location.

[1684] "Means for providing route guidance to the searched gas station or parking lot" refers to devices or software that have the function of guiding the user to their destination based on the search results.

[1685] "Means for converting voice instructions into text data" refers to a device or software that has the function of converting a user's voice input into text information.

[1686] "Means for displaying a route on a map based on the search results" refers to a device or software having a function for visually displaying a route to a searched destination on a map.

[1687] The following system configuration is used as an embodiment of the present invention. The system includes a voice recognition module, a search module, and a navigation module. These modules work together to accept voice instructions from a user, search for information on gas stations and parking lots, and provide route guidance.

[1688] Speech Recognition Module

[1689] The speech recognition module accepts the user's voice instructions and converts them into text data. Specifically, it uses a microphone to capture the user's voice and converts the voice into text using the SpeechRecognition library. This allows the user to give instructions to the system in a natural dialogue format.

[1690] Search Module

[1691] The search module searches for the nearest and cheapest gas station or the cheapest parking space that is currently available based on the text data received from the voice recognition module. Specifically, it uses the Google Places API to perform searches based on the user's current location. The search results are selected based on the most convenient results for the user.

[1692] Navigation Module

[1693] The navigation module provides route guidance to the destination based on the results obtained from the search module. Specifically, it uses the Folium library to display the route based on the search results on a map. This allows the user to visually check the route to the destination.

[1694] Hardware and Software Used

[1695] Hardware: Microphone, GPS module

[1696] Software: SpeechRecognition library, requests library, Google Places API, Folium library

[1697] Examples

[1698] When a user says, "Tell me where the nearest gas station is," the system searches for the nearest gas station from the user's current location and displays the route. The following is an example of a prompt:

[1699] Examples of prompt statements

[1700] User: "What's the closest gas station?"

[1701] System: "Voice commands please."

[1702] User: "What's the closest gas station?"

[1703] System: "Recognized text: Where is the nearest gas station?"

[1704] System: "Searching for gas stations..."

[1705] System: "Route to gas station x has been generated."

[1706] In this way, users can safely and efficiently search for gas stations and parking lots and receive intuitive route guidance while driving.

[1707] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1708] Step 1:

[1709] The user issues a voice command.

[1710] The user issues a voice command into the microphone, such as "Tell me where the nearest gas station is." The input is the user's voice, and the output is voice data.

[1711] Step 2:

[1712] The terminal captures the voice data and sends it to the voice recognition module.

[1713] The device captures the user's voice using a microphone and converts the voice data into text data using the SpeechRecognition library. The input is voice data and the output is text data.

[1714] Step 3:

[1715] The server receives the text data and sends it to the search module.

[1716] The server analyzes the text data received from the speech recognition module to understand the user's intent. The input is the text data, and the output is the search query.

[1717] Step 4:

[1718] Based on the search query, the server searches for the closest, cheapest gas stations and parking lots from the current location.

[1719] The server uses the Google Places API to search for gas stations and parking lots based on the user's current location. The input is the search query and the current location data, and the output is the search results.

[1720] Step 5:

[1721] The server sends the search results to the navigation module.

[1722] The server analyzes the search results and selects the most suitable gas station or parking lot. The input is the search results, and the output is the selected destination data.

[1723] Step 6:

[1724] The terminal receives the selected destination data and provides route guidance.

[1725] The device uses the Folium library to display routes to selected destinations on a map. The input is destination data, and the output is a map with route directions.

[1726] Step 7:

[1727] The user follows the route guidance to the destination.

[1728] The user follows the route guidance displayed on the terminal to head to a gas station or parking lot. The input is a map of the route guidance, and the output is reaching the destination.

[1729] Example 2

[1730] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1731] In conventional systems, when a user gives voice instructions, it is difficult to accurately understand the instructions and provide appropriate information. In addition, route guidance based on search results for gas stations and parking lots is insufficient, resulting in low user convenience. Furthermore, the accuracy and speed of the process of converting voice instructions into text data are also issues. There is a demand for a system that can solve these problems and provide users with quick and accurate information and route guidance.

[1732] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1733] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for converting the voice instructions into text data, a means for searching for the cheapest gas station closest to the current location, a means for transmitting the search results to the navigation module, and a means for providing route guidance to the searched gas station. This makes it possible to accurately understand voice instructions given by the user and provide appropriate information promptly. In addition, by providing optimal route guidance based on the search results, it is possible to improve user convenience.

[1734] "Natural dialogue voice instructions" refers to voice input by the user in everyday language, with the goal of the system understanding the speech and responding appropriately.

[1735] "Means for converting voice instructions into text data" refers to technology that analyzes a user's voice input and converts it into a corresponding text format.

[1736] "Current location" refers to where a user is currently located, typically determined based on location information such as GPS data.

[1737] "Method of finding nearest cheapest gas stations" refers to a search algorithm or database used to identify the nearest cheapest gas stations based on a user's current location.

[1738] "Means for transmitting search results to a navigation module" refers to a technique for transmitting information on the searched gas stations to a navigation system and generating route guidance.

[1739] "Means of route guidance" refers to technology for providing users with the optimal route to their destination, and includes map displays and voice guidance.

[1740] This invention is a system that searches for the nearest inexpensive gas station from the user's current location in response to voice instructions, and provides route guidance based on the search results. A specific embodiment of this system will be described below.

[1741] Receiving and converting voice commands

[1742] The user issues a voice command such as "Tell me where the cheapest gas station is nearby." A voice recognition module (e.g., a voice recognition API) installed in the device converts this voice command into text data. The voice recognition module analyzes the voice signal and generates the corresponding text.

[1743] Gas station search

[1744] The server receives the text data and searches for the nearest and cheapest gas station from the user's current location using a search module (e.g., map API). The search module obtains the user's current location information from GPS data and gas station price information from a database.

[1745] Providing navigation

[1746] The server sends the search results to a navigation module (e.g., navigation API) to generate optimal route guidance. The device provides this route guidance to the user. Specifically, the map and route are displayed on the smartphone screen, and voice guidance is also provided.

[1747] Examples

[1748] For example, if a user is near Tokyo Station, they may give a voice command such as, "Tell me the nearest cheap gas station." The voice recognition module converts this command into text data, and the server uses a map API to search for gas stations near Tokyo Station. The search results may reveal that "Gas Station A" is the closest and cheapest. The server uses the navigation API to generate the optimal route to "Gas Station A," and the device provides route guidance to the user. Specifically, a map and route to "Gas Station A" are displayed on the smartphone screen, and voice guidance such as, "Turn right 500 meters ahead" is given.

[1749] Examples of prompt statements

[1750] "If a user says, 'Tell me where the cheapest gas station is nearby,' how would you proceed?"

[1751] This invention allows users to quickly find the nearest, cheapest gas station by simply giving voice instructions, and furthermore, to receive optimal route guidance to that location, which greatly improves user convenience.

[1752] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1753] Step 1:

[1754] The user gives a voice command.

[1755] A user issues a voice command such as, "Tell me the nearest cheap gas station." The input is the user's voice, and the output is a voice signal.

[1756] Step 2:

[1757] The device converts the voice into text data.

[1758] A voice recognition module (e.g., a voice recognition API) installed in the device receives the user's voice instructions and converts them into text data. Specifically, it analyzes the voice signal and generates the corresponding text. The input is the voice signal, and the output is the text data.

[1759] Step 3:

[1760] The server receives the text data.

[1761] The terminal generates text data and sends it to the server. The server receives the text data and proceeds to the next processing step. The input is the text data, and the output is a receipt confirmation.

[1762] Step 4:

[1763] The server searches for gas stations.

[1764] The server analyzes the text data and obtains the user's current location information. It then uses a search module (e.g., map API) to search for the nearest, least expensive gas station from the user's current location. Specifically, it uses GPS data to identify the current location and obtains gas station price information from a database. The input is the text data and GPS data, and the output is the search results.

[1765] Step 5:

[1766] The server sends the search results to the navigation module.

[1767] The server sends the search results to a navigation module (e.g., navigation API) and generates optimal route guidance. Specifically, it calculates the optimal route from the user's current location based on the location information of gas stations obtained as search results. The input is the search results, and the output is route guidance data.

[1768] Step 6:

[1769] The terminal provides route guidance to the user.

[1770] The terminal provides the route guidance received from the navigation module to the user. Specifically, the map and route are displayed on the smartphone screen, and voice guidance is also provided. For example, specific instructions such as "Turn right 500 meters ahead" are provided by voice. The input is route guidance data, and the output is route guidance to the user.

[1771] (Application example 2)

[1772] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1773] Conventional navigation systems have difficulty efficiently searching for the nearest and cheapest gas stations and parking lots and providing real-time updated information, even when users give voice instructions. In addition, they lack sufficient coordination with autonomous vehicles and are unable to provide optimal route guidance. This can lead to users spending a lot of time and effort to reach their destination.

[1774] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1775] In this invention, the server includes a means for receiving voice instructions through natural dialogue, a means for searching for the nearest and cheapest gas station from the current location, a means for searching for the nearest and cheapest parking lot from the current location that is currently available, a means for providing route guidance to the searched gas station or parking lot, a means for updating price and location information in real time, a means for providing search results based on the voice instructions, and a means for providing optimal route guidance in cooperation with the navigation system of the autonomous vehicle. This allows the user to efficiently search for the nearest and cheapest gas station or parking lot through voice instructions, and to receive optimal route guidance based on information updated in real time.

[1776] "Voice instructions through natural dialogue" refers to voice input made by the user in natural language, similar to everyday conversation.

[1777] "Means for searching for the closest, least expensive gas station from a current location" refers to a method or device for identifying the closest, least expensive gas station based on a user's current location.

[1778] "Means for searching for the closest, currently available, and cheapest parking lot from the current location" refers to a method or device for identifying the closest, available, and cheapest parking lot based on the user's current location.

[1779] "Means for providing route guidance to a searched gas station or parking lot" refers to a method or device for providing a user with the optimal route to a gas station or parking lot that the user has searched for.

[1780] "Means for updating price and location information in real time" refers to a method or apparatus for obtaining and updating price and location information for gas stations and parking lots in real time.

[1781] "Means for providing search results based on voice instructions" refers to a method or device for generating and providing search results based on a user's voice instructions.

[1782] "Means for providing optimal route guidance in cooperation with the navigation system of an autonomous vehicle" refers to a method or device for providing optimal routes in cooperation with the navigation system of an autonomous vehicle.

[1783] The following system configuration will be described as an embodiment of the present invention.

[1784] System Configuration

[1785] The system includes a voice recognition module, a search module, a navigation module, a real-time update module, and a module that interfaces with the autonomous vehicle's navigation system.

[1786] Speech Recognition Module

[1787] The voice recognition module converts the voice into text data when the user gives a voice command through natural dialogue. Specifically, it inputs the voice using a microphone and converts the voice into text using the speech_recognition library.

[1788] Search Module

[1789] The search module searches for the nearest and cheapest gas station and the cheapest parking lot that is currently available based on the user's current location. To do this, it uses the GPS module to get the current location and the requests library to get information about gas stations and parking lots from external APIs.

[1790] Navigation Module

[1791] The navigation module provides optimal route guidance based on the search results. It uses the navigation API to calculate the shortest route from the user's current location to the destination and provides guidance.

[1792] Real-time update module

[1793] The real-time update module updates prices and location information for gas stations and parking lots in real time, so users can always get the most up-to-date search results.

[1794] Linking with autonomous vehicle navigation systems

[1795] The module, which works with the autonomous vehicle's navigation system, sends the search results to the autonomous vehicle's navigation system and provides optimal route guidance, allowing users to reach their destination efficiently through voice instructions.

[1796] Examples

[1797] For example, when a user gives a voice command such as "Tell me the nearest cheapest gas station," the voice recognition module converts the command into text data. The search module then searches for the nearest cheapest gas station from the current location. The search results are sent to the navigation module, which provides the optimal route guidance for the user. The real-time update module keeps the gas station prices and location information always up to date. By linking with the autonomous vehicle's navigation system, users can use the autonomous vehicle to reach their destination efficiently.

[1798] Examples of prompt statements

[1799] When a user gives a voice command such as "Tell me the nearest cheap gas station," the voice recognition module converts this command into text data, and the search module searches for the nearest cheap gas station from the current location. The search results are sent to the navigation module, and the system is designed to provide the optimal route guidance for the user.

[1800] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1801] Step 1:

[1802] The user gives a voice command.

[1803] The user gives voice instructions in a natural dialogue, such as "Tell me where the nearest cheap gas station is." The input is the user's voice, and the output is voice data.

[1804] Step 2:

[1805] The terminal converts the voice data into text data.

[1806] The speech recognition module uses a microphone to capture voice data and converts the voice to text using the speech_recognition library. The input is the voice data and the output is the text data.

[1807] Step 3:

[1808] The server gets the current location.

[1809] Use the device's GPS module to obtain the user's current location. The input is the GPS signal, and the output is the coordinate data of the current location.

[1810] Step 4:

[1811] The server obtains gas station information.

[1812] Use the requests library to get gas station location and price information from an external API. The input is the API request, and the output is data including gas station location and price information.

[1813] Step 5:

[1814] The server searches for the nearest cheapest gas station.

[1815] Based on the acquired gas station information, the distance from the current location and the price are calculated to identify the nearest and cheapest gas station. The input is the gas station location and price information, and the current location coordinate data, and the output is the optimal gas station information.

[1816] Step 6:

[1817] The server generates the navigation information.

[1818] Use the navigation API to generate route guidance from the current location to the best gas station. The input is the coordinate data of the current location and the location information of the best gas station, and the output is the route guidance information.

[1819] Step 7:

[1820] The terminal provides route guidance to the user.

[1821] The navigation module guides the user on the optimal route by voice or visually based on the generated route guidance information. The input is the route guidance information, and the output is navigation instructions to the user.

[1822] Step 8:

[1823] The server updates the information in real time.

[1824] The real-time update module periodically retrieves gas station prices and location information from an external API and updates the data in the system. The input is the API request and the output is the updated gas station information.

[1825] Step 9:

[1826] The server works in conjunction with the autonomous vehicle's navigation system.

[1827] The search results and route guidance information are sent to the navigation system of the autonomous vehicle, and the system controls the vehicle so that it automatically drives along the optimal route. The input is route guidance information, and the output is control instructions for the autonomous vehicle.

[1828] In addition, an emotion engine that estimates the emotion of the user may be further combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[1829] "Example 1"

[1830] As an embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotion of a user. The system includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station from the current location, a means for searching for the cheapest parking lot that is currently available from the current location, and a means for providing route guidance to the searched gas station or parking lot. The system further includes an emotion engine that recognizes the emotion of the user. The emotion engine analyzes the emotion from the user's voice and provides the result to other parts of the system. For example, if the user feels that he or she is in a hurry, the emotion engine provides the information to the search means, and the search means searches for the nearest gas station or parking lot with priority. Also, if the user feels that he or she is relaxed, the emotion engine provides the information to the route guidance means, and the route guidance means can provide services according to the user's emotion, such as suggesting a route with a better view.

[1831] "Example 2"

[1832] As an embodiment of the present invention, a system is provided that combines an emotion engine that recognizes the emotion of a user. The system includes a means for accepting voice instructions through natural dialogue, a means for searching for the cheapest gas station from the current location, a means for searching for the cheapest parking lot that is currently available from the current location, and a means for providing route guidance to the searched gas station or parking lot. The system further includes an emotion engine that recognizes the emotion of the user. The emotion engine analyzes the emotion from the user's voice and provides the result to other parts of the system. For example, if the user feels that he or she is in a hurry, the emotion engine provides the information to the search means, and the search means searches for the nearest gas station or parking lot with priority. Also, if the user feels that he or she is relaxed, the emotion engine provides the information to the route guidance means, and the route guidance means can provide services according to the user's emotion, such as suggesting a route with a better view.

[1833] The process flow of each embodiment will be described below.

[1834] "Example 1"

[1835] Step 1: The user issues a voice command to the system, such as "Find the nearest cheap gas station."

[1836] Step 2: The system's voice recognition module converts the voice instructions into text data, while the emotion engine analyzes the emotions from the user's voice.

[1837] Step 3: Based on the analysis result of the sentiment engine, the system's search module searches for the nearest cheap gas station. For example, if the user feels that he is in a hurry, it will prioritize searching for the nearest gas station.

[1838] Step 4: Based on the search results, the system's navigation module will provide route guidance to the searched gas station. For example, if the user feels relaxed, it will suggest a more scenic route.

[1839] Example 1

[1840] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1841] Conventional navigation systems can accept voice instructions from users, but they are unable to provide search results or route guidance that take the user's emotions into account. This means that there is an issue with them being unable to provide optimal guidance according to the situation, such as when the user is in a hurry or relaxed.

[1842] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1843] In this invention, the server includes a means for accepting voice instructions through natural dialogue, a means for converting the voice instructions into text data, a means for searching for the cheapest fuel supply facility closest to the current location, a means for searching for the cheapest parking facility closest to the current location that is currently available, a means for analyzing the user's emotions, a means for providing route guidance to the searched fuel supply facility or parking facility, and a means for adjusting the search results and route guidance based on the user's emotions, thereby enabling optimal search results and route guidance according to the user's emotions and circumstances.

[1844] "Natural dialogue voice instruction" refers to voice input by the user in everyday language, and is voice instruction that does not rely on specific commands or keywords.

[1845] "Means for converting voice instructions into text data" refers to a function that uses voice recognition technology to convert a user's voice input into text information.

[1846] A "fuel distribution facility" refers to a facility that supplies fuel or energy to vehicles, such as a gas station or an electric vehicle charging station.

[1847] A "parking facility" is a place for temporarily parking a vehicle, and refers to facilities including parking lots and parking meters.

[1848] "Means for analyzing user emotions" refers to technologies used to determine a user's emotional state from voice or other input data.

[1849] "Means for adjusting search results and route guidance" refers to the ability to change the content of search results and route guidance provided depending on the user's emotions and situation.

[1850] The present invention is a system that receives voice instructions from a user, analyzes emotions, searches for optimal fuel supply facilities and parking facilities, and provides appropriate route guidance. A specific embodiment of this system will be described below.

[1851] Hardware and Software Used

[1852] 1. Speech Recognition Module: Uses speech recognition technology to convert voice instructions into text data. This module uses a common speech recognition API.

[1853] Example: Speech Recognition API

[1854] 2. Search module: Uses the map API to search for the nearest cheapest fuel and parking facilities from your current location.

[1855] Example: Map Information API

[1856] 3. Navigation module: Uses the navigation API to provide route guidance to the searched facilities.

[1857] Example: Navigation API

[1858] 4. Emotion Engine: Uses emotion analysis technology to analyze emotions from the user's voice.

[1859] Example: Sentiment analysis API

[1860] Specific operation of the system

[1861] 1. Accepting user voice commands

[1862] The user issues a voice command in the car saying, "Tell me the nearest cheap gas station."

[1863] The device's microphone captures the user's voice and transmits it to the system as voice data.

[1864] 2. Voice-to-text conversion

[1865] The terminal transmits the voice data to a voice recognition module.

[1866] The speech recognition module converts voice data into text data using a speech recognition API. For example, a voice saying "Tell me where the nearest cheap gas station is" is converted into text saying "Tell me whe...

Claims

1. means for accepting voice instructions from a user; A means of obtaining the user's current location; means for searching for at least one of a fuel supply facility or an available parking facility; A means for acquiring a route from the current location to the searched fuel supply facility or the searched parking facility; means for determining an emotional state of a user using an emotion engine; means for generating a prompt sentence instructing the generation of a modified route based on the identified emotion of the user and the route; means for generating a modified route using the generated prompt sentences and the generative AI model; and means for presenting route guidance information to the user along the generated revised route.

2. The system of claim 1 , wherein the means for determining an emotional state of the user determines the emotional state of the user by analyzing received voice instructions of the user.

3. 3. The system of claim 2, wherein the means for generating a modified route generates a route to the nearest fuel facility or parking facility if the user is in a hurry and generates a scenic route if the user is relaxed.

Images