System

Abstract

To provide a system capable of searching for and setting a destination which reflect on the detailed request of a user and further unitarily performing up to reservation.SOLUTION: A system comprises: means for converting a voice input from a user into text information using a voice recognition technology; means for analyzing the text information using a natural language processing technology to grasp an instruction content of the user; means for analyzing the voice input from the user with an emotion engine to recognize an emotional state of the user; means for recommending a destination based on the recognized emotional state of the user and the instruction content of the user; means for searching for a destination based on a request from the user whose destination is recommended; means for setting a destination based on a search result; and means for providing navigation to the set destination.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional navigation systems have difficulty in searching and setting destinations that meet the user's detailed requests, and lack the functionality to make reservations based on the user's requests. [Means for solving the problem]

[0005] The present invention provides a system including a means for learning information from various information sources, a means for searching for a destination based on a user's detailed requests, a means for setting a destination based on the search results, and a means for providing navigation to the set destination, which makes it possible to centrally search for and set a destination according to a user's detailed requests, and even make a reservation. [Brief description of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16]FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 11 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0023] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] One embodiment of the present invention is an in-vehicle navigation system. This system uses a machine learning algorithm as a means for learning information from various information sources. Specifically, information is acquired from a database including restaurant rating information and information on the availability of parking spaces, and this information is learned. In addition, natural language processing technology is used as a means for searching for a destination based on the detailed requests of the user. Questions from the user are analyzed, and an appropriate destination is searched for based on the analysis. Furthermore, GPS technology is used as a means for setting the destination based on the search results. The optimal route is calculated based on the user's current location and the search results, and the destination is set. Finally, a voice guidance system is used as a means for providing navigation to the set destination. This allows the user to be safely guided to the destination even while driving.

[0029] "Example 2"

[0030] Another embodiment of the present invention is a smartphone application. This application has the same functions as the in-car navigation system described above, but also includes a function for making reservations based on a user's request. Specifically, when a user requests "make a reservation by phone," the system automatically calls the corresponding restaurant and makes the reservation. This function is realized using voice recognition technology and an automatic dialing system.

[0031] The process flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: The system learns from various sources, such as restaurant ratings and parking availability, and uses machine learning algorithms to learn from them.

[0034] Step 2: Once the user has entered their detailed request, the system analyzes it using natural language processing technology.

[0035] Step 3: Based on the analysis results, the system searches for suitable destinations.

[0036] Step 4: Based on the search results and the user's current location, the system will use GPS technology to calculate the optimal route and set the destination.

[0037] Step 5: To provide navigation to the set destination, the system guides the user using the voice guidance system.

[0038] "Example 2"

[0039] Step 1: The smartphone application has the same functions as the in-car navigation system mentioned above.

[0040] Step 2: If the user requests to "make a reservation by phone," the system will automatically call the relevant restaurant.

[0041] Step 3: The system uses voice recognition technology and an automated dialing system to make the reservation.

[0042] Example 1

[0043] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0044] Conventional navigation systems have difficulty in searching for a destination based on the user's detailed requests, and are unable to generate appropriate answers to the user's questions. Furthermore, they are unable to effectively use the user's current location or search results when setting a destination or providing navigation. This reduces user convenience and makes it difficult to ensure safety while driving.

[0045] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0046] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for converting voice input into text, a means for analyzing the user's questions using natural language processing techniques, a means for learning data using machine learning algorithms, a means for acquiring location information, and a means for generating voice guidance. This makes it possible to search for and set a destination based on the user's detailed requests, and to be safely guided to the destination even while driving.

[0047] "Source" refers to any medium or system that provides data.

[0048] "Learning means" refers to the algorithms and techniques used to analyze data and extract patterns and knowledge.

[0049] "User specific requests" refers to the detailed requirements that users have based on their specific conditions or preferences.

[0050] "Destination search means" refers to the technologies and algorithms used to search for appropriate destinations based on the user's requirements.

[0051] "Search Results" refers to destination candidates and information obtained through a search.

[0052] "Means for setting a destination" refers to the technology or algorithm used to determine an optimal destination based on the search results.

[0053] "Means for providing navigation" refers to technologies and systems that provide route guidance to help users reach their destination.

[0054] "Means of converting voice input into text" refers to technology or systems that analyze the user's voice and convert it into text information.

[0055] "Natural language processing technology" refers to techniques and algorithms for analyzing and understanding human language.

[0056] A "machine learning algorithm" refers to an algorithm that learns patterns and knowledge from data and makes predictions and classifications.

[0057] "Means of obtaining location information" refers to technologies and systems for identifying a user's current location.

[0058] "Means for generating audio guidance" refers to technology or systems for providing audio instructions to a user.

[0059] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0060] This invention relates to an in-vehicle navigation system, which searches for a destination based on the detailed requests of the user and provides the optimal route. This system operates in cooperation with a server, a terminal, and a user.

[0061] 1. Program Generation

[0062] The server generates the program for the in-car navigation system. This program is built by combining machine learning algorithms and natural language processing technology. Specifically, it uses libraries and APIs such as Python's Pandas, Scikit-learn, NLTK, SpaCy, Google (registered trademark) Maps API, and Amazon Polly.

[0063] 2. Program Processing

[0064] The server processes the data in the following steps:

[0065] 1. Data acquisition and training:

[0066] The server retrieves data from a database that contains information such as restaurant ratings and whether or not parking is available. Specifically, the server reads the data using Python's Pandas library. The server then uses Scikit-learn to train the data with machine learning algorithms. For example, the server creates a model to predict which restaurants have high ratings based on restaurant ratings.

[0067] 2. User Question Analysis:

[0068] The device receives questions from users. For example, it receives a question like, "Tell me about a highly rated restaurant nearby." The device analyzes the question using natural language processing technology. Specifically, it uses Python's NLTK and SpaCy to understand the intent of the question.

[0069] 3. Set your destination:

[0070] The server calculates the optimal route based on the user's current location and the search results. Specifically, it uses the Google Maps API to obtain the user's current location and the locations of highly rated restaurants, and calculates the optimal route.

[0071] 4. Providing navigation:

[0072] The device provides navigation to the set destination. Specifically, it uses a voice guidance system to give voice instructions to the user. For example, it uses Amazon Polly to generate voice guidance such as "Turn right."

[0073] 3. Specific Examples

[0074] As a concrete example, consider the case where a user asks, "Can you tell me about some highly rated restaurants nearby?"

[0075] 1. Data acquisition and training:

[0076] The server connects to the database and obtains restaurant rating information.

[0077] The server converts the data into a DataFrame using Pandas.

[0078] The server uses Scikit-learn's RandomForestClassifier to train a model to predict highly rated restaurants.

[0079] 2. User Question Analysis:

[0080] The user speaks to the device saying, "Tell me about some highly rated restaurants nearby."

[0081] The device converts the voice input into text.

[0082] The device uses NLTK to tokenize the text and SpaCy to analyze the intent of the question.

[0083] 3. Set your destination:

[0084] The device obtains the user's current location using the GPS sensor.

[0085] The server uses the Google Maps API to calculate a route based on the user's current location and the locations of highly rated restaurants.

[0086] The server transmits the calculation result to the terminal.

[0087] 4. Providing navigation:

[0088] The terminal starts navigation based on the route information received from the server.

[0089] The device uses Amazon Polly to generate voice guidance and provide instructions to the user.

[0090] The user drives by following the voice guidance.

[0091] Example prompt:

[0092] "Tell me about some highly rated restaurants nearby."

[0093] In this way, users are safely guided to their destination while driving.

[0094] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0095] Step 1: Data acquisition and training

[0096] The server retrieves information from the database, such as restaurant ratings and whether parking is available. As input, it uses the database connection information and a SQL query. The server converts the data into a DataFrame using Python's Pandas library. The server then trains the data with a machine learning algorithm using Scikit-learn. Specifically, it uses RandomForestClassifier to create a model that predicts highly rated restaurants. As output, it obtains the trained model.

[0097] Specific behavior:

[0098] The server connects to the database and executes SQL queries to retrieve the data.

[0099] The server converts the acquired data into a Pandas DataFrame.

[0100] The server uses Scikit-learn's RandomForestClassifier to train a model to predict restaurant ratings.

[0101] Step 2: Analyze user questions

[0102] The terminal receives a question from the user. As input, the user's voice question is used. The terminal uses speech recognition technology to convert the voice input into text. The terminal then analyzes the question using natural language processing technology. Specifically, it uses Python's NLTK and SpaCy to understand the intent of the question. As output, the analyzed text data is obtained.

[0103] Specific behavior:

[0104] The user speaks to the device saying, "Tell me about some highly rated restaurants nearby."

[0105] The device converts the voice input into text.

[0106] The device uses NLTK to tokenize the text and SpaCy to analyze the intent of the question.

[0107] Step 3: Set your destination

[0108] The server calculates the optimal route based on the user's current location and the search results. The user's current location and the parsed text data are used as input. The server uses the Google Maps API to obtain the user's current location and the locations of highly rated restaurants, and calculates the optimal route. The calculated route information is obtained as output.

[0109] Specific behavior:

[0110] The device obtains the user's current location using the GPS sensor.

[0111] The server uses the Google Maps API to calculate a route based on the user's current location and the locations of highly rated restaurants.

[0112] The server transmits the calculation result to the terminal.

[0113] Step 4: Providing navigation

[0114] The device provides navigation to the set destination. The calculated route information is used as input. The device uses a voice guidance system to give voice instructions to the user. Specifically, it uses Amazon Polly to generate voice guidance such as "Turn right". The voice guidance is obtained as output.

[0115] Specific behavior:

[0116] The terminal starts navigation based on the route information received from the server.

[0117] The device uses Amazon Polly to generate voice guidance and provide instructions to the user.

[0118] The user drives by following the voice guidance.

[0119] (Application example 1)

[0120] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0121] Conventional navigation systems have limited capabilities to search for destinations based on the user's detailed requests and lack the ability to generate appropriate answers to user questions. They also lack the ability to calculate the optimal route between the current location and the destination and provide voice guidance. This often causes inconvenience to users while traveling to their destination.

[0122] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a natural language processing means for analyzing the user's question and generating an appropriate answer, a means for calculating the optimal route between the current position and the destination, and a means for providing voice guidance. This makes it possible to search for a destination based on the user's detailed requests, calculate the optimal route, and provide voice guidance.

[0123] A "source" is a source or resource that provides data or knowledge.

[0124] A "means of learning information" is a method that uses machine learning algorithms to analyze data obtained from various sources and extract patterns and knowledge.

[0125] "User specific requests" are the detailed requirements or wishes of a user based on their specific conditions or preferences.

[0126] A "means for searching for a destination" is a method or technology for searching for an appropriate destination based on a user's request.

[0127] "Search results" are the results of a search obtained by a means for searching for a destination.

[0128] A "means for setting a destination" is a method or technology for determining a user's destination based on search results.

[0129] A "means for providing navigation" is a method or technology that provides route guidance or directions to a user to reach a destination.

[0130] "Natural language processing means" is a technology for analyzing a user's questions or requests and generating appropriate answers.

[0131] A "means for calculating a route" is a method or technique for calculating an optimal path between a current location and a destination.

[0132] "Means for providing audio guidance" means the methods or techniques for converting the calculated route information into audio and providing it to the user.

[0133] As an embodiment for carrying out the present invention, the following system is considered.

[0134] First, the server has a means of learning information from various sources. This means uses machine learning algorithms to analyze data such as restaurant reviews and the availability of parking spaces, and extract patterns and knowledge. Specifically, it obtains information from a database and learns from it.

[0135] Next, there is a means to search for a destination based on the user's detailed requests. This means uses natural language processing technology to analyze the user's question and search for an appropriate destination. For example, if a user asks, "What are some highly rated restaurants nearby?", the system will analyze the question and search for an appropriate restaurant.

[0136] In addition, there is a method to set a destination based on the search results. This method calculates the optimal route based on the user's current location and the search results, and sets the destination. Specifically, it uses the Geopy library to calculate the distance between the current location and the destination.

[0137] Finally, there is a method to provide navigation to a set destination, which uses the Google Text-to-Speech (gTTS) library to convert the calculated route information into audio and provide it to the user, allowing the user to be safely guided to their destination while driving.

[0138] As a concrete example, when a user asks, "What are some highly rated restaurants nearby?" the system works as follows:

[0139] 1. Natural language processing means analyzes the user's question and generates an appropriate answer.

[0140] 2. The means for learning information retrieves restaurant rating information from the database and analyzes it.

[0141] 3. A route calculation means calculates the optimal route between the current location and the destination.

[0142] 4. A means for providing audio guidance converts the calculated route information into audio and provides it to the user.

[0143] An example of a prompt sentence might be:

[0144] User asks: "What are some highly rated restaurants near me?"

[0145] Context: "Restaurant ratings, availability of parking, etc."

[0146] This prompt can then be used to have a generative AI model parse the question and provide an appropriate answer.

[0147] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0148] Step 1:

[0149] The server receives a question from a user. The input is the user's question, e.g., "What are some highly rated restaurants nearby?", as text data. The output is text data ready for analysis.

[0150] Step 2:

[0151] The server analyzes the user's question using natural language processing means. The input is the user's question received in step 1, and the output is an appropriate search query as the analysis result. Specifically, a generative AI model is used to analyze the question and generate an appropriate answer.

[0152] Step 3:

[0153] The server uses a means of learning information to obtain data such as restaurant rating information and availability of parking spaces from the database. The input is the search query generated in step 2, and the output is the obtained rating information, availability of parking spaces, and other data. Specifically, the server executes a query against the database to obtain the required information.

[0154] Step 4:

[0155] The server calculates the optimal route between the user's current location and the destination based on the acquired information. The input is the user's current location and the destination information acquired in step 3, and the output is the calculated optimal route information. Specifically, the Geopy library is used to calculate the distance and determine the optimal route.

[0156] Step 5:

[0157] The server converts the calculated route information into voice guidance. The input is the route information calculated in step 4, and the output is an audio file. Specifically, it uses the Google Text-to-Speech (gTTS) library to convert the text information into speech.

[0158] Step 6:

[0159] The terminal provides the generated voice guidance to the user. The input is the voice file generated in step 5, and the output is voice guidance for the user. Specifically, the terminal plays the voice file and provides navigation information to the user.

[0160] Example 2

[0161] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0162] Conventional navigation systems only provide destination setting and navigation functions, and do not have a reservation function based on the user's detailed requests, so users had to make reservations manually. In addition, there was no system that analyzed voice instructions and made reservations automatically, which made it difficult for users to use the system efficiently. This meant that users had to take the time to set the destination and make reservations separately, making it difficult to use the system efficiently.

[0163] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0164] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for analyzing the user's voice instructions, a means for extracting reservation details based on the analyzed voice instructions, a means for automatically calling the establishment for which a reservation is to be made based on the extracted reservation details, and a means for audibly conveying the reservation details over the phone to an employee of the establishment for which a reservation is to be made. This allows the user to automatically set a destination and make a reservation simply by issuing voice instructions, greatly improving convenience.

[0165] "Source" refers to any source that provides data or knowledge.

[0166] "User" refers to any individual or entity that uses the System.

[0167] "Destination" refers to the place the user wants to reach.

[0168] "Search" refers to the act of finding specific information.

[0169] "Navigation" refers to the function of providing route guidance to a destination.

[0170] "Voice instruction" refers to an instruction given by a user to the system using voice.

[0171] "Analysis" refers to the act of examining data or information in detail to understand its meaning and structure.

[0172] "Reservation details" refers to the information necessary for making a reservation (date and time, number of people, location, etc.).

[0173] "Automatic" refers to a machine or system that operates independently, without human intervention.

[0174] "Making a call" refers to the act of using a communication device to attempt to connect to another communication device.

[0175] "Communicating by voice" refers to the act of conveying information to others using voice.

[0176] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0177] The present invention is implemented as an application for a smartphone. This application is a system that automatically sets a destination and makes a reservation based on a user's voice instruction. A specific embodiment of this system will be described below.

[0178] Hardware and Software Used

[0179] The system uses the following hardware and software:

[0180] Smartphone (device)

[0181] Speech recognition technology (Google Cloud Speech-to-Text API)

[0182] Database Management System (Firebase Firestore)

[0183] Automated dialing system (Twilio API)

[0184] Speech synthesis technology (Google Cloud Text-to-Speech API)

[0185] Data processing and calculation

[0186] The user launches the smartphone application and says, "Make a reservation for four people tomorrow night at 7 p.m." The device uses the Google Cloud Speech-to-Text API to convert the user's voice instructions into text. This conversion process occurs in real time.

[0187] Next, the device analyzes the converted text and extracts the reservation details (date, time, number of people). For example, from the text "Make a reservation for four people at 7 p.m. tomorrow," the information "tomorrow at 7 p.m." and "four people" is extracted.

[0188] The device sends a request to the server to retrieve restaurant information stored in Firebase Firestore. This request includes the user's location and preferred restaurant conditions. The server receives the request, retrieves the corresponding restaurant information from Firebase Firestore, and returns it to the device. The returned information includes the restaurant's name, phone number, address, etc.

[0189] The device uses the Twilio API to automatically call the restaurant's phone number, without any user intervention. Once the call is connected, the device uses the Google Cloud Text-to-Speech API to voice the reservation details it extracted in advance to the restaurant employee. For example, it might say, "Please make a reservation for four people tomorrow night at 7 p.m."

[0190] Examples

[0191] As a concrete example, the following will show the behavior when the user instructs "Make a reservation for four people at 7pm tomorrow night."

[0192] 1. The user launches the smartphone application and gives a voice command such as, "Make a reservation for four people at 7pm tomorrow night."

[0193] 2. The device uses the Google Cloud Speech-to-Text API to convert the user's voice commands into text.

[0194] 3. The device analyzes the converted text and extracts the reservation details: "tomorrow night at 7pm" and "4 people."

[0195] 4. The device sends a request to the server to get restaurant information stored in Firebase Firestore.

[0196] 5. The server receives the request, retrieves the relevant restaurant information from Firebase Firestore, and returns it to the device.

[0197] 6. The device uses the Twilio API to automatically call the restaurant's phone number.

[0198] 7. Once the call is connected, the device uses the Google Cloud Text-to-Speech API to communicate to the restaurant, "Please make a reservation for four people tomorrow night at 7pm."

[0199] Examples of prompt statements

[0200] An example of a prompt sentence might be:

[0201] "If a user says, 'Make a reservation for 4 people tomorrow night at 7pm,' how should the application handle that?"

[0202] By inputting this prompt sentence into the generative AI model, an explanation of the processing based on the specific example above can be obtained.

[0203] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0204] Step 1:

[0205] The user issues a voice command.

[0206] The user starts the smartphone application and gives a voice command such as, "Make a reservation for four people at 7 p.m. tomorrow." This voice command becomes the input.

[0207] Step 2:

[0208] The device converts the speech to text.

[0209] The device uses voice recognition technology to convert the user's voice instructions into text. This process is performed using a voice recognition API. The input is the user's voice instructions, and the output is the instructions in text format.

[0210] Step 3:

[0211] The terminal analyzes the text and extracts the reservation details.

[0212] The terminal analyzes the converted text and extracts the reservation details (date, time, number of people). For example, from the text "Make a reservation for four people at 7 p.m. tomorrow," the information "tomorrow at 7 p.m." and "four people" is extracted. The input is instructions in text format, and the output is the reservation details data.

[0213] Step 4:

[0214] The terminal requests restaurant information from the server.

[0215] The terminal sends a request to the server to get restaurant information stored in the database management system. This request includes the user's location information and the conditions of the preferred restaurant. The input is the reservation details and the user's location information, and the output is a request for restaurant information.

[0216] Step 5:

[0217] The server returns the restaurant information.

[0218] The server receives the request, retrieves the relevant restaurant information from the database, and returns it to the terminal. The returned information includes the restaurant's name, phone number, address, etc. The input is a request for restaurant information, and the output is the restaurant information.

[0219] Step 6:

[0220] The terminal makes the call using an automatic dialing system.

[0221] The terminal then uses an automatic dialing system to automatically call the restaurant's phone number, without any user intervention: the input is the restaurant's phone number, and the output is the establishment of a telephone connection.

[0222] Step 7:

[0223] The terminal will announce the reservation details aloud.

[0224] When the phone is connected, the device uses voice synthesis technology to communicate the reservation details it extracted in advance to the restaurant by voice. For example, it may say, "Please make a reservation for four people tomorrow night at 7 p.m." The input is the reservation details data, and the output is a voice message.

[0225] (Application example 2)

[0226] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0227] Conventional navigation and reservation systems required users to operate them manually, which made them inconvenient. In addition, they lacked the functionality to utilize the user's past behavioral history to make optimal suggestions, making it difficult to respond to user requests quickly and accurately. Furthermore, there was a need to effectively combine voice recognition technology and an automatic dialing system to improve the user experience.

[0228] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0229] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for understanding the user's requests using voice recognition technology, a means for making reservations using an automatic dialing system, a means for learning the user's past behavior history and making optimal suggestions, and a means for providing voice feedback. This allows the user to search for and make reservations at a destination using only voice instructions, and to receive optimal suggestions based on the user's past behavior history.

[0230] "Source" refers to any source that provides data or knowledge.

[0231] "Means of learning" refers to the methods and techniques used to collect, analyze, and understand information.

[0232] "The user's detailed requests" refer to the specific conditions and wishes that the user has.

[0233] "Destination search means" refers to methods and technologies for searching for appropriate locations based on a user's requirements.

[0234] "Search Results" refers to the information or data obtained through a Search.

[0235] "Means for setting a destination" refers to a method or technology for determining a final destination based on a search result.

[0236] "Means for providing navigation" refers to methods and technologies for providing directions to a set destination.

[0237] "Voice recognition technology" refers to the technology that analyzes voice and converts it into text or commands.

[0238] "Automatic dialing system" refers to a system that automatically dials designated telephone numbers.

[0239] "Means for making reservations" refers to methods or technologies for automatically making reservations for specified locations or services.

[0240] "Past behavioral history" refers to the record of actions and choices a user has previously made.

[0241] "Means of making optimal suggestions" refers to methods and technologies for presenting the most suitable options to users based on their past behavioral history and current needs.

[0242] "Means for providing audio feedback" refers to methods or technologies that allow the system to communicate information or results to the user using audio.

[0243] A system for implementing the present invention has the following configuration: A server includes means for learning information from various information sources, means for searching for a destination based on a user's detailed requests, means for setting a destination based on the search results, means for providing navigation to the set destination, means for understanding a user's requests using voice recognition technology, means for making reservations using an automatic dialing system, means for learning a user's past behavior history and making optimal suggestions, and means for providing voice feedback.

[0244] Explanation of program processing

[0245] Hardware and Software

[0246] The system uses the smartphone's microphone and speaker as hardware, the speech_recognition library for voice recognition, the pyttsx3 library for voice feedback, and the requests library for data communication.

[0247] Data processing and calculation

[0248] 1. Speech recognition: When the user speaks their request into the smartphone's microphone, the speech_recognition library converts the speech into text.

[0249] 2. Request Analysis: Analyze the converted text to understand the user's request. For example, analyze the request "order a pizza."

[0250] 3. Execute the booking: Based on the request, send the order to the food delivery API using the requests library. If the order is successful, provide audio feedback using the pyttsx3 library.

[0251] 4. Learning from behavioral history: Learns the user's past behavioral history and makes optimal suggestions for future visits.

[0252] Examples

[0253] For example, if a user says "order a pizza," the system would do the following:

[0254] 1. Capture the user's voice with the microphone.

[0255] 2. Convert speech to text and get the text "Order pizza."

[0256] 3. Analyze the text to identify "pizza" orders.

[0257] 4. Send an order for "Pizza" to the food delivery API.

[0258] 5. If the order is successful, provide audio feedback saying "Your pizza order has been completed."

[0259] Examples of prompt statements

[0260] "When a user says, 'Order a pizza,' please create a program that converts the speech to text using speech recognition technology and sends the order to the food delivery API. If the order is successful, please provide feedback by voice saying, 'The pizza order has been completed.'"

[0261] In this way, users can search for and book destinations using only voice commands, and receive optimal suggestions based on their past travel history.

[0262] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0263] Step 1:

[0264] The user speaks their request into the smartphone's microphone.

[0265] Input: User's speech.

[0266] Output: Audio data captured by the microphone.

[0267] What happens: When a user says "order a pizza," the smartphone's microphone captures the audio.

[0268] Step 2:

[0269] The device converts the voice data into text using the speech_recognition library.

[0270] Input: Audio data captured by the microphone.

[0271] Output: Text data (e.g. "order a pizza").

[0272] What happens: The speech recognition engine analyzes the voice data and converts it into corresponding text.

[0273] Step 3:

[0274] The device analyzes the text data and understands the user's requests.

[0275] Input: Text data (e.g. "order a pizza").

[0276] Output: Parsed request (e.g. "order pizza").

[0277] What happens: The text analysis engine analyzes the text "order a pizza" and identifies the request "order a pizza."

[0278] Step 4:

[0279] The device uses the requests library to send an order to the food delivery API.

[0280] Input: Parsed request (e.g. "order pizza").

[0281] Output: The order request to the API and its response.

[0282] Specific behavior: Send a request to the food delivery API containing "order pizza" and receive a response indicating whether the order was successful.

[0283] Step 5:

[0284] The device provides audio feedback using the pyttsx3 library.

[0285] Input: Response from the API (e.g., order successful)

[0286] Output: Audio feedback (e.g. "Your pizza order is complete.")

[0287] What happens: The speech synthesis engine generates the message "Your pizza order has been completed" and relays it to the user through the smartphone speaker.

[0288] Step 6:

[0289] The server learns the user's past behavioral history and makes optimal suggestions for future visits.

[0290] Input: User behavioral history data.

[0291] Output: Proposed data based on the learning results.

[0292] What happens: The server analyzes the user's past ordering history and generates data to suggest the best restaurant and menu for the next time they order.

[0293] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0294] "Example 1"

[0295] One embodiment of the present invention is a navigation system incorporating an emotion engine. This system recognizes the user's emotions from the tone of voice, facial expressions, choice of words, etc., and searches for a destination taking these emotions into consideration. For example, if a user requests "find a nice restaurant nearby," the system will recommend lively restaurants if the user's tone of voice is cheerful, and quiet restaurants if the user's tone of voice is sad.

[0296] "Example 2"

[0297] Also, a smartphone application incorporating the emotion engine is one embodiment of the present invention. This application recognizes emotions from the user's text input or voice input and provides services taking them into consideration. For example, if the user inputs "I'm tired today. I want to eat something delicious," the application recognizes the emotion and recommends restaurants with a relaxing atmosphere and restaurants that serve the user's favorite dishes.

[0298] The process flow of each embodiment will be described below.

[0299] "Example 1"

[0300] Step 1: The user issues a verbal request to the navigation system.

[0301] Step 2: The emotion engine recognizes the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[0302] Step 3: Search for destinations taking into account the emotions recognized by the system.

[0303] Step 4: The system presents the search results to the user.

[0304] "Example 2"

[0305] Step 1: The user submits a request to a smartphone application via text input or voice input.

[0306] Step 2: The emotion engine recognizes emotions from the user's text or voice input.

[0307] Step 3: The application takes into account the emotions it recognizes and provides a service.

[0308] Step 4: The application presents the results of the services it provides to the user.

[0309] Example 1

[0310] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0311] Conventional navigation systems have difficulty in recommending destinations that take into account the user's detailed requests and feelings, and often do not adequately calculate optimal routes or analyze voice input. As a result, there are issues with lowering user satisfaction and making it difficult to ensure safety while driving.

[0312] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0313] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for recognizing the user's emotions and recommending a destination based thereon, a means for calculating an optimal route based on the user's current location and the search results, and a means for analyzing voice input and understanding the user's requests. This makes it possible to recommend a destination taking into account the user's detailed requests and emotions, and also calculates an optimal route and analyzes voice input, thereby improving user satisfaction and ensuring safety during driving.

[0314] An "information source" is a source location or system that provides data, such as a database that contains information such as restaurant ratings and the availability of parking spaces.

[0315] "Means of learning" refers to the process of using machine learning algorithms to analyze data based on acquired information and extract patterns and features.

[0316] "User specific requests" refers to specific requests made by users based on their particular conditions or preferences.

[0317] "Destination search means" refers to the process of searching a database for suitable destinations based on a user's request.

[0318] "Search Results" refers to destination information retrieved based on a User's request.

[0319] "Means for setting a destination" refers to the process of determining the user's destination based on the search results and registering that information in the system.

[0320] "Means for providing navigation" refers to a system or method for guiding a user to routes to a set destination.

[0321] "Means of recognizing emotions" refers to the process of analyzing and understanding the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[0322] "Method of calculating optimal route" refers to the process of calculating the most efficient route based on the user's current location and destination.

[0323] "Means for analyzing voice input" refers to the technologies and methods used to convert a user's speech into text and understand the content of that text.

[0324] The present invention relates to an in-vehicle navigation system that provides navigation by recommending optimal destinations while taking into consideration the detailed needs and feelings of the user. This system operates in cooperation with a server, a terminal, and a user.

[0325] Acquiring information and learning

[0326] The server acquires information from a database that includes information such as restaurant ratings and whether parking is available. This database is accessed, for example, through a general map service API or a rating information providing service API. The server learns the acquired information using a machine learning algorithm (for example, TENSORFLOW (registered trademark) or PyTorch). Specifically, feature quantities such as restaurant ratings and whether parking is available are extracted, and these are input into a model for learning.

[0327] User demand analysis

[0328] The device receives questions and requests from the user as voice input. For example, the user says, "Find a nice restaurant nearby." The device converts the voice into text using voice recognition technology (e.g., a general voice recognition API). The converted text is analyzed using natural language processing technology (e.g., a general natural language processing model). As a result of the analysis, the device understands that the user's request is intended to "find a nice restaurant."

[0329] Search for a destination

[0330] The server searches the database for an appropriate destination based on the analysis results. For example, it searches the database using the keyword "delicious restaurant." The server uses an emotion engine to recognize emotions from the user's tone of voice, facial expressions, and choice of words. For example, if the user's tone of voice is cheerful, it will recommend a lively restaurant, and if the tone of voice is sad, it will recommend a quiet restaurant.

[0331] Route calculation and setting

[0332] The server calculates the optimal route based on the user's current location (GPS information) and the search results. For example, the route is calculated using a general map service API. The calculated route information is sent to the device, and the destination is set.

[0333] Providing navigation

[0334] The device provides navigation to the set destination using a voice guidance system (e.g., a general navigation app). Specifically, instructions such as "Turn right at the next intersection" are given by voice. The user is safely guided to the destination while driving.

[0335] Examples

[0336] For example, if a user voice-inputs, "Find a nice restaurant nearby," the following specific actions will be taken:

[0337] 1. The device receives the user's voice and converts it into text using a speech recognition API.

[0338] 2. The device inputs the converted text into a natural language processing model and analyzes the intent of "find a good restaurant."

[0339] 3. The server receives the analysis results and searches the database for "good restaurants."

[0340] 4. The server analyzes the tone of the user's voice and recommends lively restaurants if the tone is cheerful, and quiet restaurants if the tone is depressed.

[0341] 5. The server uses the map service API to calculate the optimal route from the user's current location to the destination.

[0342] 6. The device receives the calculated route information and provides navigation to the user using a voice guidance system.

[0343] Examples of prompt statements

[0344] “If a user speaks to you to find a good restaurant nearby, design a system that recommends lively restaurants if the user’s voice tone is cheerful, and quiet restaurants if the user’s voice tone is depressed.”

[0345] In this way, optimal navigation is provided according to the user's emotions and needs.

[0346] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0347] Step 1:

[0348] Acquiring information and learning

[0349] The server retrieves information from a database that includes information such as restaurant ratings and whether parking is available.

[0350] Input: Information from a database (e.g. restaurant ratings, availability of parking)

[0351] Data processing: The acquired information is trained using a machine learning algorithm (e.g. TensorFlow, PyTorch). Specifically, features such as restaurant ratings and the availability of parking spaces are extracted and input into the model for training.

[0352] Output: The trained model

[0353] Step 2:

[0354] User demand analysis

[0355] The device receives questions and requests from the user as voice input.

[0356] Input: User's voice input (e.g. "Find some good restaurants near me")

[0357] Data processing: Convert speech to text using speech recognition technology (e.g., speech recognition API). Analyze the converted text using natural language processing technology (e.g., natural language processing model) to understand the user's request.

[0358] Output: Analysis results (e.g., the intent "find a good restaurant")

[0359] Step 3:

[0360] Search for a destination

[0361] The server searches a database for suitable destinations based on the analysis results.

[0362] Input: Analysis result (e.g., the intent "find a good restaurant")

[0363] Data processing: Search for "good restaurants" from a database. Using an emotion engine, recognize emotions from the user's tone of voice, facial expressions, and choice of words, and recommend destinations based on that.

[0364] Output: Search results (e.g. busy restaurants, quiet restaurants)

[0365] Step 4:

[0366] Route calculation and setting

[0367] The server calculates the optimal route based on the user's current location (GPS information) and the search results.

[0368] Input: User's current location (GPS information), search results

[0369] Data processing: Calculate the optimal route using a map service API.

[0370] Output: Calculated route information

[0371] Step 5:

[0372] Providing navigation

[0373] The terminal provides navigation to the set destination using a voice guidance system.

[0374] Input: Calculated route information

[0375] Data processing: Using a voice guidance system, instructions such as "Turn right at the next intersection" are given to the user.

[0376] Output: Voice guidance

[0377] What it does: Users are safely guided to their destination while driving.

[0378] (Application example 1)

[0379] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0380] Conventional navigation systems suggest destinations without considering the user's emotional state, making it difficult to provide destinations that suit the user's current mood and situation. In addition, they cannot provide optimal routes based on the user's emotions, making it difficult to improve user satisfaction.

[0381] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0382] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for recognizing the user's emotions, and a means for suggesting a destination based on the emotion recognition results, thereby enabling the suggestion of an optimal destination and navigation according to the user's emotional state.

[0383] "Source" refers to the medium or system that provides data or information.

[0384] A "means of learning" is a method for analyzing information and acquiring patterns and knowledge using machine learning algorithms.

[0385] "The user's specific requests" refer to the specific conditions and wishes of the user.

[0386] A "means for searching for a destination" is a method for searching for an appropriate destination based on a user's request.

[0387] "Search results" refers to destination candidates obtained through a search.

[0388] A "means for setting a destination" is a method for selecting and setting an optimal destination from the search results.

[0389] A "means for providing navigation" is a method for providing directions to a set destination.

[0390] A "means for recognizing emotions" is a method for analyzing and recognizing emotions from a user's tone of voice, facial expressions, etc.

[0391] "Emotion recognition result" refers to the user's emotional state obtained by the emotion recognition means.

[0392] The "means for suggesting destinations" is a method for recommending suitable destinations to a user based on emotion recognition results.

[0393] A system for implementing this invention has the following configuration. First, the server is equipped with a means for learning information from various information sources. Specifically, a machine learning algorithm (e.g., TensorFlow or PyTorch) is used to analyze data such as restaurant rating information and the availability of parking spaces, and to acquire patterns and knowledge.

[0394] Next, it has a means for searching for a destination based on the user's detailed requests. This means uses natural language processing technology (e.g., BERT or GPT-3 (registered trademark)) to analyze the user's voice command and understand the user's request. For example, if the user says, "Find a good restaurant nearby," the request is analyzed and an appropriate destination is searched for.

[0395] Furthermore, it has a means for setting a destination based on the search results. The server selects and sets the most suitable destination from the candidates obtained by the search. At this time, it takes into account the user's current location and calculates the optimal route using GPS technology.

[0396] The device also includes a means for providing navigation to the set destination. The server uses a voice guidance system to guide the user to the destination. This allows the user to reach the destination safely even while driving.

[0397] In addition, the device has a means to recognize the user's emotions. The server uses a microphone and camera to collect the user's tone of voice and facial expressions in real time, and then uses machine learning algorithms to analyze the emotions. For example, TensorFlow is used to analyze the voice data and classify the user's emotional state.

[0398] The system also has a means to suggest destinations based on emotion recognition results. The server recommends a destination that best suits the user's current emotional state based on the emotion recognition results. For example, if the user speaks in a bright tone, it recommends a lively restaurant, and if the user speaks in a calm tone, it recommends a quiet cafe.

[0399] For example, if a user says, "I want to relax today, so please find me a quiet cafe," the system will recognize that the user's tone of voice is calm and recommend a quiet cafe. It will then use GPS to calculate the optimal route and guide the user to their destination with voice guidance.

[0400] Examples of prompts include the following:

[0401] "Develop a navigation system that analyzes the user's tone of voice and facial expressions, recognizes emotions, and suggests the best destination. If the user says, 'I want to relax today, so find a quiet cafe,' the system will recognize that the user's tone of voice is calm and recommend a quiet cafe. It will then use GPS to calculate the best route and guide the user to their destination with voice guidance."

[0402] In this way, it becomes possible to suggest and navigate to optimal destinations according to the user's emotional state.

[0403] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0404] Step 1:

[0405] The server uses a microphone and a camera to collect the user's tone of voice and facial expressions in real time. The input is the user's voice and video data, and the output is pre-processed voice and video data. Specifically, it performs noise removal and facial recognition, and converts the data into a format suitable for analysis.

[0406] Step 2:

[0407] The server inputs the preprocessed audio and video data into a machine learning algorithm (e.g., TensorFlow) to recognize the user's emotions. The input is the preprocessed audio and video data, and the output is the user's emotional state (e.g., joy, sadness, anger, etc.). Specifically, the tone of the voice and facial features are analyzed to classify the emotion.

[0408] Step 3:

[0409] The server analyzes the user's voice command using natural language processing technology (e.g., BERT or GPT-3). The input is the user's voice command, and the output is text data that represents the user's request. Specifically, it analyzes commands such as "Find a good restaurant nearby" and extracts the user's request.

[0410] Step 4:

[0411] The server searches the database for an appropriate destination based on the emotion recognition result and the user's request. The input is the emotion recognition result and the user's request text, and the output is a list of destination candidates. Specifically, if the user is speaking in a bright tone, the server searches the database for lively restaurants.

[0412] Step 5:

[0413] The server selects and sets the best destination from the search results. The input is a list of candidate destinations, and the output is the set destination. Specifically, the server selects the most suitable destination taking into account the user's current location.

[0414] Step 6:

[0415] The server uses GPS technology to calculate the optimal route. The input is the user's current location and the set destination, and the output is the optimal route information. Specifically, it uses map data to calculate the shortest route and a route that takes into account traffic conditions.

[0416] Step 7:

[0417] The server uses a voice guidance system to guide the user to the destination. The input is optimal route information, and the output is voice guidance. Specifically, the server provides the user with voice instructions such as "Turn right at the next intersection."

[0418] In this way, optimal destination suggestions and navigation are realized according to the user's emotional state.

[0419] Example 2

[0420] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0421] Conventional navigation systems have difficulty in providing services that take into account the detailed requests and emotions of users. In addition, when a user requests a reservation by voice, the system lacks the function to automatically make a phone call to complete the reservation. This means that users have to make the reservation manually, which is inconvenient. Furthermore, the system does not recommend restaurants based on the user's emotions, which makes it difficult to increase user satisfaction.

[0422] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0423] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for recognizing the user's voice, a means for searching for a restaurant based on the voice recognition results, a means for automatically making a phone call, a means for completing a reservation, a means for recognizing the user's emotions, and a means for recommending a restaurant based on the emotions. This makes it possible to provide a service that takes into account the user's detailed requests and emotions, and to complete a reservation by automatically making a phone call in response to a reservation request made by voice. It is also possible to recommend restaurants based on the user's emotions, thereby increasing user satisfaction.

[0424] "Source" refers to any source that provides data or knowledge.

[0425] "Means of learning" refers to methods and techniques for analyzing data obtained from sources and accumulating knowledge.

[0426] "The user's detailed requests" refer to the specific conditions and wishes that the user has.

[0427] "Destination search means" refers to methods and technologies for searching for appropriate locations based on a user's requirements.

[0428] "Search Results" refers to the information or data obtained through a Search.

[0429] "Destination setting solution" refers to the methods or technologies that allow a user to determine where a user should visit based on search results.

[0430] "Means for providing navigation" refers to methods or technologies for providing directions or guidance to a set destination.

[0431] "Means for recognizing voice" refers to methods or technologies for analyzing a user's voice and converting it into text data.

[0432] "Means for searching for restaurants" refers to methods and technologies for searching for appropriate restaurants based on the results of voice recognition.

[0433] "Means for making automated calls" refers to methods or technologies that allow a system to automatically make calls.

[0434] "Means of completing a reservation" refers to the methods and technologies used to complete the reservation process over the telephone.

[0435] "Means for recognizing emotions" refers to methods or technologies for analyzing emotions from user input or voice.

[0436] "Restaurant recommendation means" refers to methods or techniques for suggesting suitable dining options based on recognized sentiment.

[0437] The present invention is a system for providing services that take into account the detailed requests and emotions of users. This system is implemented as a smartphone application, and performs destination search, reservation, navigation, and emotion-based restaurant recommendations based on the user's voice and text input.

[0438] Hardware and Software Used

[0439] Hardware: Smartphones (iPhone (registered trademark), ANDROID (registered trademark), etc.)

[0440] software:

[0441] Speech Recognition Technology: Uses speech recognition APIs to convert speech to text.

[0442] Restaurant search: Uses the Maps API to search for restaurants near the user's current location.

[0443] Automated Dialing System: Uses the Communications API to automatically place calls.

[0444] Emotion Recognition Technology: Use sentiment analysis API to analyze emotions from user input.

[0445] Restaurant Recommendation: Use review site APIs to recommend restaurants based on sentiment.

[0446] Specific processing of the system

[0447] Reservation function based on user requests

[0448] 1. The user launches the application on their smartphone and issues the voice command, "Make a reservation by phone."

[0449] 2. The device uses a speech recognition API to convert the user's speech into text.

[0450] 3. The device uses the map API to search for restaurants near the user's current location.

[0451] 4. The device uses a communications API to automatically call the restaurant selected from the search results.

[0452] 5. The terminal then transmits the reservation details to the restaurant via telephone, completing the reservation.

[0453] Examples:

[0454] The user says, "Call me and make a reservation."

[0455] The device uses a speech recognition API to convert speech to text.

[0456] Device: "Recognizing voice."

[0457] The device uses a map API to search for nearby restaurants.

[0458] Device: "Searching for nearby restaurants."

[0459] The device will automatically call the restaurant using a communications API.

[0460] Terminal: "Calling restaurant."

[0461] The terminal will notify the user that the reservation is complete.

[0462] Terminal: "Reservation completed."

[0463] Providing services using emotion engines

[0464] 1. The user launches a smartphone application and inputs via text or voice, "I'm tired today. I want to eat something delicious."

[0465] 2. The device uses the sentiment analysis API to recognize emotions from the user's input.

[0466] 3. The device uses review site APIs to recommend restaurants based on the user's sentiment.

[0467] Examples:

[0468] The user types, "I'm tired today. I want to eat something delicious."

[0469] The device uses a sentiment analysis API to recognize emotions.

[0470] Terminal: "Analyzing emotions."

[0471] The device uses review site APIs to recommend relaxing restaurants.

[0472] Terminal: "I'm looking for a relaxing restaurant."

[0473] Terminal: "How about this restaurant? (restaurant name)"

[0474] The above is a specific embodiment of the smartphone application of the present invention. This system makes it possible to provide services that take into account the detailed requests and emotions of the user, and can automatically make a phone call to complete a reservation in response to a voice reservation request. It can also recommend restaurants based on the user's emotions, thereby increasing user satisfaction.

[0475] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0476] Reservation function based on user requests

[0477] Processing Steps

[0478] Step 1:

[0479] The user launches the smartphone application and gives the voice command, "Make a reservation by phone."

[0480] Input: User's voice command

[0481] Output: Audio data

[0482] Specific operation: The user launches the application and gives the command "Make a reservation by phone" via voice.

[0483] Step 2:

[0484] The device uses a speech recognition API to convert the user's speech into text.

[0485] Input: Audio data

[0486] Output: Text data

[0487] Specific operation: The terminal sends voice data to the voice recognition API and receives text data.

[0488] Device: "Recognizing voice."

[0489] Step 3:

[0490] The device uses a map API to search for restaurants near the user's current location.

[0491] Input: Text data, user's current location information

[0492] Output: A list of restaurants

[0493] Specific operation: The device obtains the user's current location information and sends a query to the map API to receive a list of restaurants.

[0494] Device: "Searching for nearby restaurants."

[0495] Step 4:

[0496] The device uses a communications API to automatically call the restaurant selected from the search results.

[0497] Input: List of restaurants

[0498] Output:Call start

[0499] Specific operation: The terminal sends the phone number and call contents to the communication API and starts the call.

[0500] Terminal: "Calling restaurant."

[0501] Step 5:

[0502] The terminal then transmits the reservation details to the restaurant via telephone, completing the reservation.

[0503] Input: Start call

[0504] Output: Reservation completion notification

[0505] Specific operation: The terminal manages the call through the communication API and notifies the user that the reservation has been completed.

[0506] Terminal: "Reservation completed."

[0507] Providing services using emotion engines

[0508] Processing Steps

[0509] Step 1:

[0510] The user launches a smartphone application and inputs, by text or voice, "I'm tired today. I want to eat something delicious."

[0511] Input: User's text or voice input

[0512] Output: Text data

[0513] Specific behavior: The user launches the application and enters, via text or voice, "I'm tired today. I want to eat something delicious."

[0514] Step 2:

[0515] The device uses a sentiment analysis API to recognize emotions from user input.

[0516] Input: Text data

[0517] Output: Sentiment analysis results

[0518] Specific operation: The terminal sends text data to the sentiment analysis API and receives the sentiment analysis results.

[0519] Terminal: "Analyzing emotions."

[0520] Step 3:

[0521] The device uses review site APIs to recommend restaurants based on the user's sentiment.

[0522] Input: Sentiment analysis results

[0523] Output: A list of restaurant recommendations

[0524] Specific operation: Based on the results of the sentiment analysis, the device sends a query to the review site API and receives a list of restaurants that have a relaxing atmosphere or serve the user's favorite cuisine.

[0525] Terminal: "I'm looking for a relaxing restaurant."

[0526] Terminal: "How about this restaurant? (restaurant name)"

[0527] The above are the specific processing steps of the smartphone application of the present invention.

[0528] (Application example 2)

[0529] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0530] Conventional navigation and reservation systems were unable to consider the detailed needs and emotions of users, limiting the user experience. They also lacked the ability to input users' needs by voice, recommend optimal services based on those needs, and place orders automatically. This made it difficult to provide users with the services they wanted quickly and accurately.

[0531] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0532] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting the destination based on the search results, a means for providing navigation to the set destination, a means for recognizing a user's voice input, a means for analyzing emotions from the recognized voice input, a means for recommending an optimal service based on the results of the emotion analysis, and a means for automatically placing an order for the recommended service, thereby making it possible to provide a service that takes into account the user's requests and emotions.

[0533] "Source" refers to any source that provides data or knowledge.

[0534] A "means for learning" refers to a method or device for analyzing data obtained from a source and accumulating knowledge.

[0535] "The user's detailed requests" refer to the specific conditions and wishes that the user has.

[0536] "Destination searching means" refers to a method or device for searching for a suitable location based on a user's request.

[0537] "Search Results" refers to the collection of information obtained by a search tool.

[0538] "Means for setting a destination" refers to a method or apparatus for determining a location selected from the search results as a final destination.

[0539] "Means for providing navigation" refers to a method or device for providing directions or guidance to a set destination.

[0540] "Means for recognizing speech input" refers to a method or device for converting a user's speech into text data.

[0541] "Means for analyzing emotions" refers to a method or device for analyzing a user's emotions from voice input or text data.

[0542] "Means for recommending optimal services" refers to a method or device for suggesting the most suitable service to a user based on the results of sentiment analysis.

[0543] "Means for automated ordering" refers to a method or device for completing an order for recommended services without user intervention.

[0544] The system for implementing the present invention performs a series of processes, such as recognizing a user's voice input, analyzing emotions, recommending optimal services, and automatically placing an order. A specific embodiment of this system will be described below.

[0545] System Configuration

[0546] The system consists of the following main components:

[0547] 1. Speech Recognition Module: Converts the user's voice input into text data. Specifically, it uses the speech_recognition library.

[0548] 2. Sentiment Analysis Module: Analyzes the text data obtained from the speech recognition module and analyzes the user's sentiment. Specifically, it uses the TextBlob library.

[0549] 3. Service recommendation module: Recommends the most suitable services to users based on the results of sentiment analysis.

[0550] 4. Auto-order module: Automatically places an order for the recommended services by sending order data to an API endpoint using the requests library.

[0551] Processing flow

[0552] 1. Recognizing voice input: The user speaks their request into the smartphone. The voice recognition module converts this speech into text data.

[0553] 2. Sentiment analysis: The text data obtained from the voice recognition module is passed to the sentiment analysis module to analyze the user's emotions.

[0554] 3. Service recommendation: Based on the results of sentiment analysis, the service recommendation module selects and recommends the most suitable services to the user.

[0555] 4. Executing automatic order: The automatic order module places an order for the service selected by the service recommendation module.

[0556] Hardware and Software Used

[0557] Hardware: Smartphones, servers

[0558] Software: speech_recognition library, TextBlob library, requests library

[0559] Examples

[0560] For example, if a user says, "I'm tired today. I want to eat something delicious," the system will act as follows:

[0561] 1. The speech recognition module converts the speech into text and generates text data such as, "I'm tired today. I want to eat something delicious."

[0562] 2. The sentiment analysis module analyzes this text data and recognizes that the user is tired.

[0563] 3. The service recommendation module recommends restaurants with a relaxing atmosphere.

[0564] 4. The auto-ordering module automatically places orders from the recommended restaurants.

[0565] Examples of prompt statements

[0566] Example of a prompt to be input to a generative AI model:

[0567] If a user says, "I'm tired today. I want to eat something delicious," create a Python program that performs sentiment analysis, recommends restaurants with a relaxing atmosphere, and automatically places the order.

[0568] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0569] Step 1:

[0570] The user speaks his / her request into the smartphone. The voice input is acquired through the smartphone's microphone. The input data is the user's voice data.

[0571] Step 2:

[0572] The device's voice recognition module converts the user's voice data into text data. Specifically, it uses the speech_recognition library to analyze the voice data and generate text data. The output data is the user's voice converted into text.

[0573] Step 3:

[0574] The sentiment analysis module of the terminal receives the text data obtained from the voice recognition module and analyzes the user's sentiment. Specifically, the sentiment of the text data is analyzed using the TextBlob library. The input data is the text data, and the output data is the result of the sentiment analysis.

[0575] Step 4:

[0576] The service recommendation module of the server selects and recommends the most suitable service to the user based on the result of the sentiment analysis. The input data is the result of the sentiment analysis, and the output data is information on the recommended service. Specifically, if the sentiment is negative, a relaxing service is recommended, and if it is positive, a popular service is recommended.

[0577] Step 5:

[0578] The server's automatic ordering module receives the recommended service information obtained from the service recommendation module and automatically places an order. Specifically, it uses the requests library to send order data for the recommended service to an API endpoint. The input data is the information on the recommended service, and the output data is a notification of order completion.

[0579] Step 6:

[0580] The terminal displays a notification of order completion to the user. The input data is the notification of order completion, and the output data is a message of order completion displayed to the user. Specifically, a message indicating that the order has been completed is displayed on the screen of the smartphone.

[0581] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0582] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0583] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0584] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0585] [Second embodiment]

[0586] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0587] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0588] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0589] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0590] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0591] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0592] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0593] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0594] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0595] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0596] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0597] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0598] "Example 1"

[0599] One embodiment of the present invention is an in-vehicle navigation system. This system uses a machine learning algorithm as a means for learning information from various information sources. Specifically, information is acquired from a database including restaurant rating information and information on the availability of parking spaces, and this information is learned. In addition, natural language processing technology is used as a means for searching for a destination based on the detailed requests of the user. Questions from the user are analyzed, and an appropriate destination is searched for based on the analysis. Furthermore, GPS technology is used as a means for setting the destination based on the search results. The optimal route is calculated based on the user's current location and the search results, and the destination is set. Finally, a voice guidance system is used as a means for providing navigation to the set destination. This allows the user to be safely guided to the destination even while driving.

[0600] "Example 2"

[0601] Another embodiment of the present invention is a smartphone application. This application has the same functions as the in-car navigation system described above, but also includes a function for making reservations based on a user's request. Specifically, when a user requests "make a reservation by phone," the system automatically calls the corresponding restaurant and makes the reservation. This function is realized using voice recognition technology and an automatic dialing system.

[0602] The process flow of each embodiment will be described below.

[0603] "Example 1"

[0604] Step 1: The system learns from various sources, such as restaurant ratings and parking availability, and uses machine learning algorithms to learn from them.

[0605] Step 2: Once the user has entered their detailed request, the system analyzes it using natural language processing technology.

[0606] Step 3: Based on the analysis results, the system searches for suitable destinations.

[0607] Step 4: Based on the search results and the user's current location, the system will use GPS technology to calculate the optimal route and set the destination.

[0608] Step 5: To provide navigation to the set destination, the system guides the user using the voice guidance system.

[0609] "Example 2"

[0610] Step 1: The smartphone application has the same functions as the in-car navigation system mentioned above.

[0611] Step 2: If the user requests to "make a reservation by phone," the system will automatically call the relevant restaurant.

[0612] Step 3: The system uses voice recognition technology and an automated dialing system to make the reservation.

[0613] Example 1

[0614] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0615] Conventional navigation systems have difficulty in searching for a destination based on the user's detailed requests, and are unable to generate appropriate answers to the user's questions. Furthermore, they are unable to effectively use the user's current location or search results when setting a destination or providing navigation. This reduces user convenience and makes it difficult to ensure safety while driving.

[0616] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0617] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for converting voice input into text, a means for analyzing the user's questions using natural language processing techniques, a means for learning data using machine learning algorithms, a means for acquiring location information, and a means for generating voice guidance. This makes it possible to search for and set a destination based on the user's detailed requests, and to be safely guided to the destination even while driving.

[0618] "Source" refers to any medium or system that provides data.

[0619] "Learning means" refers to the algorithms and techniques used to analyze data and extract patterns and knowledge.

[0620] "User specific requests" refers to the detailed requirements that users have based on their specific conditions or preferences.

[0621] "Destination search means" refers to the technologies and algorithms used to search for appropriate destinations based on the user's requirements.

[0622] "Search Results" refers to destination candidates and information obtained through a search.

[0623] "Means for setting a destination" refers to the technology or algorithm used to determine an optimal destination based on the search results.

[0624] "Means for providing navigation" refers to technologies and systems that provide route guidance to help users reach their destination.

[0625] "Means of converting voice input into text" refers to technology or systems that analyze the user's voice and convert it into text information.

[0626] "Natural language processing technology" refers to techniques and algorithms for analyzing and understanding human language.

[0627] A "machine learning algorithm" refers to an algorithm that learns patterns and knowledge from data and makes predictions and classifications.

[0628] "Means of obtaining location information" refers to technologies and systems for identifying a user's current location.

[0629] "Means for generating audio guidance" refers to technology or systems for providing audio instructions to a user.

[0630] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0631] This invention relates to an in-vehicle navigation system, which searches for a destination based on the detailed requests of the user and provides the optimal route. This system operates in cooperation with a server, a terminal, and a user.

[0632] 1. Program Generation

[0633] The server generates the program for the in-car navigation system. This program is built by combining machine learning algorithms and natural language processing technology, using libraries and APIs such as Python's Pandas, Scikit-learn, NLTK, SpaCy, Google Maps API, and Amazon Polly.

[0634] 2. Program Processing

[0635] The server processes the data in the following steps:

[0636] 1. Data acquisition and training:

[0637] The server retrieves data from a database that contains information such as restaurant ratings and whether or not parking is available. Specifically, the server reads the data using Python's Pandas library. The server then uses Scikit-learn to train the data with machine learning algorithms. For example, the server creates a model to predict which restaurants have high ratings based on restaurant ratings.

[0638] 2. User Question Analysis:

[0639] The device receives questions from users. For example, it receives a question like, "Tell me about a highly rated restaurant nearby." The device analyzes the question using natural language processing technology. Specifically, it uses Python's NLTK and SpaCy to understand the intent of the question.

[0640] 3. Set your destination:

[0641] The server calculates the optimal route based on the user's current location and the search results. Specifically, it uses the Google Maps API to obtain the user's current location and the locations of highly rated restaurants, and calculates the optimal route.

[0642] 4. Providing navigation:

[0643] The device provides navigation to the set destination. Specifically, it uses a voice guidance system to give voice instructions to the user. For example, it uses Amazon Polly to generate voice guidance such as "Turn right."

[0644] 3. Specific Examples

[0645] As a concrete example, consider the case where a user asks, "Can you tell me about some highly rated restaurants nearby?"

[0646] 1. Data acquisition and training:

[0647] The server connects to the database and obtains restaurant rating information.

[0648] The server converts the data into a DataFrame using Pandas.

[0649] The server uses Scikit-learn's RandomForestClassifier to train a model to predict highly rated restaurants.

[0650] 2. User Question Analysis:

[0651] The user speaks to the device saying, "Tell me about some highly rated restaurants nearby."

[0652] The device converts the voice input into text.

[0653] The device uses NLTK to tokenize the text and SpaCy to analyze the intent of the question.

[0654] 3. Set your destination:

[0655] The device obtains the user's current location using the GPS sensor.

[0656] The server uses the Google Maps API to calculate a route based on the user's current location and the locations of highly rated restaurants.

[0657] The server transmits the calculation result to the terminal.

[0658] 4. Providing navigation:

[0659] The terminal starts navigation based on the route information received from the server.

[0660] The device uses Amazon Polly to generate voice guidance and provide instructions to the user.

[0661] The user drives by following the voice guidance.

[0662] Example prompt:

[0663] "Tell me about a highly rated restaurant nearby."

[0664] In this way, users are safely guided to their destination while driving.

[0665] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0666] Step 1: Data acquisition and training

[0667] The server retrieves information from the database, such as restaurant ratings and whether parking is available. As input, it uses the database connection information and a SQL query. The server converts the data into a DataFrame using Python's Pandas library. The server then trains the data with a machine learning algorithm using Scikit-learn. Specifically, it uses RandomForestClassifier to create a model that predicts highly rated restaurants. As output, it obtains the trained model.

[0668] Specific behavior:

[0669] The server connects to the database and executes SQL queries to retrieve the data.

[0670] The server converts the acquired data into a Pandas DataFrame.

[0671] The server uses Scikit-learn's RandomForestClassifier to train a model to predict restaurant ratings.

[0672] Step 2: Analyze user questions

[0673] The terminal receives a question from the user. As input, the user's voice question is used. The terminal uses speech recognition technology to convert the voice input into text. The terminal then analyzes the question using natural language processing technology. Specifically, it uses Python's NLTK and SpaCy to understand the intent of the question. As output, the analyzed text data is obtained.

[0674] Specific behavior:

[0675] The user speaks to the device saying, "Tell me about some highly rated restaurants nearby."

[0676] The device converts the voice input into text.

[0677] The device uses NLTK to tokenize the text and SpaCy to analyze the intent of the question.

[0678] Step 3: Set your destination

[0679] The server calculates the optimal route based on the user's current location and the search results. The user's current location and the parsed text data are used as input. The server uses the Google Maps API to obtain the user's current location and the locations of highly rated restaurants, and calculates the optimal route. The calculated route information is obtained as output.

[0680] Specific behavior:

[0681] The device obtains the user's current location using the GPS sensor.

[0682] The server uses the Google Maps API to calculate a route based on the user's current location and the locations of highly rated restaurants.

[0683] The server transmits the calculation result to the terminal.

[0684] Step 4: Providing navigation

[0685] The device provides navigation to the set destination. The calculated route information is used as input. The device uses a voice guidance system to give voice instructions to the user. Specifically, it uses Amazon Polly to generate voice guidance such as "Turn right". The voice guidance is obtained as output.

[0686] Specific behavior:

[0687] The terminal starts navigation based on the route information received from the server.

[0688] The device uses Amazon Polly to generate voice guidance and provide instructions to the user.

[0689] The user drives by following the voice guidance.

[0690] (Application example 1)

[0691] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0692] Conventional navigation systems have limited capabilities to search for destinations based on the user's detailed requests and lack the ability to generate appropriate answers to user questions. They also lack the ability to calculate the optimal route between the current location and the destination and provide voice guidance. This often causes inconvenience to users while traveling to their destination.

[0693] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a natural language processing means for analyzing the user's question and generating an appropriate answer, a means for calculating the optimal route between the current position and the destination, and a means for providing voice guidance. This makes it possible to search for a destination based on the user's detailed requests, calculate the optimal route, and provide voice guidance.

[0694] A "source" is a source or resource that provides data or knowledge.

[0695] A "means of learning information" is a method that uses machine learning algorithms to analyze data obtained from various sources and extract patterns and knowledge.

[0696] "User specific requests" are the detailed requirements or wishes of a user based on their specific conditions or preferences.

[0697] A "means for searching for a destination" is a method or technology for searching for an appropriate destination based on a user's request.

[0698] "Search results" are the results of a search obtained by a means for searching for a destination.

[0699] A "means for setting a destination" is a method or technology for determining a user's destination based on search results.

[0700] A "means for providing navigation" is a method or technology that provides route guidance or directions to a user to reach a destination.

[0701] "Natural language processing means" is a technology for analyzing a user's questions or requests and generating appropriate answers.

[0702] A "means for calculating a route" is a method or technique for calculating an optimal path between a current location and a destination.

[0703] "Means for providing audio guidance" means the methods or techniques for converting the calculated route information into audio and providing it to the user.

[0704] As an embodiment for carrying out the present invention, the following system is considered.

[0705] First, the server has a means of learning information from various sources. This means uses machine learning algorithms to analyze data such as restaurant reviews and the availability of parking spaces, and extract patterns and knowledge. Specifically, it obtains information from a database and learns from it.

[0706] Next, there is a means to search for a destination based on the user's detailed requests. This means uses natural language processing technology to analyze the user's question and search for an appropriate destination. For example, if a user asks, "What are some highly rated restaurants nearby?", the system will analyze the question and search for an appropriate restaurant.

[0707] In addition, there is a method to set a destination based on the search results. This method calculates the optimal route based on the user's current location and the search results, and sets the destination. Specifically, it uses the Geopy library to calculate the distance between the current location and the destination.

[0708] Finally, there is a method to provide navigation to a set destination, which uses the Google Text-to-Speech (gTTS) library to convert the calculated route information into audio and provide it to the user, allowing the user to be safely guided to their destination while driving.

[0709] As a concrete example, when a user asks, "What are some highly rated restaurants nearby?" the system works as follows:

[0710] 1. Natural language processing means analyzes the user's question and generates an appropriate answer.

[0711] 2. The means for learning information retrieves restaurant rating information from the database and analyzes it.

[0712] 3. A route calculation means calculates the optimal route between the current location and the destination.

[0713] 4. A means for providing audio guidance converts the calculated route information into audio and provides it to the user.

[0714] An example of a prompt sentence might be:

[0715] User asks: "What are some highly rated restaurants near me?"

[0716] Context: "Restaurant ratings, availability of parking, etc."

[0717] This prompt can then be used to have a generative AI model parse the question and provide an appropriate answer.

[0718] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0719] Step 1:

[0720] The server receives a question from a user. The input is the user's question, e.g., "What are some highly rated restaurants nearby?", as text data. The output is text data ready for analysis.

[0721] Step 2:

[0722] The server analyzes the user's question using natural language processing means. The input is the user's question received in step 1, and the output is an appropriate search query as the analysis result. Specifically, a generative AI model is used to analyze the question and generate an appropriate answer.

[0723] Step 3:

[0724] The server uses a means of learning information to obtain data such as restaurant rating information and availability of parking spaces from the database. The input is the search query generated in step 2, and the output is the obtained rating information, availability of parking spaces, and other data. Specifically, the server executes a query against the database to obtain the required information.

[0725] Step 4:

[0726] The server calculates the optimal route between the user's current location and the destination based on the acquired information. The input is the user's current location and the destination information acquired in step 3, and the output is the calculated optimal route information. Specifically, the Geopy library is used to calculate the distance and determine the optimal route.

[0727] Step 5:

[0728] The server converts the calculated route information into voice guidance. The input is the route information calculated in step 4, and the output is an audio file. Specifically, it uses the Google Text-to-Speech (gTTS) library to convert the text information into speech.

[0729] Step 6:

[0730] The terminal provides the generated voice guidance to the user. The input is the voice file generated in step 5, and the output is voice guidance for the user. Specifically, the terminal plays the voice file and provides navigation information to the user.

[0731] Example 2

[0732] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0733] Conventional navigation systems only provide destination setting and navigation functions, and do not have a reservation function based on the user's detailed requests, so users had to make reservations manually. In addition, there was no system that analyzed voice instructions and made reservations automatically, which made it difficult for users to use the system efficiently. This meant that users had to take the time to set the destination and make reservations separately, making it difficult to use the system efficiently.

[0734] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0735] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for analyzing the user's voice instructions, a means for extracting reservation details based on the analyzed voice instructions, a means for automatically making a phone call based on the extracted reservation details, and a means for audibly conveying the reservation details over the phone. This allows the user to automatically set a destination and make a reservation simply by issuing voice instructions, greatly improving convenience.

[0736] "Source" refers to any source that provides data or knowledge.

[0737] "User" refers to any individual or entity that uses the System.

[0738] "Destination" refers to the place the user wants to reach.

[0739] "Search" refers to the act of finding specific information.

[0740] "Navigation" refers to the function of providing route guidance to a destination.

[0741] "Voice instruction" refers to an instruction given by a user to the system using voice.

[0742] "Analysis" refers to the act of examining data or information in detail to understand its meaning and structure.

[0743] "Reservation details" refers to the information necessary for making a reservation (date and time, number of people, location, etc.).

[0744] "Automatic" refers to a machine or system that operates independently, without human intervention.

[0745] "Making a call" refers to the act of using a communication device to attempt to connect to another communication device.

[0746] "Communicating by voice" refers to the act of conveying information to others using voice.

[0747] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0748] The present invention is embodied as an application for a smartphone. This application is a system that automatically sets a destination and makes a reservation based on a user's voice instruction. A specific embodiment of this system will be described below.

[0749] Hardware and Software Used

[0750] The system uses the following hardware and software:

[0751] Smartphone (device)

[0752] Speech recognition technology (Google Cloud Speech-to-Text API)

[0753] Database Management System (Firebase Firestore)

[0754] Automated dialing system (Twilio API)

[0755] Speech synthesis technology (Google Cloud Text-to-Speech API)

[0756] Data processing and calculation

[0757] The user launches the smartphone application and says, "Make a reservation for four people tomorrow night at 7 p.m." The device uses the Google Cloud Speech-to-Text API to convert the user's voice instructions into text. This conversion process occurs in real time.

[0758] Next, the device analyzes the converted text and extracts the reservation details (date, time, number of people). For example, from the text "Make a reservation for four people at 7 p.m. tomorrow," the information "tomorrow at 7 p.m." and "four people" is extracted.

[0759] The device sends a request to the server to retrieve restaurant information stored in Firebase Firestore. This request includes the user's location and preferred restaurant conditions. The server receives the request, retrieves the corresponding restaurant information from Firebase Firestore, and returns it to the device. The returned information includes the restaurant's name, phone number, address, etc.

[0760] The device uses the Twilio API to automatically call the restaurant's phone number, without any user intervention. Once the call is connected, the device uses the Google Cloud Text-to-Speech API to communicate the reservation details it extracted in advance to the restaurant. For example, it might say, "Please make a reservation for four people tomorrow night at 7 p.m."

[0761] Examples

[0762] As a concrete example, the following will show the behavior when the user instructs "Make a reservation for four people at 7pm tomorrow night."

[0763] 1. The user launches the smartphone application and gives a voice command such as, "Make a reservation for four people at 7pm tomorrow night."

[0764] 2. The device uses the Google Cloud Speech-to-Text API to convert the user's voice commands into text.

[0765] 3. The device analyzes the converted text and extracts the reservation details: "tomorrow night at 7pm" and "4 people."

[0766] 4. The device sends a request to the server to get restaurant information stored in Firebase Firestore.

[0767] 5. The server receives the request, retrieves the relevant restaurant information from Firebase Firestore, and returns it to the device.

[0768] 6. The device uses the Twilio API to automatically call the restaurant's phone number.

[0769] 7. Once the call is connected, the device uses the Google Cloud Text-to-Speech API to communicate to the restaurant, "Please make a reservation for four people tomorrow night at 7pm."

[0770] Examples of prompt statements

[0771] An example of a prompt sentence might be:

[0772] "If a user says, 'Make a reservation for 4 people tomorrow night at 7pm,' how should the application handle that?"

[0773] By inputting this prompt sentence into the generative AI model, an explanation of the processing based on the specific example above can be obtained.

[0774] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0775] Step 1:

[0776] The user issues a voice command.

[0777] The user starts the smartphone application and gives a voice command such as, "Make a reservation for four people at 7 p.m. tomorrow." This voice command becomes the input.

[0778] Step 2:

[0779] The device converts the speech to text.

[0780] The device uses voice recognition technology to convert the user's voice instructions into text. This process is performed using a voice recognition API. The input is the user's voice instructions, and the output is the instructions in text format.

[0781] Step 3:

[0782] The terminal analyzes the text and extracts the reservation details.

[0783] The terminal analyzes the converted text and extracts the reservation details (date, time, number of people). For example, from the text "Make a reservation for four people at 7 p.m. tomorrow," the information "tomorrow at 7 p.m." and "four people" is extracted. The input is instructions in text format, and the output is the reservation details data.

[0784] Step 4:

[0785] The terminal requests restaurant information from the server.

[0786] The terminal sends a request to the server to get restaurant information stored in the database management system. This request includes the user's location information and the conditions of the preferred restaurant. The input is the reservation details and the user's location information, and the output is a request for restaurant information.

[0787] Step 5:

[0788] The server returns the restaurant information.

[0789] The server receives the request, retrieves the relevant restaurant information from the database, and returns it to the terminal. The returned information includes the restaurant's name, phone number, address, etc. The input is a request for restaurant information, and the output is the restaurant information.

[0790] Step 6:

[0791] The terminal makes the call using an automatic dialing system.

[0792] The terminal then uses an automatic dialing system to automatically call the restaurant's phone number, without any user intervention: the input is the restaurant's phone number, and the output is the establishment of a telephone connection.

[0793] Step 7:

[0794] The terminal will announce the reservation details aloud.

[0795] When the phone is connected, the device uses voice synthesis technology to communicate the reservation details it extracted in advance to the restaurant by voice. For example, it may say, "Please make a reservation for four people tomorrow night at 7 p.m." The input is the reservation details data, and the output is a voice message.

[0796] (Application example 2)

[0797] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0798] Conventional navigation and reservation systems required users to operate them manually, which made them inconvenient. In addition, they lacked the functionality to utilize the user's past behavioral history to make optimal suggestions, making it difficult to respond to user requests quickly and accurately. Furthermore, there was a need to effectively combine voice recognition technology and an automatic dialing system to improve the user experience.

[0799] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0800] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for understanding the user's requests using voice recognition technology, a means for making reservations using an automatic dialing system, a means for learning the user's past behavior history and making optimal suggestions, and a means for providing voice feedback. This allows the user to search for and make reservations at a destination using only voice instructions, and to receive optimal suggestions based on the user's past behavior history.

[0801] "Source" refers to any source that provides data or knowledge.

[0802] "Means of learning" refers to the methods and techniques used to collect, analyze, and understand information.

[0803] "The user's detailed requests" refer to the specific conditions and wishes that the user has.

[0804] "Destination search means" refers to methods and technologies for searching for appropriate locations based on a user's requirements.

[0805] "Search Results" refers to the information or data obtained through a Search.

[0806] "Means for setting a destination" refers to a method or technology for determining a final destination based on a search result.

[0807] "Means for providing navigation" refers to methods and technologies for providing directions to a set destination.

[0808] "Voice recognition technology" refers to the technology that analyzes voice and converts it into text or commands.

[0809] "Automatic dialing system" refers to a system that automatically dials designated telephone numbers.

[0810] "Means for making reservations" refers to methods or technologies for automatically making reservations for specified locations or services.

[0811] "Past behavioral history" refers to the record of actions and choices a user has previously made.

[0812] "Means of making optimal suggestions" refers to methods and technologies for presenting the most suitable options to users based on their past behavioral history and current needs.

[0813] "Means for providing audio feedback" refers to methods or technologies that allow the system to communicate information or results to the user using audio.

[0814] A system for implementing the present invention has the following configuration: A server includes means for learning information from various information sources, means for searching for a destination based on a user's detailed requests, means for setting a destination based on the search results, means for providing navigation to the set destination, means for understanding a user's requests using voice recognition technology, means for making reservations using an automatic dialing system, means for learning a user's past behavior history and making optimal suggestions, and means for providing voice feedback.

[0815] Explanation of program processing

[0816] Hardware and Software

[0817] The system uses the smartphone's microphone and speaker as hardware, the speech_recognition library for voice recognition, the pyttsx3 library for voice feedback, and the requests library for data communication.

[0818] Data processing and calculation

[0819] 1. Speech recognition: When the user speaks their request into the smartphone's microphone, the speech_recognition library converts the speech into text.

[0820] 2. Request Analysis: Analyze the converted text to understand the user's request. For example, analyze the request "order a pizza."

[0821] 3. Execute the booking: Based on the request, send the order to the food delivery API using the requests library. If the order is successful, provide audio feedback using the pyttsx3 library.

[0822] 4. Learning from behavioral history: Learns the user's past behavioral history and makes optimal suggestions for future visits.

[0823] Examples

[0824] For example, if a user says "order a pizza," the system would do the following:

[0825] 1. Capture the user's voice with the microphone.

[0826] 2. Convert speech to text and get the text "Order pizza."

[0827] 3. Analyze the text to identify "pizza" orders.

[0828] 4. Send an order for "Pizza" to the food delivery API.

[0829] 5. If the order is successful, provide audio feedback saying "Your pizza order has been completed."

[0830] Examples of prompt statements

[0831] When a user says "Order a pizza," create a program that converts the speech to text using speech recognition technology and sends the order to the food delivery API. If the order is successful, provide feedback by voice saying "Pizza order completed."

[0832] In this way, users can search for and book destinations using only voice commands, and receive optimal suggestions based on their past travel history.

[0833] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0834] Step 1:

[0835] The user speaks their request into the smartphone's microphone.

[0836] Input: User's speech.

[0837] Output: Audio data captured by the microphone.

[0838] What happens: When a user says "order a pizza," the smartphone's microphone captures the audio.

[0839] Step 2:

[0840] The device converts the voice data into text using the speech_recognition library.

[0841] Input: Audio data captured by the microphone.

[0842] Output: Text data (e.g. "order a pizza").

[0843] What happens: The speech recognition engine analyzes the voice data and converts it into corresponding text.

[0844] Step 3:

[0845] The device analyzes the text data and understands the user's requests.

[0846] Input: Text data (e.g. "order a pizza").

[0847] Output: Parsed request (e.g. "order pizza").

[0848] What happens: The text analysis engine analyzes the text "order a pizza" and identifies the request "order a pizza."

[0849] Step 4:

[0850] The device uses the requests library to send an order to the food delivery API.

[0851] Input: Parsed request (e.g. "order pizza").

[0852] Output: The order request to the API and its response.

[0853] Specific behavior: Send a request to the food delivery API containing "order pizza" and receive a response indicating whether the order was successful.

[0854] Step 5:

[0855] The device provides audio feedback using the pyttsx3 library.

[0856] Input: The response from the API (e.g., order successful).

[0857] Output: Audio feedback (e.g. "Your pizza order is complete").

[0858] What happens: The speech synthesis engine generates the message "Your pizza order has been completed" and relays it to the user through the smartphone speaker.

[0859] Step 6:

[0860] The server learns the user's past behavioral history and makes optimal suggestions for future visits.

[0861] Input: User behavioral history data.

[0862] Output: Proposed data based on the learning results.

[0863] What happens: The server analyzes the user's past ordering history and generates data to suggest the best restaurant and menu for the next time they order.

[0864] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0865] "Example 1"

[0866] One embodiment of the present invention is a navigation system incorporating an emotion engine. This system recognizes the user's emotions from the tone of voice, facial expressions, choice of words, etc., and searches for a destination taking these emotions into consideration. For example, if a user requests "find a nice restaurant nearby," the system will recommend lively restaurants if the user's tone of voice is cheerful, and quiet restaurants if the user's tone of voice is sad.

[0867] "Example 2"

[0868] Also, a smartphone application incorporating the emotion engine is one embodiment of the present invention. This application recognizes emotions from the user's text input or voice input and provides services taking them into consideration. For example, if the user inputs "I'm tired today. I want to eat something delicious," the application recognizes the emotion and recommends restaurants with a relaxing atmosphere and restaurants that serve the user's favorite dishes.

[0869] The process flow of each embodiment will be described below.

[0870] "Example 1"

[0871] Step 1: The user issues a verbal request to the navigation system.

[0872] Step 2: The emotion engine recognizes the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[0873] Step 3: Search for destinations taking into account the emotions recognized by the system.

[0874] Step 4: The system presents the search results to the user.

[0875] "Example 2"

[0876] Step 1: The user submits a request to a smartphone application via text input or voice input.

[0877] Step 2: The emotion engine recognizes emotions from the user's text or voice input.

[0878] Step 3: The application takes into account the emotions it recognizes and provides a service.

[0879] Step 4: The application presents the results of the services it provides to the user.

[0880] Example 1

[0881] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0882] Conventional navigation systems have difficulty in recommending destinations that take into account the user's detailed requests and feelings, and often do not adequately calculate optimal routes or analyze voice input. As a result, there are issues with lowering user satisfaction and making it difficult to ensure safety while driving.

[0883] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0884] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for recognizing the user's emotions and recommending a destination based thereon, a means for calculating an optimal route based on the user's current location and the search results, and a means for analyzing voice input and understanding the user's requests. This makes it possible to recommend a destination taking into account the user's detailed requests and emotions, and also calculates an optimal route and analyzes voice input, thereby improving user satisfaction and ensuring safety during driving.

[0885] An "information source" is a source location or system that provides data, such as a database that contains information such as restaurant ratings and the availability of parking spaces.

[0886] "Means of learning" refers to the process of using machine learning algorithms to analyze data based on acquired information and extract patterns and features.

[0887] "User specific requests" refers to specific requests made by users based on their particular conditions or preferences.

[0888] "Destination search means" refers to the process of searching a database for suitable destinations based on a user's request.

[0889] "Search Results" refers to destination information retrieved based on a User's request.

[0890] "Means for setting a destination" refers to the process of determining the user's destination based on the search results and registering that information in the system.

[0891] "Means for providing navigation" refers to a system or method for guiding a user to routes to a set destination.

[0892] "Means of recognizing emotions" refers to the process of analyzing and understanding the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[0893] "Method of calculating optimal route" refers to the process of calculating the most efficient route based on the user's current location and destination.

[0894] "Means for analyzing voice input" refers to the technologies and methods used to convert a user's speech into text and understand the content of that text.

[0895] The present invention relates to an in-vehicle navigation system that provides navigation by recommending optimal destinations while taking into consideration the detailed needs and feelings of the user. This system operates in cooperation with a server, a terminal, and a user.

[0896] Acquiring information and learning

[0897] The server obtains information from a database that contains information such as restaurant ratings and whether or not parking is available. This database is accessed, for example, through a general map service API or a rating information providing service API. The server uses a machine learning algorithm (for example, TensorFlow or PyTorch) to learn from the obtained information. Specifically, it extracts features such as the restaurant's ratings and whether or not parking is available, and inputs them into a model for learning.

[0898] User demand analysis

[0899] The device receives questions and requests from the user as voice input. For example, the user says, "Find a nice restaurant nearby." The device converts the voice into text using voice recognition technology (e.g., a general voice recognition API). The converted text is analyzed using natural language processing technology (e.g., a general natural language processing model). As a result of the analysis, the device understands that the user's request is intended to "find a nice restaurant."

[0900] Search for a destination

[0901] The server searches the database for an appropriate destination based on the analysis results. For example, it searches the database using the keyword "delicious restaurant." The server uses an emotion engine to recognize emotions from the user's tone of voice, facial expressions, and choice of words. For example, if the user's tone of voice is cheerful, it will recommend a lively restaurant, and if the tone of voice is sad, it will recommend a quiet restaurant.

[0902] Route calculation and setting

[0903] The server calculates the optimal route based on the user's current location (GPS information) and the search results. For example, the route is calculated using a general map service API. The calculated route information is sent to the device, and the destination is set.

[0904] Providing navigation

[0905] The device provides navigation to the set destination using a voice guidance system (e.g., a general navigation app). Specifically, instructions such as "Turn right at the next intersection" are given by voice. The user is safely guided to the destination while driving.

[0906] Examples

[0907] For example, if a user voice-inputs, "Find a nice restaurant nearby," the following specific actions will be taken:

[0908] 1. The device receives the user's voice and converts it into text using a speech recognition API.

[0909] 2. The device inputs the converted text into a natural language processing model and analyzes the intent of "find a good restaurant."

[0910] 3. The server receives the analysis results and searches the database for "good restaurants."

[0911] 4. The server analyzes the tone of the user's voice and recommends lively restaurants if the tone is cheerful, and quiet restaurants if the tone is depressed.

[0912] 5. The server uses the map service API to calculate the optimal route from the user's current location to the destination.

[0913] 6. The device receives the calculated route information and provides navigation to the user using a voice guidance system.

[0914] Examples of prompt statements

[0915] “If a user speaks to you to find a good restaurant nearby, design a system that recommends lively restaurants if the user’s voice tone is cheerful, and quiet restaurants if the user’s voice tone is depressed.”

[0916] In this way, optimal navigation is provided according to the user's emotions and needs.

[0917] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0918] Step 1:

[0919] Acquiring information and learning

[0920] The server retrieves information from a database that includes information such as restaurant ratings and whether parking is available.

[0921] Input: Information from a database (e.g. restaurant ratings, availability of parking)

[0922] Data processing: The acquired information is trained using a machine learning algorithm (e.g. TensorFlow, PyTorch). Specifically, features such as restaurant ratings and the availability of parking spaces are extracted and input into the model for training.

[0923] Output: The trained model

[0924] Step 2:

[0925] User demand analysis

[0926] The device receives questions and requests from the user as voice input.

[0927] Input: User's voice input (e.g. "Find some good restaurants near me")

[0928] Data processing: Convert speech to text using speech recognition technology (e.g., speech recognition API). Analyze the converted text using natural language processing technology (e.g., natural language processing model) to understand the user's request.

[0929] Output: Analysis results (e.g., the intent "find a good restaurant")

[0930] Step 3:

[0931] Search for a destination

[0932] The server searches a database for suitable destinations based on the analysis results.

[0933] Input: Analysis result (e.g., the intent "find a good restaurant")

[0934] Data processing: Search for "good restaurants" from a database. Using an emotion engine, recognize emotions from the user's tone of voice, facial expressions, and choice of words, and recommend destinations based on that.

[0935] Output: Search results (e.g. busy restaurants, quiet restaurants)

[0936] Step 4:

[0937] Route calculation and setting

[0938] The server calculates the optimal route based on the user's current location (GPS information) and the search results.

[0939] Input: User's current location (GPS information), search results

[0940] Data processing: Calculate the optimal route using a map service API.

[0941] Output: Calculated route information

[0942] Step 5:

[0943] Providing navigation

[0944] The terminal provides navigation to the set destination using a voice guidance system.

[0945] Input: Calculated route information

[0946] Data processing: Using a voice guidance system, instructions such as "Turn right at the next intersection" are given to the user.

[0947] Output: Voice guidance

[0948] What it does: Users are safely guided to their destination while driving.

[0949] (Application example 1)

[0950] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0951] Conventional navigation systems suggest destinations without considering the user's emotional state, making it difficult to provide destinations that suit the user's current mood and situation. In addition, they cannot provide optimal routes based on the user's emotions, making it difficult to improve user satisfaction.

[0952] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0953] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for recognizing the user's emotions, and a means for suggesting a destination based on the emotion recognition results, thereby enabling the suggestion of an optimal destination and navigation according to the user's emotional state.

[0954] "Source" refers to the medium or system that provides data or information.

[0955] A "means of learning" is a method for analyzing information and acquiring patterns and knowledge using machine learning algorithms.

[0956] "The user's specific requests" refer to the specific conditions and wishes of the user.

[0957] A "means for searching for a destination" is a method for searching for an appropriate destination based on a user's request.

[0958] "Search results" refers to destination candidates obtained through a search.

[0959] A "means for setting a destination" is a method for selecting and setting an optimal destination from the search results.

[0960] A "means for providing navigation" is a method for providing directions to a set destination.

[0961] A "means for recognizing emotions" is a method for analyzing and recognizing emotions from a user's tone of voice, facial expressions, etc.

[0962] "Emotion recognition result" refers to the user's emotional state obtained by the emotion recognition means.

[0963] The "means for suggesting destinations" is a method for recommending suitable destinations to a user based on emotion recognition results.

[0964] A system for implementing this invention has the following configuration. First, the server is equipped with a means for learning information from various information sources. Specifically, a machine learning algorithm (e.g., TensorFlow or PyTorch) is used to analyze data such as restaurant rating information and the availability of parking spaces, and to acquire patterns and knowledge.

[0965] Next, it has a means to search for a destination based on the user's detailed requests. This means uses natural language processing technology (e.g., BERT and GPT-3) to analyze the user's voice commands and understand the user's requests. For example, if the user says, "Find a good restaurant nearby," the request is analyzed and an appropriate destination is searched for.

[0966] Furthermore, it has a means to set a destination based on the search results. The server selects and sets the most suitable destination from the candidates obtained by the search. At this time, it takes into account the user's current location and calculates the optimal route using GPS technology.

[0967] The device also includes a means for providing navigation to the set destination. The server uses a voice guidance system to guide the user to the destination. This allows the user to reach the destination safely even while driving.

[0968] In addition, the device has a means to recognize the user's emotions. The server uses a microphone and camera to collect the user's tone of voice and facial expressions in real time, and then uses machine learning algorithms to analyze the emotions. For example, TensorFlow is used to analyze the voice data and classify the user's emotional state.

[0969] The system also has a means to suggest destinations based on emotion recognition results. The server recommends a destination that best suits the user's current emotional state based on the emotion recognition results. For example, if the user speaks in a bright tone, it recommends a lively restaurant, and if the user speaks in a calm tone, it recommends a quiet cafe.

[0970] For example, if a user says, "I want to relax today, so please find me a quiet cafe," the system will recognize that the user's tone of voice is calm and recommend a quiet cafe. It will then use GPS to calculate the optimal route and guide the user to their destination with voice guidance.

[0971] Examples of prompts include the following:

[0972] "Develop a navigation system that analyzes the user's tone of voice and facial expressions, recognizes emotions, and suggests the best destination. If the user says, 'I want to relax today, so find a quiet cafe,' the system will recognize that the user's tone of voice is calm and recommend a quiet cafe. It will then use GPS to calculate the best route and guide the user to their destination with voice guidance."

[0973] In this way, it becomes possible to suggest and navigate to optimal destinations according to the user's emotional state.

[0974] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0975] Step 1:

[0976] The server uses a microphone and a camera to collect the user's tone of voice and facial expressions in real time. The input is the user's voice and video data, and the output is pre-processed voice and video data. Specifically, it performs noise removal and facial recognition, and converts the data into a format suitable for analysis.

[0977] Step 2:

[0978] The server inputs the preprocessed audio and video data into a machine learning algorithm (e.g., TensorFlow) to recognize the user's emotions. The input is the preprocessed audio and video data, and the output is the user's emotional state (e.g., joy, sadness, anger, etc.). Specifically, the tone of the voice and facial features are analyzed to classify the emotion.

[0979] Step 3:

[0980] The server analyzes the user's voice command using natural language processing technology (e.g., BERT or GPT-3). The input is the user's voice command, and the output is text data that represents the user's request. Specifically, it analyzes commands such as "Find a good restaurant nearby" and extracts the user's request.

[0981] Step 4:

[0982] The server searches the database for an appropriate destination based on the emotion recognition result and the user's request. The input is the emotion recognition result and the user's request text, and the output is a list of destination candidates. Specifically, if the user is speaking in a bright tone, the server searches the database for lively restaurants.

[0983] Step 5:

[0984] The server selects and sets the best destination from the search results. The input is a list of candidate destinations, and the output is the set destination. Specifically, the server selects the most suitable destination taking into account the user's current location.

[0985] Step 6:

[0986] The server uses GPS technology to calculate the optimal route. The input is the user's current location and the set destination, and the output is the optimal route information. Specifically, it uses map data to calculate the shortest route and a route that takes into account traffic conditions.

[0987] Step 7:

[0988] The server uses a voice guidance system to guide the user to the destination. The input is optimal route information, and the output is voice guidance. Specifically, the server provides the user with voice instructions such as "Turn right at the next intersection."

[0989] In this way, optimal destination suggestions and navigation are realized according to the user's emotional state.

[0990] Example 2

[0991] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0992] Conventional navigation systems have difficulty in providing services that take into account the detailed requests and emotions of users. In addition, when a user requests a reservation by voice, the system lacks the function to automatically make a phone call to complete the reservation. This means that users have to make the reservation manually, which is inconvenient. Furthermore, the system does not recommend restaurants based on the user's emotions, which makes it difficult to increase user satisfaction.

[0993] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0994] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for recognizing the user's voice, a means for searching for a restaurant based on the voice recognition results, a means for automatically making a phone call, a means for completing a reservation, a means for recognizing the user's emotions, and a means for recommending a restaurant based on the emotions. This makes it possible to provide a service that takes into account the user's detailed requests and emotions, and to complete a reservation by automatically making a phone call in response to a reservation request made by voice. It is also possible to recommend restaurants based on the user's emotions, thereby increasing user satisfaction.

[0995] "Source" refers to any source that provides data or knowledge.

[0996] "Means of learning" refers to methods and techniques for analyzing data obtained from sources and accumulating knowledge.

[0997] "The user's detailed requests" refer to the specific conditions and wishes that the user has.

[0998] "Destination search means" refers to methods and technologies for searching for appropriate locations based on a user's requirements.

[0999] "Search Results" refers to the information or data obtained through a Search.

[1000] "Destination setting solution" refers to the methods or technologies that allow a user to determine where a user should visit based on search results.

[1001] "Means for providing navigation" refers to methods or technologies for providing directions or guidance to a set destination.

[1002] "Means for recognizing voice" refers to methods or technologies for analyzing a user's voice and converting it into text data.

[1003] "Means for searching for restaurants" refers to methods and technologies for searching for appropriate restaurants based on the results of voice recognition.

[1004] "Means for making automated calls" refers to methods or technologies that allow a system to automatically make calls.

[1005] "Means of completing a reservation" refers to the methods and technologies used to complete the reservation process over the telephone.

[1006] "Means for recognizing emotions" refers to methods or technologies for analyzing emotions from user input or voice.

[1007] "Restaurant recommendation means" refers to methods or techniques for suggesting suitable dining options based on recognized sentiment.

[1008] The present invention is a system for providing services that take into account the detailed requests and emotions of users. This system is implemented as a smartphone application, and performs destination search, reservation, navigation, and emotion-based restaurant recommendations based on the user's voice and text input.

[1009] Hardware and Software Used

[1010] Hardware: Smartphone (iPhone, Android device, etc.)

[1011] software:

[1012] Speech Recognition Technology: Uses speech recognition APIs to convert speech to text.

[1013] Restaurant search: Uses the Maps API to search for restaurants near the user's current location.

[1014] Automated Dialing System: Uses the Communications API to automatically place calls.

[1015] Emotion Recognition Technology: Use sentiment analysis API to analyze emotions from user input.

[1016] Restaurant Recommendation: Use review site APIs to recommend restaurants based on sentiment.

[1017] Specific processing of the system

[1018] Reservation function based on user requests

[1019] 1. The user launches the application on their smartphone and issues the voice command, "Make a reservation by phone."

[1020] 2. The device uses a speech recognition API to convert the user's speech into text.

[1021] 3. The device uses the map API to search for restaurants near the user's current location.

[1022] 4. The device uses a communications API to automatically call the restaurant selected from the search results.

[1023] 5. The terminal then transmits the reservation details to the restaurant via telephone, completing the reservation.

[1024] Examples:

[1025] The user says, "Call me and make a reservation."

[1026] The device uses a speech recognition API to convert speech to text.

[1027] Device: "Recognizing voice."

[1028] The device uses a map API to search for nearby restaurants.

[1029] Device: "Searching for nearby restaurants."

[1030] The device will automatically call the restaurant using a communications API.

[1031] Terminal: "Calling restaurant."

[1032] The terminal will notify the user that the reservation is complete.

[1033] Terminal: "Reservation completed."

[1034] Providing services using emotion engines

[1035] 1. The user launches a smartphone application and inputs via text or voice, "I'm tired today. I want to eat something delicious."

[1036] 2. The device uses the sentiment analysis API to recognize emotions from the user's input.

[1037] 3. The device uses review site APIs to recommend restaurants based on the user's sentiment.

[1038] Examples:

[1039] The user types, "I'm tired today. I want to eat something delicious."

[1040] The device uses a sentiment analysis API to recognize emotions.

[1041] Terminal: "Analyzing emotions."

[1042] The device uses review site APIs to recommend relaxing restaurants.

[1043] Terminal: "I'm looking for a relaxing restaurant."

[1044] Terminal: "How about this restaurant? (restaurant name)"

[1045] The above is a specific embodiment of the smartphone application of the present invention. This system makes it possible to provide services that take into account the detailed requests and emotions of the user, and can automatically make a phone call to complete a reservation in response to a voice reservation request. It can also recommend restaurants based on the user's emotions, thereby increasing user satisfaction.

[1046] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1047] Reservation function based on user requests

[1048] Processing Steps

[1049] Step 1:

[1050] The user launches the smartphone application and gives the voice command, "Make a reservation by phone."

[1051] Input: User's voice command

[1052] Output: Audio data

[1053] Specific operation: The user launches the application and gives the command "Make a reservation by phone" via voice.

[1054] Step 2:

[1055] The device uses a speech recognition API to convert the user's speech into text.

[1056] Input: Audio data

[1057] Output: Text data

[1058] Specific operation: The terminal sends voice data to the voice recognition API and receives text data.

[1059] Device: "Recognizing voice."

[1060] Step 3:

[1061] The device uses a map API to search for restaurants near the user's current location.

[1062] Input: Text data, user's current location information

[1063] Output: A list of restaurants

[1064] Specific operation: The device obtains the user's current location information and sends a query to the map API to receive a list of restaurants.

[1065] Device: "Searching for nearby restaurants."

[1066] Step 4:

[1067] The device uses a communications API to automatically call the restaurant selected from the search results.

[1068] Input: List of restaurants

[1069] Output:Call start

[1070] Specific operation: The terminal sends the phone number and call contents to the communication API and starts the call.

[1071] Terminal: "Calling restaurant."

[1072] Step 5:

[1073] The terminal then transmits the reservation details to the restaurant via telephone, completing the reservation.

[1074] Input: Start call

[1075] Output: Reservation completion notification

[1076] Specific operation: The terminal manages the call through the communication API and notifies the user that the reservation has been completed.

[1077] Terminal: "Reservation completed."

[1078] Providing services using emotion engines

[1079] Processing Steps

[1080] Step 1:

[1081] The user launches a smartphone application and inputs, via text or voice, "I'm tired today. I want to eat something delicious."

[1082] Input: User's text or voice input

[1083] Output: Text data

[1084] Specific behavior: The user launches the application and enters, via text or voice, "I'm tired today. I want to eat something delicious."

[1085] Step 2:

[1086] The device uses a sentiment analysis API to recognize emotions from user input.

[1087] Input: Text data

[1088] Output: Sentiment analysis results

[1089] Specific operation: The terminal sends text data to the sentiment analysis API and receives the sentiment analysis results.

[1090] Terminal: "Analyzing emotions."

[1091] Step 3:

[1092] The device uses review site APIs to recommend restaurants based on user sentiment.

[1093] Input: Sentiment analysis results

[1094] Output: A list of restaurant recommendations

[1095] How it works: Based on the sentiment analysis results, the device sends a query to the review site API and receives a list of restaurants that offer a relaxing atmosphere or the user's favorite cuisine.

[1096] Terminal: "I'm looking for a relaxing restaurant."

[1097] Terminal: "How about this restaurant? (restaurant name)"

[1098] The above are the specific processing steps of the smartphone application of the present invention.

[1099] (Application example 2)

[1100] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[1101] Conventional navigation and reservation systems were unable to consider the detailed needs and emotions of users, limiting the user experience. They also lacked the ability to input users' needs by voice, recommend optimal services based on those needs, and place orders automatically. This made it difficult to provide users with the services they wanted quickly and accurately.

[1102] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1103] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting the destination based on the search results, a means for providing navigation to the set destination, a means for recognizing a user's voice input, a means for analyzing emotions from the recognized voice input, a means for recommending an optimal service based on the results of the emotion analysis, and a means for automatically placing an order for the recommended service, thereby making it possible to provide a service that takes into account the user's requests and emotions.

[1104] "Source" refers to any source that provides data or knowledge.

[1105] A "means for learning" refers to a method or device for analyzing data obtained from a source and accumulating knowledge.

[1106] "The user's detailed requests" refer to the specific conditions and wishes that the user has.

[1107] "Destination searching means" refers to a method or device for searching for a suitable location based on a user's request.

[1108] "Search Results" refers to the collection of information obtained by a search tool.

[1109] "Means for setting a destination" refers to a method or apparatus for determining a location selected from the search results as a final destination.

[1110] "Means for providing navigation" refers to a method or device for providing directions or guidance to a set destination.

[1111] "Means for recognizing speech input" refers to a method or device for converting a user's speech into text data.

[1112] "Means for analyzing emotions" refers to a method or device for analyzing a user's emotions from voice input or text data.

[1113] "Means for recommending optimal services" refers to a method or device for suggesting the most suitable service to a user based on the results of sentiment analysis.

[1114] "Means for automated ordering" refers to a method or device for completing an order for recommended services without user intervention.

[1115] The system for implementing the present invention performs a series of processes, such as recognizing a user's voice input, analyzing emotions, recommending optimal services, and automatically placing an order. A specific embodiment of this system will be described below.

[1116] System Configuration

[1117] The system consists of the following main components:

[1118] 1. Speech Recognition Module: Converts the user's voice input into text data. Specifically, it uses the speech_recognition library.

[1119] 2. Sentiment Analysis Module: Analyzes the text data obtained from the speech recognition module and analyzes the user's sentiment. Specifically, it uses the TextBlob library.

[1120] 3. Service recommendation module: Recommends the most suitable services to users based on the results of sentiment analysis.

[1121] 4. Auto-order module: Automatically places an order for the recommended services by sending order data to an API endpoint using the requests library.

[1122] Processing flow

[1123] 1. Recognizing voice input: The user speaks their request into the smartphone. The voice recognition module converts this speech into text data.

[1124] 2. Sentiment analysis: The text data obtained from the voice recognition module is passed to the sentiment analysis module to analyze the user's emotions.

[1125] 3. Service recommendation: Based on the results of sentiment analysis, the service recommendation module selects and recommends the most suitable services to the user.

[1126] 4. Executing automatic order: The automatic order module places an order for the service selected by the service recommendation module.

[1127] Hardware and Software Used

[1128] Hardware: Smartphones, servers

[1129] Software: speech_recognition library, TextBlob library, requests library

[1130] Examples

[1131] For example, if a user says, "I'm tired today. I want to eat something delicious," the system will act as follows:

[1132] 1. The speech recognition module converts the speech into text and generates text data such as, "I'm tired today. I want to eat something delicious."

[1133] 2. The sentiment analysis module analyzes this text data and recognizes that the user is tired.

[1134] 3. The service recommendation module recommends restaurants with a relaxing atmosphere.

[1135] 4. The auto-ordering module automatically places orders from the recommended restaurants.

[1136] Examples of prompt statements

[1137] Example of a prompt to be input to a generative AI model:

[1138] If a user says, "I'm tired today. I want to eat something delicious," create a Python program that performs sentiment analysis, recommends restaurants with a relaxing atmosphere, and automatically places the order.

[1139] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1140] Step 1:

[1141] The user speaks their request into the smartphone. The voice input is acquired through the smartphone's microphone. The input data is the user's voice data.

[1142] Step 2:

[1143] The device's voice recognition module converts the user's voice data into text data. Specifically, it uses the speech_recognition library to analyze the voice data and generate text data. The output data is the user's voice converted into text.

[1144] Step 3:

[1145] The sentiment analysis module of the terminal receives the text data obtained from the voice recognition module and analyzes the user's sentiment. Specifically, the sentiment of the text data is analyzed using the TextBlob library. The input data is the text data, and the output data is the result of the sentiment analysis.

[1146] Step 4:

[1147] The service recommendation module of the server selects and recommends the most suitable service to the user based on the result of the sentiment analysis. The input data is the result of the sentiment analysis, and the output data is information on the recommended service. Specifically, if the sentiment is negative, a relaxing service is recommended, and if it is positive, a popular service is recommended.

[1148] Step 5:

[1149] The server's automatic ordering module receives the recommended service information obtained from the service recommendation module and automatically places an order. Specifically, it uses the requests library to send order data for the recommended service to an API endpoint. The input data is the information on the recommended service, and the output data is a notification of order completion.

[1150] Step 6:

[1151] The terminal displays a notification of order completion to the user. The input data is the notification of order completion, and the output data is a message of order completion displayed to the user. Specifically, a message indicating that the order has been completed is displayed on the screen of the smartphone.

[1152] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1153] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1154] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1155] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1156] [Third embodiment]

[1157] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1158] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1159] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1160] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1161] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1162] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1163] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1164] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1165] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1166] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1167] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1168] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1169] "Example 1"

[1170] One embodiment of the present invention is an in-vehicle navigation system. This system uses a machine learning algorithm as a means for learning information from various information sources. Specifically, information is acquired from a database including restaurant rating information and information on the availability of parking spaces, and this information is learned. In addition, natural language processing technology is used as a means for searching for a destination based on the detailed requests of the user. Questions from the user are analyzed, and an appropriate destination is searched for based on the analysis. Furthermore, GPS technology is used as a means for setting the destination based on the search results. The optimal route is calculated based on the user's current location and the search results, and the destination is set. Finally, a voice guidance system is used as a means for providing navigation to the set destination. This allows the user to be safely guided to the destination even while driving.

[1171] "Example 2"

[1172] Another embodiment of the present invention is a smartphone application. This application has the same functions as the in-car navigation system described above, but also includes a function for making reservations based on a user's request. Specifically, when a user requests "make a reservation by phone," the system automatically calls the corresponding restaurant and makes the reservation. This function is realized using voice recognition technology and an automatic dialing system.

[1173] The process flow of each embodiment will be described below.

[1174] "Example 1"

[1175] Step 1: The system learns from various sources, such as restaurant ratings and parking availability, and uses machine learning algorithms to learn from them.

[1176] Step 2: Once the user has entered their detailed request, the system analyzes it using natural language processing technology.

[1177] Step 3: Based on the analysis results, the system searches for suitable destinations.

[1178] Step 4: Based on the search results and the user's current location, the system will use GPS technology to calculate the optimal route and set the destination.

[1179] Step 5: To provide navigation to the set destination, the system guides the user using the voice guidance system.

[1180] "Example 2"

[1181] Step 1: The smartphone application has the same functions as the in-car navigation system mentioned above.

[1182] Step 2: If the user requests to "make a reservation by phone," the system will automatically call the relevant restaurant.

[1183] Step 3: The system uses voice recognition technology and an automated dialing system to make the reservation.

[1184] Example 1

[1185] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1186] Conventional navigation systems have difficulty in searching for a destination based on the user's detailed requests, and are unable to generate appropriate answers to the user's questions. Furthermore, they are unable to effectively use the user's current location or search results when setting a destination or providing navigation. This reduces user convenience and makes it difficult to ensure safety while driving.

[1187] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1188] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for converting voice input into text, a means for analyzing the user's questions using natural language processing techniques, a means for learning data using machine learning algorithms, a means for acquiring location information, and a means for generating voice guidance. This makes it possible to search for and set a destination based on the user's detailed requests, and to be safely guided to the destination even while driving.

[1189] "Source" refers to any medium or system that provides data.

[1190] "Learning means" refers to the algorithms and techniques used to analyze data and extract patterns and knowledge.

[1191] "User specific requests" refers to the detailed requirements that users have based on their specific conditions or preferences.

[1192] "Destination search means" refers to the technologies and algorithms used to search for appropriate destinations based on the user's requirements.

[1193] "Search Results" refers to destination candidates and information obtained through a search.

[1194] "Means for setting a destination" refers to the technology or algorithm used to determine an optimal destination based on the search results.

[1195] "Means for providing navigation" refers to technologies and systems that provide route guidance to help users reach their destination.

[1196] "Means of converting voice input into text" refers to technology or systems that analyze the user's voice and convert it into text information.

[1197] "Natural language processing technology" refers to techniques and algorithms for analyzing and understanding human language.

[1198] A "machine learning algorithm" refers to an algorithm that learns patterns and knowledge from data and makes predictions and classifications.

[1199] "Means of obtaining location information" refers to technologies and systems for identifying a user's current location.

[1200] "Means for generating audio guidance" refers to technology or systems for providing audio instructions to a user.

[1201] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1202] This invention relates to an in-vehicle navigation system, which searches for a destination based on the detailed requests of the user and provides the optimal route. This system operates in cooperation with a server, a terminal, and a user.

[1203] 1. Program Generation

[1204] The server generates the program for the in-car navigation system. This program is built by combining machine learning algorithms and natural language processing technology, using libraries and APIs such as Python's Pandas, Scikit-learn, NLTK, SpaCy, Google Maps API, and Amazon Polly.

[1205] 2. Program Processing

[1206] The server processes the data in the following steps:

[1207] 1. Data acquisition and training:

[1208] The server retrieves data from a database that contains information such as restaurant ratings and whether or not parking is available. Specifically, the server reads the data using Python's Pandas library. The server then uses Scikit-learn to train the data with machine learning algorithms. For example, the server creates a model to predict which restaurants have high ratings based on restaurant ratings.

[1209] 2. User Question Analysis:

[1210] The device receives questions from users. For example, it receives a question like, "Tell me about a highly rated restaurant nearby." The device analyzes the question using natural language processing technology. Specifically, it uses Python's NLTK and SpaCy to understand the intent of the question.

[1211] 3. Set your destination:

[1212] The server calculates the optimal route based on the user's current location and the search results. Specifically, it uses the Google Maps API to obtain the user's current location and the locations of highly rated restaurants, and calculates the optimal route.

[1213] 4. Providing navigation:

[1214] The device provides navigation to the set destination. Specifically, it uses a voice guidance system to give voice instructions to the user. For example, it uses Amazon Polly to generate voice guidance such as "Turn right."

[1215] 3. Specific Examples

[1216] As a concrete example, consider the case where a user asks, "Can you tell me about some highly rated restaurants nearby?"

[1217] 1. Data acquisition and training:

[1218] The server connects to the database and obtains restaurant rating information.

[1219] The server converts the data into a DataFrame using Pandas.

[1220] The server uses Scikit-learn's RandomForestClassifier to train a model to predict highly rated restaurants.

[1221] 2. User Question Analysis:

[1222] The user speaks to the device saying, "Tell me about some highly rated restaurants nearby."

[1223] The device converts the voice input into text.

[1224] The device uses NLTK to tokenize the text and SpaCy to analyze the intent of the question.

[1225] 3. Set your destination:

[1226] The device obtains the user's current location using the GPS sensor.

[1227] The server uses the Google Maps API to calculate a route based on the user's current location and the locations of highly rated restaurants.

[1228] The server transmits the calculation result to the terminal.

[1229] 4. Providing navigation:

[1230] The terminal starts navigation based on the route information received from the server.

[1231] The device uses Amazon Polly to generate voice guidance and provide instructions to the user.

[1232] The user drives by following the voice guidance.

[1233] Example prompt:

[1234] "Tell me about a highly rated restaurant nearby."

[1235] In this way, users are safely guided to their destination while driving.

[1236] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1237] Step 1: Data acquisition and training

[1238] The server retrieves information from the database, such as restaurant ratings and whether parking is available. As input, it uses the database connection information and a SQL query. The server converts the data into a DataFrame using Python's Pandas library. The server then trains the data with a machine learning algorithm using Scikit-learn. Specifically, it uses RandomForestClassifier to create a model that predicts highly rated restaurants. As output, it obtains the trained model.

[1239] Specific behavior:

[1240] The server connects to the database and executes SQL queries to retrieve the data.

[1241] The server converts the acquired data into a Pandas DataFrame.

[1242] The server uses Scikit-learn's RandomForestClassifier to train a model to predict restaurant ratings.

[1243] Step 2: Analyze user questions

[1244] The terminal receives a question from the user. As input, the user's voice question is used. The terminal uses speech recognition technology to convert the voice input into text. The terminal then analyzes the question using natural language processing technology. Specifically, it uses Python's NLTK and SpaCy to understand the intent of the question. As output, the analyzed text data is obtained.

[1245] Specific behavior:

[1246] The user speaks to the device saying, "Tell me about some highly rated restaurants nearby."

[1247] The device converts the voice input into text.

[1248] The device uses NLTK to tokenize the text and SpaCy to analyze the intent of the question.

[1249] Step 3: Set your destination

[1250] The server calculates the optimal route based on the user's current location and the search results. The user's current location and the parsed text data are used as input. The server uses the Google Maps API to obtain the user's current location and the locations of highly rated restaurants, and calculates the optimal route. The calculated route information is obtained as output.

[1251] Specific behavior:

[1252] The device obtains the user's current location using the GPS sensor.

[1253] The server uses the Google Maps API to calculate a route based on the user's current location and the locations of highly rated restaurants.

[1254] The server transmits the calculation result to the terminal.

[1255] Step 4: Providing navigation

[1256] The device provides navigation to the set destination. The calculated route information is used as input. The device uses a voice guidance system to give voice instructions to the user. Specifically, it uses Amazon Polly to generate voice guidance such as "Turn right". The voice guidance is obtained as output.

[1257] Specific behavior:

[1258] The terminal starts navigation based on the route information received from the server.

[1259] The device uses Amazon Polly to generate voice guidance and provide instructions to the user.

[1260] The user drives by following the voice guidance.

[1261] (Application example 1)

[1262] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1263] Conventional navigation systems have limited capabilities to search for destinations based on the user's detailed requests and lack the ability to generate appropriate answers to user questions. They also lack the ability to calculate the optimal route between the current location and the destination and provide voice guidance. This often causes inconvenience to users while traveling to their destination.

[1264] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a natural language processing means for analyzing the user's question and generating an appropriate answer, a means for calculating the optimal route between the current position and the destination, and a means for providing voice guidance. This makes it possible to search for a destination based on the user's detailed requests, calculate the optimal route, and provide voice guidance.

[1265] A "source" is a source or resource that provides data or knowledge.

[1266] A "means of learning information" is a method that uses machine learning algorithms to analyze data obtained from various sources and extract patterns and knowledge.

[1267] "User specific requests" are the detailed requirements or wishes of a user based on their specific conditions or preferences.

[1268] A "means for searching for a destination" is a method or technology for searching for an appropriate destination based on a user's request.

[1269] "Search results" are the results of a search obtained by a means for searching for a destination.

[1270] A "means for setting a destination" is a method or technology for determining a user's destination based on search results.

[1271] A "means for providing navigation" is a method or technology that provides route guidance or directions to a user to reach a destination.

[1272] "Natural language processing means" is a technology for analyzing a user's questions or requests and generating appropriate answers.

[1273] A "means for calculating a route" is a method or technique for calculating an optimal path between a current location and a destination.

[1274] "Means for providing audio guidance" means the methods or techniques for converting the calculated route information into audio and providing it to the user.

[1275] As an embodiment for carrying out the present invention, the following system is considered.

[1276] First, the server has a means of learning information from various sources. This means uses machine learning algorithms to analyze data such as restaurant reviews and the availability of parking spaces, and extract patterns and knowledge. Specifically, it obtains information from a database and learns from it.

[1277] Next, there is a means to search for a destination based on the user's detailed requests. This means uses natural language processing technology to analyze the user's question and search for an appropriate destination. For example, if a user asks, "What are some highly rated restaurants nearby?", the system will analyze the question and search for an appropriate restaurant.

[1278] In addition, there is a method to set a destination based on the search results. This method calculates the optimal route based on the user's current location and the search results, and sets the destination. Specifically, it uses the Geopy library to calculate the distance between the current location and the destination.

[1279] Finally, there is a method to provide navigation to a set destination, which uses the Google Text-to-Speech (gTTS) library to convert the calculated route information into audio and provide it to the user, allowing the user to be safely guided to their destination while driving.

[1280] As a concrete example, when a user asks, "What are some highly rated restaurants nearby?" the system works as follows:

[1281] 1. Natural language processing means analyzes the user's question and generates an appropriate answer.

[1282] 2. The means for learning information retrieves restaurant rating information from the database and analyzes it.

[1283] 3. A route calculation means calculates the optimal route between the current location and the destination.

[1284] 4. A means for providing audio guidance converts the calculated route information into audio and provides it to the user.

[1285] An example of a prompt sentence might be:

[1286] User asks: "What are some highly rated restaurants near me?"

[1287] Context: "Restaurant ratings, availability of parking, etc."

[1288] This prompt can then be used to have a generative AI model parse the question and provide an appropriate answer.

[1289] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1290] Step 1:

[1291] The server receives a question from a user. The input is the user's question, e.g., "What are some highly rated restaurants nearby?", as text data. The output is text data ready for analysis.

[1292] Step 2:

[1293] The server analyzes the user's question using natural language processing means. The input is the user's question received in step 1, and the output is an appropriate search query as the analysis result. Specifically, a generative AI model is used to analyze the question and generate an appropriate answer.

[1294] Step 3:

[1295] The server uses a means of learning information to obtain data such as restaurant rating information and availability of parking spaces from the database. The input is the search query generated in step 2, and the output is the obtained rating information, availability of parking spaces, and other data. Specifically, the server executes a query against the database to obtain the required information.

[1296] Step 4:

[1297] The server calculates the optimal route between the user's current location and the destination based on the acquired information. The input is the user's current location and the destination information acquired in step 3, and the output is the calculated optimal route information. Specifically, the Geopy library is used to calculate the distance and determine the optimal route.

[1298] Step 5:

[1299] The server converts the calculated route information into voice guidance. The input is the route information calculated in step 4, and the output is an audio file. Specifically, it uses the Google Text-to-Speech (gTTS) library to convert the text information into speech.

[1300] Step 6:

[1301] The terminal provides the generated voice guidance to the user. The input is the voice file generated in step 5, and the output is voice guidance for the user. Specifically, the terminal plays the voice file and provides navigation information to the user.

[1302] Example 2

[1303] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1304] Conventional navigation systems only provide destination setting and navigation functions, and do not have a reservation function based on the user's detailed requests, so users had to make reservations manually. In addition, there was no system that analyzed voice instructions and made reservations automatically, which made it difficult for users to use the system efficiently. This meant that users had to take the time to set the destination and make reservations separately, making it difficult to use the system efficiently.

[1305] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1306] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for analyzing the user's voice instructions, a means for extracting reservation details based on the analyzed voice instructions, a means for automatically making a phone call based on the extracted reservation details, and a means for audibly conveying the reservation details over the phone. This allows the user to automatically set a destination and make a reservation simply by issuing voice instructions, greatly improving convenience.

[1307] "Source" refers to any source that provides data or knowledge.

[1308] "User" refers to any individual or entity that uses the System.

[1309] "Destination" refers to the place the user wants to reach.

[1310] "Search" refers to the act of finding specific information.

[1311] "Navigation" refers to the function of providing route guidance to a destination.

[1312] "Voice instruction" refers to an instruction given by a user to the system using voice.

[1313] "Analysis" refers to the act of examining data or information in detail to understand its meaning and structure.

[1314] "Reservation details" refers to the information necessary for making a reservation (date and time, number of people, location, etc.).

[1315] "Automatic" refers to a machine or system that operates independently, without human intervention.

[1316] "Making a call" refers to the act of using a communication device to attempt to connect to another communication device.

[1317] "Communicating by voice" refers to the act of conveying information to others using voice.

[1318] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1319] The present invention is embodied as an application for a smartphone. This application is a system that automatically sets a destination and makes a reservation based on a user's voice instruction. A specific embodiment of this system will be described below.

[1320] Hardware and Software Used

[1321] The system uses the following hardware and software:

[1322] Smartphone (device)

[1323] Speech recognition technology (Google Cloud Speech-to-Text API)

[1324] Database Management System (Firebase Firestore)

[1325] Automated dialing system (Twilio API)

[1326] Speech synthesis technology (Google Cloud Text-to-Speech API)

[1327] Data processing and calculation

[1328] The user launches the smartphone application and says, "Make a reservation for four people tomorrow night at 7 p.m." The device uses the Google Cloud Speech-to-Text API to convert the user's voice instructions into text. This conversion process occurs in real time.

[1329] Next, the device analyzes the converted text and extracts the reservation details (date, time, number of people). For example, from the text "Make a reservation for four people at 7 p.m. tomorrow," the information "tomorrow at 7 p.m." and "four people" is extracted.

[1330] The device sends a request to the server to retrieve restaurant information stored in Firebase Firestore. This request includes the user's location and preferred restaurant conditions. The server receives the request, retrieves the corresponding restaurant information from Firebase Firestore, and returns it to the device. The returned information includes the restaurant's name, phone number, address, etc.

[1331] The device uses the Twilio API to automatically call the restaurant's phone number, without any user intervention. Once the call is connected, the device uses the Google Cloud Text-to-Speech API to communicate the reservation details it extracted in advance to the restaurant. For example, it might say, "Please make a reservation for four people tomorrow night at 7 p.m."

[1332] Examples

[1333] As a concrete example, the following will show the behavior when the user instructs "Make a reservation for four people at 7pm tomorrow night."

[1334] 1. The user launches the smartphone application and gives a voice command such as, "Make a reservation for four people at 7pm tomorrow night."

[1335] 2. The device uses the Google Cloud Speech-to-Text API to convert the user's voice commands into text.

[1336] 3. The device analyzes the converted text and extracts the reservation details: "tomorrow night at 7pm" and "4 people."

[1337] 4. The device sends a request to the server to get restaurant information stored in Firebase Firestore.

[1338] 5. The server receives the request, retrieves the relevant restaurant information from Firebase Firestore, and returns it to the device.

[1339] 6. The device uses the Twilio API to automatically call the restaurant's phone number.

[1340] 7. Once the call is connected, the device uses the Google Cloud Text-to-Speech API to communicate to the restaurant, "Please make a reservation for four people tomorrow night at 7pm."

[1341] Examples of prompt statements

[1342] An example of a prompt sentence might be:

[1343] "If a user says, 'Make a reservation for 4 people tomorrow night at 7pm,' how should the application handle that?"

[1344] By inputting this prompt sentence into the generative AI model, an explanation of the processing based on the specific example above can be obtained.

[1345] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1346] Step 1:

[1347] The user issues a voice command.

[1348] The user starts the smartphone application and gives a voice command such as, "Make a reservation for four people at 7 p.m. tomorrow." This voice command becomes the input.

[1349] Step 2:

[1350] The device converts the speech to text.

[1351] The device uses voice recognition technology to convert the user's voice instructions into text. This process is performed using a voice recognition API. The input is the user's voice instructions, and the output is the instructions in text format.

[1352] Step 3:

[1353] The terminal analyzes the text and extracts the reservation details.

[1354] The terminal analyzes the converted text and extracts the reservation details (date, time, number of people). For example, from the text "Make a reservation for four people at 7 p.m. tomorrow," the information "tomorrow at 7 p.m." and "four people" is extracted. The input is instructions in text format, and the output is the reservation details data.

[1355] Step 4:

[1356] The terminal requests restaurant information from the server.

[1357] The terminal sends a request to the server to get restaurant information stored in the database management system. This request includes the user's location information and the conditions of the preferred restaurant. The input is the reservation details and the user's location information, and the output is a request for restaurant information.

[1358] Step 5:

[1359] The server returns the restaurant information.

[1360] The server receives the request, retrieves the relevant restaurant information from the database, and returns it to the terminal. The returned information includes the restaurant's name, phone number, address, etc. The input is a request for restaurant information, and the output is the restaurant information.

[1361] Step 6:

[1362] The terminal makes the call using an automatic dialing system.

[1363] The terminal then uses an automatic dialing system to automatically call the restaurant's phone number, without any user intervention: the input is the restaurant's phone number, and the output is the establishment of a telephone connection.

[1364] Step 7:

[1365] The terminal will announce the reservation details aloud.

[1366] When the phone is connected, the device uses voice synthesis technology to communicate the reservation details it extracted in advance to the restaurant by voice. For example, it may say, "Please make a reservation for four people tomorrow night at 7 p.m." The input is the reservation details data, and the output is a voice message.

[1367] (Application example 2)

[1368] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1369] Conventional navigation and reservation systems required users to operate them manually, which made them less convenient. In addition, they lacked the functionality to utilize the user's past behavioral history to make optimal suggestions, making it difficult to respond to user requests quickly and accurately. Furthermore, there was a need to effectively combine voice recognition technology and an automatic dialing system to improve the user experience.

[1370] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1371] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for understanding the user's requests using voice recognition technology, a means for making reservations using an automatic dialing system, a means for learning the user's past behavior history and making optimal suggestions, and a means for providing voice feedback. This allows the user to search for and make reservations at a destination using only voice instructions, and to receive optimal suggestions based on the user's past behavior history.

[1372] "Source" refers to any source that provides data or knowledge.

[1373] "Means of learning" refers to the methods and techniques used to collect, analyze, and understand information.

[1374] "The user's detailed requests" refer to the specific conditions and wishes that the user has.

[1375] "Destination search means" refers to methods and technologies for searching for appropriate locations based on a user's requirements.

[1376] "Search Results" refers to the information or data obtained through a Search.

[1377] "Means for setting a destination" refers to a method or technology for determining a final destination based on a search result.

[1378] "Means for providing navigation" refers to methods and technologies for providing directions to a set destination.

[1379] "Voice recognition technology" refers to the technology that analyzes voice and converts it into text or commands.

[1380] "Automatic dialing system" refers to a system that automatically dials designated telephone numbers.

[1381] "Means for making reservations" refers to methods or technologies for automatically making reservations for specified locations or services.

[1382] "Past behavioral history" refers to the record of actions and choices a user has previously made.

[1383] "Means of making optimal suggestions" refers to methods and technologies for presenting the most suitable options to users based on their past behavioral history and current needs.

[1384] "Means for providing audio feedback" refers to methods or technologies that allow the system to communicate information or results to the user using audio.

[1385] A system for implementing the present invention has the following configuration: A server includes means for learning information from various information sources, means for searching for a destination based on a user's detailed requests, means for setting a destination based on the search results, means for providing navigation to the set destination, means for understanding a user's requests using voice recognition technology, means for making reservations using an automatic dialing system, means for learning a user's past behavior history and making optimal suggestions, and means for providing voice feedback.

[1386] Explanation of program processing

[1387] Hardware and Software

[1388] The system uses the smartphone's microphone and speaker as hardware, the speech_recognition library for voice recognition, the pyttsx3 library for voice feedback, and the requests library for data communication.

[1389] Data processing and calculation

[1390] 1. Speech recognition: When the user speaks their request into the smartphone's microphone, the speech_recognition library converts the speech into text.

[1391] 2. Request Analysis: Analyze the converted text to understand the user's request. For example, analyze the request "order a pizza."

[1392] 3. Execute the booking: Based on the request, send the order to the food delivery API using the requests library. If the order is successful, provide audio feedback using the pyttsx3 library.

[1393] 4. Learning from behavioral history: Learns the user's past behavioral history and makes optimal suggestions for future visits.

[1394] Examples

[1395] For example, if a user says "order a pizza," the system would do the following:

[1396] 1. Capture the user's voice with the microphone.

[1397] 2. Convert speech to text and get the text "Order pizza."

[1398] 3. Analyze the text to identify "pizza" orders.

[1399] 4. Send an order for "Pizza" to the food delivery API.

[1400] 5. If the order is successful, provide audio feedback saying "Your pizza order has been completed."

[1401] Examples of prompt statements

[1402] "When a user says, 'Order a pizza,' please create a program that converts the speech to text using speech recognition technology and sends the order to the food delivery API. If the order is successful, please provide feedback by voice saying, 'The pizza order has been completed.'"

[1403] In this way, users can search for and book destinations using only voice commands, and receive optimal suggestions based on their past travel history.

[1404] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1405] Step 1:

[1406] The user speaks their request into the smartphone's microphone.

[1407] Input: User's speech.

[1408] Output: Audio data captured by the microphone.

[1409] What happens: When a user says "order a pizza," the smartphone's microphone captures the audio.

[1410] Step 2:

[1411] The device converts the voice data into text using the speech_recognition library.

[1412] Input: Audio data captured by the microphone.

[1413] Output: Text data (e.g. "order a pizza").

[1414] What happens: The speech recognition engine analyzes the voice data and converts it into corresponding text.

[1415] Step 3:

[1416] The device analyzes the text data and understands the user's requests.

[1417] Input: Text data (e.g. "order a pizza").

[1418] Output: Parsed request (e.g. "order pizza").

[1419] What happens: The text analysis engine analyzes the text "order a pizza" and identifies the request "order a pizza."

[1420] Step 4:

[1421] The device uses the requests library to send an order to the food delivery API.

[1422] Input: Parsed request (e.g. "order pizza").

[1423] Output: The order request to the API and its response.

[1424] Specific behavior: Send a request to the food delivery API containing "order pizza" and receive a response indicating whether the order was successful.

[1425] Step 5:

[1426] The device provides audio feedback using the pyttsx3 library.

[1427] Input: Response from the API (e.g. order successful).

[1428] Output: Audio feedback (e.g. "Your pizza order is complete").

[1429] What happens: The speech synthesis engine generates the message "Your pizza order has been completed" and relays it to the user through the smartphone speaker.

[1430] Step 6:

[1431] The server learns the user's past behavioral history and makes optimal suggestions for future visits.

[1432] Input: User behavioral history data.

[1433] Output: Proposed data based on the learning results.

[1434] What happens: The server analyzes the user's past ordering history and generates data to suggest the best restaurant and menu for the next time they order.

[1435] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1436] "Example 1"

[1437] One embodiment of the present invention is a navigation system incorporating an emotion engine. This system recognizes the user's emotions from the tone of voice, facial expressions, choice of words, etc., and searches for a destination taking these emotions into consideration. For example, if a user requests "find a nice restaurant nearby," the system will recommend lively restaurants if the user's tone of voice is cheerful, and quiet restaurants if the user's tone of voice is sad.

[1438] "Example 2"

[1439] Also, a smartphone application incorporating the emotion engine is one embodiment of the present invention. This application recognizes emotions from the user's text input or voice input and provides services taking them into consideration. For example, if the user inputs "I'm tired today. I want to eat something delicious," the application recognizes the emotion and recommends restaurants with a relaxing atmosphere and restaurants that serve the user's favorite dishes.

[1440] The process flow of each embodiment will be described below.

[1441] "Example 1"

[1442] Step 1: The user issues a verbal request to the navigation system.

[1443] Step 2: The emotion engine recognizes the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[1444] Step 3: Search for destinations taking into account the emotions recognized by the system.

[1445] Step 4: The system presents the search results to the user.

[1446] "Example 2"

[1447] Step 1: The user submits a request to a smartphone application via text input or voice input.

[1448] Step 2: The emotion engine recognizes emotions from the user's text or voice input.

[1449] Step 3: The application takes into account the emotions it recognizes and provides a service.

[1450] Step 4: The application presents the results of the services it provides to the user.

[1451] Example 1

[1452] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1453] Conventional navigation systems have difficulty in recommending destinations that take into account the user's detailed requests and feelings, and often do not adequately calculate optimal routes or analyze voice input. As a result, there are issues with lowering user satisfaction and making it difficult to ensure safety while driving.

[1454] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1455] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for recognizing the user's emotions and recommending a destination based thereon, a means for calculating an optimal route based on the user's current location and the search results, and a means for analyzing voice input and understanding the user's requests. This makes it possible to recommend a destination taking into account the user's detailed requests and emotions, and also calculates an optimal route and analyzes voice input, thereby improving user satisfaction and ensuring safety during driving.

[1456] An "information source" is a source location or system that provides data, such as a database that contains information such as restaurant ratings and the availability of parking spaces.

[1457] "Means of learning" refers to the process of using machine learning algorithms to analyze data based on acquired information and extract patterns and features.

[1458] "User specific requests" refers to specific requests made by users based on their particular conditions or preferences.

[1459] "Destination search means" refers to the process of searching a database for suitable destinations based on a user's request.

[1460] "Search Results" refers to destination information retrieved based on a User's request.

[1461] "Means for setting a destination" refers to the process of determining the user's destination based on the search results and registering that information in the system.

[1462] "Means for providing navigation" refers to a system or method for guiding a user to routes to a set destination.

[1463] "Means of recognizing emotions" refers to the process of analyzing and understanding the user's emotions from their tone of voice, facial expressions, choice of words, etc.

[1464] "Method of calculating optimal route" refers to the process of calculating the most efficient route based on the user's current location and destination.

[1465] "Means for analyzing voice input" refers to the technologies and methods used to convert a user's speech into text and understand the content of that text.

[1466] The present invention relates to an in-vehicle navigation system that provides navigation by recommending optimal destinations while taking into consideration the detailed needs and feelings of the user. This system operates in cooperation with a server, a terminal, and a user.

[1467] Acquiring information and learning

[1468] The server obtains information from a database that contains information such as restaurant ratings and whether or not parking is available. This database is accessed, for example, through a general map service API or a rating information providing service API. The server uses a machine learning algorithm (for example, TensorFlow or PyTorch) to learn from the obtained information. Specifically, it extracts features such as the restaurant's ratings and whether or not parking is available, and inputs them into a model for learning.

[1469] User demand analysis

[1470] The device receives questions and requests from the user as voice input. For example, the user says, "Find a nice restaurant nearby." The device converts the voice into text using voice recognition technology (e.g., a general voice recognition API). The converted text is analyzed using natural language processing technology (e.g., a general natural language processing model). As a result of the analysis, the device understands that the user's request is intended to "find a nice restaurant."

[1471] Search for a destination

[1472] The server searches the database for an appropriate destination based on the analysis results. For example, it searches the database using the keyword "delicious restaurant." The server uses an emotion engine to recognize emotions from the user's tone of voice, facial expressions, and choice of words. For example, if the user's tone of voice is cheerful, it will recommend a lively restaurant, and if the tone of voice is sad, it will recommend a quiet restaurant.

[1473] Route calculation and setting

[1474] The server calculates the optimal route based on the user's current location (GPS information) and the search results. For example, the route is calculated using a general map service API. The calculated route information is sent to the device, and the destination is set.

[1475] Providing navigation

[1476] The device provides navigation to the set destination using a voice guidance system (e.g., a general navigation app). Specifically, instructions such as "Turn right at the next intersection" are given by voice. The user is safely guided to the destination while driving.

[1477] Examples

[1478] For example, if a user voice-inputs, "Find a nice restaurant nearby," the following specific actions will be taken:

[1479] 1. The device receives the user's voice and converts it into text using a speech recognition API.

[1480] 2. The device inputs the converted text into a natural language processing model and analyzes the intent of "find a good restaurant."

[1481] 3. The server receives the analysis results and searches the database for "good restaurants."

[1482] 4. The server analyzes the tone of the user's voice and recommends lively restaurants if the tone is cheerful, and quiet restaurants if the tone is depressed.

[1483] 5. The server uses the map service API to calculate the optimal route from the user's current location to the destination.

[1484] 6. The device receives the calculated route information and provides navigation to the user using a voice guidance system.

[1485] Examples of prompt statements

[1486] “If a user speaks to you to find a good restaurant nearby, design a system that recommends lively restaurants if the user’s voice tone is cheerful, and quiet restaurants if the user’s voice tone is depressed.”

[1487] In this way, optimal navigation is provided according to the user's emotions and needs.

[1488] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1489] Step 1:

[1490] Acquiring information and learning

[1491] The server retrieves information from a database that includes information such as restaurant ratings and whether parking is available.

[1492] Input: Information from a database (e.g. restaurant ratings, availability of parking)

[1493] Data processing: The acquired information is trained using a machine learning algorithm (e.g. TensorFlow, PyTorch). Specifically, features such as restaurant ratings and the availability of parking spaces are extracted and input into the model for training.

[1494] Output: The trained model

[1495] Step 2:

[1496] User demand analysis

[1497] The device receives questions and requests from the user as voice input.

[1498] Input: User's voice input (e.g. "Find some good restaurants near me")

[1499] Data processing: Convert speech to text using speech recognition technology (e.g., speech recognition API). Analyze the converted text using natural language processing technology (e.g., natural language processing model) to understand the user's request.

[1500] Output: Analysis results (e.g., the intent "find a good restaurant")

[1501] Step 3:

[1502] Search for a destination

[1503] The server searches a database for suitable destinations based on the analysis results.

[1504] Input: Analysis result (e.g., the intent "find a good restaurant")

[1505] Data processing: Search for "good restaurants" from a database. Using an emotion engine, recognize emotions from the user's tone of voice, facial expressions, and choice of words, and recommend destinations based on that.

[1506] Output: Search results (e.g. busy restaurants, quiet restaurants)

[1507] Step 4:

[1508] Route calculation and setting

[1509] The server calculates the optimal route based on the user's current location (GPS information) and the search results.

[1510] Input: User's current location (GPS information), search results

[1511] Data processing: Calculate the optimal route using a map service API.

[1512] Output: Calculated route information

[1513] Step 5:

[1514] Providing navigation

[1515] The terminal provides navigation to the set destination using a voice guidance system.

[1516] Input: Calculated route information

[1517] Data processing: Using a voice guidance system, instructions such as "Turn right at the next intersection" are given to the user.

[1518] Output: Voice guidance

[1519] What it does: Users are safely guided to their destination while driving.

[1520] (Application example 1)

[1521] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1522] Conventional navigation systems suggest destinations without considering the user's emotional state, making it difficult to provide destinations that suit the user's current mood and situation. In addition, they cannot provide optimal routes based on the user's emotions, making it difficult to improve user satisfaction.

[1523] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1524] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for recognizing the user's emotions, and a means for suggesting a destination based on the emotion recognition results, thereby enabling the suggestion of an optimal destination and navigation according to the user's emotional state.

[1525] "Source" refers to the medium or system that provides data or information.

[1526] A "means of learning" is a method for analyzing information and acquiring patterns and knowledge using machine learning algorithms.

[1527] "The user's specific requests" refer to the specific conditions and wishes of the user.

[1528] A "means for searching for a destination" is a method for searching for an appropriate destination based on a user's request.

[1529] "Search results" refers to destination candidates obtained through a search.

[1530] A "means for setting a destination" is a method for selecting and setting an optimal destination from the search results.

[1531] A "means for providing navigation" is a method for providing directions to a set destination.

[1532] A "means for recognizing emotions" is a method for analyzing and recognizing emotions from a user's tone of voice, facial expressions, etc.

[1533] "Emotion recognition result" refers to the user's emotional state obtained by the emotion recognition means.

[1534] The "means for suggesting destinations" is a method for recommending suitable destinations to a user based on emotion recognition results.

[1535] A system for implementing this invention has the following configuration. First, the server is equipped with a means for learning information from various information sources. Specifically, a machine learning algorithm (e.g., TensorFlow or PyTorch) is used to analyze data such as restaurant rating information and the availability of parking spaces, and to acquire patterns and knowledge.

[1536] Next, it has a means to search for a destination based on the user's detailed requests. This means uses natural language processing technology (e.g., BERT and GPT-3) to analyze the user's voice commands and understand the user's requests. For example, if the user says, "Find a good restaurant nearby," the request is analyzed and an appropriate destination is searched for.

[1537] Furthermore, it has a means to set a destination based on the search results. The server selects and sets the most suitable destination from the candidates obtained by the search. At this time, it takes into account the user's current location and calculates the optimal route using GPS technology.

[1538] The device also includes a means for providing navigation to the set destination. The server uses a voice guidance system to guide the user to the destination. This allows the user to reach the destination safely even while driving.

[1539] In addition, the device has a means to recognize the user's emotions. The server uses a microphone and camera to collect the user's tone of voice and facial expressions in real time, and then uses machine learning algorithms to analyze the emotions. For example, TensorFlow is used to analyze the voice data and classify the user's emotional state.

[1540] The system also has a means to suggest destinations based on emotion recognition results. The server recommends a destination that best suits the user's current emotional state based on the emotion recognition results. For example, if the user speaks in a bright tone, it recommends a lively restaurant, and if the user speaks in a calm tone, it recommends a quiet cafe.

[1541] For example, if a user says, "I want to relax today, so please find me a quiet cafe," the system will recognize that the user's tone of voice is calm and recommend a quiet cafe. It will then use GPS to calculate the optimal route and guide the user to their destination with voice guidance.

[1542] Examples of prompts include the following:

[1543] "Develop a navigation system that analyzes the user's tone of voice and facial expressions, recognizes emotions, and suggests the best destination. If the user says, 'I want to relax today, so find a quiet cafe,' the system will recognize that the user's tone of voice is calm and recommend a quiet cafe. It will then use GPS to calculate the best route and guide the user to their destination with voice guidance."

[1544] In this way, it becomes possible to suggest and navigate to optimal destinations according to the user's emotional state.

[1545] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1546] Step 1:

[1547] The server uses a microphone and a camera to collect the user's tone of voice and facial expressions in real time. The input is the user's voice and video data, and the output is pre-processed voice and video data. Specifically, it performs noise removal and facial recognition, and converts the data into a format suitable for analysis.

[1548] Step 2:

[1549] The server inputs the preprocessed audio and video data into a machine learning algorithm (e.g., TensorFlow) to recognize the user's emotions. The input is the preprocessed audio and video data, and the output is the user's emotional state (e.g., joy, sadness, anger, etc.). Specifically, the tone of the voice and facial features are analyzed to classify the emotion.

[1550] Step 3:

[1551] The server analyzes the user's voice command using natural language processing technology (e.g., BERT or GPT-3). The input is the user's voice command, and the output is text data that represents the user's request. Specifically, it analyzes commands such as "Find a good restaurant nearby" and extracts the user's request.

[1552] Step 4:

[1553] The server searches the database for an appropriate destination based on the emotion recognition result and the user's request. The input is the emotion recognition result and the user's request text, and the output is a list of destination candidates. Specifically, if the user is speaking in a bright tone, the server searches the database for lively restaurants.

[1554] Step 5:

[1555] The server selects and sets the best destination from the search results. The input is a list of candidate destinations, and the output is the set destination. Specifically, the server selects the most suitable destination taking into account the user's current location.

[1556] Step 6:

[1557] The server uses GPS technology to calculate the optimal route. The input is the user's current location and the set destination, and the output is the optimal route information. Specifically, it uses map data to calculate the shortest route and a route that takes into account traffic conditions.

[1558] Step 7:

[1559] The server uses a voice guidance system to guide the user to the destination. The input is optimal route information, and the output is voice guidance. Specifically, the server provides the user with voice instructions such as "Turn right at the next intersection."

[1560] In this way, optimal destination suggestions and navigation are realized according to the user's emotional state.

[1561] Example 2

[1562] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1563] Conventional navigation systems have difficulty in providing services that take into account the detailed requests and emotions of users. In addition, when a user requests a reservation by voice, the system lacks the function to automatically make a phone call to complete the reservation. This means that users have to make the reservation manually, which is inconvenient. Furthermore, the system does not recommend restaurants based on the user's emotions, which makes it difficult to increase user satisfaction.

[1564] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1565] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for recognizing the user's voice, a means for searching for a restaurant based on the voice recognition results, a means for automatically making a phone call, a means for completing a reservation, a means for recognizing the user's emotions, and a means for recommending a restaurant based on the emotions. This makes it possible to provide a service that takes into account the user's detailed requests and emotions, and to complete a reservation by automatically making a phone call in response to a reservation request made by voice. It is also possible to recommend restaurants based on the user's emotions, thereby increasing user satisfaction.

[1566] "Source" refers to any source that provides data or knowledge.

[1567] "Means of learning" refers to methods and techniques for analyzing data obtained from sources and accumulating knowledge.

[1568] "The user's detailed requests" refer to the specific conditions and wishes that the user has.

[1569] "Destination search means" refers to methods and technologies for searching for appropriate locations based on a user's requirements.

[1570] "Search Results" refers to the information or data obtained through a Search.

[1571] "Destination setting solution" refers to the methods or technologies that allow a user to determine where a user should visit based on search results.

[1572] "Means for providing navigation" refers to methods or technologies for providing directions or guidance to a set destination.

[1573] "Means for recognizing voice" refers to methods or technologies for analyzing a user's voice and converting it into text data.

[1574] "Means for searching for restaurants" refers to methods and technologies for searching for appropriate restaurants based on the results of voice recognition.

[1575] "Means for making automated calls" refers to methods or technologies that allow a system to automatically make calls.

[1576] "Means of completing a reservation" refers to the methods and technologies used to complete the reservation process over the telephone.

[1577] "Means for recognizing emotions" refers to methods or technologies for analyzing emotions from user input or voice.

[1578] "Restaurant recommendation means" refers to methods or techniques for suggesting suitable dining options based on recognized sentiment.

[1579] The present invention is a system for providing services that take into account the detailed requests and emotions of users. This system is implemented as a smartphone application, and performs destination search, reservation, navigation, and emotion-based restaurant recommendations based on the user's voice and text input.

[1580] Hardware and Software Used

[1581] Hardware: Smartphone (iPhone, Android device, etc.)

[1582] software:

[1583] Speech Recognition Technology: Uses speech recognition APIs to convert speech to text.

[1584] Restaurant search: Uses the Maps API to search for restaurants near the user's current location.

[1585] Automated Dialing System: Uses the Communications API to automatically place calls.

[1586] Emotion Recognition Technology: Use sentiment analysis API to analyze emotions from user input.

[1587] Restaurant Recommendation: Use review site APIs to recommend restaurants based on sentiment.

[1588] Specific processing of the system

[1589] Reservation function based on user requests

[1590] 1. The user launches the application on their smartphone and issues the voice command, "Make a reservation by phone."

[1591] 2. The device uses a speech recognition API to convert the user's speech into text.

[1592] 3. The device uses the map API to search for restaurants near the user's current location.

[1593] 4. The device uses a communications API to automatically call the restaurant selected from the search results.

[1594] 5. The terminal then transmits the reservation details to the restaurant via telephone, completing the reservation.

[1595] Examples:

[1596] The user says, "Please call me to make a reservation."

[1597] The device uses a speech recognition API to convert speech to text.

[1598] Device: "Recognizing voice."

[1599] The device uses a map API to search for nearby restaurants.

[1600] Device: "Searching for nearby restaurants."

[1601] The device will automatically call the restaurant using a communications API.

[1602] Terminal: "Calling restaurant."

[1603] The terminal will notify the user that the reservation is complete.

[1604] Terminal: "Reservation completed."

[1605] Providing services using emotion engines

[1606] 1. The user launches a smartphone application and inputs via text or voice, "I'm tired today. I want to eat something delicious."

[1607] 2. The device uses the sentiment analysis API to recognize emotions from the user's input.

[1608] 3. The device uses review site APIs to recommend restaurants based on the user's sentiment.

[1609] Examples:

[1610] The user types, "I'm tired today. I want to eat something delicious."

[1611] The device uses a sentiment analysis API to recognize emotions.

[1612] Terminal: "Analyzing emotions."

[1613] The device uses review site APIs to recommend relaxing restaurants.

[1614] Terminal: "I'm looking for a relaxing restaurant."

[1615] Terminal: "How about this restaurant? (restaurant name)"

[1616] The above is a specific embodiment of the smartphone application of the present invention. This system makes it possible to provide services that take into account the detailed requests and emotions of the user, and can automatically make a phone call to complete a reservation in response to a voice reservation request. It can also recommend restaurants based on the user's emotions, thereby increasing user satisfaction.

[1617] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1618] Reservation function based on user requests

[1619] Processing Steps

[1620] Step 1:

[1621] The user launches the smartphone application and gives the voice command, "Make a reservation by phone."

[1622] Input: User's voice command

[1623] Output: Audio data

[1624] Specific operation: The user launches the application and gives the command "Make a reservation by phone" via voice.

[1625] Step 2:

[1626] The device uses a speech recognition API to convert the user's speech into text.

[1627] Input: Audio data

[1628] Output: Text data

[1629] Specific operation: The terminal sends voice data to the voice recognition API and receives text data.

[1630] Device: "Recognizing voice."

[1631] Step 3:

[1632] The device uses a map API to search for restaurants near the user's current location.

[1633] Input: Text data, user's current location information

[1634] Output: A list of restaurants

[1635] Specific operation: The device obtains the user's current location information and sends a query to the map API to receive a list of restaurants.

[1636] Device: "Searching for nearby restaurants."

[1637] Step 4:

[1638] The device uses a communications API to automatically call the restaurant selected from the search results.

[1639] Input: List of restaurants

[1640] Output:Call start

[1641] Specific operation: The terminal sends the phone number and call contents to the communication API and starts the call.

[1642] Terminal: "Calling restaurant."

[1643] Step 5:

[1644] The terminal then transmits the reservation details to the restaurant via telephone, completing the reservation.

[1645] Input: Start call

[1646] Output: Reservation completion notification

[1647] Specific operation: The terminal manages the call through the communication API and notifies the user that the reservation has been completed.

[1648] Terminal: "Reservation completed."

[1649] Providing services using emotion engines

[1650] Processing Steps

[1651] Step 1:

[1652] The user launches a smartphone application and inputs, via text or voice, "I'm tired today. I want to eat something delicious."

[1653] Input: User's text or voice input

[1654] Output: Text data

[1655] Specific behavior: The user launches the application and enters, via text or voice, "I'm tired today. I want to eat something delicious."

[1656] Step 2:

[1657] The device uses a sentiment analysis API to recognize emotions from user input.

[1658] Input: Text data

[1659] Output: Sentiment analysis results

[1660] Specific operation: The terminal sends text data to the sentiment analysis API and receives the sentiment analysis results.

[1661] Terminal: "Analyzing emotions."

[1662] Step 3:

[1663] The device uses review site APIs to recommend restaurants based on the user's sentiment.

[1664] Input: Sentiment analysis results

[1665] Output: A list of restaurant recommendations

[1666] Specific operation: Based on the results of the sentiment analysis, the device sends a query to the review site API and receives a list of restaurants that have a relaxing atmosphere or serve the user's favorite cuisine.

[1667] Terminal: "I'm looking for a relaxing restaurant."

[1668] Terminal: "How about this restaurant? (restaurant name)"

[1669] The above are the specific processing steps of the smartphone application of the present invention.

[1670] (Application example 2)

[1671] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1672] Conventional navigation and reservation systems were unable to consider the detailed needs and feelings of users, limiting the user experience. In addition, they lacked the functionality to recommend optimal services based on users' voice input and automatically place orders. This made it difficult to provide the services users wanted quickly and accurately.

[1673] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1674] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting the destination based on the search results, a means for providing navigation to the set destination, a means for recognizing a user's voice input, a means for analyzing emotions from the recognized voice input, a means for recommending an optimal service based on the results of the emotion analysis, and a means for automatically placing an order for the recommended service, thereby making it possible to provide a service that takes into account the user's requests and emotions.

[1675] "Source" refers to any source that provides data or knowledge.

[1676] A "means for learning" refers to a method or device for analyzing data obtained from a source and accumulating knowledge.

[1677] "The user's detailed requests" refer to the specific conditions and wishes that the user has.

[1678] "Destination searching means" refers to a method or device for searching for a suitable location based on a user's request.

[1679] "Search Results" refers to the collection of information obtained by a search tool.

[1680] "Means for setting a destination" refers to a method or apparatus for determining a location selected from the search results as a final destination.

[1681] "Means for providing navigation" refers to a method or device for providing directions or guidance to a set destination.

[1682] "Means for recognizing speech input" refers to a method or device for converting a user's speech into text data.

[1683] "Means for analyzing emotions" refers to a method or device for analyzing a user's emotions from voice input or text data.

[1684] "Means for recommending optimal services" refers to a method or device for suggesting the most suitable service to a user based on the results of sentiment analysis.

[1685] "Means for automated ordering" refers to a method or device for completing an order for recommended services without user intervention.

[1686] The system for implementing the present invention performs a series of processes, such as recognizing a user's voice input, analyzing emotions, recommending optimal services, and automatically placing an order. A specific embodiment of this system will be described below.

[1687] System Configuration

[1688] The system consists of the following main components:

[1689] 1. Speech Recognition Module: Converts the user's voice input into text data. Specifically, it uses the speech_recognition library.

[1690] 2. Sentiment Analysis Module: Analyzes the text data obtained from the speech recognition module and analyzes the user's sentiment. Specifically, it uses the TextBlob library.

[1691] 3. Service recommendation module: Recommends the most suitable services to users based on the results of sentiment analysis.

[1692] 4. Auto-order module: Automatically places an order for the recommended services by sending order data to an API endpoint using the requests library.

[1693] Processing flow

[1694] 1. Recognizing voice input: The user speaks their request into the smartphone. The voice recognition module converts this speech into text data.

[1695] 2. Sentiment analysis: The text data obtained from the voice recognition module is passed to the sentiment analysis module to analyze the user's emotions.

[1696] 3. Service recommendation: Based on the results of sentiment analysis, the service recommendation module selects and recommends the most suitable services to the user.

[1697] 4. Executing automatic order: The automatic order module places an order for the service selected by the service recommendation module.

[1698] Hardware and Software Used

[1699] Hardware: Smartphones, servers

[1700] Software: speech_recognition library, TextBlob library, requests library

[1701] Examples

[1702] For example, if a user says, "I'm tired today. I want to eat something delicious," the system will act as follows:

[1703] 1. The speech recognition module converts the speech into text and generates text data such as, "I'm tired today. I want to eat something delicious."

[1704] 2. The sentiment analysis module analyzes this text data and recognizes that the user is tired.

[1705] 3. The service recommendation module recommends restaurants with a relaxing atmosphere.

[1706] 4. The auto-ordering module automatically places orders from the recommended restaurants.

[1707] Examples of prompt statements

[1708] Example of a prompt to be input to a generative AI model:

[1709] "If a user says, 'I'm tired today. I want to eat something delicious,' create a Python program that performs sentiment analysis, recommends restaurants with a relaxing atmosphere, and automatically places the order."

[1710] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1711] Step 1:

[1712] The user speaks his / her request into the smartphone. The voice input is acquired through the smartphone's microphone. The input data is the user's voice data.

[1713] Step 2:

[1714] The device's voice recognition module converts the user's voice data into text data. Specifically, it uses the speech_recognition library to analyze the voice data and generate text data. The output data is the user's voice converted into text.

[1715] Step 3:

[1716] The sentiment analysis module of the terminal receives the text data obtained from the voice recognition module and analyzes the user's sentiment. Specifically, the sentiment of the text data is analyzed using the TextBlob library. The input data is the text data, and the output data is the result of the sentiment analysis.

[1717] Step 4:

[1718] The service recommendation module of the server selects and recommends the most suitable service to the user based on the result of the sentiment analysis. The input data is the result of the sentiment analysis, and the output data is information on the recommended service. Specifically, if the sentiment is negative, a relaxing service is recommended, and if it is positive, a popular service is recommended.

[1719] Step 5:

[1720] The server's automatic ordering module receives the recommended service information obtained from the service recommendation module and automatically places an order. Specifically, it uses the requests library to send order data for the recommended service to an API endpoint. The input data is the information on the recommended service, and the output data is a notification of order completion.

[1721] Step 6:

[1722] The terminal displays a notification of order completion to the user. The input data is the notification of order completion, and the output data is a message of order completion displayed to the user. Specifically, a message indicating that the order has been completed is displayed on the screen of the smartphone.

[1723] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1724] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1725] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1726] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1727] [Fourth embodiment]

[1728] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1729] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1730] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure.

[1731] The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1732] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1733] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1734] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1735] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1736] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1737] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1738] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1739] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1740] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1741] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1742] "Example 1"

[1743] One embodiment of the present invention is an in-vehicle navigation system. This system uses a machine learning algorithm as a means for learning information from various information sources. Specifically, information is acquired from a database including restaurant rating information and information on the availability of parking spaces, and this information is learned. In addition, natural language processing technology is used as a means for searching for a destination based on the detailed requests of the user. Questions from the user are analyzed, and an appropriate destination is searched for based on the analysis. Furthermore, GPS technology is used as a means for setting the destination based on the search results. The optimal route is calculated based on the user's current location and the search results, and the destination is set. Finally, a voice guidance system is used as a means for providing navigation to the set destination. This allows the user to be safely guided to the destination even while driving.

[1744] "Example 2"

[1745] Another embodiment of the present invention is a smartphone application. This application has the same functions as the in-car navigation system described above, but also includes a function for making reservations based on a user's request. Specifically, when a user requests "make a reservation by phone," the system automatically calls the corresponding restaurant and makes the reservation. This function is realized using voice recognition technology and an automatic dialing system.

[1746] The process flow of each embodiment will be described below.

[1747] "Example 1"

[1748] Step 1: The system learns from various sources, such as restaurant ratings and parking availability, and uses machine learning algorithms to learn from them.

[1749] Step 2: Once the user has entered their detailed request, the system analyzes it using natural language processing technology.

[1750] Step 3: Based on the analysis results, the system searches for suitable destinations.

[1751] Step 4: Based on the search results and the user's current location, the system will use GPS technology to calculate the optimal route and set the destination.

[1752] Step 5: To provide navigation to the set destination, the system guides the user using the voice guidance system.

[1753] "Example 2"

[1754] Step 1: The smartphone application has the same functions as the in-car navigation system mentioned above.

[1755] Step 2: If the user requests to "make a reservation by phone," the system will automatically call the relevant restaurant.

[1756] Step 3: The system uses voice recognition technology and an automated dialing system to make the reservation.

[1757] Example 1

[1758] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1759] Conventional navigation systems have difficulty in searching for a destination based on the user's detailed requests, and are unable to generate appropriate answers to the user's questions. Furthermore, they are unable to effectively use the user's current location or search results when setting a destination or providing navigation. This reduces user convenience and makes it difficult to ensure safety while driving.

[1760] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1761] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for converting voice input into text, a means for analyzing the user's questions using natural language processing techniques, a means for learning data using machine learning algorithms, a means for acquiring location information, and a means for generating voice guidance. This makes it possible to search for and set a destination based on the user's detailed requests, and to be safely guided to the destination even while driving.

[1762] "Source" refers to any medium or system that provides data.

[1763] "Learning means" refers to the algorithms and techniques used to analyze data and extract patterns and knowledge.

[1764] "User specific requests" refers to the detailed requirements that users have based on their specific conditions or preferences.

[1765] "Destination search means" refers to the technologies and algorithms used to search for appropriate destinations based on the user's requirements.

[1766] "Search Results" refers to destination candidates and information obtained through a search.

[1767] "Means for setting a destination" refers to the technology or algorithm used to determine an optimal destination based on the search results.

[1768] "Means for providing navigation" refers to technologies and systems that provide route guidance to help users reach their destination.

[1769] "Means of converting voice input into text" refers to technology or systems that analyze the user's voice and convert it into text information.

[1770] "Natural language processing technology" refers to techniques and algorithms for analyzing and understanding human language.

[1771] A "machine learning algorithm" refers to an algorithm that learns patterns and knowledge from data and makes predictions and classifications.

[1772] "Means of obtaining location information" refers to technologies and systems for identifying a user's current location.

[1773] "Means for generating audio guidance" refers to technology or systems for providing audio instructions to a user.

[1774] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1775] This invention relates to an in-vehicle navigation system, which searches for a destination based on the detailed requests of the user and provides the optimal route. This system operates in cooperation with a server, a terminal, and a user.

[1776] 1. Program Generation

[1777] The server generates the program for the in-car navigation system. This program is built by combining machine learning algorithms and natural language processing technology, using libraries and APIs such as Python's Pandas, Scikit-learn, NLTK, SpaCy, Google Maps API, and Amazon Polly.

[1778] 2. Program Processing

[1779] The server processes the data in the following steps:

[1780] 1. Data acquisition and training:

[1781] The server retrieves data from a database that contains information such as restaurant ratings and whether or not parking is available. Specifically, the server reads the data using Python's Pandas library. The server then uses Scikit-learn to train the data with machine learning algorithms. For example, the server creates a model to predict which restaurants have high ratings based on restaurant ratings.

[1782] 2. User Question Analysis:

[1783] The device receives questions from users. For example, it receives a question like, "Tell me about a highly rated restaurant nearby." The device analyzes the question using natural language processing technology. Specifically, it uses Python's NLTK and SpaCy to understand the intent of the question.

[1784] 3. Set your destination:

[1785] The server calculates the optimal route based on the user's current location and the search results. Specifically, it uses the Google Maps API to obtain the user's current location and the locations of highly rated restaurants, and calculates the optimal route.

[1786] 4. Providing navigation:

[1787] The device provides navigation to the set destination. Specifically, it uses a voice guidance system to give voice instructions to the user. For example, it uses Amazon Polly to generate voice guidance such as "Turn right."

[1788] 3. Specific Examples

[1789] As a concrete example, consider the case where a user asks, "Can you tell me about some highly rated restaurants nearby?"

[1790] 1. Data acquisition and training:

[1791] The server connects to the database and obtains restaurant rating information.

[1792] The server converts the data into a DataFrame using Pandas.

[1793] The server uses Scikit-learn's RandomForestClassifier to train a model to predict highly rated restaurants.

[1794] 2. User Question Analysis:

[1795] The user speaks to the device saying, "Tell me about some highly rated restaurants nearby."

[1796] The device converts the voice input into text.

[1797] The device uses NLTK to tokenize the text and SpaCy to analyze the intent of the question.

[1798] 3. Set your destination:

[1799] The device obtains the user's current location using the GPS sensor.

[1800] The server uses the Google Maps API to calculate a route based on the user's current location and the locations of highly rated restaurants.

[1801] The server transmits the calculation result to the terminal.

[1802] 4. Providing navigation:

[1803] The terminal starts navigation based on the route information received from the server.

[1804] The device uses Amazon Polly to generate voice guidance and provide instructions to the user.

[1805] The user drives by following the voice guidance.

[1806] Example prompt:

[1807] "Tell me about some highly rated restaurants nearby."

[1808] In this way, users are safely guided to their destination while driving.

[1809] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1810] Step 1: Data acquisition and training

[1811] The server retrieves information from the database, such as restaurant ratings and whether parking is available. As input, it uses the database connection information and a SQL query. The server converts the data into a DataFrame using Python's Pandas library. The server then trains the data with a machine learning algorithm using Scikit-learn. Specifically, it uses RandomForestClassifier to create a model that predicts highly rated restaurants. As output, it obtains the trained model.

[1812] Specific behavior:

[1813] The server connects to the database and executes SQL queries to retrieve the data.

[1814] The server converts the acquired data into a Pandas DataFrame.

[1815] The server uses Scikit-learn's RandomForestClassifier to train a model to predict restaurant ratings.

[1816] Step 2: Analyze user questions

[1817] The terminal receives a question from the user. As input, the user's voice question is used. The terminal uses speech recognition technology to convert the voice input into text. The terminal then analyzes the question using natural language processing technology. Specifically, it uses Python's NLTK and SpaCy to understand the intent of the question. As output, the analyzed text data is obtained.

[1818] Specific behavior:

[1819] The user speaks to the device, saying, "Tell me about some highly rated restaurants nearby."

[1820] The device converts the voice input into text.

[1821] The device uses NLTK to tokenize the text and SpaCy to analyze the intent of the question.

[1822] Step 3: Set your destination

[1823] The server calculates the optimal route based on the user's current location and the search results. The user's current location and the parsed text data are used as input. The server uses the Google Maps API to obtain the user's current location and the locations of highly rated restaurants, and calculates the optimal route. The calculated route information is obtained as output.

[1824] Specific behavior:

[1825] The device obtains the user's current location using the GPS sensor.

[1826] The server uses the Google Maps API to calculate a route based on the user's current location and the locations of highly rated restaurants.

[1827] The server transmits the calculation result to the terminal.

[1828] Step 4: Providing navigation

[1829] The device provides navigation to the set destination. The calculated route information is used as input. The device uses a voice guidance system to give voice instructions to the user. Specifically, it uses Amazon Polly to generate voice guidance such as "Turn right". The voice guidance is obtained as output.

[1830] Specific behavior:

[1831] The terminal starts navigation based on the route information received from the server.

[1832] The device uses Amazon Polly to generate voice guidance and provide instructions to the user.

[1833] The user drives by following the voice guidance.

[1834] (Application example 1)

[1835] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1836] Conventional navigation systems have limited capabilities to search for destinations based on the user's detailed requests and lack the ability to generate appropriate answers to user questions. They also lack the ability to calculate the optimal route between the current location and the destination and provide voice guidance. This often causes inconvenience to users while traveling to their destination.

[1837] The identification process by the identification processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means. In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a natural language processing means for analyzing the user's question and generating an appropriate answer, a means for calculating the optimal route between the current position and the destination, and a means for providing voice guidance. This makes it possible to search for a destination based on the user's detailed requests, calculate the optimal route, and provide voice guidance.

[1838] A "source" is a source or resource that provides data or knowledge.

[1839] A "means of learning information" is a method that uses machine learning algorithms to analyze data obtained from various sources and extract patterns and knowledge.

[1840] "User specific requests" are the detailed requirements or wishes of a user based on their specific conditions or preferences.

[1841] A "means for searching for a destination" is a method or technology for searching for an appropriate destination based on a user's request.

[1842] "Search results" are the results of a search obtained by a means for searching for a destination.

[1843] A "means for setting a destination" is a method or technology for determining a user's destination based on search results.

[1844] A "means for providing navigation" is a method or technology that provides route guidance or directions to a user to reach a destination.

[1845] "Natural language processing means" is a technology for analyzing a user's questions or requests and generating appropriate answers.

[1846] A "means for calculating a route" is a method or technique for calculating an optimal path between a current location and a destination.

[1847] "Means for providing audio guidance" means the methods or techniques for converting the calculated route information into audio and providing it to the user.

[1848] As an embodiment for carrying out the present invention, the following system is considered.

[1849] First, the server has a means of learning information from various sources. This means uses machine learning algorithms to analyze data such as restaurant reviews and the availability of parking spaces, and extract patterns and knowledge. Specifically, it obtains information from a database and learns from it.

[1850] Next, there is a means to search for a destination based on the user's detailed requests. This means uses natural language processing technology to analyze the user's question and search for an appropriate destination. For example, if a user asks, "What are some highly rated restaurants nearby?", the system will analyze the question and search for an appropriate restaurant.

[1851] In addition, there is a method to set a destination based on the search results. This method calculates the optimal route based on the user's current location and the search results, and sets the destination. Specifically, it uses the Geopy library to calculate the distance between the current location and the destination.

[1852] Finally, there is a method to provide navigation to a set destination, which uses the Google Text-to-Speech (gTTS) library to convert the calculated route information into audio and provide it to the user, allowing the user to be safely guided to their destination while driving.

[1853] As a concrete example, when a user asks, "What are some highly rated restaurants nearby?" the system works as follows:

[1854] 1. Natural language processing means analyzes the user's question and generates an appropriate answer.

[1855] 2. The means for learning information retrieves restaurant rating information from the database and analyzes it.

[1856] 3. A route calculation means calculates the optimal route between the current location and the destination.

[1857] 4. A means for providing audio guidance converts the calculated route information into audio and provides it to the user.

[1858] An example of a prompt sentence might be:

[1859] User asks: "What are some highly rated restaurants near me?"

[1860] Context: "Restaurant ratings, availability of parking, etc."

[1861] This prompt can then be used to have a generative AI model parse the question and provide an appropriate answer.

[1862] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1863] Step 1:

[1864] The server receives a question from a user. The input is the user's question, e.g., "What are some highly rated restaurants nearby?", as text data. The output is text data ready for analysis.

[1865] Step 2:

[1866] The server analyzes the user's question using natural language processing means. The input is the user's question received in step 1, and the output is an appropriate search query as the analysis result. Specifically, a generative AI model is used to analyze the question and generate an appropriate answer.

[1867] Step 3:

[1868] The server uses a means of learning information to obtain data such as restaurant rating information and availability of parking spaces from the database. The input is the search query generated in step 2, and the output is the obtained rating information, availability of parking spaces, and other data. Specifically, the server executes a query against the database to obtain the required information.

[1869] Step 4:

[1870] The server calculates the optimal route between the user's current location and the destination based on the acquired information. The input is the user's current location and the destination information acquired in step 3, and the output is the calculated optimal route information. Specifically, the Geopy library is used to calculate the distance and determine the optimal route.

[1871] Step 5:

[1872] The server converts the calculated route information into voice guidance. The input is the route information calculated in step 4, and the output is an audio file. Specifically, it uses the Google Text-to-Speech (gTTS) library to convert the text information into speech.

[1873] Step 6:

[1874] The terminal provides the generated voice guidance to the user. The input is the voice file generated in step 5, and the output is voice guidance for the user. Specifically, the terminal plays the voice file and provides navigation information to the user.

[1875] Example 2

[1876] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1877] Conventional navigation systems only provide destination setting and navigation functions, and do not have a reservation function based on the user's detailed requests, so users had to make reservations manually. In addition, there was no system that analyzed voice instructions and made reservations automatically, which made it difficult for users to use the system efficiently. This meant that users had to take the time to set the destination and make reservations separately, making it difficult to use the system efficiently.

[1878] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1879] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for analyzing the user's voice instructions, a means for extracting reservation details based on the analyzed voice instructions, a means for automatically making a phone call based on the extracted reservation details, and a means for audibly conveying the reservation details over the phone. This allows the user to automatically set a destination and make a reservation simply by issuing voice instructions, greatly improving convenience.

[1880] "Source" refers to any source that provides data or knowledge.

[1881] "User" refers to any individual or entity that uses the System.

[1882] "Destination" refers to the place the user wants to reach.

[1883] "Search" refers to the act of finding specific information.

[1884] "Navigation" refers to the function of providing route guidance to a destination.

[1885] "Voice instruction" refers to an instruction given by a user to the system using voice.

[1886] "Analysis" refers to the act of examining data or information in detail to understand its meaning and structure.

[1887] "Reservation details" refers to the information necessary for making a reservation (date and time, number of people, location, etc.).

[1888] "Automatic" refers to a machine or system that operates independently, without human intervention.

[1889] "Making a call" refers to the act of using a communication device to attempt to connect to another communication device.

[1890] "Communicating by voice" refers to the act of conveying information to others using voice.

[1891] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1892] The present invention is embodied as an application for a smartphone. This application is a system that automatically sets a destination and makes a reservation based on a user's voice instruction. A specific embodiment of this system will be described below.

[1893] Hardware and Software Used

[1894] The system uses the following hardware and software:

[1895] Smartphone (device)

[1896] Speech recognition technology (Google Cloud Speech-to-Text API)

[1897] Database Management System (Firebase Firestore)

[1898] Automated dialing system (Twilio API)

[1899] Speech synthesis technology (Google Cloud Text-to-Speech API)

[1900] Data processing and calculation

[1901] The user launches the smartphone application and says, "Make a reservation for four people tomorrow night at 7 p.m." The device uses the Google Cloud Speech-to-Text API to convert the user's voice instructions into text. This conversion process occurs in real time.

[1902] Next, the device analyzes the converted text and extracts the reservation details (date, time, number of people). For example, from the text "Make a reservation for four people at 7 p.m. tomorrow," the information "tomorrow at 7 p.m." and "four people" is extracted.

[1903] The device sends a request to the server to retrieve restaurant information stored in Firebase Firestore. This request includes the user's location and preferred restaurant conditions. The server receives the request, retrieves the corresponding restaurant information from Firebase Firestore, and returns it to the device. The returned information includes the restaurant's name, phone number, address, etc.

[1904] The device uses the Twilio API to automatically call the restaurant's phone number, without any user intervention. Once the call is connected, the device uses the Google Cloud Text-to-Speech API to communicate the reservation details it extracted in advance to the restaurant. For example, it might say, "Please make a reservation for four people tomorrow night at 7 p.m."

[1905] Examples

[1906] As a concrete example, the following will show the behavior when the user instructs "Make a reservation for four people at 7pm tomorrow night."

[1907] 1. The user launches the smartphone application and gives a voice command such as, "Make a reservation for four people at 7pm tomorrow night."

[1908] 2. The device uses the Google Cloud Speech-to-Text API to convert the user's voice commands into text.

[1909] 3. The device analyzes the converted text and extracts the reservation details: "tomorrow night at 7pm" and "4 people."

[1910] 4. The device sends a request to the server to get restaurant information stored in Firebase Firestore.

[1911] 5. The server receives the request, retrieves the relevant restaurant information from Firebase Firestore, and returns it to the device.

[1912] 6. The device uses the Twilio API to automatically call the restaurant's phone number.

[1913] 7. Once the call is connected, the device uses the Google Cloud Text-to-Speech API to communicate to the restaurant, "Please make a reservation for four people tomorrow night at 7pm."

[1914] Examples of prompt statements

[1915] An example of a prompt sentence might be:

[1916] "If a user says, 'Make a reservation for 4 people tomorrow night at 7pm,' how should the application handle that?"

[1917] By inputting this prompt sentence into the generative AI model, an explanation of the processing based on the specific example above can be obtained.

[1918] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1919] Step 1:

[1920] The user issues a voice command.

[1921] The user starts the smartphone application and gives a voice command such as, "Make a reservation for four people at 7 p.m. tomorrow." This voice command becomes the input.

[1922] Step 2:

[1923] The device converts the speech to text.

[1924] The device uses voice recognition technology to convert the user's voice instructions into text. This process is performed using a voice recognition API. The input is the user's voice instructions, and the output is the instructions in text format.

[1925] Step 3:

[1926] The terminal analyzes the text and extracts the reservation details.

[1927] The terminal analyzes the converted text and extracts the reservation details (date, time, number of people). For example, from the text "Make a reservation for four people at 7 p.m. tomorrow," the information "tomorrow at 7 p.m." and "four people" is extracted. The input is instructions in text format, and the output is the reservation details data.

[1928] Step 4:

[1929] The terminal requests restaurant information from the server.

[1930] The terminal sends a request to the server to get restaurant information stored in the database management system. This request includes the user's location information and the conditions of the preferred restaurant. The input is the reservation details and the user's location information, and the output is a request for restaurant information.

[1931] Step 5:

[1932] The server returns the restaurant information.

[1933] The server receives the request, retrieves the relevant restaurant information from the database, and returns it to the terminal. The returned information includes the restaurant's name, phone number, address, etc. The input is a request for restaurant information, and the output is the restaurant information.

[1934] Step 6:

[1935] The terminal makes the call using an automatic dialing system.

[1936] The terminal then uses an automatic dialing system to automatically call the restaurant's phone number, without any user intervention: the input is the restaurant's phone number, and the output is the establishment of a telephone connection.

[1937] Step 7:

[1938] The terminal will announce the reservation details aloud.

[1939] When the phone is connected, the device uses voice synthesis technology to communicate the reservation details it extracted in advance to the restaurant by voice. For example, it may say, "Please make a reservation for four people tomorrow night at 7 p.m." The input is the reservation details data, and the output is a voice message.

[1940] (Application example 2)

[1941] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1942] Conventional navigation and reservation systems required users to operate them manually, which made them inconvenient. In addition, they lacked the functionality to utilize the user's past behavioral history to make optimal suggestions, making it difficult to respond to user requests quickly and accurately. Furthermore, there was a need to effectively combine voice recognition technology and an automatic dialing system to improve the user experience.

[1943] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1944] In this invention, the server includes a means for learning information from various information sources, a means for searching for a destination based on the user's detailed requests, a means for setting a destination based on the search results, a means for providing navigation to the set destination, a means for understanding the user's requests using voice recognition technology, a means for making reservations using an automatic dialing system, a means for learning the user's past behavior history and making optimal suggestions, and a means for providing voice feedback. This allows the user to search for and make reservations at a destination using only voice instructions, and to receive optimal suggestions based on the user's past behavior history.

[1945] "Source" refers to any source that provides data or knowledge.

[1946] "Means of learning" refers to the methods and techniques used to collect, analyze, and understand information.

[1947] "The user's detailed requests" refer to the specific conditions and wishes that the user has.

[1948] "Destination search means" refers to methods and technologies for searching for appropriate locations based on a user's requirements.

[1949] "Search Results" refers to the information or data obtained through a Search.

[1950] "Means for setting a destination" refers to a method or technology for determining a final destination based on a search result.

[1951] "Means for providing navigation" refers to methods and technologies for providing directions to a set destination.

[1952] "Voice recognition technology" refers to the technology that analyzes voice and converts it into text or commands.

[1953] "Automatic dialing system" refers to a system that automatically dials designated telephone numbers.

[1954] "Means for making reservations" refers to methods or technologies for automatically making reservations for specified locations or services.

[1955] "Past behavioral history" refers to the record of actions and choices a user has previously made.

[1956] "Means of making optimal suggestions" refers to methods and technologies for presenting the most suitable options to users based on their past behavioral history and current needs.

[1957] "Means for providing audio feedback" refers to methods or technologies that allow the system to communicate information or results to the user using audio.

[1958] A system for implementing the present invention has the following configuration: A server includes means for learning information from various information sources, means for searching for a destination based on a user's detailed requests, means for setting a destination based on the search results, means for providing navigation to the set destination, means for understanding a user's requests using voice recognition technology, means for making reservations using an automatic dialing system, means for learning a user's past behavior history and making optimal suggestions, and means for providing voice feedback.

[1959] Explanation of program processing

[1960] Hardware and Software

[1961] The system uses the smartphone's microphone and speaker as hardware, the speech_recognition library for voice recognition, the pyttsx3 library for voice feedback, and the requests library for data communication.

[1962] Data processing and calculation

[1963] 1. Speech recognition: When the user speaks their request into the smartphone's microphone, the speech_recognition library converts the speech into text.

[1964] 2. Request Analysis: Analyze the converted text to understand the user's request. For example, analyze the request "order a pizza."

[1965] 3. Execute the booking: Based on the request, send the order to the food delivery API using the requests library. If the order is successful, provide audio feedback using the pyttsx3 library.

[1966] 4. Learning from behavioral history: Learns the user's past behavioral history and makes optimal suggestions for future visits.

[1967] Examples

[1968] For example, if a user says "order a pizza," the system would do the following:

[1969] 1. Capture the user's voice with the microphone.

[1970] 2. Convert speech to text and get the text "Order pizza."

[1971] 3. Analyze the text to identify "...

Claims

1. A means for converting voice input from a user into text information using voice recognition technology; A means for understanding the content of a user's instructions by analyzing text information using natural language processing technology; means for recognizing an emotional state of a user by analyzing speech input from the user with an emotion engine; means for recommending destinations based on the recognized emotional state of the user and the user's instructions; A means for searching for a destination based on a request from a user who has recommended the destination; A means for setting a destination based on the search results; a means for providing navigation to a set destination; A system including:

2. a means for analyzing a question received from a user by using a natural language processing means and generating a prompt sentence instructing generation of an answer to the analyzed question from the user; A means for generating an answer to a question from a user using the generated prompt sentence and the generative AI model; a means for outputting the generated answer to a user by voice; The system of claim 1 further comprising:

3. A means for extracting a reservation instruction included in the instruction content from a user; means for automatically calling the establishment to be reserved based on the extracted reservation instructions; A means for completing a reservation by verbally conveying the reservation details to an employee of the store to which the reservation is made; Once the reservation is complete, a means of notifying the user that the reservation is complete; The system of claim 1 further comprising:

Images