system

Abstract

Provide a system. The method includes: means for receiving natural language input from a user; means for analyzing the received natural language input to generate search parameters; means for retrieving relevant information from a database based on the generated search parameters; a means for filtering and ranking the relevant information obtained; and means for displaying the filtered and ranked information to a user.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional map search systems have difficulty obtaining appropriate search results unless the user has clear information such as a specific address or store name. Furthermore, when users are seeking new discoveries or encounters, there is no search method that meets their needs. There is a need for a system that can solve these problems and provide information based on the user's specific needs and interests. [Means for solving the problem]

[0005] The present invention proposes a system for receiving natural language input from a user and analyzing the input to generate search parameters, which specifically includes the following means:

[0006] A means of receiving natural language input from a user

[0007] A means of parsing the received natural language input and generating search parameters (using a natural language processing model)

[0008] A means of retrieving relevant information from a database based on generated search parameters

[0009] A means of filtering and ranking the relevant information obtained

[0010] A means of displaying filtered and ranked information to users

[0011] This system allows users to search flexibly based on their current location and interests, rather than relying on addresses or store names, providing new discoveries and encounters.

[0012] "Natural language input from a user" is natural language input uttered by a user to a system in the form of voice or text.

[0013] A "receiving means" is an interface, device, or program for capturing natural language input sent by a user.

[0014] The "means for analyzing" refers to an algorithm or natural language processing model for understanding the received natural language input and extracting the necessary information and parameters.

[0015] "Search parameters" are elements that serve as search criteria generated from parsed natural language input, and specifically include location information, distance, entity attributes, and popularity.

[0016] A "database" is a structured collection of information that stores specific information in an organized manner and allows it to be accessed and managed.

[0017] "Relevant Information" means information retrieved from a database based on search parameters, including geographic entities and their details that correspond to the user's request.

[0018] A "filtering means" is an algorithm or method for sorting the relevant information obtained based on specific criteria.

[0019] A "ranking means" is an algorithm or method for ranking the filtered information based on a particular criterion.

[0020] A "means for displaying" is an interface or device for visually or audibly presenting the filtered and ranked information to a user.

[0021] A "natural language processing model" is a machine learning algorithm or deep learning model that analyzes natural language input provided by a user and understands its meaning and intent. [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] The system of the present invention includes a series of processes for searching and acquiring related information based on a user's natural language input and providing the results. The operation of the system is described below with specific examples.

[0044] Receiving user input

[0045] User: Asks questions in natural language to a smartphone or car navigation system with the application installed. For example, the user might type, "Tell me about some popular restaurants within five minutes of my current location where a family of four can spend half a day."

[0046] Sending input to the server

[0047] Terminal: In the case of voice input, the voice recognition function is used to convert it into text, and in the case of text input, the user's input is sent to the server as text data as is.

[0048] Natural Language Processing and Parameter Generation

[0049] Server: Sends the received text input to a natural language processing (NLP) model to analyze the user's intent, such as extracting parameters like current location, travel time (distance), family-friendly, store, popularity, etc.

[0050] For example, the NLP model understands that the user wants to know information about fun things to do with a family of four within a five-minute walk from their current location.

[0051] Retrieving information from a database

[0052] Server: Based on the parsed parameters, retrieve relevant information from geographic databases and external APIs (e.g., map APIs, review site APIs). For example, retrieve data on popular family-friendly restaurants and amusement spots within the specified area.

[0053] Filtering and Ranking

[0054] Server: From the acquired information, select the information that best suits the user's intent and needs. For example, filtering for family-friendly facilities with recent high ratings.

[0055] The selected information is then ranked based on evaluation criteria (recent popularity, number of reviews, etc.).

[0056] Sending and Displaying Information

[0057] Server: Formats the filtered and ranked results and sends them to the device.

[0058] Terminal: Displays the received results visually to the user and can also provide audio information if required.

[0059] For example, it displays detailed information (location, ratings, recent reviews) about "Restaurant A" which is three minutes away from the user's current location, and "Amusement Park B" which is four minutes away.

[0060] User Interactions

[0061] Users can use the information provided to learn more or request directions.

[0062] On the device: Displays detailed information and directions upon user request.

[0063] The system allows users to easily search and discover geographic entities that match specific criteria, providing new place discovery and fun experiences.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] User: In the chat window of their smartphone or car navigation app, they type a question in natural language: "Can you tell me about some popular restaurants within five minutes of our current location that a family of four can enjoy for half a day?"

[0067] Step 2:

[0068] Device: In the case of voice input, the voice recognition function is used to convert it into text, and in the case of text input, the input is received as is. An API call is made to send the received input data to the server.

[0069] Step 3:

[0070] Server: Sends the received text data to a natural language processing (NLP) model, which analyzes the text, extracts user intent, and generates search parameters (such as current location, distance, entity attributes, and popularity).

[0071] Step 4:

[0072] Server: Retrieves GPS information sent from the device, identifies the user's current location (latitude and longitude), calculates the distance traveled within the next 5 minutes (a walking distance of about 400 meters), and constructs a search query.

[0073] Step 5:

[0074] Server: Using the constructed search query, retrieves relevant information from geographic databases and external APIs (e.g., map APIs, review site APIs). Retrieved information includes popular family-friendly restaurants and amusement spots within the specified area.

[0075] Step 6:

[0076] Server: Based on the retrieved relevant information, the server filters the information according to the user's request, for example, selecting family-friendly spots with recent high ratings.

[0077] Step 7:

[0078] Server: Ranks the filtered information based on criteria (recent popularity, number of reviews, etc.). This results in raw data being shaped and sorted in order of most relevance to the user.

[0079] Step 8:

[0080] Server: Formats the ranked results and sends them to the user's device. The results are provided in a user-friendly format (e.g., JSON format).

[0081] Step 9:

[0082] Device: The device displays the received results in a user interface, highlighting spots within the specified area on a map and allowing users to tap on the spots for more information.

[0083] Step 10:

[0084] User: Selects points of interest from the presented information and requests more information and directions.

[0085] Step 11:

[0086] Device: Sends a request to the server again in response to additional requests from the user (more information or directions).

[0087] Step 12:

[0088] Server: Retrieves detailed data about the selected spot from a database or external API and sends it to the device.

[0089] Step 13:

[0090] Terminal: The device displays the received detailed data in a user interface, including reviews, opening hours, and services offered by the spot.

[0091] In this way, through the series of processes from step 1 to step 13, users can obtain and confirm the most appropriate information by asking questions in natural language.

[0092] Example 1

[0093] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0094] Conventional geographic information search systems have difficulty accurately analyzing users' natural language input and quickly providing the information most relevant to their intent. Furthermore, they suffer from insufficient ranking of search results and detailed information, which hinders user experience. Furthermore, the lack of voice input support and route guidance functions also contributes to a lack of user convenience.

[0095] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0096] In this invention, the server includes means for receiving a user's natural language input, means for analyzing the received natural language input to generate search parameters, and means for acquiring related information from information sources based on the generated search parameters. This enables quick search for and acquisition of information most suited to the user's intent. The server also includes means for filtering and ranking the acquired information and means for displaying the filtered and ranked information to the user. This allows the user to easily browse the most suited information. Furthermore, by including means for providing detailed information and route guidance in response to additional user requests, user convenience is significantly improved. In addition, by including means for converting a user's voice input into text data and means for analyzing the user's intent using a natural language processing model, it is possible to support voice input and analyze intent with high accuracy.

[0097] "User" means a person who uses the system to search for information.

[0098] "Natural language input" is a format in which users input information using everyday language.

[0099] "Server" means a computer system with a central function that receives input from users and performs analysis, retrieval, filtering, ranking, and display of relevant information.

[0100] "Means" refers to a specific method or device by which a system performs a specific function or operation.

[0101] "Search parameters" refer to the conditions or criteria used in a search that are parsed from a user's natural language input.

[0102] "Sources" refer to databases and external API services that provide relevant information.

[0103] "Relevant Information" refers to information useful to the User that is obtained from a source based on the search parameters.

[0104] "Filtering" is the process of selecting necessary information from the acquired related information.

[0105] "Ranking" is the process of ranking the filtered information based on specific criteria.

[0106] "More Information" refers to more specific and detailed information provided in response to a User's additional request.

[0107] "Route guidance" refers to navigation information that shows the route to a destination specified by the user.

[0108] "Means for converting voice input into text data" refers to technology that converts a user's voice input into a string of characters.

[0109] A "natural language processing model" is an artificial intelligence model that analyzes a user's natural language input and understands its meaning and intent.

[0110] The system of the present invention includes a series of processes for searching and acquiring related information based on natural language input from a user and providing the results. Below, an embodiment of this system will be described with specific examples.

[0111] First, the user uses an application installed on their smartphone or car navigation system to input a question in natural language, for example, "Tell me about a popular restaurant within five minutes of my current location where a family of four can spend half a day," either by voice or text.

[0112] In the case of voice input, the device converts the voice into text using voice recognition software (e.g., Google (registered trademark) voice recognition API). In the case of text input, the device processes the voice as text data as is. The converted or acquired text data is sent to the server.

[0113] The server sends the received text data to a natural language processing model (e.g., Google Cloud Natural Language API) to analyze the user's intent, extracting parameters from the text such as current location, travel time, family-friendly, and highly rated restaurants.

[0114] Next, based on the analysis results, the server retrieves relevant information from geographic databases and external APIs (e.g., map APIs or review site APIs). It creates queries to gather the necessary information. For example, it retrieves information about popular family-friendly restaurants and amusement facilities within a five-minute walk from the user's current location.

[0115] The server filters the information it receives and selects the information that best suits the user's intent and requirements. After filtering, it ranks the information based on evaluation criteria (e.g., recent popularity, number of reviews).

[0116] The server formats the ranked information and sends it to the device. The device displays this information visually to the user. It also has the ability to provide audio information if needed. For example, the device might display detailed information (location, ratings, recent reviews) about "Restaurant A" that is three minutes away from the user's current location, or "Amusement Park B" that is four minutes away.

[0117] Users can view more details or request route guidance from the information provided. The device will then display the details or route guidance as requested by the user. For example, it can launch Google Maps and display directions to a specified location.

[0118] As a concrete example, if a user inputs the prompt "Tell me about a highly rated cafe nearby," the device converts this speech into text and sends it to the server. The server analyzes it using a natural language processing model and extracts the parameters "highly rated cafe" and "nearby." Next, it retrieves information about highly rated cafes near the user's current location from a geographic information database and a rating site API. This information is filtered and ranked, and finally, information about "Cafe C" and "Cafe D" is displayed to the user. If the user requests "Tell me the way to Cafe C," the device launches a map app and displays route guidance.

[0119] This system allows users to easily find the right location for their needs, providing a more convenient and comfortable experience.

[0120] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0121] Step 1:

[0122] The user launches an application installed on their smartphone or car navigation system and asks for the information they need in natural language. For example, they might say, "Tell me about a popular restaurant within five minutes of my current location where a family of four can spend half a day." Input: A question in natural language (voice or text). Output: The content of the user's question.

[0123] Step 2:

[0124] The device converts voice input into text data using voice recognition software (e.g., voice recognition API). In the case of text input, it is processed as text data as is. Since text data is generated through voice recognition, the content of the user's question becomes clear. Input: Voice data (in the case of voice input). Output: Text data.

[0125] Step 3:

[0126] The terminal sends the generated text data to the server. A data transmission protocol is used to deliver the text data to the server quickly and reliably. Input: Text data (question content). Output: Sending text data to the server.

[0127] Step 4:

[0128] The server sends the received text data to a natural language processing model (e.g., natural language processing API) to analyze the user's intent. Search parameters such as current location, travel time, family-friendly, and highly rated shops are extracted from the text data. Input: Text data. Output: Search parameters.

[0129] Step 5:

[0130] The server retrieves related information from geographic information databases and external APIs (e.g., map APIs, review site APIs) based on the search parameters obtained as a result of the analysis. Specifically, it collects information on popular family-friendly restaurants and amusement facilities that can be enjoyed within a five-minute walk from the user's current location. Input: Search parameters. Output: Related information data.

[0131] Step 6:

[0132] The server filters the retrieved related information and selects information based on specific criteria (e.g., highly rated restaurants, family-friendly restaurants, etc.). It then ranks the filtered information based on evaluation criteria (e.g., recent popularity, number of reviews). Input: Related information data. Output: Filtered and ranked information.

[0133] Step 7:

[0134] The server formats the ranked results and sends them to the terminal. The display format is adjusted to make it easy for the user to understand. Input: Ranked information. Output: Formatted information.

[0135] Step 8:

[0136] The device displays the received formatted information to the user visually and audibly. For example, detailed information (location, rating, recent reviews) for "Restaurant A" and "Amusement Park B" is displayed to the user. Input: Formatted information. Output: Display to the user (visual, audio).

[0137] Step 9:

[0138] Users can request more details or directions from the information provided. For example, they might enter commands like "Tell me more about Restaurant A" or "Show me directions to Amusement Park B." Input: A request for more information. Output: A request for more information or directions.

[0139] Step 10:

[0140] The device displays more information or directions at the user's request, for example, launching a maps app to show routes to a specified location. Input: Request for more information. Output: More information or directions.

[0141] (Application example 1)

[0142] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0143] Conventional information search systems that use natural language input lack a mechanism for efficiently analyzing user voice input and generating appropriate search parameters. Furthermore, there are no concrete examples of providing users with the acquired information visually or audibly, or of acquiring information from external APIs, making it difficult to provide accurate information in real time. This results in problems such as reduced user convenience and satisfaction.

[0144] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0145] In this invention, the server includes means for receiving natural language input from a user using voice recognition hardware, means for analyzing the received natural language input and generating search parameters using a natural language processing model, means for retrieving related information from an external API based on the generated search parameters, means for filtering and ranking the retrieved related information, and means for displaying the filtered and ranked information to the user audibly and visually, thereby enabling efficient analysis of the user's voice input, retrieval of information from the external API, and provision of appropriate information visually and audibly to the user in real time.

[0146] "Speech recognition hardware" means a device for receiving a user's voice input and converting it into voice data.

[0147] "Natural language input" refers to text or voice instructions or queries given by a user in the language they normally use.

[0148] A "natural language processing model" is an algorithm or program that analyzes received natural language input and understands the user's intent and meaning.

[0149] "Search parameters" are conditions or criteria generated based on user input for retrieving relevant information from databases or APIs.

[0150] An "external API" is a protocol for accessing data or functionality through an interface provided by an external service or application.

[0151] "Relevant information" refers to useful data retrieved from databases and external APIs based on the user's search parameters.

[0152] "Filtering" is the process of selecting relevant information obtained based on certain criteria and removing unnecessary information.

[0153] "Ranking" is the process of evaluating the filtered relevant information based on specific criteria and sorting it according to priority.

[0154] "Visual presentation means" refers to a method or device for displaying acquired information in a manner that appeals to the user's visual sense.

[0155] "Means for providing audio information" refers to a method or device for providing acquired information in a form that appeals to the user's auditory sense.

[0156] The present invention relates to a system for searching for information based on a user's natural language input and providing the search results. Specific embodiments are described below.

[0157] First, the main hardware components of the system are the speech recognition hardware, server, and terminal. Specifically, they are a microphone for users to input speech, the "SpeechRecognition" speech recognition software library for converting input speech into text, and an NLP model for natural language processing. The "requests" library is used for API requests.

[0158] Receiving audio input

[0159] The user gives instructions in natural language through speech recognition hardware (microphone), for example, "Tell me about nearby family-friendly attractions."

[0160] Speech-to-text

[0161] On-device speech recognition software (SpeechRecognition) is used to convert the received voice input into text, which is then sent to the server.

[0162] natural language analysis

[0163] The server sends the received text to a natural language processing model (NLP model) to analyze the user's intent. The NLP model analyzes the user's request and extracts search parameters (e.g., current location, family-friendly, attractions).

[0164] Obtaining information from external APIs

[0165] Based on the generated search parameters, the server calls external APIs (e.g., geographic information APIs or review site APIs) to retrieve relevant information, thereby gathering information that matches the specified criteria (e.g., family-friendly attractions).

[0166] Information filtering and ranking

[0167] The acquired information is filtered and ranked on the server. The filtering selects information that matches the user's intent, and the ranking prioritizes it based on criteria such as the number of reviews and rating points.

[0168] Providing information

[0169] Finally, the server sends the filtered and ranked results to the device, which then presents them to the user via audio and visual displays, such as a list of attractions near the user's current location and highly rated spots, and audio guidance when needed.

[0170] Specific examples

[0171] When a user says, "Tell me the route to the next rest stop" in their car, the system recognizes the speech and a natural language processing model extracts parameters such as "rest stop" and "route." Next, it uses a geographic information API to obtain information on the best rest stops, filters it, and ranks them. As a result, the best rest stop candidates are displayed on the user's navigation system. At the same time, a voice guide tells the user, "The best rest stop that can be reached within 15 minutes from your current location is XX."

[0172] Prompt Sentence Examples

[0173] "Where's a nearby park where I can relax?"

[0174] "Tell me the route to the next rest stop."

[0175] This allows users to obtain optimal information in real time, enabling efficient navigation.

[0176] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0177] Step 1: Receiving Audio Input

[0178] The device receives the user's voice input through the voice recognition hardware (microphone). For example, the user might say, "Tell me about nearby family attractions." The input is analog voice data, which is converted to digital data in the next step.

[0179] Step 2: Speech to text

[0180] The device uses speech recognition software (SpeechRecognition library) to convert the received voice data into digital text. The input is voice data, and the output is text data such as "Tell me about nearby family-friendly attractions." This conversion is performed by the speech recognition software's algorithm.

[0181] Step 3: Natural Language Analysis

[0182] The server receives the text data sent from the device and analyzes it using a natural language processing model (NLP model). Specifically, it extracts search parameters from the text. The input is the text "Tell me about nearby attractions that families can enjoy," and the output is search parameters such as "current location," "family-friendly," and "attractions."

[0183] Step 4: Retrieving information from an external API

[0184] Based on the generated search parameters, the server calls external APIs (such as geographical information APIs and review site APIs) to retrieve relevant information. The input is the search parameters, and the output is data about family-friendly attractions within the specified range. The retrieved information includes location, ratings, and recent reviews.

[0185] Step 5: Filtering and ranking information

[0186] The server filters and ranks the acquired information. Filtering selects only information that matches criteria such as "family-friendly" or "highly rated." Next, ranking prioritizes the information based on the number of reviews and ratings. The input is the acquired attraction information, and the output is a filtered and ranked list of attractions.

[0187] Step 6: Provide information

[0188] The server sends the filtered and ranked information to the terminal, which then presents it to the user audibly and visually. The input is the filtered and ranked list of attractions, and the output is the specific attraction information displayed to the user. Specifically, the terminal will provide a voice prompt saying, "Attractions that can be reached within 15 minutes from your current location are XX," and also display the information on a map.

[0189] The above are the specific steps for carrying out the invention. This processing flow allows users to obtain optimal information in real time, realizing efficient navigation.

[0190] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0191] The system of the present invention uses a method to recognize natural language input from a user and the user's emotions and provide relevant information. By combining this system with an emotion engine, it is possible to provide more personalized information according to the user's emotions. The operation of the system is explained below with a concrete example.

[0192] Receiving user input

[0193] User: In the chat window of a smartphone or car navigation app, type a question in natural language, such as, "Tell me about a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day." As the user types, the system detects the user's emotions from the voice and text.

[0194] Sending input to the server

[0195] Terminal: Recognizes the user's voice input and converts it into text, or receives text input as is, and sends the input data and emotion data to the server.

[0196] Natural Language Processing and Emotion Recognition

[0197] Server: The received text data is sent to a natural language processing (NLP) model to analyze the user's intent. At the same time, the emotion engine analyzes the user's emotion data and identifies the user's emotion. For example, if the user is feeling stressed, it will be recognized as "tension" or "fatigue."

[0198] Parameter generation and tuning

[0199] Server: Generates search parameters from the parsed natural language input and adjusts them based on the user's emotional data. For example, if the user is tired, parameters that prioritize "relaxing places" and "quiet environments" are added.

[0200] Retrieving information from a database

[0201] Server: Based on the adjusted search parameters, retrieves relevant information from geographic databases and external APIs, such as data on popular family-friendly, relaxing restaurants and attractions within the specified radius.

[0202] Filtering and Ranking

[0203] Server: Based on the retrieved relevant information, it filters and ranks the information that best matches the specified emotional state. Evaluation criteria include recent popularity, user reviews, and emotional fitness.

[0204] Sending and Displaying Information

[0205] Server: Formats the filtered and ranked results and sends them to the user's device.

[0206] Device: The received results are displayed in the user interface. Spots within the specified area are highlighted on the map, and users can tap on them to view more information. Additionally, special recommendations based on the user's emotions are also displayed.

[0207] User Interactions

[0208] Users can select points of interest from the information presented and request more information or directions.

[0209] Device: Sends these additional requests back to the server to retrieve more information and directions to provide to the user.

[0210] In this way, the system of the present invention combines natural language input from users with sentiment analysis to provide optimal information, enabling users to obtain more personalized search results, leading to new discoveries and a richer experience.

[0211] The processing flow will be explained below.

[0212] Step 1:

[0213] User: In the chat window of their smartphone or car navigation app, they type a question in natural language, such as, "Tell me about a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day." As they type, the speech recognition or text analysis module works to detect emotions from the user's voice and text.

[0214] Step 2:

[0215] Device: In the case of voice input, the device uses the voice recognition function to convert the input into text, and in the case of text input, the device receives the input as text data. An API call is made to send the received natural language data and emotion data to the server.

[0216] Step 3:

[0217] Server: Sends the received text data to a natural language processing (NLP) model to analyze the user's intent. At the same time, the emotion engine analyzes the user's emotion data and identifies the user's emotion as "tension," "fatigue," "joy," etc.

[0218] Step 4:

[0219] Server: Generates search parameters (e.g., current location, distance, entity attributes, popularity) from the analysis results of the NLP model. Additionally, adjusts specific search parameters to match the user's emotional state based on the results of the emotion engine. For example, if the user is tired, add parameters that prioritize "places where you can relax" and "quiet environments."

[0220] Step 5:

[0221] Server: Based on the generated and adjusted search parameters, retrieve relevant information from geographic databases and external APIs (e.g., map APIs, review site APIs). For example, retrieve data on popular family-friendly, relaxing restaurants and attractions within the specified radius.

[0222] Step 6:

[0223] Server: Filters the retrieved relevant information based on the user's request and emotional state, for example, selecting family-friendly spots with a particularly "relaxing" or "quiet" environment.

[0224] Step 7:

[0225] Server: Ranks the filtered information based on criteria (e.g., recent popularity, number of reviews, emotional fitness), with specific weightings applied based on the user's emotional state.

[0226] Step 8:

[0227] Server: Formats the ranked results and sends them to the user's device. The result data is formatted in a user-friendly format (e.g., JSON format).

[0228] Step 9:

[0229] Device: The received results are displayed in the user interface. Spots within the specified area are highlighted on a map, and users can tap on them to view more information. Special recommendations based on the user's emotions are also displayed.

[0230] Step 10:

[0231] Users can select points of interest from the information presented and request more information or directions.

[0232] Step 11:

[0233] Device: Sends the user's additional request (e.g., more information or directions) back to the server.

[0234] Step 12:

[0235] Server: Retrieves detailed data about the selected spot from a database or external API and sends the latest detailed data to the device.

[0236] Step 13:

[0237] Terminal: The device displays the received detailed data in a user interface, including the location's reviews, opening hours, and services offered.

[0238] Through these steps, the system of the present invention can provide optimal information by taking into account the user's natural language input and emotions, allowing the user to obtain more personalized search results and enjoy a comfortable and novel experience.

[0239] Example 2

[0240] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0241] Conventional systems analyze users' natural language input to provide information, but they are unable to provide information that takes into account the user's emotions, making it difficult to provide personalized information that meets the user's needs.In addition, they are unable to adjust search parameters based on the user's emotions, making it difficult to provide more appropriate information.

[0242] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving natural language input from a user, means for analyzing the received natural language input to generate search parameters, means for analyzing emotion data of the user and identifying the emotion, means for adjusting the search parameters based on the analyzed emotion data, means for acquiring related information from a database based on the generated and adjusted search parameters, means for filtering and ranking the acquired related information, and means for displaying the filtered and ranked information to the user. This makes it possible to provide personalized information according to the user's emotions.

[0243] "Natural language input" refers to data entered as voice or text in a language that the user normally uses.

[0244] "Search parameters" are criteria or conditions for information search that are extracted by analyzing the user's natural language input.

[0245] "Emotional Data" means data that indicates an emotional state detected from a user's input or behavior.

[0246] "Emotion identification" refers to analyzing a user's emotion data to identify a specific emotional state (e.g., joy, sadness, tension, fatigue, etc.).

[0247] "Means for adjusting" refers to a method or device for modifying search parameters and providing appropriate information based on the identified emotion.

[0248] A "database" is a structured collection of data that allows for efficient retrieval, storage, and management of information.

[0249] "Filtering" refers to the process of removing unnecessary information from acquired related information according to specific criteria.

[0250] "Ranking" refers to the process of ordering filtered information based on predetermined evaluation criteria.

[0251] "Means for displaying to the user" refers to a method or device for visually displaying the final selected information on the user's terminal.

[0252] The present invention is a system that analyzes data and emotion data input by a user in natural language and provides optimal information based on the analysis. This system can be implemented using the following hardware and software.

[0253] First, the user inputs natural language using a device such as a smartphone or car navigation system, such as "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day at." This input includes both voice and text input. In the case of voice input, the device uses a speech recognition system to convert speech into text. Specifically, a speech recognition tool such as the Google Cloud Speech-to-Text API is used.

[0254] The device then sends the received input data to a server, which uses Apache® OpenNLP or a similar natural language processing (NLP) model to analyze the received text data and Microsoft® Azure® Emotion API or a similar emotion analysis tool to analyze the user's emotional data and identify their emotions.

[0255] The server generates search parameters based on the analyzed natural language input and emotional data. The search parameters are then adjusted according to the emotional data. For example, if the user feels "tired," parameters that prioritize "relaxing places" and "quiet environments" are added.

[0256] The server then retrieves relevant information from geographic databases and external APIs (e.g., Google Places API) based on the generated and adjusted search parameters, and the retrieved information is filtered and ranked based on evaluation criteria such as user emotional relevance and popularity.

[0257] The server sends the filtered and ranked information to the device, which displays it in a user interface. Spots are indicated on a map with markers, and users can tap them to view more information.

[0258] For example, a user types into their smartphone, "I'm tired, can you recommend a quiet cafe nearby?" Based on this prompt, the system will recognize the user's emotion as "tired" and add the search parameter "quiet cafe." Finally, it will suggest a quiet cafe near the user's current location, which the user can view on a map.

[0259] In this way, the system of the present invention can combine the user's natural language input with emotion analysis to provide information that best suits the user's emotional state, allowing the user to obtain more personalized information and enjoy a pleasant and beneficial experience.

[0260] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0261] Step 1: Receiving User Input

[0262] User: Using the chat function on their smartphone or car navigation system, they input information in natural language, such as, "Please tell me about some popular restaurants within five minutes of our current location where a family of four can spend half a day."

[0263] Input: The user types a question into the device by voice or text.

[0264] What it does: Users speak into their smartphone's microphone or type a question into a text field.

[0265] Step 2: Sending input to the server

[0266] Device: For voice input, convert the input voice into text using the Google Cloud Speech-to-Text API, etc. Then, send the input data (text) to the server.

[0267] Data processing: Converting speech to text and structuring it as natural language input from the user.

[0268] Output: Structured text data.

[0269] What it does: The device converts the user's voice into text and sends that text to the server.

[0270] Step 3: Natural Language Processing and Emotion Recognition

[0271] Server: Analyzes the received text data and extracts the user's intent using a natural language processing (NLP) model such as Apache OpenNLP, and identifies the user's emotions using a sentiment analysis tool (e.g., Microsoft Azure Emotion API).

[0272] Data Computing: Analyzes natural language input to extract keywords and intent, while simultaneously analyzing sentiment data to identify emotional states.

[0273] Output: Search parameters and emotional state based on user intent.

[0274] How it works: The server sends the text "Tell me about popular spots these days" to a natural language processing model to extract search parameters for the destination. It also uses an emotion analysis model to identify emotional states such as "tired."

[0275] Step 4: Generate and adjust parameters

[0276] Server: Generates search parameters from the parsed natural language input and adjusts the search parameters based on sentiment data.

[0277] Data processing: Generate search parameters and adjust priorities according to emotional state.

[0278] Output: The adjusted search parameters.

[0279] Specific behavior: The server adds conditions to the generated parameters that prioritize "relaxing places" and "quiet environments."

[0280] Step 5: Retrieving information from the database

[0281] Server: Based on the adjusted search parameters, retrieves relevant information from geographic databases and external APIs (e.g., Google Places API).

[0282] Data Calculation: Generates database queries based on parameters to retrieve relevant information.

[0283] Output: A list of the relevant information retrieved.

[0284] Specific operation: The server retrieves information about places that meet certain conditions, such as "quiet cafes," from a database or API.

[0285] Step 6: Filtering and ranking

[0286] Server: Filters the retrieved relevant information to best match the emotional state and ranks it based on evaluation criteria.

[0287] Data processing: Filter the acquired information and rank it based on emotional fitness and popularity.

[0288] Output: A list of filtered and ranked information.

[0289] How it works: The server filters the information it receives, lists places that match criteria such as "relaxing," and ranks them based on evaluation criteria.

[0290] Step 7: Send and view information

[0291] Server: Formats the filtered and ranked results and sends them to the user's device.

[0292] Device: The received results are displayed on the user interface. Spots within the specified area are indicated on the map with markers, and users can tap on a spot to view more information.

[0293] Output: Information displayed on the user interface.

[0294] Specific operation: The location of the cafe will be displayed on a map on the smartphone screen, and detailed information will be displayed in a pop-up.

[0295] Step 8: User Interaction

[0296] User: Users can select points of interest from the information presented and request more information or directions.

[0297] Device: Sends additional requests to the server again to get more information and directions to provide to the user.

[0298] Data calculations: Generate detailed information and directions based on user requests.

[0299] Output: Detailed information and directions.

[0300] Specific operation: When a user requests "Tell me the route to this cafe," the device sends a request to the server and the optimal route is displayed.

[0301] As a result, the entire system becomes a process that provides optimal information based on the user's emotions.

[0302] (Application example 2)

[0303] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0304] In recent years, there has been an increasing demand to meet diverse consumer needs and personalize daily shopping experiences. Conventional systems provide information based on the user's natural language input. However, these systems do not take into account the user's emotional state, which means they are unable to fully enhance the user's psychological satisfaction. Therefore, there is a need for a system that can detect the user's emotions and provide optimal information based on them.

[0305] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a natural language input and emotion data from a user, means for analyzing the received natural language input and emotion data to generate search parameters, and means for retrieving related information from a database based on the generated search parameters. This makes it possible to provide personalized related information based on the user's natural language input and emotion data.

[0306] "User" refers to an individual consumer who uses the system.

[0307] "Natural language input" refers to language data that is input in the form of everyday conversation.

[0308] "Emotional data" refers to data that indicates a user's psychological state analyzed based on their input and behavior.

[0309] "Search parameters" refer to criteria used to retrieve information from a database.

[0310] "Database" refers to a collection of data that stores related information and makes it searchable and retrievalable.

[0311] "Filtering" refers to the process of selecting necessary information from a large amount of information according to specific criteria.

[0312] "Ranking" refers to the process of ranking selected information according to certain criteria.

[0313] A "natural language processing model" refers to an algorithm or system that analyzes natural language and understands user intent.

[0314] An "emotion analysis engine" refers to an algorithm or system for analyzing a user's emotional state from their input and behavior.

[0315] The system of the present invention analyzes a user's natural language input and emotion data, and provides information suited to the user's needs. The system includes the following means.

[0316] 1. Input receiving means

[0317] A user uses a smartphone application to provide natural language input via voice or text, for example, a query of the form "I want to relax today, can you tell me where the nearest cafe is?" This input can be received as voice or directly as text.

[0318] 2. Data transmission method

[0319] The user's natural language input and accompanying emotional data are sent from the smartphone device to a server, where voice input is converted into text using speech recognition technology, and then an emotional analysis engine identifies the user's emotional state.

[0320] 3. Analysis and search parameter generation method

[0321] The server analyzes the received text data using a natural language processing (NLP) model to extract the user's intent. Furthermore, a sentiment analysis engine analyzes the emotional data to identify the user's specific emotional state, such as "I want to relax" or "I'm tired." Search parameters are generated based on this information.

[0322] 4. Database information acquisition method

[0323] Based on the search parameters, the server retrieves relevant information from external databases and APIs, such as the location, opening hours, and reputation of cafes. The retrieved information is also adjusted according to emotional data. For example, if a user is looking for a relaxing space, quiet cafes will be prioritized.

[0324] 5. Information filtering and ranking methods

[0325] The retrieved relevant information is filtered on the server, and information that best matches the user's emotional state is prioritized. Other criteria, such as reputation and popularity, are also taken into account when ranking the information. For example, cafes that are rated as "quiet" and have high reviews will be ranked higher.

[0326] 6. Information display means

[0327] The filtered and ranked results are formatted and sent to the user's smartphone, where they can view a map view and detailed information within the application interface.

[0328] Hardware and software used

[0329] Hardware: Smartphones, servers

[0330] Software: speech recognition technology (e.g., Google Speech-to-Text API), natural language processing models (e.g., spaCy), sentiment analysis engines (e.g., Hugging Face Transformers)

[0331] Examples of specific examples and prompts

[0332] As a concrete example, suppose a user types into a smartphone application, "I want to relax today, so tell me where the nearest cafe is."

[0333] Example prompt sentence:

[0334] I want to relax today, so please tell me about a nearby cafe.

[0335] A cloud-based server receives the prompt and performs natural language processing and sentiment analysis to suggest the best cafe to match the user's preferences. This allows users to find the location and products that best suit their emotional state, resulting in a highly satisfying experience.

[0336] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0337] Step 1:

[0338] Receiving user input

[0339] A user uses a smartphone application to input natural language. The input format is text or voice, for example, "I want to relax today, so please tell me about a nearby cafe." The input data is received by the device. Input: User's natural language input (text or voice), Output: Received input data.

[0340] Step 2:

[0341] Speech-to-text conversion (for voice input)

[0342] For voice input, speech recognition technology is used to convert the voice data into text. For example, Google Speech-to-Text API is used to convert the voice into text data. Input: Voice data, Output: Text data.

[0343] Step 3:

[0344] Natural Language Processing

[0345] The received text data is analyzed using a natural language processing model (e.g., spaCy) on the server to extract the user's intent. This analysis identifies important keywords and propositions. Input: text data, output: keywords and propositions.

[0346] Step 4:

[0347] Emotion analysis

[0348] The server uses an emotion analysis engine (e.g., Hugging Face Transformers) to identify the emotional state from the user's text input. For example, it determines whether the input indicates the emotion "relaxed." Input: text data, Output: emotion data (e.g., "relaxed").

[0349] Step 5:

[0350] Search parameter generation

[0351] Generate search parameters based on the results of natural language processing and sentiment analysis. For example, set specific search conditions such as "relaxing cafes" or "nearby places." Input: Keywords, sentiment data, output: search parameters.

[0352] Step 6:

[0353] Database information acquisition

[0354] The server retrieves relevant information from external databases or APIs based on the search parameters. For example, it collects the location, opening hours, and reputation of the cafe. Input: Search parameters, Output: Related information data.

[0355] Step 7:

[0356] Information Filtering

[0357] The acquired related information is filtered on the server, and the information that best matches the user's emotional state is selected. For example, "a quiet cafe with a good reputation" is selected as the filtering result. Input: related information data, Output: filtered information.

[0358] Step 8:

[0359] Information Ranking

[0360] The filtered information is ranked based on criteria such as reputation and popularity. For example, cafes with high reviews are ranked higher. Input: Filtered information, Output: Ranked information.

[0361] Step 9:

[0362] Information transmission

[0363] The server formats the ranking information and sends it to the user's smartphone. A map display and detailed information are displayed on the user's screen. Input: Ranking information, Output: Information displayed on the smartphone.

[0364] Step 10:

[0365] User Interaction

[0366] The user can review the information sent and request points of interest or route guidance. The device again communicates with the server to provide additional information or route guidance. Input: User's additional request, Output: Additional information or route guidance.

[0367] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0368] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0369] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0370] [Second embodiment]

[0371] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0372] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0373] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0374] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0375] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0376] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0377] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0378] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0379] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0380] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0381] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0382] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0383] The system of the present invention includes a series of processes for searching and acquiring related information based on a user's natural language input and providing the results. The operation of the system is described below with specific examples.

[0384] Receiving user input

[0385] User: Asks questions in natural language to a smartphone or car navigation system with the application installed. For example, the user might type, "Tell me about some popular restaurants within five minutes of my current location where a family of four can spend half a day."

[0386] Sending input to the server

[0387] Terminal: In the case of voice input, the voice recognition function is used to convert it into text, and in the case of text input, the user's input is sent to the server as text data as is.

[0388] Natural Language Processing and Parameter Generation

[0389] Server: Sends the received text input to a natural language processing (NLP) model to analyze the user's intent, such as extracting parameters like current location, travel time (distance), family-friendly, store, popularity, etc.

[0390] For example, the NLP model understands that the user wants to know information about fun things to do with a family of four within a five-minute walk from their current location.

[0391] Retrieving information from a database

[0392] Server: Based on the parsed parameters, retrieve relevant information from geographic databases and external APIs (e.g., map APIs, review site APIs). For example, retrieve data on popular family-friendly restaurants and amusement spots within the specified area.

[0393] Filtering and Ranking

[0394] Server: From the acquired information, select the information that best suits the user's intent and needs. For example, filtering for family-friendly facilities with recent high ratings.

[0395] The selected information is then ranked based on evaluation criteria (recent popularity, number of reviews, etc.).

[0396] Sending and Displaying Information

[0397] Server: Formats the filtered and ranked results and sends them to the device.

[0398] Terminal: Displays the received results visually to the user and can also provide audio information if required.

[0399] For example, it displays detailed information (location, ratings, recent reviews) about "Restaurant A" which is three minutes away from the user's current location, and "Amusement Park B" which is four minutes away.

[0400] User Interactions

[0401] Users can use the information provided to learn more or request directions.

[0402] On the device: Displays detailed information and directions upon user request.

[0403] The system allows users to easily search and discover geographic entities that match specific criteria, providing new place discovery and fun experiences.

[0404] The processing flow will be explained below.

[0405] Step 1:

[0406] User: In the chat window of their smartphone or car navigation app, they type a question in natural language: "Can you tell me about some popular restaurants within five minutes of our current location that a family of four can enjoy for half a day?"

[0407] Step 2:

[0408] Device: In the case of voice input, the voice recognition function is used to convert it into text, and in the case of text input, the input is received as is. An API call is made to send the received input data to the server.

[0409] Step 3:

[0410] Server: Sends the received text data to a natural language processing (NLP) model, which analyzes the text, extracts user intent, and generates search parameters (such as current location, distance, entity attributes, and popularity).

[0411] Step 4:

[0412] Server: Retrieves GPS information sent from the device, identifies the user's current location (latitude and longitude), calculates the distance traveled within the next 5 minutes (a walking distance of about 400 meters), and constructs a search query.

[0413] Step 5:

[0414] Server: Using the constructed search query, retrieves relevant information from geographic databases and external APIs (e.g., map APIs, review site APIs). Retrieved information includes popular family-friendly restaurants and amusement spots within the specified area.

[0415] Step 6:

[0416] Server: Based on the retrieved relevant information, the server filters the information according to the user's request, for example, selecting family-friendly spots with recent high ratings.

[0417] Step 7:

[0418] Server: Ranks the filtered information based on criteria (recent popularity, number of reviews, etc.). This results in raw data being shaped and sorted in order of most relevance to the user.

[0419] Step 8:

[0420] Server: Formats the ranked results and sends them to the user's device. The results are provided in a user-friendly format (e.g., JSON format).

[0421] Step 9:

[0422] Device: The device displays the received results in a user interface, highlighting spots within the specified area on a map and allowing users to tap on the spots for more information.

[0423] Step 10:

[0424] User: Selects points of interest from the presented information and requests more information and directions.

[0425] Step 11:

[0426] Device: Sends a request to the server again in response to additional requests from the user (more information or directions).

[0427] Step 12:

[0428] Server: Retrieves detailed data about the selected spot from a database or external API and sends it to the device.

[0429] Step 13:

[0430] Terminal: The device displays the received detailed data in a user interface, including reviews, opening hours, and services offered by the spot.

[0431] In this way, through the series of processes from step 1 to step 13, users can obtain and confirm the most appropriate information by asking questions in natural language.

[0432] Example 1

[0433] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0434] Conventional geographic information search systems have difficulty accurately analyzing users' natural language input and quickly providing the information most relevant to their intent. Furthermore, they suffer from insufficient ranking of search results and detailed information, which hinders user experience. Furthermore, the lack of voice input support and route guidance functions also contributes to a lack of user convenience.

[0435] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0436] In this invention, the server includes means for receiving a user's natural language input, means for analyzing the received natural language input to generate search parameters, and means for acquiring related information from information sources based on the generated search parameters. This enables quick search for and acquisition of information most suited to the user's intent. The server also includes means for filtering and ranking the acquired information and means for displaying the filtered and ranked information to the user. This allows the user to easily browse the most suited information. Furthermore, by including means for providing detailed information and route guidance in response to additional user requests, user convenience is significantly improved. In addition, by including means for converting a user's voice input into text data and means for analyzing the user's intent using a natural language processing model, it is possible to support voice input and analyze intent with high accuracy.

[0437] "User" means a person who uses the system to search for information.

[0438] "Natural language input" is a format in which users input information using everyday language.

[0439] "Server" means a computer system with a central function that receives input from users and performs analysis, retrieval, filtering, ranking, and display of relevant information.

[0440] "Means" refers to a specific method or device by which a system performs a specific function or operation.

[0441] "Search parameters" refer to the conditions or criteria used in a search that are parsed from a user's natural language input.

[0442] "Sources" refer to databases and external API services that provide relevant information.

[0443] "Relevant Information" refers to information useful to the User that is obtained from a source based on the search parameters.

[0444] "Filtering" is the process of selecting necessary information from the acquired related information.

[0445] "Ranking" is the process of ranking the filtered information based on specific criteria.

[0446] "More Information" refers to more specific and detailed information provided in response to a User's additional request.

[0447] "Route guidance" refers to navigation information that shows the route to a destination specified by the user.

[0448] "Means for converting voice input into text data" refers to technology that converts a user's voice input into a string of characters.

[0449] A "natural language processing model" is an artificial intelligence model that analyzes a user's natural language input and understands its meaning and intent.

[0450] The system of the present invention includes a series of processes for searching and acquiring related information based on natural language input from a user and providing the results. Below, an embodiment of this system will be described with specific examples.

[0451] First, the user uses an application installed on their smartphone or car navigation system to input a question in natural language, for example, "Tell me about a popular restaurant within five minutes of my current location where a family of four can spend half a day," either by voice or text.

[0452] In the case of voice input, the device converts the voice into text using voice recognition software (e.g., Google Speech Recognition API). In the case of text input, the device processes the voice as text data as is. The converted or acquired text data is sent to the server.

[0453] The server sends the received text data to a natural language processing model (e.g., Google Cloud Natural Language API) to analyze the user's intent, extracting parameters from the text such as current location, travel time, family-friendly, and highly rated restaurants.

[0454] Next, based on the analysis results, the server retrieves relevant information from geographic databases and external APIs (e.g., map APIs or review site APIs). It creates queries to gather the necessary information. For example, it retrieves information about popular family-friendly restaurants and amusement facilities within a five-minute walk from the user's current location.

[0455] The server filters the information it receives and selects the information that best suits the user's intent and requirements. After filtering, it ranks the information based on evaluation criteria (e.g., recent popularity, number of reviews).

[0456] The server formats the ranked information and sends it to the device. The device displays this information visually to the user. It also has the ability to provide audio information if needed. For example, the device might display detailed information (location, ratings, recent reviews) about "Restaurant A" that is three minutes away from the user's current location, or "Amusement Park B" that is four minutes away.

[0457] Users can view more details or request route guidance from the information provided. The device will then display the details or route guidance as requested by the user. For example, it can launch Google Maps and display directions to a specified location.

[0458] As a concrete example, if a user inputs the prompt "Tell me about a highly rated cafe nearby," the device converts this speech into text and sends it to the server. The server analyzes it using a natural language processing model and extracts the parameters "highly rated cafe" and "nearby." Next, it retrieves information about highly rated cafes near the user's current location from a geographic information database and a rating site API. This information is filtered and ranked, and finally, information about "Cafe C" and "Cafe D" is displayed to the user. If the user requests "Tell me the way to Cafe C," the device launches a map app and displays route guidance.

[0459] This system allows users to easily find the right location for their needs, providing a more convenient and comfortable experience.

[0460] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0461] Step 1:

[0462] The user launches an application installed on their smartphone or car navigation system and asks for the information they need in natural language. For example, they might say, "Tell me about a popular restaurant within five minutes of my current location where a family of four can spend half a day." Input: A question in natural language (voice or text). Output: The content of the user's question.

[0463] Step 2:

[0464] The device converts voice input into text data using voice recognition software (e.g., voice recognition API). In the case of text input, it is processed as text data as is. Since text data is generated through voice recognition, the content of the user's question becomes clear. Input: Voice data (in the case of voice input). Output: Text data.

[0465] Step 3:

[0466] The terminal sends the generated text data to the server. A data transmission protocol is used to deliver the text data to the server quickly and reliably. Input: Text data (question content). Output: Sending text data to the server.

[0467] Step 4:

[0468] The server sends the received text data to a natural language processing model (e.g., natural language processing API) to analyze the user's intent. Search parameters such as current location, travel time, family-friendly, and highly rated shops are extracted from the text data. Input: Text data. Output: Search parameters.

[0469] Step 5:

[0470] The server retrieves related information from geographic information databases and external APIs (e.g., map APIs, review site APIs) based on the search parameters obtained as a result of the analysis. Specifically, it collects information on popular family-friendly restaurants and amusement facilities that can be enjoyed within a five-minute walk from the user's current location. Input: Search parameters. Output: Related information data.

[0471] Step 6:

[0472] The server filters the retrieved related information and selects information based on specific criteria (e.g., highly rated restaurants, family-friendly restaurants, etc.). It then ranks the filtered information based on evaluation criteria (e.g., recent popularity, number of reviews). Input: Related information data. Output: Filtered and ranked information.

[0473] Step 7:

[0474] The server formats the ranked results and sends them to the terminal. The display format is adjusted to make it easy for the user to understand. Input: Ranked information. Output: Formatted information.

[0475] Step 8:

[0476] The device displays the received formatted information to the user visually and audibly. For example, detailed information (location, rating, recent reviews) for "Restaurant A" and "Amusement Park B" is displayed to the user. Input: Formatted information. Output: Display to the user (visual, audio).

[0477] Step 9:

[0478] Users can request more details or directions from the information provided. For example, they might enter commands like "Tell me more about Restaurant A" or "Show me directions to Amusement Park B." Input: A request for more information. Output: A request for more information or directions.

[0479] Step 10:

[0480] The device displays more information or directions at the user's request, for example, launching a maps app to show routes to a specified location. Input: Request for more information. Output: More information or directions.

[0481] (Application example 1)

[0482] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0483] Conventional information search systems that use natural language input lack a mechanism for efficiently analyzing user voice input and generating appropriate search parameters. Furthermore, there are no concrete examples of providing users with the acquired information visually or audibly, or of acquiring information from external APIs, making it difficult to provide accurate information in real time. This results in problems such as reduced user convenience and satisfaction.

[0484] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0485] In this invention, the server includes means for receiving natural language input from a user using voice recognition hardware, means for analyzing the received natural language input and generating search parameters using a natural language processing model, means for retrieving related information from an external API based on the generated search parameters, means for filtering and ranking the retrieved related information, and means for displaying the filtered and ranked information to the user audibly and visually, thereby enabling efficient analysis of the user's voice input, retrieval of information from the external API, and provision of appropriate information visually and audibly to the user in real time.

[0486] "Speech recognition hardware" means a device for receiving a user's voice input and converting it into voice data.

[0487] "Natural language input" refers to text or voice instructions or queries given by a user in the language they normally use.

[0488] A "natural language processing model" is an algorithm or program that analyzes received natural language input and understands the user's intent and meaning.

[0489] "Search parameters" are conditions or criteria generated based on user input for retrieving relevant information from databases or APIs.

[0490] An "external API" is a protocol for accessing data or functionality through an interface provided by an external service or application.

[0491] "Relevant information" refers to useful data retrieved from databases and external APIs based on the user's search parameters.

[0492] "Filtering" is the process of selecting relevant information obtained based on certain criteria and removing unnecessary information.

[0493] "Ranking" is the process of evaluating the filtered relevant information based on specific criteria and sorting it according to priority.

[0494] "Visual presentation means" refers to a method or device for displaying acquired information in a manner that appeals to the user's visual sense.

[0495] "Means for providing audio information" refers to a method or device for providing acquired information in a form that appeals to the user's auditory sense.

[0496] The present invention relates to a system for searching for information based on a user's natural language input and providing the search results. Specific embodiments are described below.

[0497] First, the main hardware components of the system are the speech recognition hardware, server, and terminal. Specifically, they are a microphone for users to input speech, the "SpeechRecognition" speech recognition software library for converting input speech into text, and an NLP model for natural language processing. The "requests" library is used for API requests.

[0498] Receiving audio input

[0499] The user gives instructions in natural language through speech recognition hardware (microphone), for example, "Tell me about nearby family-friendly attractions."

[0500] Speech-to-text

[0501] On-device speech recognition software (SpeechRecognition) is used to convert the received voice input into text, which is then sent to the server.

[0502] natural language analysis

[0503] The server sends the received text to a natural language processing model (NLP model) to analyze the user's intent. The NLP model analyzes the user's request and extracts search parameters (e.g., current location, family-friendly, attractions).

[0504] Obtaining information from external APIs

[0505] Based on the generated search parameters, the server calls external APIs (e.g., geographic information APIs or review site APIs) to retrieve relevant information, thereby gathering information that matches the specified criteria (e.g., family-friendly attractions).

[0506] Information filtering and ranking

[0507] The acquired information is filtered and ranked on the server. The filtering selects information that matches the user's intent, and the ranking prioritizes it based on criteria such as the number of reviews and rating points.

[0508] Providing information

[0509] Finally, the server sends the filtered and ranked results to the device, which then presents them to the user via audio and visual displays, such as a list of attractions near the user's current location and highly rated spots, and audio guidance when needed.

[0510] Specific examples

[0511] When a user says, "Tell me the route to the next rest stop" in their car, the system recognizes the speech and a natural language processing model extracts parameters such as "rest stop" and "route." Next, it uses a geographic information API to obtain information on the best rest stops, filters it, and ranks them. As a result, the best rest stop candidates are displayed on the user's navigation system. At the same time, a voice guide tells the user, "The best rest stop that can be reached within 15 minutes from your current location is XX."

[0512] Prompt Sentence Examples

[0513] "Where's a nearby park where I can relax?"

[0514] "Tell me the route to the next rest stop."

[0515] This allows users to obtain optimal information in real time, enabling efficient navigation.

[0516] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0517] Step 1: Receiving Audio Input

[0518] The device receives the user's voice input through the voice recognition hardware (microphone). For example, the user might say, "Tell me about nearby family attractions." The input is analog voice data, which is converted to digital data in the next step.

[0519] Step 2: Speech to text

[0520] The device uses speech recognition software (SpeechRecognition library) to convert the received voice data into digital text. The input is voice data, and the output is text data such as "Tell me about nearby family-friendly attractions." This conversion is performed by the speech recognition software's algorithm.

[0521] Step 3: Natural Language Analysis

[0522] The server receives the text data sent from the device and analyzes it using a natural language processing model (NLP model). Specifically, it extracts search parameters from the text. The input is the text "Tell me about nearby attractions that families can enjoy," and the output is search parameters such as "current location," "family-friendly," and "attractions."

[0523] Step 4: Retrieving information from an external API

[0524] Based on the generated search parameters, the server calls external APIs (such as geographical information APIs and review site APIs) to retrieve relevant information. The input is the search parameters, and the output is data about family-friendly attractions within the specified range. The retrieved information includes location, ratings, and recent reviews.

[0525] Step 5: Filtering and ranking information

[0526] The server filters and ranks the acquired information. Filtering selects only information that matches criteria such as "family-friendly" or "highly rated." Next, ranking prioritizes the information based on the number of reviews and ratings. The input is the acquired attraction information, and the output is a filtered and ranked list of attractions.

[0527] Step 6: Provide information

[0528] The server sends the filtered and ranked information to the terminal, which then presents it to the user audibly and visually. The input is the filtered and ranked list of attractions, and the output is the specific attraction information displayed to the user. Specifically, the terminal will provide a voice prompt saying, "Attractions that can be reached within 15 minutes from your current location are XX," and also display the information on a map.

[0529] The above are the specific steps for carrying out the invention. This processing flow allows users to obtain optimal information in real time, realizing efficient navigation.

[0530] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0531] The system of the present invention uses a method to recognize natural language input from a user and the user's emotions and provide relevant information. By combining this system with an emotion engine, it is possible to provide more personalized information according to the user's emotions. The operation of the system is explained below with a concrete example.

[0532] Receiving user input

[0533] User: In the chat window of a smartphone or car navigation app, type a question in natural language, such as, "Tell me about a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day." As the user types, the system detects the user's emotions from the voice and text.

[0534] Sending input to the server

[0535] Terminal: Recognizes the user's voice input and converts it into text, or receives text input as is, and sends the input data and emotion data to the server.

[0536] Natural Language Processing and Emotion Recognition

[0537] Server: The received text data is sent to a natural language processing (NLP) model to analyze the user's intent. At the same time, the emotion engine analyzes the user's emotion data and identifies the user's emotion. For example, if the user is feeling stressed, it will be recognized as "tension" or "fatigue."

[0538] Parameter generation and tuning

[0539] Server: Generates search parameters from the parsed natural language input and adjusts them based on the user's emotional data. For example, if the user is tired, parameters that prioritize "relaxing places" and "quiet environments" are added.

[0540] Retrieving information from a database

[0541] Server: Based on the adjusted search parameters, retrieves relevant information from geographic databases and external APIs, such as data on popular family-friendly, relaxing restaurants and attractions within the specified radius.

[0542] Filtering and Ranking

[0543] Server: Based on the retrieved relevant information, it filters and ranks the information that best matches the specified emotional state. Evaluation criteria include recent popularity, user reviews, and emotional fitness.

[0544] Sending and Displaying Information

[0545] Server: Formats the filtered and ranked results and sends them to the user's device.

[0546] Device: The received results are displayed in the user interface. Spots within the specified area are highlighted on the map, and users can tap on them to view more information. Additionally, special recommendations based on the user's emotions are also displayed.

[0547] User Interactions

[0548] Users can select points of interest from the information presented and request more information or directions.

[0549] Device: Sends these additional requests back to the server to retrieve more information and directions to provide to the user.

[0550] In this way, the system of the present invention combines natural language input from users with sentiment analysis to provide optimal information, enabling users to obtain more personalized search results, leading to new discoveries and a richer experience.

[0551] The processing flow will be explained below.

[0552] Step 1:

[0553] User: In the chat window of their smartphone or car navigation app, they type a question in natural language, such as, "Tell me about a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day." As they type, the speech recognition or text analysis module works to detect emotions from the user's voice and text.

[0554] Step 2:

[0555] Device: In the case of voice input, the device uses the voice recognition function to convert the input into text, and in the case of text input, the device receives the input as text data. An API call is made to send the received natural language data and emotion data to the server.

[0556] Step 3:

[0557] Server: Sends the received text data to a natural language processing (NLP) model to analyze the user's intent. At the same time, the emotion engine analyzes the user's emotion data and identifies the user's emotion as "tension," "fatigue," "joy," etc.

[0558] Step 4:

[0559] Server: Generates search parameters (e.g., current location, distance, entity attributes, popularity) from the analysis results of the NLP model. Additionally, adjusts specific search parameters to match the user's emotional state based on the results of the emotion engine. For example, if the user is tired, add parameters that prioritize "places where you can relax" and "quiet environments."

[0560] Step 5:

[0561] Server: Based on the generated and adjusted search parameters, retrieve relevant information from geographic databases and external APIs (e.g., map APIs, review site APIs). For example, retrieve data on popular family-friendly, relaxing restaurants and attractions within the specified radius.

[0562] Step 6:

[0563] Server: Filters the retrieved relevant information based on the user's request and emotional state, for example, selecting family-friendly spots with a particularly "relaxing" or "quiet" environment.

[0564] Step 7:

[0565] Server: Ranks the filtered information based on criteria (e.g., recent popularity, number of reviews, emotional fitness), with specific weightings applied based on the user's emotional state.

[0566] Step 8:

[0567] Server: Formats the ranked results and sends them to the user's device. The result data is formatted in a user-friendly format (e.g., JSON format).

[0568] Step 9:

[0569] Device: The received results are displayed in the user interface. Spots within the specified area are highlighted on a map, and users can tap on them to view more information. Special recommendations based on the user's emotions are also displayed.

[0570] Step 10:

[0571] Users can select points of interest from the information presented and request more information or directions.

[0572] Step 11:

[0573] Device: Sends the user's additional request (e.g., more information or directions) back to the server.

[0574] Step 12:

[0575] Server: Retrieves detailed data about the selected spot from a database or external API and sends the latest detailed data to the device.

[0576] Step 13:

[0577] Terminal: The device displays the received detailed data in a user interface, including the location's reviews, opening hours, and services offered.

[0578] Through these steps, the system of the present invention can provide optimal information by taking into account the user's natural language input and emotions, allowing the user to obtain more personalized search results and enjoy a comfortable and novel experience.

[0579] Example 2

[0580] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0581] Conventional systems analyze users' natural language input to provide information, but they are unable to provide information that takes into account the user's emotions, making it difficult to provide personalized information that meets the user's needs.In addition, they are unable to adjust search parameters based on the user's emotions, making it difficult to provide more appropriate information.

[0582] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving natural language input from a user, means for analyzing the received natural language input to generate search parameters, means for analyzing emotion data of the user and identifying the emotion, means for adjusting the search parameters based on the analyzed emotion data, means for acquiring related information from a database based on the generated and adjusted search parameters, means for filtering and ranking the acquired related information, and means for displaying the filtered and ranked information to the user. This makes it possible to provide personalized information according to the user's emotions.

[0583] "Natural language input" refers to data entered as voice or text in a language that the user normally uses.

[0584] "Search parameters" are criteria or conditions for information search that are extracted by analyzing the user's natural language input.

[0585] "Emotional Data" means data that indicates an emotional state detected from a user's input or behavior.

[0586] "Emotion identification" refers to analyzing a user's emotion data to identify a specific emotional state (e.g., joy, sadness, tension, fatigue, etc.).

[0587] "Means for adjusting" refers to a method or device for modifying search parameters and providing appropriate information based on the identified emotion.

[0588] A "database" is a structured collection of data that allows for efficient retrieval, storage, and management of information.

[0589] "Filtering" refers to the process of removing unnecessary information from acquired related information according to specific criteria.

[0590] "Ranking" refers to the process of ordering filtered information based on predetermined evaluation criteria.

[0591] "Means for displaying to the user" refers to a method or device for visually displaying the final selected information on the user's terminal.

[0592] The present invention is a system that analyzes data and emotion data input by a user in natural language and provides optimal information based on the analysis. This system can be implemented using the following hardware and software.

[0593] First, the user inputs natural language using a device such as a smartphone or car navigation system, such as "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day at." This input includes both voice and text input. In the case of voice input, the device uses a speech recognition system to convert speech into text. Specifically, a speech recognition tool such as the Google Cloud Speech-to-Text API is used.

[0594] The device then sends the received input data to a server, which uses Apache OpenNLP or a similar natural language processing (NLP) model to analyze the received text data and Microsoft Azure Emotion API or a similar emotion analysis tool to analyze the user's emotional data and identify their emotions.

[0595] The server generates search parameters based on the analyzed natural language input and emotional data. The search parameters are then adjusted according to the emotional data. For example, if the user feels "tired," parameters that prioritize "relaxing places" and "quiet environments" are added.

[0596] The server then retrieves relevant information from geographic databases and external APIs (e.g., Google Places API) based on the generated and adjusted search parameters, and the retrieved information is filtered and ranked based on evaluation criteria such as user emotional relevance and popularity.

[0597] The server sends the filtered and ranked information to the device, which displays it in a user interface. Spots are indicated on a map with markers, and users can tap them to view more information.

[0598] For example, a user types into their smartphone, "I'm tired, can you recommend a quiet cafe nearby?" Based on this prompt, the system will recognize the user's emotion as "tired" and add the search parameter "quiet cafe." Finally, it will suggest a quiet cafe near the user's current location, which the user can view on a map.

[0599] In this way, the system of the present invention can combine the user's natural language input with emotion analysis to provide information that best suits the user's emotional state, allowing the user to obtain more personalized information and enjoy a pleasant and beneficial experience.

[0600] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0601] Step 1: Receiving User Input

[0602] User: Using the chat function on their smartphone or car navigation system, they input information in natural language, such as, "Please tell me about some popular restaurants within five minutes of our current location where a family of four can spend half a day."

[0603] Input: The user types a question into the device by voice or text.

[0604] What it does: Users speak into their smartphone's microphone or type a question into a text field.

[0605] Step 2: Sending input to the server

[0606] Device: For voice input, convert the input voice into text using the Google Cloud Speech-to-Text API, etc. Then, send the input data (text) to the server.

[0607] Data processing: Converting speech to text and structuring it as natural language input from the user.

[0608] Output: Structured text data.

[0609] What it does: The device converts the user's voice into text and sends that text to the server.

[0610] Step 3: Natural Language Processing and Emotion Recognition

[0611] Server: Analyzes the received text data and extracts the user's intent using a natural language processing (NLP) model such as Apache OpenNLP, and identifies the user's emotions using a sentiment analysis tool (e.g., Microsoft Azure Emotion API).

[0612] Data Computing: Analyzes natural language input to extract keywords and intent, while simultaneously analyzing sentiment data to identify emotional states.

[0613] Output: Search parameters and emotional state based on user intent.

[0614] How it works: The server sends the text "Tell me about popular spots these days" to a natural language processing model to extract search parameters for the destination. It also uses an emotion analysis model to identify emotional states such as "tired."

[0615] Step 4: Generate and adjust parameters

[0616] Server: Generates search parameters from the parsed natural language input and adjusts the search parameters based on sentiment data.

[0617] Data processing: Generate search parameters and adjust priorities according to emotional state.

[0618] Output: The adjusted search parameters.

[0619] Specific behavior: The server adds conditions to the generated parameters that prioritize "relaxing places" and "quiet environments."

[0620] Step 5: Retrieving information from the database

[0621] Server: Based on the adjusted search parameters, retrieves relevant information from geographic databases and external APIs (e.g., Google Places API).

[0622] Data Calculation: Generates database queries based on parameters to retrieve relevant information.

[0623] Output: A list of the relevant information retrieved.

[0624] Specific operation: The server retrieves information about places that meet certain conditions, such as "quiet cafes," from a database or API.

[0625] Step 6: Filtering and ranking

[0626] Server: Filters the retrieved relevant information to best match the emotional state and ranks it based on evaluation criteria.

[0627] Data processing: Filter the acquired information and rank it based on emotional fitness and popularity.

[0628] Output: A list of filtered and ranked information.

[0629] How it works: The server filters the information it receives, lists places that match criteria such as "relaxing," and ranks them based on evaluation criteria.

[0630] Step 7: Send and view information

[0631] Server: Formats the filtered and ranked results and sends them to the user's device.

[0632] Device: The received results are displayed on the user interface. Spots within the specified area are indicated on the map with markers, and users can tap on a spot to view more information.

[0633] Output: Information displayed on the user interface.

[0634] Specific operation: The location of the cafe will be displayed on a map on the smartphone screen, and detailed information will be displayed in a pop-up.

[0635] Step 8: User Interaction

[0636] User: Users can select points of interest from the information presented and request more information or directions.

[0637] Device: Sends additional requests to the server again to get more information and directions to provide to the user.

[0638] Data calculations: Generate detailed information and directions based on user requests.

[0639] Output: Detailed information and directions.

[0640] Specific operation: When a user requests "Tell me the route to this cafe," the device sends a request to the server and the optimal route is displayed.

[0641] As a result, the entire system becomes a process that provides optimal information based on the user's emotions.

[0642] (Application example 2)

[0643] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0644] In recent years, there has been an increasing demand to meet diverse consumer needs and personalize daily shopping experiences. Conventional systems provide information based on the user's natural language input. However, these systems do not take into account the user's emotional state, which means they are unable to fully enhance the user's psychological satisfaction. Therefore, there is a need for a system that can detect the user's emotions and provide optimal information based on them.

[0645] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a natural language input and emotion data from a user, means for analyzing the received natural language input and emotion data to generate search parameters, and means for retrieving related information from a database based on the generated search parameters. This makes it possible to provide personalized related information based on the user's natural language input and emotion data.

[0646] "User" refers to an individual consumer who uses the system.

[0647] "Natural language input" refers to language data that is input in the form of everyday conversation.

[0648] "Emotional data" refers to data that indicates a user's psychological state analyzed based on their input and behavior.

[0649] "Search parameters" refer to criteria used to retrieve information from a database.

[0650] "Database" refers to a collection of data that stores related information and makes it searchable and retrievalable.

[0651] "Filtering" refers to the process of selecting necessary information from a large amount of information according to specific criteria.

[0652] "Ranking" refers to the process of ranking selected information according to certain criteria.

[0653] A "natural language processing model" refers to an algorithm or system that analyzes natural language and understands user intent.

[0654] An "emotion analysis engine" refers to an algorithm or system for analyzing a user's emotional state from their input and behavior.

[0655] The system of the present invention analyzes a user's natural language input and emotion data, and provides information suited to the user's needs. The system includes the following means.

[0656] 1. Input receiving means

[0657] A user uses a smartphone application to provide natural language input via voice or text, for example, a query of the form "I want to relax today, can you tell me where the nearest cafe is?" This input can be received as voice or directly as text.

[0658] 2. Data transmission method

[0659] The user's natural language input and accompanying emotional data are sent from the smartphone device to a server, where voice input is converted into text using speech recognition technology, and then an emotional analysis engine identifies the user's emotional state.

[0660] 3. Analysis and search parameter generation method

[0661] The server analyzes the received text data using a natural language processing (NLP) model to extract the user's intent. Furthermore, a sentiment analysis engine analyzes the emotional data to identify the user's specific emotional state, such as "I want to relax" or "I'm tired." Search parameters are generated based on this information.

[0662] 4. Database information acquisition method

[0663] Based on the search parameters, the server retrieves relevant information from external databases and APIs, such as the location, opening hours, and reputation of cafes. The retrieved information is also adjusted according to emotional data. For example, if a user is looking for a relaxing space, quiet cafes will be prioritized.

[0664] 5. Information filtering and ranking methods

[0665] The retrieved relevant information is filtered on the server, and information that best matches the user's emotional state is prioritized. Other criteria, such as reputation and popularity, are also taken into account when ranking the information. For example, cafes that are rated as "quiet" and have high reviews will be ranked higher.

[0666] 6. Information display means

[0667] The filtered and ranked results are formatted and sent to the user's smartphone, where they can view a map view and detailed information within the application interface.

[0668] Hardware and software used

[0669] Hardware: Smartphones, servers

[0670] Software: speech recognition technology (e.g., Google Speech-to-Text API), natural language processing models (e.g., spaCy), sentiment analysis engines (e.g., Hugging Face Transformers)

[0671] Examples of specific examples and prompts

[0672] As a concrete example, suppose a user types into a smartphone application, "I want to relax today, so tell me where the nearest cafe is."

[0673] Example prompt sentence:

[0674] I want to relax today, so please tell me about a nearby cafe.

[0675] A cloud-based server receives the prompt and performs natural language processing and sentiment analysis to suggest the best cafe to match the user's preferences. This allows users to find the location and products that best suit their emotional state, resulting in a highly satisfying experience.

[0676] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0677] Step 1:

[0678] Receiving user input

[0679] A user uses a smartphone application to input natural language. The input format is text or voice, for example, "I want to relax today, so please tell me about a nearby cafe." The input data is received by the device. Input: User's natural language input (text or voice), Output: Received input data.

[0680] Step 2:

[0681] Speech-to-text conversion (for voice input)

[0682] For voice input, speech recognition technology is used to convert the voice data into text. For example, Google Speech-to-Text API is used to convert the voice into text data. Input: Voice data, Output: Text data.

[0683] Step 3:

[0684] Natural Language Processing

[0685] The received text data is analyzed using a natural language processing model (e.g., spaCy) on the server to extract the user's intent. This analysis identifies important keywords and propositions. Input: text data, output: keywords and propositions.

[0686] Step 4:

[0687] Emotion analysis

[0688] The server uses an emotion analysis engine (e.g., Hugging Face Transformers) to identify the emotional state from the user's text input. For example, it determines whether the input indicates the emotion "relaxed." Input: text data, Output: emotion data (e.g., "relaxed").

[0689] Step 5:

[0690] Search parameter generation

[0691] Generate search parameters based on the results of natural language processing and sentiment analysis. For example, set specific search conditions such as "relaxing cafes" or "nearby places." Input: Keywords, sentiment data, output: search parameters.

[0692] Step 6:

[0693] Database information acquisition

[0694] The server retrieves relevant information from external databases or APIs based on the search parameters. For example, it collects the location, opening hours, and reputation of the cafe. Input: Search parameters, Output: Related information data.

[0695] Step 7:

[0696] Information Filtering

[0697] The acquired related information is filtered on the server, and the information that best matches the user's emotional state is selected. For example, "a quiet cafe with a good reputation" is selected as the filtering result. Input: related information data, Output: filtered information.

[0698] Step 8:

[0699] Information Ranking

[0700] The filtered information is ranked based on criteria such as reputation and popularity. For example, cafes with high reviews are ranked higher. Input: Filtered information, Output: Ranked information.

[0701] Step 9:

[0702] Information transmission

[0703] The server formats the ranking information and sends it to the user's smartphone. A map display and detailed information are displayed on the user's screen. Input: Ranking information, Output: Information displayed on the smartphone.

[0704] Step 10:

[0705] User Interaction

[0706] The user can review the information sent and request points of interest or route guidance. The device again communicates with the server to provide additional information or route guidance. Input: User's additional request, Output: Additional information or route guidance.

[0707] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0708] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0709] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0710] [Third embodiment]

[0711] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0712] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0713] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0714] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0715] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0716] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0717] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0718] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0719] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0720] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0721] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0722] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0723] The system of the present invention includes a series of processes for searching and acquiring related information based on a user's natural language input and providing the results. The operation of the system is described below with specific examples.

[0724] Receiving user input

[0725] User: Asks questions in natural language to a smartphone or car navigation system with the application installed. For example, the user might type, "Tell me about some popular restaurants within five minutes of my current location where a family of four can spend half a day."

[0726] Sending input to the server

[0727] Terminal: In the case of voice input, the voice recognition function is used to convert it into text, and in the case of text input, the user's input is sent to the server as text data as is.

[0728] Natural Language Processing and Parameter Generation

[0729] Server: Sends the received text input to a natural language processing (NLP) model to analyze the user's intent, such as extracting parameters like current location, travel time (distance), family-friendly, store, popularity, etc.

[0730] For example, the NLP model understands that the user wants to know information about fun things to do with a family of four within a five-minute walk from their current location.

[0731] Retrieving information from a database

[0732] Server: Based on the parsed parameters, retrieve relevant information from geographic databases and external APIs (e.g., map APIs, review site APIs). For example, retrieve data on popular family-friendly restaurants and amusement spots within the specified area.

[0733] Filtering and Ranking

[0734] Server: From the acquired information, select the information that best suits the user's intent and needs. For example, filtering for family-friendly facilities with recent high ratings.

[0735] The selected information is then ranked based on evaluation criteria (recent popularity, number of reviews, etc.).

[0736] Sending and Displaying Information

[0737] Server: Formats the filtered and ranked results and sends them to the device.

[0738] Terminal: Displays the received results visually to the user and can also provide audio information if required.

[0739] For example, it displays detailed information (location, ratings, recent reviews) about "Restaurant A" which is three minutes away from the user's current location, and "Amusement Park B" which is four minutes away.

[0740] User Interactions

[0741] Users can use the information provided to learn more or request directions.

[0742] On the device: Displays detailed information and directions upon user request.

[0743] The system allows users to easily search and discover geographic entities that match specific criteria, providing new place discovery and fun experiences.

[0744] The processing flow will be explained below.

[0745] Step 1:

[0746] User: In the chat window of their smartphone or car navigation app, they type a question in natural language: "Can you tell me about some popular restaurants within five minutes of our current location that a family of four can enjoy for half a day?"

[0747] Step 2:

[0748] Device: In the case of voice input, the voice recognition function is used to convert it into text, and in the case of text input, the input is received as is. An API call is made to send the received input data to the server.

[0749] Step 3:

[0750] Server: Sends the received text data to a natural language processing (NLP) model, which analyzes the text, extracts user intent, and generates search parameters (such as current location, distance, entity attributes, and popularity).

[0751] Step 4:

[0752] Server: Retrieves GPS information sent from the device, identifies the user's current location (latitude and longitude), calculates the distance traveled within the next 5 minutes (a walking distance of about 400 meters), and constructs a search query.

[0753] Step 5:

[0754] Server: Using the constructed search query, retrieves relevant information from geographic databases and external APIs (e.g., map APIs, review site APIs). Retrieved information includes popular family-friendly restaurants and amusement spots within the specified area.

[0755] Step 6:

[0756] Server: Based on the retrieved relevant information, the server filters the information according to the user's request, for example, selecting family-friendly spots with recent high ratings.

[0757] Step 7:

[0758] Server: Ranks the filtered information based on criteria (recent popularity, number of reviews, etc.). This results in raw data being shaped and sorted in order of most relevance to the user.

[0759] Step 8:

[0760] Server: Formats the ranked results and sends them to the user's device. The results are provided in a user-friendly format (e.g., JSON format).

[0761] Step 9:

[0762] Device: The device displays the received results in a user interface, highlighting spots within the specified area on a map and allowing users to tap on the spots for more information.

[0763] Step 10:

[0764] User: Selects points of interest from the presented information and requests more information and directions.

[0765] Step 11:

[0766] Device: Sends a request to the server again in response to additional requests from the user (more information or directions).

[0767] Step 12:

[0768] Server: Retrieves detailed data about the selected spot from a database or external API and sends it to the device.

[0769] Step 13:

[0770] Terminal: The device displays the received detailed data in a user interface, including reviews, opening hours, and services offered by the spot.

[0771] In this way, through the series of processes from step 1 to step 13, users can obtain and confirm the most appropriate information by asking questions in natural language.

[0772] Example 1

[0773] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0774] Conventional geographic information search systems have difficulty accurately analyzing users' natural language input and quickly providing the information most relevant to their intent. Furthermore, they suffer from insufficient ranking of search results and detailed information, which hinders user experience. Furthermore, the lack of voice input support and route guidance functions also contributes to a lack of user convenience.

[0775] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0776] In this invention, the server includes means for receiving a user's natural language input, means for analyzing the received natural language input to generate search parameters, and means for acquiring related information from information sources based on the generated search parameters. This enables quick search for and acquisition of information most suited to the user's intent. The server also includes means for filtering and ranking the acquired information and means for displaying the filtered and ranked information to the user. This allows the user to easily browse the most suited information. Furthermore, by including means for providing detailed information and route guidance in response to additional user requests, user convenience is significantly improved. In addition, by including means for converting a user's voice input into text data and means for analyzing the user's intent using a natural language processing model, it is possible to support voice input and analyze intent with high accuracy.

[0777] "User" means a person who uses the system to search for information.

[0778] "Natural language input" is a format in which users input information using everyday language.

[0779] "Server" means a computer system with a central function that receives input from users and performs analysis, retrieval, filtering, ranking, and display of relevant information.

[0780] "Means" refers to a specific method or device by which a system performs a specific function or operation.

[0781] "Search parameters" refer to the conditions or criteria used in a search that are parsed from a user's natural language input.

[0782] "Sources" refer to databases and external API services that provide relevant information.

[0783] "Relevant Information" refers to information useful to the User that is obtained from a source based on the search parameters.

[0784] "Filtering" is the process of selecting necessary information from the acquired related information.

[0785] "Ranking" is the process of ranking the filtered information based on specific criteria.

[0786] "More Information" refers to more specific and detailed information provided in response to a User's additional request.

[0787] "Route guidance" refers to navigation information that shows the route to a destination specified by the user.

[0788] "Means for converting voice input into text data" refers to technology that converts a user's voice input into a string of characters.

[0789] A "natural language processing model" is an artificial intelligence model that analyzes a user's natural language input and understands its meaning and intent.

[0790] The system of the present invention includes a series of processes for searching and acquiring related information based on natural language input from a user and providing the results. Below, an embodiment of this system will be described with specific examples.

[0791] First, the user uses an application installed on their smartphone or car navigation system to input a question in natural language, for example, "Tell me about a popular restaurant within five minutes of my current location where a family of four can spend half a day," either by voice or text.

[0792] In the case of voice input, the device converts the voice into text using voice recognition software (e.g., Google Speech Recognition API). In the case of text input, the device processes the voice as text data as is. The converted or acquired text data is sent to the server.

[0793] The server sends the received text data to a natural language processing model (e.g., Google Cloud Natural Language API) to analyze the user's intent, extracting parameters from the text such as current location, travel time, family-friendly, and highly rated restaurants.

[0794] Next, based on the analysis results, the server retrieves relevant information from geographic databases and external APIs (e.g., map APIs or review site APIs). It creates queries to gather the necessary information. For example, it retrieves information about popular family-friendly restaurants and amusement facilities within a five-minute walk from the user's current location.

[0795] The server filters the information it receives and selects the information that best suits the user's intent and requirements. After filtering, it ranks the information based on evaluation criteria (e.g., recent popularity, number of reviews).

[0796] The server formats the ranked information and sends it to the device. The device displays this information visually to the user. It also has the ability to provide audio information if needed. For example, the device might display detailed information (location, ratings, recent reviews) about "Restaurant A" that is three minutes away from the user's current location, or "Amusement Park B" that is four minutes away.

[0797] Users can view more details or request route guidance from the information provided. The device will then display the details or route guidance as requested by the user. For example, it can launch Google Maps and display directions to a specified location.

[0798] As a concrete example, if a user inputs the prompt "Tell me about a highly rated cafe nearby," the device converts this speech into text and sends it to the server. The server analyzes it using a natural language processing model and extracts the parameters "highly rated cafe" and "nearby." Next, it retrieves information about highly rated cafes near the user's current location from a geographic information database and a rating site API. This information is filtered and ranked, and finally, information about "Cafe C" and "Cafe D" is displayed to the user. If the user requests "Tell me the way to Cafe C," the device launches a map app and displays route guidance.

[0799] This system allows users to easily find the right location for their needs, providing a more convenient and comfortable experience.

[0800] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0801] Step 1:

[0802] The user launches an application installed on their smartphone or car navigation system and asks for the information they need in natural language. For example, they might say, "Tell me about a popular restaurant within five minutes of my current location where a family of four can spend half a day." Input: A question in natural language (voice or text). Output: The content of the user's question.

[0803] Step 2:

[0804] The device converts voice input into text data using voice recognition software (e.g., voice recognition API). In the case of text input, it is processed as text data as is. Since text data is generated through voice recognition, the content of the user's question becomes clear. Input: Voice data (in the case of voice input). Output: Text data.

[0805] Step 3:

[0806] The terminal sends the generated text data to the server. A data transmission protocol is used to deliver the text data to the server quickly and reliably. Input: Text data (question content). Output: Sending text data to the server.

[0807] Step 4:

[0808] The server sends the received text data to a natural language processing model (e.g., natural language processing API) to analyze the user's intent. Search parameters such as current location, travel time, family-friendly, and highly rated shops are extracted from the text data. Input: Text data. Output: Search parameters.

[0809] Step 5:

[0810] The server retrieves related information from geographic information databases and external APIs (e.g., map APIs, review site APIs) based on the search parameters obtained as a result of the analysis. Specifically, it collects information on popular family-friendly restaurants and amusement facilities that can be enjoyed within a five-minute walk from the user's current location. Input: Search parameters. Output: Related information data.

[0811] Step 6:

[0812] The server filters the retrieved related information and selects information based on specific criteria (e.g., highly rated restaurants, family-friendly restaurants, etc.). It then ranks the filtered information based on evaluation criteria (e.g., recent popularity, number of reviews). Input: Related information data. Output: Filtered and ranked information.

[0813] Step 7:

[0814] The server formats the ranked results and sends them to the terminal. The display format is adjusted to make it easy for the user to understand. Input: Ranked information. Output: Formatted information.

[0815] Step 8:

[0816] The device displays the received formatted information to the user visually and audibly. For example, detailed information (location, rating, recent reviews) for "Restaurant A" and "Amusement Park B" is displayed to the user. Input: Formatted information. Output: Display to the user (visual, audio).

[0817] Step 9:

[0818] Users can request more details or directions from the information provided. For example, they might enter commands like "Tell me more about Restaurant A" or "Show me directions to Amusement Park B." Input: A request for more information. Output: A request for more information or directions.

[0819] Step 10:

[0820] The device displays more information or directions at the user's request, for example, launching a maps app to show routes to a specified location. Input: Request for more information. Output: More information or directions.

[0821] (Application example 1)

[0822] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0823] Conventional information search systems that use natural language input lack a mechanism for efficiently analyzing user voice input and generating appropriate search parameters. Furthermore, there are no concrete examples of providing users with the acquired information visually or audibly, or of acquiring information from external APIs, making it difficult to provide accurate information in real time. This results in problems such as reduced user convenience and satisfaction.

[0824] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0825] In this invention, the server includes means for receiving natural language input from a user using voice recognition hardware, means for analyzing the received natural language input and generating search parameters using a natural language processing model, means for retrieving related information from an external API based on the generated search parameters, means for filtering and ranking the retrieved related information, and means for displaying the filtered and ranked information to the user audibly and visually, thereby enabling efficient analysis of the user's voice input, retrieval of information from the external API, and provision of appropriate information visually and audibly to the user in real time.

[0826] "Speech recognition hardware" means a device for receiving a user's voice input and converting it into voice data.

[0827] "Natural language input" refers to text or voice instructions or queries given by a user in the language they normally use.

[0828] A "natural language processing model" is an algorithm or program that analyzes received natural language input and understands the user's intent and meaning.

[0829] "Search parameters" are conditions or criteria generated based on user input for retrieving relevant information from databases or APIs.

[0830] An "external API" is a protocol for accessing data or functionality through an interface provided by an external service or application.

[0831] "Relevant information" refers to useful data retrieved from databases and external APIs based on the user's search parameters.

[0832] "Filtering" is the process of selecting relevant information obtained based on certain criteria and removing unnecessary information.

[0833] "Ranking" is the process of evaluating the filtered relevant information based on specific criteria and sorting it according to priority.

[0834] "Visual presentation means" refers to a method or device for displaying acquired information in a manner that appeals to the user's visual sense.

[0835] "Means for providing audio information" refers to a method or device for providing acquired information in a form that appeals to the user's auditory sense.

[0836] The present invention relates to a system for searching for information based on a user's natural language input and providing the search results. Specific embodiments are described below.

[0837] First, the main hardware components of the system are the speech recognition hardware, server, and terminal. Specifically, they are a microphone for users to input speech, the "SpeechRecognition" speech recognition software library for converting input speech into text, and an NLP model for natural language processing. The "requests" library is used for API requests.

[0838] Receiving audio input

[0839] The user gives instructions in natural language through speech recognition hardware (microphone), for example, "Tell me about nearby family-friendly attractions."

[0840] Speech-to-text

[0841] On-device speech recognition software (SpeechRecognition) is used to convert the received voice input into text, which is then sent to the server.

[0842] natural language analysis

[0843] The server sends the received text to a natural language processing model (NLP model) to analyze the user's intent. The NLP model analyzes the user's request and extracts search parameters (e.g., current location, family-friendly, attractions).

[0844] Obtaining information from external APIs

[0845] Based on the generated search parameters, the server calls external APIs (e.g., geographic information APIs or review site APIs) to retrieve relevant information, thereby gathering information that matches the specified criteria (e.g., family-friendly attractions).

[0846] Information filtering and ranking

[0847] The acquired information is filtered and ranked on the server. The filtering selects information that matches the user's intent, and the ranking prioritizes it based on criteria such as the number of reviews and rating points.

[0848] Providing information

[0849] Finally, the server sends the filtered and ranked results to the device, which then presents them to the user via audio and visual displays, such as a list of attractions near the user's current location and highly rated spots, and audio guidance when needed.

[0850] Specific examples

[0851] When a user says, "Tell me the route to the next rest stop" in their car, the system recognizes the speech and a natural language processing model extracts parameters such as "rest stop" and "route." Next, it uses a geographic information API to obtain information on the best rest stops, filters it, and ranks them. As a result, the best rest stop candidates are displayed on the user's navigation system. At the same time, a voice guide tells the user, "The best rest stop that can be reached within 15 minutes from your current location is XX."

[0852] Prompt Sentence Examples

[0853] "Where's a nearby park where I can relax?"

[0854] "Tell me the route to the next rest stop."

[0855] This allows users to obtain optimal information in real time, enabling efficient navigation.

[0856] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0857] Step 1: Receiving Audio Input

[0858] The device receives the user's voice input through the voice recognition hardware (microphone). For example, the user might say, "Tell me about nearby family attractions." The input is analog voice data, which is converted to digital data in the next step.

[0859] Step 2: Speech to text

[0860] The device uses speech recognition software (SpeechRecognition library) to convert the received voice data into digital text. The input is voice data, and the output is text data such as "Tell me about nearby family-friendly attractions." This conversion is performed by the speech recognition software's algorithm.

[0861] Step 3: Natural Language Analysis

[0862] The server receives the text data sent from the device and analyzes it using a natural language processing model (NLP model). Specifically, it extracts search parameters from the text. The input is the text "Tell me about nearby attractions that families can enjoy," and the output is search parameters such as "current location," "family-friendly," and "attractions."

[0863] Step 4: Retrieving information from an external API

[0864] Based on the generated search parameters, the server calls external APIs (such as geographical information APIs and review site APIs) to retrieve relevant information. The input is the search parameters, and the output is data about family-friendly attractions within the specified range. The retrieved information includes location, ratings, and recent reviews.

[0865] Step 5: Filtering and ranking information

[0866] The server filters and ranks the acquired information. Filtering selects only information that matches criteria such as "family-friendly" or "highly rated." Next, ranking prioritizes the information based on the number of reviews and ratings. The input is the acquired attraction information, and the output is a filtered and ranked list of attractions.

[0867] Step 6: Provide information

[0868] The server sends the filtered and ranked information to the terminal, which then presents it to the user audibly and visually. The input is the filtered and ranked list of attractions, and the output is the specific attraction information displayed to the user. Specifically, the terminal will provide a voice prompt saying, "Attractions that can be reached within 15 minutes from your current location are XX," and also display the information on a map.

[0869] The above are the specific steps for carrying out the invention. This processing flow allows users to obtain optimal information in real time, realizing efficient navigation.

[0870] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0871] The system of the present invention uses a method to recognize natural language input from a user and the user's emotions and provide relevant information. By combining this system with an emotion engine, it is possible to provide more personalized information according to the user's emotions. The operation of the system is explained below with a concrete example.

[0872] Receiving user input

[0873] User: In the chat window of a smartphone or car navigation app, type a question in natural language, such as, "Tell me about a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day." As the user types, the system detects the user's emotions from the voice and text.

[0874] Sending input to the server

[0875] Terminal: Recognizes the user's voice input and converts it into text, or receives text input as is, and sends the input data and emotion data to the server.

[0876] Natural Language Processing and Emotion Recognition

[0877] Server: The received text data is sent to a natural language processing (NLP) model to analyze the user's intent. At the same time, the emotion engine analyzes the user's emotion data and identifies the user's emotion. For example, if the user is feeling stressed, it will be recognized as "tension" or "fatigue."

[0878] Parameter generation and tuning

[0879] Server: Generates search parameters from the parsed natural language input and adjusts them based on the user's emotional data. For example, if the user is tired, parameters that prioritize "relaxing places" and "quiet environments" are added.

[0880] Retrieving information from a database

[0881] Server: Based on the adjusted search parameters, retrieves relevant information from geographic databases and external APIs, such as data on popular family-friendly, relaxing restaurants and attractions within the specified radius.

[0882] Filtering and Ranking

[0883] Server: Based on the retrieved relevant information, it filters and ranks the information that best matches the specified emotional state. Evaluation criteria include recent popularity, user reviews, and emotional fitness.

[0884] Sending and Displaying Information

[0885] Server: Formats the filtered and ranked results and sends them to the user's device.

[0886] Device: The received results are displayed in the user interface. Spots within the specified area are highlighted on the map, and users can tap on them to view more information. Additionally, special recommendations based on the user's emotions are also displayed.

[0887] User Interactions

[0888] Users can select points of interest from the information presented and request more information or directions.

[0889] Device: Sends these additional requests back to the server to retrieve more information and directions to provide to the user.

[0890] In this way, the system of the present invention combines natural language input from users with sentiment analysis to provide optimal information, enabling users to obtain more personalized search results, leading to new discoveries and a richer experience.

[0891] The processing flow will be explained below.

[0892] Step 1:

[0893] User: In the chat window of their smartphone or car navigation app, they type a question in natural language, such as, "Tell me about a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day." As they type, the speech recognition or text analysis module works to detect emotions from the user's voice and text.

[0894] Step 2:

[0895] Device: In the case of voice input, the device uses the voice recognition function to convert the input into text, and in the case of text input, the device receives the input as text data. An API call is made to send the received natural language data and emotion data to the server.

[0896] Step 3:

[0897] Server: Sends the received text data to a natural language processing (NLP) model to analyze the user's intent. At the same time, the emotion engine analyzes the user's emotion data and identifies the user's emotion as "tension," "fatigue," "joy," etc.

[0898] Step 4:

[0899] Server: Generates search parameters (e.g., current location, distance, entity attributes, popularity) from the analysis results of the NLP model. Additionally, adjusts specific search parameters to match the user's emotional state based on the results of the emotion engine. For example, if the user is tired, add parameters that prioritize "places where you can relax" and "quiet environments."

[0900] Step 5:

[0901] Server: Based on the generated and adjusted search parameters, retrieve relevant information from geographic databases and external APIs (e.g., map APIs, review site APIs). For example, retrieve data on popular family-friendly, relaxing restaurants and attractions within the specified radius.

[0902] Step 6:

[0903] Server: Filters the retrieved relevant information based on the user's request and emotional state, for example, selecting family-friendly spots with a particularly "relaxing" or "quiet" environment.

[0904] Step 7:

[0905] Server: Ranks the filtered information based on criteria (e.g., recent popularity, number of reviews, emotional fitness), with specific weightings applied based on the user's emotional state.

[0906] Step 8:

[0907] Server: Formats the ranked results and sends them to the user's device. The result data is formatted in a user-friendly format (e.g., JSON format).

[0908] Step 9:

[0909] Device: The received results are displayed in the user interface. Spots within the specified area are highlighted on a map, and users can tap on them to view more information. Special recommendations based on the user's emotions are also displayed.

[0910] Step 10:

[0911] Users can select points of interest from the information presented and request more information or directions.

[0912] Step 11:

[0913] Device: Sends the user's additional request (e.g., more information or directions) back to the server.

[0914] Step 12:

[0915] Server: Retrieves detailed data about the selected spot from a database or external API and sends the latest detailed data to the device.

[0916] Step 13:

[0917] Terminal: The device displays the received detailed data in a user interface, including the location's reviews, opening hours, and services offered.

[0918] Through these steps, the system of the present invention can provide optimal information by taking into account the user's natural language input and emotions, allowing the user to obtain more personalized search results and enjoy a comfortable and novel experience.

[0919] Example 2

[0920] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0921] Conventional systems analyze users' natural language input to provide information, but they are unable to provide information that takes into account the user's emotions, making it difficult to provide personalized information that meets the user's needs.In addition, they are unable to adjust search parameters based on the user's emotions, making it difficult to provide more appropriate information.

[0922] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving natural language input from a user, means for analyzing the received natural language input to generate search parameters, means for analyzing emotion data of the user and identifying the emotion, means for adjusting the search parameters based on the analyzed emotion data, means for acquiring related information from a database based on the generated and adjusted search parameters, means for filtering and ranking the acquired related information, and means for displaying the filtered and ranked information to the user. This makes it possible to provide personalized information according to the user's emotions.

[0923] "Natural language input" refers to data entered as voice or text in a language that the user normally uses.

[0924] "Search parameters" are criteria or conditions for information search that are extracted by analyzing the user's natural language input.

[0925] "Emotional Data" means data that indicates an emotional state detected from a user's input or behavior.

[0926] "Emotion identification" refers to analyzing a user's emotion data to identify a specific emotional state (e.g., joy, sadness, tension, fatigue, etc.).

[0927] "Means for adjusting" refers to a method or device for modifying search parameters and providing appropriate information based on the identified emotion.

[0928] A "database" is a structured collection of data that allows for efficient retrieval, storage, and management of information.

[0929] "Filtering" refers to the process of removing unnecessary information from acquired related information according to specific criteria.

[0930] "Ranking" refers to the process of ordering filtered information based on predetermined evaluation criteria.

[0931] "Means for displaying to the user" refers to a method or device for visually displaying the final selected information on the user's terminal.

[0932] The present invention is a system that analyzes data and emotion data input by a user in natural language and provides optimal information based on the analysis. This system can be implemented using the following hardware and software.

[0933] First, the user inputs natural language using a device such as a smartphone or car navigation system, such as "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day at." This input includes both voice and text input. In the case of voice input, the device uses a speech recognition system to convert speech into text. Specifically, a speech recognition tool such as the Google Cloud Speech-to-Text API is used.

[0934] The device then sends the received input data to a server, which uses Apache OpenNLP or a similar natural language processing (NLP) model to analyze the received text data and Microsoft Azure Emotion API or a similar emotion analysis tool to analyze the user's emotional data and identify their emotions.

[0935] The server generates search parameters based on the analyzed natural language input and emotional data. The search parameters are then adjusted according to the emotional data. For example, if the user feels "tired," parameters that prioritize "relaxing places" and "quiet environments" are added.

[0936] The server then retrieves relevant information from geographic databases and external APIs (e.g., Google Places API) based on the generated and adjusted search parameters, and the retrieved information is filtered and ranked based on evaluation criteria such as user emotional relevance and popularity.

[0937] The server sends the filtered and ranked information to the device, which displays it in a user interface. Spots are indicated on a map with markers, and users can tap them to view more information.

[0938] For example, a user types into their smartphone, "I'm tired, can you recommend a quiet cafe nearby?" Based on this prompt, the system will recognize the user's emotion as "tired" and add the search parameter "quiet cafe." Finally, it will suggest a quiet cafe near the user's current location, which the user can view on a map.

[0939] In this way, the system of the present invention can combine the user's natural language input with emotion analysis to provide information that best suits the user's emotional state, allowing the user to obtain more personalized information and enjoy a pleasant and beneficial experience.

[0940] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0941] Step 1: Receiving User Input

[0942] User: Using the chat function on their smartphone or car navigation system, they input information in natural language, such as, "Please tell me about some popular restaurants within five minutes of our current location where a family of four can spend half a day."

[0943] Input: The user types a question into the device by voice or text.

[0944] What it does: Users speak into their smartphone's microphone or type a question into a text field.

[0945] Step 2: Sending input to the server

[0946] Device: For voice input, convert the input voice into text using the Google Cloud Speech-to-Text API, etc. Then, send the input data (text) to the server.

[0947] Data processing: Converting speech to text and structuring it as natural language input from the user.

[0948] Output: Structured text data.

[0949] What it does: The device converts the user's voice into text and sends that text to the server.

[0950] Step 3: Natural Language Processing and Emotion Recognition

[0951] Server: Analyzes the received text data and extracts the user's intent using a natural language processing (NLP) model such as Apache OpenNLP, and identifies the user's emotions using a sentiment analysis tool (e.g., Microsoft Azure Emotion API).

[0952] Data Computing: Analyzes natural language input to extract keywords and intent, while simultaneously analyzing sentiment data to identify emotional states.

[0953] Output: Search parameters and emotional state based on user intent.

[0954] How it works: The server sends the text "Tell me about popular spots these days" to a natural language processing model to extract search parameters for the destination. It also uses an emotion analysis model to identify emotional states such as "tired."

[0955] Step 4: Generate and adjust parameters

[0956] Server: Generates search parameters from the parsed natural language input and adjusts the search parameters based on sentiment data.

[0957] Data processing: Generate search parameters and adjust priorities according to emotional state.

[0958] Output: The adjusted search parameters.

[0959] Specific behavior: The server adds conditions to the generated parameters that prioritize "relaxing places" and "quiet environments."

[0960] Step 5: Retrieving information from the database

[0961] Server: Based on the adjusted search parameters, retrieves relevant information from geographic databases and external APIs (e.g., Google Places API).

[0962] Data Calculation: Generates database queries based on parameters to retrieve relevant information.

[0963] Output: A list of the relevant information retrieved.

[0964] Specific operation: The server retrieves information about places that meet certain conditions, such as "quiet cafes," from a database or API.

[0965] Step 6: Filtering and ranking

[0966] Server: Filters the retrieved relevant information to best match the emotional state and ranks it based on evaluation criteria.

[0967] Data processing: Filter the acquired information and rank it based on emotional fitness and popularity.

[0968] Output: A list of filtered and ranked information.

[0969] How it works: The server filters the information it receives, lists places that match criteria such as "relaxing," and ranks them based on evaluation criteria.

[0970] Step 7: Send and view information

[0971] Server: Formats the filtered and ranked results and sends them to the user's device.

[0972] Device: The received results are displayed on the user interface. Spots within the specified area are indicated on the map with markers, and users can tap on a spot to view more information.

[0973] Output: Information displayed on the user interface.

[0974] Specific operation: The location of the cafe will be displayed on a map on the smartphone screen, and detailed information will be displayed in a pop-up.

[0975] Step 8: User Interaction

[0976] User: Users can select points of interest from the information presented and request more information or directions.

[0977] Device: Sends additional requests to the server again to get more information and directions to provide to the user.

[0978] Data calculations: Generate detailed information and directions based on user requests.

[0979] Output: Detailed information and directions.

[0980] Specific operation: When a user requests "Tell me the route to this cafe," the device sends a request to the server and the optimal route is displayed.

[0981] As a result, the entire system becomes a process that provides optimal information based on the user's emotions.

[0982] (Application example 2)

[0983] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0984] In recent years, there has been an increasing demand to meet diverse consumer needs and personalize daily shopping experiences. Conventional systems provide information based on the user's natural language input. However, these systems do not take into account the user's emotional state, which means they are unable to fully enhance the user's psychological satisfaction. Therefore, there is a need for a system that can detect the user's emotions and provide optimal information based on them.

[0985] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a natural language input and emotion data from a user, means for analyzing the received natural language input and emotion data to generate search parameters, and means for retrieving related information from a database based on the generated search parameters. This makes it possible to provide personalized related information based on the user's natural language input and emotion data.

[0986] "User" refers to an individual consumer who uses the system.

[0987] "Natural language input" refers to language data that is input in the form of everyday conversation.

[0988] "Emotional data" refers to data that indicates a user's psychological state analyzed based on their input and behavior.

[0989] "Search parameters" refer to criteria used to retrieve information from a database.

[0990] "Database" refers to a collection of data that stores related information and makes it searchable and retrievalable.

[0991] "Filtering" refers to the process of selecting necessary information from a large amount of information according to specific criteria.

[0992] "Ranking" refers to the process of ranking selected information according to certain criteria.

[0993] A "natural language processing model" refers to an algorithm or system that analyzes natural language and understands user intent.

[0994] An "emotion analysis engine" refers to an algorithm or system for analyzing a user's emotional state from their input and behavior.

[0995] The system of the present invention analyzes a user's natural language input and emotion data, and provides information suited to the user's needs. The system includes the following means.

[0996] 1. Input receiving means

[0997] A user uses a smartphone application to provide natural language input via voice or text, for example, a query of the form "I want to relax today, can you tell me where the nearest cafe is?" This input can be received as voice or directly as text.

[0998] 2. Data transmission method

[0999] The user's natural language input and accompanying emotional data are sent from the smartphone device to a server, where voice input is converted into text using speech recognition technology, and then an emotional analysis engine identifies the user's emotional state.

[1000] 3. Analysis and search parameter generation method

[1001] The server analyzes the received text data using a natural language processing (NLP) model to extract the user's intent. Furthermore, a sentiment analysis engine analyzes the emotional data to identify the user's specific emotional state, such as "I want to relax" or "I'm tired." Search parameters are generated based on this information.

[1002] 4. Database information acquisition method

[1003] Based on the search parameters, the server retrieves relevant information from external databases and APIs, such as the location, opening hours, and reputation of cafes. The retrieved information is also adjusted according to emotional data. For example, if a user is looking for a relaxing space, quiet cafes will be prioritized.

[1004] 5. Information filtering and ranking methods

[1005] The retrieved relevant information is filtered on the server, and information that best matches the user's emotional state is prioritized. Other criteria, such as reputation and popularity, are also taken into account when ranking the information. For example, cafes that are rated as "quiet" and have high reviews will be ranked higher.

[1006] 6. Information display means

[1007] The filtered and ranked results are formatted and sent to the user's smartphone, where they can view a map view and detailed information within the application interface.

[1008] Hardware and software used

[1009] Hardware: Smartphones, servers

[1010] Software: speech recognition technology (e.g., Google Speech-to-Text API), natural language processing models (e.g., spaCy), sentiment analysis engines (e.g., Hugging Face Transformers)

[1011] Examples of specific examples and prompts

[1012] As a concrete example, suppose a user types into a smartphone application, "I want to relax today, so tell me where the nearest cafe is."

[1013] Example prompt sentence:

[1014] I want to relax today, so please tell me about a nearby cafe.

[1015] A cloud-based server receives the prompt and performs natural language processing and sentiment analysis to suggest the best cafe to match the user's preferences. This allows users to find the location and products that best suit their emotional state, resulting in a highly satisfying experience.

[1016] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1017] Step 1:

[1018] Receiving user input

[1019] A user uses a smartphone application to input natural language. The input format is text or voice, for example, "I want to relax today, so please tell me about a nearby cafe." The input data is received by the device. Input: User's natural language input (text or voice), Output: Received input data.

[1020] Step 2:

[1021] Speech-to-text conversion (for voice input)

[1022] For voice input, speech recognition technology is used to convert the voice data into text. For example, Google Speech-to-Text API is used to convert the voice into text data. Input: Voice data, Output: Text data.

[1023] Step 3:

[1024] Natural Language Processing

[1025] The received text data is analyzed using a natural language processing model (e.g., spaCy) on the server to extract the user's intent. This analysis identifies important keywords and propositions. Input: text data, output: keywords and propositions.

[1026] Step 4:

[1027] Emotion analysis

[1028] The server uses an emotion analysis engine (e.g., Hugging Face Transformers) to identify the emotional state from the user's text input. For example, it determines whether the input indicates the emotion "relaxed." Input: text data, Output: emotion data (e.g., "relaxed").

[1029] Step 5:

[1030] Search parameter generation

[1031] Generate search parameters based on the results of natural language processing and sentiment analysis. For example, set specific search conditions such as "relaxing cafes" or "nearby places." Input: Keywords, sentiment data, output: search parameters.

[1032] Step 6:

[1033] Database information acquisition

[1034] The server retrieves relevant information from external databases or APIs based on the search parameters. For example, it collects the location, opening hours, and reputation of the cafe. Input: Search parameters, Output: Related information data.

[1035] Step 7:

[1036] Information Filtering

[1037] The acquired related information is filtered on the server, and the information that best matches the user's emotional state is selected. For example, "a quiet cafe with a good reputation" is selected as the filtering result. Input: related information data, Output: filtered information.

[1038] Step 8:

[1039] Information Ranking

[1040] The filtered information is ranked based on criteria such as reputation and popularity. For example, cafes with high reviews are ranked higher. Input: Filtered information, Output: Ranked information.

[1041] Step 9:

[1042] Information transmission

[1043] The server formats the ranking information and sends it to the user's smartphone. A map display and detailed information are displayed on the user's screen. Input: Ranking information, Output: Information displayed on the smartphone.

[1044] Step 10:

[1045] User Interaction

[1046] The user can review the information sent and request points of interest or route guidance. The device again communicates with the server to provide additional information or route guidance. Input: User's additional request, Output: Additional information or route guidance.

[1047] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1048] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1049] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1050] [Fourth embodiment]

[1051] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1052] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1053] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1054] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1055] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1056] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1057] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1058] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1059] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1060] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1061] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1062] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1063] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1064] The system of the present invention includes a series of processes for searching and acquiring related information based on a user's natural language input and providing the results. The operation of the system is described below with specific examples.

[1065] Receiving user input

[1066] User: Asks questions in natural language to a smartphone or car navigation system with the application installed. For example, the user might type, "Tell me about some popular restaurants within five minutes of my current location where a family of four can spend half a day."

[1067] Sending input to the server

[1068] Terminal: In the case of voice input, the voice recognition function is used to convert it into text, and in the case of text input, the user's input is sent to the server as text data as is.

[1069] Natural Language Processing and Parameter Generation

[1070] Server: Sends the received text input to a natural language processing (NLP) model to analyze the user's intent, such as extracting parameters like current location, travel time (distance), family-friendly, store, popularity, etc.

[1071] For example, the NLP model understands that the user wants to know information about fun things to do with a family of four within a five-minute walk from their current location.

[1072] Retrieving information from a database

[1073] Server: Based on the parsed parameters, retrieve relevant information from geographic databases and external APIs (e.g., map APIs, review site APIs). For example, retrieve data on popular family-friendly restaurants and amusement spots within the specified area.

[1074] Filtering and Ranking

[1075] Server: From the acquired information, select the information that best suits the user's intent and needs. For example, filtering for family-friendly facilities with recent high ratings.

[1076] The selected information is then ranked based on evaluation criteria (recent popularity, number of reviews, etc.).

[1077] Sending and Displaying Information

[1078] Server: Formats the filtered and ranked results and sends them to the device.

[1079] Terminal: Displays the received results visually to the user and can also provide audio information if required.

[1080] For example, it displays detailed information (location, ratings, recent reviews) about "Restaurant A" which is three minutes away from the user's current location, and "Amusement Park B" which is four minutes away.

[1081] User Interactions

[1082] Users can use the information provided to learn more or request directions.

[1083] On the device: Displays detailed information and directions upon user request.

[1084] The system allows users to easily search and discover geographic entities that match specific criteria, providing new place discovery and fun experiences.

[1085] The processing flow will be explained below.

[1086] Step 1:

[1087] User: In the chat window of their smartphone or car navigation app, they type a question in natural language: "Can you tell me about some popular restaurants within five minutes of our current location that a family of four can enjoy for half a day?"

[1088] Step 2:

[1089] Device: In the case of voice input, the voice recognition function is used to convert it into text, and in the case of text input, the input is received as is. An API call is made to send the received input data to the server.

[1090] Step 3:

[1091] Server: Sends the received text data to a natural language processing (NLP) model, which analyzes the text, extracts user intent, and generates search parameters (such as current location, distance, entity attributes, and popularity).

[1092] Step 4:

[1093] Server: Retrieves GPS information sent from the device, identifies the user's current location (latitude and longitude), calculates the distance traveled within the next 5 minutes (a walking distance of about 400 meters), and constructs a search query.

[1094] Step 5:

[1095] Server: Using the constructed search query, retrieves relevant information from geographic databases and external APIs (e.g., map APIs, review site APIs). Retrieved information includes popular family-friendly restaurants and amusement spots within the specified area.

[1096] Step 6:

[1097] Server: Based on the retrieved relevant information, the server filters the information according to the user's request, for example, selecting family-friendly spots with recent high ratings.

[1098] Step 7:

[1099] Server: Ranks the filtered information based on criteria (recent popularity, number of reviews, etc.). This results in raw data being shaped and sorted in order of most relevance to the user.

[1100] Step 8:

[1101] Server: Formats the ranked results and sends them to the user's device. The results are provided in a user-friendly format (e.g., JSON format).

[1102] Step 9:

[1103] Device: The device displays the received results in a user interface, highlighting spots within the specified area on a map and allowing users to tap on the spots for more information.

[1104] Step 10:

[1105] User: Selects points of interest from the presented information and requests more information and directions.

[1106] Step 11:

[1107] Device: Sends a request to the server again in response to additional requests from the user (more information or directions).

[1108] Step 12:

[1109] Server: Retrieves detailed data about the selected spot from a database or external API and sends it to the device.

[1110] Step 13:

[1111] Terminal: The device displays the received detailed data in a user interface, including reviews, opening hours, and services offered by the spot.

[1112] In this way, through the series of processes from step 1 to step 13, users can obtain and confirm the most appropriate information by asking questions in natural language.

[1113] Example 1

[1114] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1115] Conventional geographic information search systems have difficulty accurately analyzing users' natural language input and quickly providing the information most relevant to their intent. Furthermore, they suffer from insufficient ranking of search results and detailed information, which hinders user experience. Furthermore, the lack of voice input support and route guidance functions also contributes to a lack of user convenience.

[1116] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1117] In this invention, the server includes means for receiving a user's natural language input, means for analyzing the received natural language input to generate search parameters, and means for acquiring related information from information sources based on the generated search parameters. This enables quick search for and acquisition of information most suited to the user's intent. The server also includes means for filtering and ranking the acquired information and means for displaying the filtered and ranked information to the user. This allows the user to easily browse the most suited information. Furthermore, by including means for providing detailed information and route guidance in response to additional user requests, user convenience is significantly improved. In addition, by including means for converting a user's voice input into text data and means for analyzing the user's intent using a natural language processing model, it is possible to support voice input and analyze intent with high accuracy.

[1118] "User" means a person who uses the system to search for information.

[1119] "Natural language input" is a format in which users input information using everyday language.

[1120] "Server" means a computer system with a central function that receives input from users and performs analysis, retrieval, filtering, ranking, and display of relevant information.

[1121] "Means" refers to a specific method or device by which a system performs a specific function or operation.

[1122] "Search parameters" refer to the conditions or criteria used in a search that are parsed from a user's natural language input.

[1123] "Sources" refer to databases and external API services that provide relevant information.

[1124] "Relevant Information" refers to information useful to the User that is obtained from a source based on the search parameters.

[1125] "Filtering" is the process of selecting necessary information from the acquired related information.

[1126] "Ranking" is the process of ranking the filtered information based on specific criteria.

[1127] "More Information" refers to more specific and detailed information provided in response to a User's additional request.

[1128] "Route guidance" refers to navigation information that shows the route to a destination specified by the user.

[1129] "Means for converting voice input into text data" refers to technology that converts a user's voice input into a string of characters.

[1130] A "natural language processing model" is an artificial intelligence model that analyzes a user's natural language input and understands its meaning and intent.

[1131] The system of the present invention includes a series of processes for searching and acquiring related information based on natural language input from a user and providing the results. Below, an embodiment of this system will be described with specific examples.

[1132] First, the user uses an application installed on their smartphone or car navigation system to input a question in natural language, for example, "Tell me about a popular restaurant within five minutes of my current location where a family of four can spend half a day," either by voice or text.

[1133] In the case of voice input, the device converts the voice into text using voice recognition software (e.g., Google Speech Recognition API). In the case of text input, the device processes the voice as text data as is. The converted or acquired text data is sent to the server.

[1134] The server sends the received text data to a natural language processing model (e.g., Google Cloud Natural Language API) to analyze the user's intent, extracting parameters from the text such as current location, travel time, family-friendly, and highly rated restaurants.

[1135] Next, based on the analysis results, the server retrieves relevant information from geographic databases and external APIs (e.g., map APIs or review site APIs). It creates queries to gather the necessary information. For example, it retrieves information about popular family-friendly restaurants and amusement facilities within a five-minute walk from the user's current location.

[1136] The server filters the information it receives and selects the information that best suits the user's intent and requirements. After filtering, it ranks the information based on evaluation criteria (e.g., recent popularity, number of reviews).

[1137] The server formats the ranked information and sends it to the device. The device displays this information visually to the user. It also has the ability to provide audio information if needed. For example, the device might display detailed information (location, ratings, recent reviews) about "Restaurant A" that is three minutes away from the user's current location, or "Amusement Park B" that is four minutes away.

[1138] Users can view more details or request route guidance from the information provided. The device will then display the details or route guidance as requested by the user. For example, it can launch Google Maps and display directions to a specified location.

[1139] As a concrete example, if a user inputs the prompt "Tell me about a highly rated cafe nearby," the device converts this speech into text and sends it to the server. The server analyzes it using a natural language processing model and extracts the parameters "highly rated cafe" and "nearby." Next, it retrieves information about highly rated cafes near the user's current location from a geographic information database and a rating site API. This information is filtered and ranked, and finally, information about "Cafe C" and "Cafe D" is displayed to the user. If the user requests "Tell me the way to Cafe C," the device launches a map app and displays route guidance.

[1140] This system allows users to easily find the right location for their needs, providing a more convenient and comfortable experience.

[1141] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1142] Step 1:

[1143] The user launches an application installed on their smartphone or car navigation system and asks for the information they need in natural language. For example, they might say, "Tell me about a popular restaurant within five minutes of my current location where a family of four can spend half a day." Input: A question in natural language (voice or text). Output: The content of the user's question.

[1144] Step 2:

[1145] The device converts voice input into text data using voice recognition software (e.g., voice recognition API). In the case of text input, it is processed as text data as is. Since text data is generated through voice recognition, the content of the user's question becomes clear. Input: Voice data (in the case of voice input). Output: Text data.

[1146] Step 3:

[1147] The terminal sends the generated text data to the server. A data transmission protocol is used to deliver the text data to the server quickly and reliably. Input: Text data (question content). Output: Sending text data to the server.

[1148] Step 4:

[1149] The server sends the received text data to a natural language processing model (e.g., natural language processing API) to analyze the user's intent. Search parameters such as current location, travel time, family-friendly, and highly rated shops are extracted from the text data. Input: Text data. Output: Search parameters.

[1150] Step 5:

[1151] The server retrieves related information from geographic information databases and external APIs (e.g., map APIs, review site APIs) based on the search parameters obtained as a result of the analysis. Specifically, it collects information on popular family-friendly restaurants and amusement facilities that can be enjoyed within a five-minute walk from the user's current location. Input: Search parameters. Output: Related information data.

[1152] Step 6:

[1153] The server filters the retrieved related information and selects information based on specific criteria (e.g., highly rated restaurants, family-friendly restaurants, etc.). It then ranks the filtered information based on evaluation criteria (e.g., recent popularity, number of reviews). Input: Related information data. Output: Filtered and ranked information.

[1154] Step 7:

[1155] The server formats the ranked results and sends them to the terminal. The display format is adjusted to make it easy for the user to understand. Input: Ranked information. Output: Formatted information.

[1156] Step 8:

[1157] The device displays the received formatted information to the user visually and audibly. For example, detailed information (location, rating, recent reviews) for "Restaurant A" and "Amusement Park B" is displayed to the user. Input: Formatted information. Output: Display to the user (visual, audio).

[1158] Step 9:

[1159] Users can request more details or directions from the information provided. For example, they might enter commands like "Tell me more about Restaurant A" or "Show me directions to Amusement Park B." Input: A request for more information. Output: A request for more information or directions.

[1160] Step 10:

[1161] The device displays more information or directions at the user's request, for example, launching a maps app to show routes to a specified location. Input: Request for more information. Output: More information or directions.

[1162] (Application example 1)

[1163] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1164] Conventional information search systems that use natural language input lack a mechanism for efficiently analyzing user voice input and generating appropriate search parameters. Furthermore, there are no concrete examples of providing users with the acquired information visually or audibly, or of acquiring information from external APIs, making it difficult to provide accurate information in real time. This results in problems such as reduced user convenience and satisfaction.

[1165] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1166] In this invention, the server includes means for receiving natural language input from a user using voice recognition hardware, means for analyzing the received natural language input and generating search parameters using a natural language processing model, means for retrieving related information from an external API based on the generated search parameters, means for filtering and ranking the retrieved related information, and means for displaying the filtered and ranked information to the user audibly and visually, thereby enabling efficient analysis of the user's voice input, retrieval of information from the external API, and provision of appropriate information visually and audibly to the user in real time.

[1167] "Speech recognition hardware" means a device for receiving a user's voice input and converting it into voice data.

[1168] "Natural language input" refers to text or voice instructions or queries given by a user in the language they normally use.

[1169] A "natural language processing model" is an algorithm or program that analyzes received natural language input and understands the user's intent and meaning.

[1170] "Search parameters" are conditions or criteria generated based on user input for retrieving relevant information from databases or APIs.

[1171] An "external API" is a protocol for accessing data or functionality through an interface provided by an external service or application.

[1172] "Relevant information" refers to useful data retrieved from databases and external APIs based on the user's search parameters.

[1173] "Filtering" is the process of selecting relevant information obtained based on certain criteria and removing unnecessary information.

[1174] "Ranking" is the process of evaluating the filtered relevant information based on specific criteria and sorting it according to priority.

[1175] "Visual presentation means" refers to a method or device for displaying acquired information in a manner that appeals to the user's visual sense.

[1176] "Means for providing audio information" refers to a method or device for providing acquired information in a form that appeals to the user's auditory sense.

[1177] The present invention relates to a system for searching for information based on a user's natural language input and providing the search results. Specific embodiments are described below.

[1178] First, the main hardware components of the system are the speech recognition hardware, server, and terminal. Specifically, they are a microphone for users to input speech, the "SpeechRecognition" speech recognition software library for converting input speech into text, and an NLP model for natural language processing. The "requests" library is used for API requests.

[1179] Receiving audio input

[1180] The user gives instructions in natural language through speech recognition hardware (microphone), for example, "Tell me about nearby family-friendly attractions."

[1181] Speech-to-text

[1182] On-device speech recognition software (SpeechRecognition) is used to convert the received voice input into text, which is then sent to the server.

[1183] natural language analysis

[1184] The server sends the received text to a natural language processing model (NLP model) to analyze the user's intent. The NLP model analyzes the user's request and extracts search parameters (e.g., current location, family-friendly, attractions).

[1185] Obtaining information from external APIs

[1186] Based on the generated search parameters, the server calls external APIs (e.g., geographic information APIs or review site APIs) to retrieve relevant information, thereby gathering information that matches the specified criteria (e.g., family-friendly attractions).

[1187] Information filtering and ranking

[1188] The acquired information is filtered and ranked on the server. The filtering selects information that matches the user's intent, and the ranking prioritizes it based on criteria such as the number of reviews and rating points.

[1189] Providing information

[1190] Finally, the server sends the filtered and ranked results to the device, which then presents them to the user via audio and visual displays, such as a list of attractions near the user's current location and highly rated spots, and audio guidance when needed.

[1191] Specific examples

[1192] When a user says, "Tell me the route to the next rest stop" in their car, the system recognizes the speech and a natural language processing model extracts parameters such as "rest stop" and "route." Next, it uses a geographic information API to obtain information on the best rest stops, filters it, and ranks them. As a result, the best rest stop candidates are displayed on the user's navigation system. At the same time, a voice guide tells the user, "The best rest stop that can be reached within 15 minutes from your current location is XX."

[1193] Prompt Sentence Examples

[1194] "Where's a nearby park where I can relax?"

[1195] "Tell me the route to the next rest stop."

[1196] This allows users to obtain optimal information in real time, enabling efficient navigation.

[1197] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1198] Step 1: Receiving Audio Input

[1199] The device receives the user's voice input through the voice recognition hardware (microphone). For example, the user might say, "Tell me about nearby family attractions." The input is analog voice data, which is converted to digital data in the next step.

[1200] Step 2: Speech to text

[1201] The device uses speech recognition software (SpeechRecognition library) to convert the received voice data into digital text. The input is voice data, and the output is text data such as "Tell me about nearby family-friendly attractions." This conversion is performed by the speech recognition software's algorithm.

[1202] Step 3: Natural Language Analysis

[1203] The server receives the text data sent from the device and analyzes it using a natural language processing model (NLP model). Specifically, it extracts search parameters from the text. The input is the text "Tell me about nearby attractions that families can enjoy," and the output is search parameters such as "current location," "family-friendly," and "attractions."

[1204] Step 4: Retrieving information from an external API

[1205] Based on the generated search parameters, the server calls external APIs (such as geographical information APIs and review site APIs) to retrieve relevant information. The input is the search parameters, and the output is data about family-friendly attractions within the specified range. The retrieved information includes location, ratings, and recent reviews.

[1206] Step 5: Filtering and ranking information

[1207] The server filters and ranks the acquired information. Filtering selects only information that matches criteria such as "family-friendly" or "highly rated." Next, ranking prioritizes the information based on the number of reviews and ratings. The input is the acquired attraction information, and the output is a filtered and ranked list of attractions.

[1208] Step 6: Provide information

[1209] The server sends the filtered and ranked information to the terminal, which then presents it to the user audibly and visually. The input is the filtered and ranked list of attractions, and the output is the specific attraction information displayed to the user. Specifically, the terminal will provide a voice prompt saying, "Attractions that can be reached within 15 minutes from your current location are XX," and also display the information on a map.

[1210] The above are the specific steps for carrying out the invention. This processing flow allows users to obtain optimal information in real time, realizing efficient navigation.

[1211] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1212] The system of the present invention uses a method to recognize natural language input from a user and the user's emotions and provide relevant information. By combining this system with an emotion engine, it is possible to provide more personalized information according to the user's emotions. The operation of the system is explained below with a concrete example.

[1213] Receiving user input

[1214] User: In the chat window of a smartphone or car navigation app, type a question in natural language, such as, "Tell me about a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day." As the user types, the system detects the user's emotions from the voice and text.

[1215] Sending input to the server

[1216] Terminal: Recognizes the user's voice input and converts it into text, or receives text input as is, and sends the input data and emotion data to the server.

[1217] Natural Language Processing and Emotion Recognition

[1218] Server: The received text data is sent to a natural language processing (NLP) model to analyze the user's intent. At the same time, the emotion engine analyzes the user's emotion data and identifies the user's emotion. For example, if the user is feeling stressed, it will be recognized as "tension" or "fatigue."

[1219] Parameter generation and tuning

[1220] Server: Generates search parameters from the parsed natural language input and adjusts them based on the user's emotional data. For example, if the user is tired, parameters that prioritize "relaxing places" and "quiet environments" are added.

[1221] Retrieving information from a database

[1222] Server: Based on the adjusted search parameters, retrieves relevant information from geographic databases and external APIs, such as data on popular family-friendly, relaxing restaurants and attractions within the specified radius.

[1223] Filtering and Ranking

[1224] Server: Based on the retrieved relevant information, it filters and ranks the information that best matches the specified emotional state. Evaluation criteria include recent popularity, user reviews, and emotional fitness.

[1225] Sending and Displaying Information

[1226] Server: Formats the filtered and ranked results and sends them to the user's device.

[1227] Device: The received results are displayed in the user interface. Spots within the specified area are highlighted on the map, and users can tap on them to view more information. Additionally, special recommendations based on the user's emotions are also displayed.

[1228] User Interactions

[1229] Users can select points of interest from the information presented and request more information or directions.

[1230] Device: Sends these additional requests back to the server to retrieve more information and directions to provide to the user.

[1231] In this way, the system of the present invention combines natural language input from users with sentiment analysis to provide optimal information, enabling users to obtain more personalized search results, leading to new discoveries and a richer experience.

[1232] The processing flow will be explained below.

[1233] Step 1:

[1234] User: In the chat window of their smartphone or car navigation app, they type a question in natural language, such as, "Tell me about a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day." As they type, the speech recognition or text analysis module works to detect emotions from the user's voice and text.

[1235] Step 2:

[1236] Device: In the case of voice input, the device uses the voice recognition function to convert the input into text, and in the case of text input, the device receives the input as text data. An API call is made to send the received natural language data and emotion data to the server.

[1237] Step 3:

[1238] Server: Sends the received text data to a natural language processing (NLP) model to analyze the user's intent. At the same time, the emotion engine analyzes the user's emotion data and identifies the user's emotion as "tension," "fatigue," "joy," etc.

[1239] Step 4:

[1240] Server: Generates search parameters (e.g., current location, distance, entity attributes, popularity) from the analysis results of the NLP model. Additionally, adjusts specific search parameters to match the user's emotional state based on the results of the emotion engine. For example, if the user is tired, add parameters that prioritize "places where you can relax" and "quiet environments."

[1241] Step 5:

[1242] Server: Based on the generated and adjusted search parameters, retrieve relevant information from geographic databases and external APIs (e.g., map APIs, review site APIs). For example, retrieve data on popular family-friendly, relaxing restaurants and attractions within the specified radius.

[1243] Step 6:

[1244] Server: Filters the retrieved relevant information based on the user's request and emotional state, for example, selecting family-friendly spots with a particularly "relaxing" or "quiet" environment.

[1245] Step 7:

[1246] Server: Ranks the filtered information based on criteria (e.g., recent popularity, number of reviews, emotional fitness), with specific weightings applied based on the user's emotional state.

[1247] Step 8:

[1248] Server: Formats the ranked results and sends them to the user's device. The result data is formatted in a user-friendly format (e.g., JSON format).

[1249] Step 9:

[1250] Device: The received results are displayed in the user interface. Spots within the specified area are highlighted on a map, and users can tap on them to view more information. Special recommendations based on the user's emotions are also displayed.

[1251] Step 10:

[1252] Users can select points of interest from the information presented and request more information or directions.

[1253] Step 11:

[1254] Device: Sends the user's additional request (e.g., more information or directions) back to the server.

[1255] Step 12:

[1256] Server: Retrieves detailed data about the selected spot from a database or external API and sends the latest detailed data to the device.

[1257] Step 13:

[1258] Terminal: The device displays the received detailed data in a user interface, including the location's reviews, opening hours, and services offered.

[1259] Through these steps, the system of the present invention can provide optimal information by taking into account the user's natural language input and emotions, allowing the user to obtain more personalized search results and enjoy a comfortable and novel experience.

[1260] Example 2

[1261] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1262] Conventional systems analyze users' natural language input to provide information, but they are unable to provide information that takes into account the user's emotions, making it difficult to provide personalized information that meets the user's needs.In addition, they are unable to adjust search parameters based on the user's emotions, making it difficult to provide more appropriate information.

[1263] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving natural language input from a user, means for analyzing the received natural language input to generate search parameters, means for analyzing emotion data of the user and identifying the emotion, means for adjusting the search parameters based on the analyzed emotion data, means for acquiring related information from a database based on the generated and adjusted search parameters, means for filtering and ranking the acquired related information, and means for displaying the filtered and ranked information to the user. This makes it possible to provide personalized information according to the user's emotions.

[1264] "Natural language input" refers to data entered as voice or text in a language that the user normally uses.

[1265] "Search parameters" are criteria or conditions for information search that are extracted by analyzing the user's natural language input.

[1266] "Emotional Data" means data that indicates an emotional state detected from a user's input or behavior.

[1267] "Emotion identification" refers to analyzing a user's emotion data to identify a specific emotional state (e.g., joy, sadness, tension, fatigue, etc.).

[1268] "Means for adjusting" refers to a method or device for modifying search parameters and providing appropriate information based on the identified emotion.

[1269] A "database" is a structured collection of data that allows for efficient retrieval, storage, and management of information.

[1270] "Filtering" refers to the process of removing unnecessary information from acquired related information according to specific criteria.

[1271] "Ranking" refers to the process of ordering filtered information based on predetermined evaluation criteria.

[1272] "Means for displaying to the user" refers to a method or device for visually displaying the final selected information on the user's terminal.

[1273] The present invention is a system that analyzes data and emotion data input by a user in natural language and provides optimal information based on the analysis. This system can be implemented using the following hardware and software.

[1274] First, the user inputs natural language using a device such as a smartphone or car navigation system, such as "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day at." This input includes both voice and text input. In the case of voice input, the device uses a speech recognition system to convert speech into text. Specifically, a speech recognition tool such as the Google Cloud Speech-to-Text API is used.

[1275] The device then sends the received input data to a server, which uses Apache OpenNLP or a similar natural language processing (NLP) model to analyze the received text data and Microsoft Azure Emotion API or a similar emotion analysis tool to analyze the user's emotional data and identify their emotions.

[1276] The server generates search parameters based on the analyzed natural language input and emotional data. The search parameters are then adjusted according to the emotional data. For example, if the user feels "tired," parameters that prioritize "relaxing places" and "quiet environments" are added.

[1277] The server then retrieves relevant information from geographic databases and external APIs (e.g., Google Places API) based on the generated and adjusted search parameters, and the retrieved information is filtered and ranked based on evaluation criteria such as user emotional relevance and popularity.

[1278] The server sends the filtered and ranked information to the device, which displays it in a user interface. Spots are indicated on a map with markers, and users can tap them to view more information.

[1279] For example, a user types into their smartphone, "I'm tired, can you recommend a quiet cafe nearby?" Based on this prompt, the system will recognize the user's emotion as "tired" and add the search parameter "quiet cafe." Finally, it will suggest a quiet cafe near the user's current location, which the user can view on a map.

[1280] In this way, the system of the present invention can combine the user's natural language input with emotion analysis to provide information that best suits the user's emotional state, allowing the user to obtain more personalized information and enjoy a pleasant and beneficial experience.

[1281] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1282] Step 1: Receiving User Input

[1283] User: Using the chat function on their smartphone or car navigation system, they input information in natural language, such as, "Please tell me about some popular restaurants within five minutes of our current location where a family of four can spend half a day."

[1284] Input: The user types a question into the device by voice or text.

[1285] What it does: Users speak into their smartphone's microphone or type a question into a text field.

[1286] Step 2: Sending input to the server

[1287] Device: For voice input, convert the input voice into text using the Google Cloud Speech-to-Text API, etc. Then, send the input data (text) to the server.

[1288] Data processing: Converting speech to text and structuring it as natural language input from the user.

[1289] Output: Structured text data.

[1290] What it does: The device converts the user's voice into text and sends that text to the server.

[1291] Step 3: Natural Language Processing and Emotion Recognition

[1292] Server: Analyzes the received text data and extracts the user's intent using a natural language processing (NLP) model such as Apache OpenNLP, and identifies the user's emotions using a sentiment analysis tool (e.g., Microsoft Azure Emotion API).

[1293] Data Computing: Analyzes natural language input to extract keywords and intent, while simultaneously analyzing sentiment data to identify emotional states.

[1294] Output: Search parameters and emotional state based on user intent.

[1295] How it works: The server sends the text "Tell me about popular spots these days" to a natural language processing model to extract search parameters for the destination. It also uses an emotion analysis model to identify emotional states such as "tired."

[1296] Step 4: Generate and adjust parameters

[1297] Server: Generates search parameters from the parsed natural language input and adjusts the search parameters based on sentiment data.

[1298] Data processing: Generate search parameters and adjust priorities according to emotional state.

[1299] Output: The adjusted search parameters.

[1300] Specific behavior: The server adds conditions to the generated parameters that prioritize "relaxing places" and "quiet environments."

[1301] Step 5: Retrieving information from the database

[1302] Server: Based on the adjusted search parameters, retrieves relevant information from geographic databases and external APIs (e.g., Google Places API).

[1303] Data Calculation: Generates database queries based on parameters to retrieve relevant information.

[1304] Output: A list of the relevant information retrieved.

[1305] Specific operation: The server retrieves information about places that meet certain conditions, such as "quiet cafes," from a database or API.

[1306] Step 6: Filtering and ranking

[1307] Server: Filters the retrieved relevant information to best match the emotional state and ranks it based on evaluation criteria.

[1308] Data processing: Filter the acquired information and rank it based on emotional fitness and popularity.

[1309] Output: A list of filtered and ranked information.

[1310] How it works: The server filters the information it receives, lists places that match criteria such as "relaxing," and ranks them based on evaluation criteria.

[1311] Step 7: Send and view information

[1312] Server: Formats the filtered and ranked results and sends them to the user's device.

[1313] Device: The received results are displayed on the user interface. Spots within the specified area are indicated on the map with markers, and users can tap on a spot to view more information.

[1314] Output: Information displayed on the user interface.

[1315] Specific operation: The location of the cafe will be displayed on a map on the smartphone screen, and detailed information will be displayed in a pop-up.

[1316] Step 8: User Interaction

[1317] User: Users can select points of interest from the information presented and request more information or directions.

[1318] Device: Sends additional requests to the server again to get more information and directions to provide to the user.

[1319] Data calculations: Generate detailed information and directions based on user requests.

[1320] Output: Detailed information and directions.

[1321] Specific operation: When a user requests "Tell me the route to this cafe," the device sends a request to the server and the optimal route is displayed.

[1322] As a result, the entire system becomes a process that provides optimal information based on the user's emotions.

[1323] (Application example 2)

[1324] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1325] In recent years, there has been an increasing demand to meet diverse consumer needs and personalize daily shopping experiences. Conventional systems provide information based on the user's natural language input. However, these systems do not take into account the user's emotional state, which means they are unable to fully enhance the user's psychological satisfaction. Therefore, there is a need for a system that can detect the user's emotions and provide optimal information based on them.

[1326] The specification processing by the specification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving a natural language input and emotion data from a user, means for analyzing the received natural language input and emotion data to generate search parameters, and means for retrieving related information from a database based on the generated search parameters. This makes it possible to provide personalized related information based on the user's natural language input and emotion data.

[1327] "User" refers to an individual consumer who uses the system.

[1328] "Natural language input" refers to language data that is input in the form of everyday conversation.

[1329] "Emotional data" refers to data that indicates a user's psychological state analyzed based on their input and behavior.

[1330] "Search parameters" refer to criteria used to retrieve information from a database.

[1331] "Database" refers to a collection of data that stores related information and makes it searchable and retrievalable.

[1332] "Filtering" refers to the process of selecting necessary information from a large amount of information according to specific criteria.

[1333] "Ranking" refers to the process of ranking selected information according to certain criteria.

[1334] A "natural language processing model" refers to an algorithm or system that analyzes natural language and understands user intent.

[1335] An "emotion analysis engine" refers to an algorithm or system for analyzing a user's emotional state from their input and behavior.

[1336] The system of the present invention analyzes a user's natural language input and emotion data, and provides information suited to the user's needs. The system includes the following means.

[1337] 1. Input receiving means

[1338] A user uses a smartphone application to provide natural language input via voice or text, for example, a query of the form "I want to relax today, can you tell me where the nearest cafe is?" This input can be received as voice or directly as text.

[1339] 2. Data transmission method

[1340] The user's natural language input and accompanying emotional data are sent from the smartphone device to a server, where voice input is converted into text using speech recognition technology, and then an emotional analysis engine identifies the user's emotional state.

[1341] 3. Analysis and search parameter generation method

[1342] The server analyzes the received text data using a natural language processing (NLP) model to extract the user's intent. Furthermore, a sentiment analysis engine analyzes the emotional data to identify the user's specific emotional state, such as "I want to relax" or "I'm tired." Search parameters are generated based on this information.

[1343] 4. Database information acquisition method

[1344] Based on the search parameters, the server retrieves relevant information from external databases and APIs, such as the location, opening hours, and reputation of cafes. The retrieved information is also adjusted according to emotional data. For example, if a user is looking for a relaxing space, quiet cafes will be prioritized.

[1345] 5. Information filtering and ranking methods

[1346] The retrieved relevant information is filtered on the server, and information that best matches the user's emotional state is prioritized. Other criteria, such as reputation and popularity, are also taken into account when ranking the information. For example, cafes that are rated as "quiet" and have high reviews will be ranked higher.

[1347] 6. Information display means

[1348] The filtered and ranked results are formatted and sent to the user's smartphone, where they can view a map view and detailed information within the application interface.

[1349] Hardware and software used

[1350] Hardware: Smartphones, servers

[1351] Software: speech recognition technology (e.g., Google Speech-to-Text API), natural language processing models (e.g., spaCy), sentiment analysis engines (e.g., Hugging Face Transformers)

[1352] Examples of specific examples and prompts

[1353] As a concrete example, suppose a user types into a smartphone application, "I want to relax today, so tell me where the nearest cafe is."

[1354] Example prompt sentence:

[1355] I want to relax today, so please tell me about a nearby cafe.

[1356] A cloud-based server receives the prompt and performs natural language processing and sentiment analysis to suggest the best cafe to match the user's preferences. This allows users to find the location and products that best suit their emotional state, resulting in a highly satisfying experience.

[1357] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1358] Step 1:

[1359] Receiving user input

[1360] A user uses a smartphone application to input natural language. The input format is text or voice, for example, "I want to relax today, so please tell me about a nearby cafe." The input data is received by the device. Input: User's natural language input (text or voice), Output: Received input data.

[1361] Step 2:

[1362] Speech-to-text conversion (for voice input)

[1363] For voice input, speech recognition technology is used to convert the voice data into text. For example, Google Speech-to-Text API is used to convert the voice into text data. Input: Voice data, Output: Text data.

[1364] Step 3:

[1365] Natural Language Processing

[1366] The received text data is analyzed using a natural language processing model (e.g., spaCy) on the server to extract the user's intent. This analysis identifies important keywords and propositions. Input: text data, output: keywords and propositions.

[1367] Step 4:

[1368] Emotion analysis

[1369] The server uses an emotion analysis engine (e.g., Hugging Face Transformers) to identify the emotional state from the user's text input. For example, it determines whether the input indicates the emotion "relaxed." Input: text data, Output: emotion data (e.g., "relaxed").

[1370] Step 5:

[1371] Search parameter generation

[1372] Generate search parameters based on the results of natural language processing and sentiment analysis. For example, set specific search conditions such as "relaxing cafes" or "nearby places." Input: Keywords, sentiment data, output: search parameters.

[1373] Step 6:

[1374] Database information acquisition

[1375] The server retrieves relevant information from external databases or APIs based on the search parameters. For example, it collects the location, opening hours, and reputation of the cafe. Input: Search parameters, Output: Related information data.

[1376] Step 7:

[1377] Information Filtering

[1378] The acquired related information is filtered on the server, and the information that best matches the user's emotional state is selected. For example, "a quiet cafe with a good reputation" is selected as the filtering result. Input: related information data, Output: filtered information.

[1379] Step 8:

[1380] Information Ranking

[1381] The filtered information is ranked based on criteria such as reputation and popularity. For example, cafes with high reviews are ranked higher. Input: Filtered information, Output: Ranked information.

[1382] Step 9:

[1383] Information transmission

[1384] The server formats the ranking information and sends it to the user's smartphone. A map display and detailed information are displayed on the user's screen. Input: Ranking information, Output: Information displayed on the smartphone.

[1385] Step 10:

[1386] User Interaction

[1387] The user can review the information sent and request points of interest or route guidance. The device again communicates with the server to provide additional information or route guidance. Input: User's additional request, Output: Additional information or route guidance.

[1388] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1389] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1390] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1391] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1392] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1393] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1394] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1395] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1396] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1397] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1398] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1399] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1400] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1401] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1402] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1403] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1404] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1405] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1406] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1407] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1408] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1409] The following is further disclosed regarding the above embodiment.

[1410] (Claim 1)

[1411] means for receiving natural language input from a user;

[1412] means for analyzing the received natural language input to generate search parameters;

[1413] means for retrieving relevant information from a database based on the generated search parameters;

[1414] a means for filtering and ranking the relevant information obtained;

[1415] The system includes a means for displaying the filtered and ranked information to a user.

[1416] (Claim 2)

[1417] The system of claim 1 , wherein the search parameters are generated based on a user's current location, distance, entity attributes, and popularity.

[1418] (Claim 3)

[1419] 10. The system of claim 1, wherein the means for analyzing the received natural language input uses a natural language processing model.

[1420] "Example 1"

[1421] (Claim 1)

[1422] means for receiving natural language input from a user;

[1423] means for analyzing the received natural language input to generate search parameters;

[1424] means for retrieving relevant information from sources based on the generated search parameters;

[1425] a means for filtering and ranking the relevant information obtained;

[1426] a means for displaying the filtered and ranked information to a user; and

[1427] a means of providing further information and directions upon the user's request;

[1428] a means for converting a user's voice input into text data;

[1429] A means of analyzing user intent using a natural language processing model; and

[1430] A system including:

[1431] (Claim 2)

[1432] The system of claim 1 , wherein the search parameters are generated based on the user's current location, travel time, target attributes, and popularity.

[1433] (Claim 3)

[1434] 10. The system of claim 1, wherein the means for analyzing the received natural language input uses a natural language processing model.

[1435] "Application Example 1"

[1436] Extracting new parts

[1437] From the application example, we can extract the following novel parts:

[1438] 1. A voice input method using voice recognition hardware.

[1439] 2. A means of obtaining information from external APIs.

[1440] 3. A means of generating parameters using a natural language processing model (NLP model).

[1441] 4. Means of presenting the retrieved information to the user audibly and visually.

[1442] Rewrite to fit new inventions

[1443] (Claim 1)

[1444] means for receiving natural language input from a user using speech recognition hardware;

[1445] a means using a natural language processing model for analyzing received natural language input and generating search parameters;

[1446] means for retrieving relevant information from an external API based on the generated search parameters;

[1447] a means for filtering and ranking the relevant information obtained;

[1448] The system includes means for audibly and visually displaying the filtered and ranked information to the user.

[1449] (Claim 2)

[1450] The system of claim 1 , wherein the search parameters are generated based on a user's current location, travel time, entity attributes, and ratings.

[1451] (Claim 3)

[1452] 10. The system of claim 1, wherein the system analyzes the received natural language input using a generative AI model.

[1453] "Example 2: Combining Emotion Engines"

[1454] (Claim 1)

[1455] means for receiving natural language input from a user;

[1456] means for analyzing the received natural language input to generate search parameters;

[1457] A means for analyzing the user's emotional data and identifying the emotion;

[1458] means for adjusting search parameters based on the analyzed emotion data;

[1459] means for retrieving relevant information from a database based on the generated and adjusted search parameters;

[1460] a means for filtering and ranking the relevant information obtained;

[1461] The system includes a means for displaying the filtered and ranked information to a user.

[1462] (Claim 2)

[1463] The system of claim 1 , wherein the search parameters are generated and adjusted based on the user's current location, distance, object attributes, popularity, and emotional fitness.

[1464] (Claim 3)

[1465] 10. The system of claim 1, wherein the means for analyzing the received natural language input uses a natural language processing model.

[1466] "Application example 2 when combining emotion engines"

[1467] (Claim 1)

[1468] means for receiving natural language input from a user;

[1469] means for analyzing the received natural language input and user emotion data to generate search parameters;

[1470] means for retrieving relevant information from a database based on the generated search parameters;

[1471] means for filtering and ranking the retrieved relevant information based on the user's emotional state;

[1472] The system includes a means for displaying the filtered and ranked information to a user.

[1473] (Claim 2)

[1474] 10. The system of claim 1, wherein the search parameters are generated based on a user's current location, distance, location attributes, popularity, and a user's emotional state.

[1475] (Claim 3)

[1476] 10. The system of claim 1, wherein the means for analyzing the received natural language input and user emotion data comprises a natural language processing model and a sentiment analysis engine. [Explanation of symbols]

[1477] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving natural language input from a user; means for analyzing the received natural language input to generate search parameters; means for retrieving relevant information from a database based on the generated search parameters; a means for filtering and ranking the relevant information obtained; and means for displaying the filtered and ranked information to a user.

2. The system of claim 1 , wherein the search parameters are generated based on a user's current location, distance, entity attributes, and popularity.

3. 10. The system of claim 1, wherein the means for analyzing the received natural language input uses a natural language processing model.

Images