System

Abstract

A system is provided.SOLUTION: A system comprising: means for acquiring conversation content in a chat application; means for analyzing the acquired conversation content to extract a keyword and a need; means for searching for a recommended facility based on the extracted keyword and need; means for plotting the searched recommended facility on a map; and means for displaying the plotted map data in the chat application.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When planning a trip or outing, gathering information from multiple websites and map applications can be time-consuming and laborious. Furthermore, when planning with multiple people, coordinating opinions and sharing information can be cumbersome, making efficient planning difficult. Given this background, there was a need for a system that could automatically suggest recommended facilities based on conversations in chat applications and display them on a map, allowing for efficient and intuitive planning. [Means for solving the problem]

[0005] The present invention relates to a system including a means for acquiring conversation content in a chat application, a means for analyzing the acquired conversation content to extract keywords and needs, a means for searching for recommended facilities based on the extracted keywords and needs, a means for plotting the searched recommended facilities on a map, and a means for displaying the plotted map data within the chat application. This allows users to efficiently plan trips and outings through chat, and makes it easy to confirm and share information. Furthermore, by including a means for displaying detailed information about recommended facilities as needed and a navigation means for seamlessly switching between the chat screen and the map display screen, the system provides an even more user-friendly and intuitive user interface.

[0006] "Chat application" means software or online service that enables users to exchange text messages.

[0007] "Conversation capture" refers to the process of monitoring and collecting text messages of conversations that take place within a chat application.

[0008] "Analysis" refers to the process of extracting information from conversations and understanding their meaning.

[0009] "Keywords" are important words or phrases in a conversation that indicate a particular theme or need.

[0010] "Needs" refers to information that indicates a specific feature or requirement that a user desires.

[0011] "Recommended facilities" refers to places such as stores and tourist attractions that are suggested based on analyzed keywords and needs.

[0012] "Search" refers to the process of finding relevant information from databases or external services based on specific keywords or needs.

[0013] "Plot" refers to displaying the location of the searched recommended facility on a map.

[0014] "Map data" refers to data that includes geographic information and location information on a map.

[0015] "Detailed Information" refers to additional information about a Recommended Facility, such as its address, opening hours, ratings, and reviews.

[0016] "Navigation" refers to the functionality of providing an interface that allows users to seamlessly move between different screens (e.g., chat screen and map screen). [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The present invention is a system that analyzes the content of conversations when a user plans a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map.

[0039] System configuration and operation overview

[0040] 1. Acquiring chat contents

[0041] When a user discusses travel plans with a friend in a chat application, the device captures the text data of the conversation in real time, temporarily stores the conversation content, and then transmits it to the server.

[0042] 2. Conversation Analysis

[0043] The server receives the conversation and analyzes it using a pre-trained natural language processing (NLP) model, which extracts important keywords from the conversation (e.g., "Fukuoka" and "Mizutaki") and the user's needs.

[0044] 3. Recommended stores and facilities

[0045] Based on the analysis results, the server searches for related facilities. For example, based on keywords such as "Fukuoka" and "Mizutaki," it searches for highly rated Mizutaki restaurants in the Fukuoka area. From the search results, it generates a list of recommended facilities.

[0046] 4. Mapping and Display

[0047] Based on the information in the generated list of recommended facilities (e.g., store name, address, rating), the server obtains the coordinates of the facilities and plots them on a map. The plotted map data is sent to the device and displayed within the chat application.

[0048] 5. User confirmation and selection

[0049] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[0050] 6. Seamlessly switch between chat and maps

[0051] The device provides a navigation method that seamlessly switches between the chat screen and the map display screen, allowing users to smoothly check and share information.

[0052] Specific examples

[0053] For example, suppose user A and user B are planning a trip to Fukuoka. The following conversation takes place:

[0054] User A: "Let's go to Fukuoka. What do you want to eat?"

[0055] User B: "I want to eat mizutaki."

[0056] Once this conversation has taken place, the system will operate as follows:

[0057] 1. The device acquires the conversation content in real time and sends it to the server.

[0058] 2. The server analyzes the conversation and extracts the keywords "Fukuoka" and "Mizutaki."

[0059] 3. The server searches for mizutaki restaurants in Fukuoka based on these keywords and generates a list of recommendations.

[0060] 4. The server retrieves the coordinates of the listed stores and plots them on the map data.

[0061] 5. The plotted map data is sent to the device and displayed within the chat app.

[0062] 6. When User B taps on a store on the map, they can view detailed information on their device.

[0063] This system allows users A and B to efficiently plan their trips, making it extremely easy to check and share information.

[0064] The processing flow will be explained below.

[0065] Step 1:

[0066] The device monitors text data of conversations in a chat application in real time and acquires the content of the conversation when a sending or receiving event occurs.

[0067] Step 2:

[0068] The device temporarily stores the acquired conversation content in memory and formats it for transmission to the server.

[0069] Step 3:

[0070] The device sends the formatted conversation content to the server via an API endpoint.

[0071] Step 4:

[0072] The server receives the conversation data at the API endpoint and parses it into the appropriate format.

[0073] Step 5:

[0074] The server inputs the received conversation data into a natural language processing (NLP) model to analyze and extract keywords and needs.

[0075] Step 6:

[0076] The server searches for relevant facilities and stores using an internal database or an external API (e.g., Google Places API) based on the extracted keywords and needs.

[0077] Step 7:

[0078] Based on the search results, the server generates a list of recommended facilities and stores, taking into account ratings and popularity.

[0079] Step 8:

[0080] The server obtains the coordinate information of each facility in the recommendation list and plots it on the map data.

[0081] Step 9:

[0082] The server formats the plotted map data for transmission to the terminal.

[0083] Step 10:

[0084] The terminal displays the received map data within the chat application.

[0085] Step 11:

[0086] The user looks at the displayed map and checks the suggested facilities and stores.

[0087] Step 12:

[0088] When a user selects a specific facility or store, the device displays detailed information about that facility (address, opening hours, ratings, reviews, etc.).

[0089] Step 13:

[0090] The device provides navigation buttons and gestures that allow users to seamlessly switch between the chat screen and the map screen.

[0091] Step 14:

[0092] When the device returns to the chat screen, it continues to display the latest conversation content, and when it returns to the map screen, it redisplays the map information that was previously displayed.

[0093] Example 1

[0094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] In conventional chat applications, it was difficult for users to efficiently obtain information on recommended facilities when planning trips or outings. Furthermore, few systems had the functionality to search for related facilities based on conversation content and display them on a map, resulting in low user convenience. Furthermore, users were unable to seamlessly switch between the chat screen and the map display screen, resulting in a poor user experience.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0097] In this invention, the server includes means for acquiring conversation content in a chat application, means for analyzing the acquired conversation content to extract keywords and needs, means for searching for recommended facilities based on the extracted keywords and needs, means for acquiring coordinate information of the searched recommended facilities and plotting it on a map, and means for displaying the plotted map data within the chat application, thereby enabling users to efficiently search for and display recommended facilities based on the conversation content within the chat application and make travel plans.

[0098] A "chat application" is software that allows users to exchange text messages and communicate.

[0099] "Conversation content" refers to the content of text messages exchanged between users within a chat application.

[0100] The "means for acquiring" refers to a function or process for collecting and storing conversation content within a chat application.

[0101] "Means for analysis" refers to techniques or methods for analyzing acquired text data and extracting useful information.

[0102] "Keywords" refer to particularly important words or phrases in the conversation content, and are used as search and recommendation criteria.

[0103] "Needs" refer to the requirements, desires, and needs of users, and are elements that are revealed through analysis.

[0104] "Recommended facilities" refers to places, stores, and services suggested based on the user's conversation content.

[0105] "Search means" refers to methods and techniques for searching the Internet for relevant facilities and information based on specified keywords or needs.

[0106] "Plotting means" refers to the technology or method for displaying the location of the searched facility on a map.

[0107] "Map data" refers to digital map data that includes information indicating locations and areas, and includes location information and display information.

[0108] "Navigation means" refers to functions and UI components that allow users to smoothly move between the chat screen and the map display screen.

[0109] "Detailed Information" refers to additional information about the recommended establishment, including address, opening hours, reviews, etc.

[0110] The present invention provides a system that analyzes conversations when a user is planning a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. Specific embodiments of this system are described below.

[0111] 1. Acquiring chat contents

[0112] The device monitors the text entered by the user in the chat application in real time. The device can be a smartphone or tablet. When the user sends a message, the text is stored in the device's temporary storage and then sent to a server via the Internet. The data is encrypted using the HTTPS protocol.

[0113] 2. Conversation Analysis

[0114] The server receives the text data sent from the device and automatically analyzes it using natural language processing (NLP) technology. Services such as Google Cloud Natural Language API and Amazon Comprehend are used. The server uses an NLP model to analyze the text and extract keywords (e.g., "Fukuoka" and "Mizutaki") and the user's needs.

[0115] 3. Recommended stores and facilities

[0116] Based on the analysis results, the server searches for relevant facilities. Based on the generated search query (e.g., "Fukuoka Mizutaki restaurants with high ratings"), it sends a request to an external API (e.g., Google Places, Yelp API) to obtain information on recommended facilities.

[0117] 4. Mapping and Display

[0118] The server obtains the coordinate information (latitude and longitude) of each store based on the acquired store information. The Google Maps Geocoding API is used for this purpose. The coordinate information is then plotted on map data. The plotted map data is sent to the device and displayed within the chat app. The map display uses the Google Maps or Mapbox API.

[0119] 5. User confirmation and selection

[0120] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[0121] 6. Seamlessly switch between chat and maps

[0122] The device provides a seamless navigation method for switching between chat and map views, using Android and iOS user interface components, allowing users to seamlessly view and share information.

[0123] Specific examples

[0124] For example, suppose user A and user B are planning a trip to Fukuoka. The following conversation takes place:

[0125] User A: "Let's go to Fukuoka. What do you want to eat?"

[0126] User B: "I want to eat mizutaki."

[0127] Once this conversation has taken place, the system will operate as follows:

[0128] 1. The device acquires the conversation content in real time and sends it to the server.

[0129] 2. The server analyzes the conversation and extracts the keywords "Fukuoka" and "Mizutaki."

[0130] 3. The server searches for mizutaki restaurants in Fukuoka based on these keywords and generates a list of recommendations.

[0131] 4. The server retrieves the coordinates of the listed stores and plots them on the map data.

[0132] 5. The plotted map data is sent to the device and displayed within the chat app.

[0133] 6. When User B taps on a store on the map, they can view detailed information on their device.

[0134] This system allows users A and B to efficiently plan their trips, making it extremely easy to check and share information.

[0135] Prompt Sentence Examples

[0136] "Please simulate a situation where User A and User B are planning a trip to Fukuoka and are looking for a Mizutaki restaurant. Analyze the conversation, search for recommended Mizutaki restaurants in Fukuoka, and display them on a map."

[0137] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0138] Step 1: Get chat transcript

[0139] The device monitors and acquires messages entered by the user in the chat application in real time. When the user sends a message, the text is stored in the device's temporary storage. The input is the text message entered by the user, and the output is the text data stored on the device. The stored text data is encrypted using the HTTPS protocol and sent to the server.

[0140] Specific working example:

[0141] When a user types "Let's go to Fukuoka. What do you want to eat?" into the chat and presses the send button, the text is saved on the device and then sent to the server.

[0142] Step 2: Analyzing the conversation

[0143] The server receives the text data sent from the device and analyzes it using the Google Cloud Natural Language API or Amazon Comprehend. The input is the text data received by the server, and the output is the keywords extracted by the analysis and the user's needs. The server then uses this information to identify important keywords (e.g., "Fukuoka" or "Mizutaki") and the user's needs.

[0144] Specific working example:

[0145] The server analyzes the text data "Let's go to Fukuoka. What do you want to eat?" and extracts the keywords "Fukuoka" and "want to eat."

[0146] Step 3: Proposing recommended stores and facilities

[0147] The server searches for related facilities using external APIs (Google Places and Yelp APIs) based on the extracted keywords. The input is the keywords and needs, and the output is a list of recommended facilities. Specifically, a search is performed using the keywords "Fukuoka" and "Mizutaki" to identify highly rated Mizutaki restaurants.

[0148] Specific working example:

[0149] The server calls the Google Places API using the keywords "Fukuoka" and "Mizutaki" and lists recommended Mizutaki restaurants in Fukuoka.

[0150] Step 4: Mapping and display

[0151] The server obtains the coordinate information (latitude and longitude) of the acquired store information and plots it on map data. The input is a list of recommended facilities, and the output is the plotted map data. The coordinate information is obtained using the Google Maps Geocoding API, and plotted on the map using the Google Maps or Mapbox API.

[0152] Specific working example:

[0153] The server calls the Geocoding API using the address information of the recommended mizutaki restaurant, obtains the latitude and longitude, and uses that information to place a marker on the map data.

[0154] Step 5: Confirm and select users

[0155] Users check the plotted map in the chat app on their device and select the store they are interested in. The input is the plotted map data, and the output is detailed information about the store selected by the user. When the user taps a marker on the map, detailed information (address, business hours, reviews) is displayed.

[0156] Specific working example:

[0157] When users tap on a specific mizutaki restaurant displayed on the map, detailed information about the restaurant will be displayed, and they can check its opening hours and reviews.

[0158] Step 6: Seamlessly move between chat and maps

[0159] The device provides a function to seamlessly switch between the chat screen and the map display screen. The input is a user operation, and the output is switching between the chat screen and the map screen. It is implemented using Android and iOS user interface components, and tab bars and swipe gestures are applied.

[0160] Specific working example:

[0161] When a user swipes from the chat screen to the map screen, the screen smoothly switches to the map display, and when they swipe again, they return to the chat screen.

[0162] (Application example 1)

[0163] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0164] In today's information-driven society, when planning trips or outings, users often find it difficult to find the best locations and facilities among the vast amount of information available. Furthermore, there are few systems that effectively extract information about destinations and facilities from conversations within chat applications and suggest them in real time, which hinders the efficiency of planning. Furthermore, the suggested information is not presented in a format that is intuitively understandable, rather than just in text, resulting in a lack of user convenience.

[0165] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0166] In this invention, the server includes means for acquiring conversation content in a chat application, means for analyzing the acquired conversation content to extract keywords and needs, means for searching for recommended facilities based on the extracted keywords and needs, means for acquiring location information of the plotted recommended facilities in a geographic information system, and means for displaying the plotted map data within the chat application, thereby enabling users to efficiently plan trips and outings and to check suggestions and detailed information about recommended facilities in real time.

[0167] A "chat application" is software that allows users to communicate in real time using text messages.

[0168] "Conversation" refers to the text messages sent and received by users within a chat application.

[0169] "Keywords" are important words and phrases extracted from the conversation content.

[0170] "Needs" are the requests and desires that users express through their conversations.

[0171] "Recommended facilities" are places and stores selected based on the user's keywords and needs.

[0172] The "search method" refers to the functions and algorithms used to identify recommended facilities.

[0173] The "means for plotting on a map" is a function for displaying the location information of recommended facilities on map data.

[0174] A "geographic information system" is a system for processing and visualizing location information.

[0175] "Location information of recommended facilities" is information including the latitude and longitude of the location of the recommended facilities.

[0176] "Detailed information" refers to additional information about the recommended facility, such as its name, address, rating, latitude, and longitude.

[0177] "Navigation means" is a function that allows the user to smoothly switch between the chat screen and the map display screen.

[0178] This invention is a system that analyzes conversations when a user is planning a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. This system is realized using a chat application, a natural language processing (NLP) model, a geographic information system (GIS), and a navigation means.

[0179] System configuration and operation overview

[0180] 1. Acquiring chat contents

[0181] When users discuss travel or outing plans within the chat application, the device acquires text data of the conversation in real time, temporarily stores it, and then transmits it to the server.

[0182] 2. Conversation Analysis

[0183] The server analyzes the captured conversation using a natural language processing (NLP) model (e.g., a pre-trained model from the Transformers library) to extract important keywords (e.g., "Shibuya" and "sushi") and user needs. This analysis detects specific requests, such as travel destinations and meals, from the conversation.

[0184] 3. Search for recommended facilities

[0185] The server searches for recommended establishments based on the extracted keywords and user needs. For example, based on the keywords "Shibuya" and "sushi," it searches for highly rated sushi restaurants in the Shibuya area using a geographic information system (e.g., Google Maps API).

[0186] 4. Mapping and Display

[0187] The server obtains the latitude and longitude coordinates of the facilities based on the information in the list of recommended facilities generated from the search results (e.g., store name, address, rating), and plots them on map data. The plotted map data is sent to the device and displayed within the chat application.

[0188] 5. User confirmation and selection

[0189] Users can view the suggested facilities on a map displayed within the chat application, and if they find one that interests them, they can view its details (e.g., address, opening hours, reviews) on their device.

[0190] 6. Seamlessly switch between chat and maps

[0191] The device provides a navigation means for users to seamlessly switch between the chat screen and the map display screen, updating the display in real time according to the content of the conversation.

[0192] Specific examples

[0193] For example, when a user says, "I'm looking for good sushi in Shibuya," the system works as follows:

[0194] The device acquires conversations containing the keywords "Shibuya" and "sushi" and sends them to the server.

[0195] The server analyzes the conversation and extracts the keywords "Shibuya" and "sushi."

[0196] Based on these keywords, the server uses a geographic information system to search for highly rated sushi restaurants.

[0197] The server obtains the coordinate information of the searched sushi restaurant and plots it on the map data.

[0198] The device displays the plotted map data within the chat application, allowing the user to view detailed information.

[0199] Prompt Sentence Examples

[0200] When a user says, "I'm looking for good sushi in Shibuya," the system should extract the keywords "Shibuya" and "sushi," search for related sushi restaurants, and display them on a map.

[0201] Using this system, users can intuitively and efficiently plan outings based on chat conversations. In addition, the information suggested by the system is visualized on a map, allowing users to easily select destinations and stores.

[0202] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0203] Step 1: Get chat transcript

[0204] The device captures text data of conversations in real time when users discuss travel or outing plans within a chat application. The captured text data is temporarily stored within the device and then sent to a server. The input is the user's chat messages, and the output is the text data of these messages.

[0205] Step 2: Analyzing the conversation

[0206] The server receives the text data of the conversation sent from the device and analyzes it using a natural language processing (NLP) model. This process extracts important keywords and user needs from the conversation. The input is text data, and the output is keywords and needs. Specific data processing involves the use of the NLP model for Named Entity Recognition (NER) and keyword extraction.

[0207] Step 3: Find recommended facilities

[0208] The server uses a geographic information system (GIS) to search for relevant facilities based on the extracted keywords and needs. For example, it searches for highly rated sushi restaurants based on the keywords "Shibuya" and "sushi." The input is the keywords and needs, and the output is a list of recommended facilities. As a specific example, it uses the Google Maps API to search for locations and obtain their rating information.

[0209] Step 4: Geographical information acquisition and mapping

[0210] The server obtains location information (latitude and longitude) from a geographic information system based on the address information of the searched recommended facility and plots it on map data. The input is the address information of the recommended facility, and the output is the plotted map data. The location information is processed using libraries such as Geopy.

[0211] Step 5: View detailed information about recommended facilities

[0212] The terminal receives the plotted map data sent from the server and displays it within the chat application. It also allows users to check detailed information (name, address, rating, latitude, longitude) of recommended facilities on the map. The input is the plotted map data, and the output is the facility information and detailed information displayed on the map.

[0213] Step 6: Seamlessly move between chat and maps

[0214] The device provides a navigation mechanism that allows users to seamlessly switch between chat and map views. It also updates the display based on real-time conversation content. Input is user interface manipulation, and output is an updated screen display. Specific operations include smooth switching and real-time updates of the user interface.

[0215] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0216] The present invention is a system that analyzes conversations when a user plans a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. Furthermore, an embodiment of the system will be described, which combines an emotion engine that recognizes the user's emotions and recommends and displays facilities based on the user's emotions.

[0217] System configuration and operation overview

[0218] 1. Acquiring chat contents

[0219] When a user discusses travel plans with a friend in a chat application, the device captures the text data of the conversation in real time, temporarily stores the conversation content, and then transmits it to the server.

[0220] 2. Conversation Analysis

[0221] The server receives the conversation and analyzes it using a pre-trained natural language processing (NLP) model, which extracts important keywords from the conversation (e.g., "Fukuoka" and "Mizutaki") and the user's needs.

[0222] 3. Emotional Recognition

[0223] At the same time, the server uses an emotion engine to analyze and recognize the user's emotions from the conversation content. For example, if the user seems to be having a fun conversation, the emotion of "joy" is recognized.

[0224] 4. Recommended stores and facilities

[0225] The server searches for relevant facilities based on the analyzed keywords, needs, and user emotions. For example, if the emotions "Fukuoka," "Mizutaki," and "Joy" are recognized, the server searches for highly rated Mizutaki restaurants in the Fukuoka area and prioritizes a list of the most suitable restaurants based on the user's emotions.

[0226] 5. Mapping and Display

[0227] Based on the information in the generated list of recommended facilities (e.g., store name, address, rating), the server obtains the coordinates of the facilities and plots them on a map. The plotted map data is sent to the device and displayed within the chat application.

[0228] 6. User confirmation and selection

[0229] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[0230] 7. Seamlessly switch between chat and maps

[0231] The device provides a navigation method that seamlessly switches between the chat screen and the map display screen, allowing users to smoothly check and share information.

[0232] Specific examples

[0233] For example, suppose user A and user B are planning a trip to Fukuoka. The following conversation takes place:

[0234] User A: "Let's go to Fukuoka. What do you want to eat?"

[0235] User B: "I want to eat mizutaki."

[0236] Once this conversation has taken place, the system will operate as follows:

[0237] 1. The device acquires the conversation content in real time and sends it to the server.

[0238] 2. The server analyzes the conversation and extracts the keywords "Fukuoka" and "Mizutaki."

[0239] 3. The server uses an emotion engine to recognize the user's emotions. If the conversation seems enjoyable, the emotion of "joy" is recognized.

[0240] 4. The server searches for mizutaki restaurants in Fukuoka based on the keywords and the recognized emotions, and generates a list of recommendations. At this time, it prioritizes the restaurant that best matches the user's emotions (e.g., "Hakata Kamaitachi").

[0241] 5. The server retrieves the coordinates of the listed stores and plots them on the map data.

[0242] 6. The plotted map data is sent to the device and displayed within the chat app.

[0243] 7. When User B taps on a store on the map, detailed information can be viewed on the device.

[0244] This system allows users A and B to efficiently plan their trips, making it extremely easy to confirm and share information. Furthermore, it improves the user experience by making suggestions based on the user's emotions.

[0245] The processing flow will be explained below.

[0246] Step 1:

[0247] The device monitors text data of conversations in a chat application in real time and acquires the content of the conversation when a sending or receiving event occurs.

[0248] Step 2:

[0249] The device temporarily stores the acquired conversation content in memory and formats it for transmission to the server.

[0250] Step 3:

[0251] The device sends the formatted conversation content to the server via an API endpoint.

[0252] Step 4:

[0253] The server receives the conversation data at the API endpoint and parses it into the appropriate format.

[0254] Step 5:

[0255] The server inputs the received conversation data into a natural language processing (NLP) model to analyze and extract keywords and needs.

[0256] Step 6:

[0257] At the same time, the server uses an emotion engine to analyze and recognize the user's emotions (e.g., "happiness," "surprise," "dissatisfaction," etc.) from the conversation data.

[0258] Step 7:

[0259] The server searches for relevant facilities and stores using an internal database or an external API (e.g., Google Places API) based on the extracted keywords, needs, and recognized user sentiment.

[0260] Step 8:

[0261] Based on the search results, the server generates a list of recommended facilities and stores, taking into account ratings, popularity, and recognized user emotions.

[0262] Step 9:

[0263] The server obtains the coordinate information of each facility in the recommendation list and plots it on the map data.

[0264] Step 10:

[0265] The server formats the plotted map data for transmission to the terminal.

[0266] Step 11:

[0267] The terminal displays the received map data within the chat application.

[0268] Step 12:

[0269] The user looks at the displayed map and checks the suggested facilities and stores.

[0270] Step 13:

[0271] When a user selects a specific facility or store, the device displays detailed information about that facility (address, opening hours, ratings, reviews, etc.).

[0272] Step 14:

[0273] The device provides navigation buttons and gestures that allow users to seamlessly switch between the chat screen and the map screen.

[0274] Step 15:

[0275] When the device returns to the chat screen, it continues to display the latest conversation content, and when it returns to the map screen, it redisplays the map information that was previously displayed.

[0276] Example 2

[0277] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0278] In conventional travel and outing planning, users have to search for information on the Internet and select appropriate facilities, which takes time and effort to obtain and select information.Furthermore, there is also the problem that the user experience is not sufficiently improved because it is not possible to suggest facilities based on the user's emotions and needs.

[0279] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0280] In this invention, the server includes means for acquiring conversation content between users in a chat application, means for the server to analyze the acquired conversation content and extract keywords and needs, and means for the server to search for facilities to recommend based on the extracted keywords and needs and adjust the recommendation ranking of the facilities based on the user's emotions. This enables efficient travel and outing planning by automatically searching for facilities that match the user's emotions and needs and displaying them on a map within the chat application.

[0281] "User" refers to an individual who uses the chat application to plan trips and outings.

[0282] "Server" refers to the central processing unit that analyzes the conversation content of the chat application and searches for and adjusts recommended facilities based on keywords, user needs, and emotions.

[0283] "Terminal" refers to a device (e.g., smartphone, tablet, or PC) that is directly operated by the user, has a chat application installed, acquires conversation content, and sends and receives data to and from the server.

[0284] A "chat application" is software used by users to communicate, and refers to an application that allows users to send and receive text messages in real time.

[0285] "Conversation content" refers to the content of text messages exchanged between users within a chat application, including information about travel and outings.

[0286] "Keywords" are important words and phrases extracted from conversations and used to identify a user's needs and destinations.

[0287] "Needs" refer to the desires and requirements that a user has regarding travel or outings.

[0288] "Emotion engine" refers to software or a module for analyzing and recognizing a user's emotions from the content of a conversation.

[0289] "Recommended facilities" refers to facilities such as travel destinations and restaurants that are suggested based on the user's keywords and needs.

[0290] "Recommendation ranking" refers to the priority of a recommended facility within the list, and is adjusted based on the user's feelings, ratings, etc.

[0291] "Map data" refers to map information on which the location information of recommended facilities is plotted.

[0292] The present invention is a system that analyzes conversations when a user plans a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. Furthermore, an embodiment of the system is described below, which combines an emotion engine that recognizes the user's emotions and recommends and displays facilities based on the user's emotions.

[0293] System configuration and operation

[0294] Get chat transcript

[0295] When a user discusses travel plans with a friend in a chat application, the device captures the text data of the conversation in real time, temporarily stores the conversation content, and then transmits it to the server.

[0296] Conversation analysis

[0297] The server receives the conversation and analyzes it using a pre-trained natural language processing (NLP) model (such as BERT). Through analysis, important keywords (e.g., "city name," "type of cuisine") and user needs are extracted from the conversation.

[0298] Emotion recognition

[0299] At the same time, the server uses an emotion engine (such as Affectiva) to analyze and recognize the user's emotions from the conversation content. For example, if the user is having a fun conversation, the emotion of "joy" is recognized.

[0300] Recommended stores and facilities

[0301] Based on the analyzed keywords, needs, and user sentiment, the server uses the Google Places API to search for related facilities. For example, if the "city name," "type of cuisine," and "pleasure" sentiment are recognized, the server searches for related facilities with high ratings and prioritizes the most suitable stores from the search results.

[0302] Mapping and Display

[0303] Based on the information in the generated list of recommended facilities (e.g., store name, address, rating), the server uses the Google Maps API to obtain the coordinates of the facilities and plots them on a map. The plotted map data is sent to the device and displayed within the chat application.

[0304] User confirmation and selection

[0305] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[0306] Seamlessly switch between chat and maps

[0307] The device provides a navigation method that seamlessly switches between the chat screen and the map display screen, allowing users to smoothly check and share information.

[0308] Specific examples

[0309] For example, suppose user A and user B are planning a trip to a certain city. The following conversation takes place:

[0310] User A: "Let's go to City Name. What do you want to eat?"

[0311] User B: "I want to try a variety of dishes."

[0312] Once this conversation has taken place, the system will operate as follows:

[0313] 1. The device acquires the conversation content in real time and sends it to the server.

[0314] 2. The server analyzes the conversation and extracts keywords such as "city name" and "type of cuisine."

[0315] 3. The server uses an emotion engine to recognize the user's emotions. If the conversation seems enjoyable, the emotion of "joy" is recognized.

[0316] 4. The server searches for establishments in the city that serve the type of cuisine based on the keywords and the recognized emotion, and generates a list of recommendations, prioritizing establishments that best fit the user's emotion.

[0317] 5. The server retrieves the coordinates of the listed stores and plots them on the map data.

[0318] 6. The plotted map data is sent to the device and displayed within the chat app.

[0319] 7. When User B taps on a store on the map, detailed information can be viewed on the device.

[0320] This system allows users A and B to efficiently plan their trips, making it extremely easy to confirm and share information. Furthermore, it improves the user experience by making suggestions based on the user's emotions.

[0321] Example prompts to input to the generative AI model

[0322] By inputting prompts like the following into the generative AI model, we can get an explanation of the system's behavior:

[0323] "Describe a system in a chat application for planning trips that analyzes conversations, suggests recommended facilities, and displays them on a map. Include recommending and displaying facilities based on the user's emotions."

[0324] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0325] Step 1: Get chat transcript

[0326] When a user starts a conversation on a chat application, the device collects the text data of the conversation in real time. Specifically, the user sends a message such as "Let's go to City Name. What do you want to eat?" The device receives this message and temporarily stores it. The input is the text message that the user sends on the application, and the output is the stored text data.

[0327] Step 2: Send the conversation

[0328] The terminal transmits the temporarily saved text data to the server. Specifically, the terminal transmits the temporarily saved text data to the server via the network. The input is the saved text data, and the output is the transmitted text data.

[0329] Step 3: Analyzing the conversation

[0330] The text data received by the server is analyzed using a pre-trained natural language processing (NLP) model (e.g., BERT). Specifically, the received text data is input into the NLP model, which extracts important keywords such as "city name" and "type of cuisine" as well as the user's needs. The input is the received text data, and the output is the extracted keywords and needs.

[0331] Step 4: Recognize emotions

[0332] The server uses an emotion engine (e.g., Affectiva) to analyze the user's emotions from the conversation content. Specifically, the extracted keywords and text data are input into the emotion engine to recognize the user's emotions (e.g., joy). The input is the extracted keywords and text data, and the output is the recognized emotion.

[0333] Step 5: Proposing recommended stores and facilities

[0334] The server uses the Google Places API to search for related facilities based on the extracted keywords and recognized emotions. Specifically, it searches for highly rated facilities based on the criteria of "city name," "type of cuisine," and "pleasure," and generates a list of recommendations. The input is the extracted keywords and recognized emotions, and the output is the generated list of recommended facilities.

[0335] Step 6: Obtain facility coordinates

[0336] The server uses the Google Maps API to obtain coordinate information for facilities from the list of recommended facilities. Specifically, it searches for coordinate information from the facility name or address and obtains coordinate information to plot on map data. The input is the list of recommended facilities, and the output is the obtained coordinate information.

[0337] Step 7: Generate map data

[0338] The server plots the acquired coordinate information onto map data. Specifically, it plots the facility locations on a map to generate visually easy-to-understand map data. The input is the acquired coordinate information, and the output is the plotted map data.

[0339] Step 8: Displaying map data

[0340] The terminal displays the plotted map data received from the server within the chat application. Specifically, the map data is reflected in the user interface, allowing the user to view facilities on the map. The input is the plotted map data, and the output is the displayed map interface.

[0341] Step 9: View facility details

[0342] When a user taps on a facility of interest on the map, the device displays detailed information about that facility (address, business hours, reviews, etc.). Specifically, the device obtains detailed information according to the user's operation and displays it on the interface. The input is the user's tap operation, and the output is the displayed detailed facility information.

[0343] Step 10: Seamlessly move between chat and maps

[0344] The device provides a navigation method that allows seamless switching between the chat screen and the map display screen. Specifically, the user can smoothly move back and forth between the chat screen and the map display screen using the tabs at the top of the screen. The input is the user's navigation operation, and the output is the switched screen.

[0345] (Application example 2)

[0346] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0347] Conventional chat applications lack the functionality to analyze conversation content and suggest suitable facilities when users plan trips or outings, making planning cumbersome and inefficient. Furthermore, suggestions do not reflect the user's emotions, making it difficult to provide a better experience. Furthermore, the inability to effectively present users with location information and detailed information about recommended facilities also delays planning.

[0348] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0349] In this invention, the server includes means for acquiring conversation content in a chat application, means for analyzing the acquired conversation content to extract keywords and needs, means for analyzing and recognizing user emotions from the conversation content using an emotion engine, means for searching for recommended facilities based on the extracted keywords, needs, and emotions, means for plotting the searched recommended facilities on a map, means for displaying the plotted map data within the chat application, means for displaying detailed information about the recommended facilities, and navigation means for seamlessly switching between the chat screen and the map display screen. This makes it possible to suggest optimal facilities based on the user's conversation content and emotions, making planning more efficient and convenient.

[0350] A "chat application" is an application that allows users to send and receive text messages in real time.

[0351] The "means for acquiring conversation content" is a function for acquiring text data of conversations taking place within a chat application in real time.

[0352] "Means for extracting keywords and needs" is a function for recognizing and extracting important phrases and user requests from the acquired conversation content.

[0353] An "emotion engine" is software or a component that analyzes a user's emotional state from the content of a conversation and recognizes specific emotions.

[0354] The "means for searching for recommended facilities" is a function for searching for relevant facilities from a database or the Internet based on the extracted keywords, needs, and emotions.

[0355] "Means for plotting on a map" is a function for obtaining the location information of the searched facility and marking it on a digital map.

[0356] The "means for displaying" is a function for visually displaying the plotted map data within the chat application.

[0357] "Means for displaying detailed information about recommended facilities" is a function for providing users with specific information about recommended facilities (e.g., address, business hours, reviews).

[0358] "Seamless switching navigation means" is a function that provides operability for switching between the chat screen and the map display screen without delay.

[0359] System configuration and operation overview

[0360] The present invention is a travel planning support system for smartphones. This system is composed of the following main components:

[0361] 1. Acquiring chat contents

[0362] When a user is chatting with another user about travel or outing plans within a chat application, the device captures the text data of the conversation in real time. This data is temporarily stored on the device and then sent to a server.

[0363] 2. Conversation Analysis

[0364] The server analyzes the captured conversation using natural language processing (NLP) technology, specifically using the Bidirectional Encoder Representations from Transformers (BERT) model to extract important keywords and user needs from the conversation.

[0365] 3. Emotional Recognition

[0366] The server uses an emotion engine to analyze and recognize the user's emotions from the conversation content. It uses the VADER (Valence Aware Dictionary and sEntiment Reasoner) algorithm to determine the positivity, negativity, and intensity of each emotion.

[0367] 4. Recommended facilities

[0368] The server searches a database or the Internet for relevant facilities based on the extracted keywords, needs, and recognized emotions, and prioritizes and lists the facilities that best fit the user's emotions from the search results.

[0369] 5. Mapping and Display

[0370] The location information of the recommended establishments is retrieved and plotted on a digital map. The map data is generated using the folium library, sent to the device, and displayed visually within the chat application.

[0371] 6. User confirmation and selection

[0372] Users can view suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its details (address, opening hours, reviews) on their device.

[0373] 7. Seamlessly switch between chat and maps

[0374] The device provides a navigation method that allows users to seamlessly switch between the chat screen and the map display screen, allowing them to smoothly check and share information.

[0375] Specific examples

[0376] For example, if a user has the following conversation:

[0377] User A: "Let's go to Kyoto. What do you want to eat?"

[0378] User B: "I want to eat ramen."

[0379] In this case, the system behaves as follows:

[0380] 1. The device acquires the conversation content in real time and sends it to the server.

[0381] 2. The server uses the BERT model to extract the keywords "Kyoto" and "ramen."

[0382] 3. The server uses the VADER algorithm to recognize the emotion "joy" if the conversation is positive.

[0383] 4. The server searches for highly rated ramen restaurants in the Kyoto area based on keywords and sentiment, and lists the most suitable restaurants.

[0384] 5. The server retrieves the coordinates of the listed stores and plots them on a map using the folium library.

[0385] 6. The plotted map data is sent to the device and displayed within the chat app.

[0386] This allows users to efficiently plan their trips, makes it much easier to check and share information, and improves the user experience by providing suggestions based on user sentiment.

[0387] Prompt Sentence Examples

[0388] For example, the following prompt might be used to search for a recommended cafe in Kumamoto:

[0389] text

[0390] Are there any good cafes in Kumamoto?

[0391] Based on this, the system analyzes the keywords "Kumamoto" and "cafe" along with positive emotions, suggests the most suitable cafe, and displays it on a map.

[0392] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0393] Step 1:

[0394] Acquiring conversation content

[0395] The terminal acquires text data of conversations taking place within a chat application in real time. This data is temporarily stored on the terminal. The specific input is a text message entered by the user, and the output is the acquired text data of the conversation.

[0396] Step 2:

[0397] Sending conversation transcripts

[0398] The terminal sends the acquired text data to the server. The input is the temporarily saved text data, and the output is the data to be sent to the server. Specifically, the data is sent to the server via an HTTP request.

[0399] Step 3:

[0400] Conversation analysis

[0401] The server analyzes the received conversation content using natural language processing (NLP) technology. Specifically, it uses the BERT model to extract keywords and needs from the input text data. The input is the text data sent to the server, and the output is the extracted keywords and needs.

[0402] Step 4:

[0403] Emotion recognition

[0404] The server uses an emotion engine (VADER algorithm) to analyze and recognize the user's emotions from the conversation content. The input is the text data sent to the server, and the output is the recognized emotion data. Specific operations include emotion scoring and analysis.

[0405] Step 5:

[0406] Search for recommended facilities

[0407] The server searches for relevant facilities from a database or the Internet based on the extracted keywords, needs, and recognized emotions. The input is the extracted keywords and emotion data, and the output is a list of recommended facilities. Specific operations include executing API requests and database queries.

[0408] Step 6:

[0409] Facility location and mapping

[0410] The server acquires the location information of the recommended facilities and plots it on a digital map. The input is a list of recommended facilities, and the output is the plotted map data. Specific operations include geocoding and generating map data.

[0411] Step 7:

[0412] Sending map data

[0413] The server sends the plotted map data to the device. The input is the plotted map data, and the output is the data to be sent to the device. Specifically, the server sends the map data to the device via an HTTP request.

[0414] Step 8:

[0415] Displaying the map

[0416] The terminal visually displays the received map data within the chat application. The input is the map data sent from the server, and the output is the visually displayed map.

[0417] Step 9:

[0418] Displaying detailed facility information

[0419] The user views the map displayed in the chat app to check the suggested facilities. When the user selects a facility of interest, its details (address, opening hours, reviews) are displayed on the device. The input is the ID of the selected facility, and the output is the facility details.

[0420] Step 10:

[0421] Seamless switching between chat and maps

[0422] The device provides a navigation method that allows seamless switching between the chat screen and the map display screen. The input is the user's operation, and the output is the screen after switching. Specifically, the operation smoothly switches between user interfaces.

[0423] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0424] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0425] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0426] [Second embodiment]

[0427] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0428] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0429] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0430] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0431] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0432] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0433] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0434] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0435] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0436] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0437] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0438] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0439] The present invention is a system that analyzes the content of conversations when a user plans a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map.

[0440] System configuration and operation overview

[0441] 1. Acquiring chat contents

[0442] When a user discusses travel plans with a friend in a chat application, the device captures the text data of the conversation in real time, temporarily stores the conversation content, and then transmits it to the server.

[0443] 2. Conversation Analysis

[0444] The server receives the conversation and analyzes it using a pre-trained natural language processing (NLP) model, which extracts important keywords from the conversation (e.g., "Fukuoka" and "Mizutaki") and the user's needs.

[0445] 3. Recommended stores and facilities

[0446] Based on the analysis results, the server searches for related facilities. For example, based on keywords such as "Fukuoka" and "Mizutaki," it searches for highly rated Mizutaki restaurants in the Fukuoka area. From the search results, it generates a list of recommended facilities.

[0447] 4. Mapping and Display

[0448] Based on the information in the generated list of recommended facilities (e.g., store name, address, rating), the server obtains the coordinates of the facilities and plots them on a map. The plotted map data is sent to the device and displayed within the chat application.

[0449] 5. User confirmation and selection

[0450] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[0451] 6. Seamlessly switch between chat and maps

[0452] The device provides a navigation method that seamlessly switches between the chat screen and the map display screen, allowing users to smoothly check and share information.

[0453] Specific examples

[0454] For example, suppose user A and user B are planning a trip to Fukuoka. The following conversation takes place:

[0455] User A: "Let's go to Fukuoka. What do you want to eat?"

[0456] User B: "I want to eat mizutaki."

[0457] Once this conversation has taken place, the system will operate as follows:

[0458] 1. The device acquires the conversation content in real time and sends it to the server.

[0459] 2. The server analyzes the conversation and extracts the keywords "Fukuoka" and "Mizutaki."

[0460] 3. The server searches for mizutaki restaurants in Fukuoka based on these keywords and generates a list of recommendations.

[0461] 4. The server retrieves the coordinates of the listed stores and plots them on the map data.

[0462] 5. The plotted map data is sent to the device and displayed within the chat app.

[0463] 6. When User B taps on a store on the map, they can view detailed information on their device.

[0464] This system allows users A and B to efficiently plan their trips, making it extremely easy to check and share information.

[0465] The processing flow will be explained below.

[0466] Step 1:

[0467] The device monitors text data of conversations in a chat application in real time and acquires the content of the conversation when a sending or receiving event occurs.

[0468] Step 2:

[0469] The device temporarily stores the acquired conversation content in memory and formats it for transmission to the server.

[0470] Step 3:

[0471] The device sends the formatted conversation content to the server via an API endpoint.

[0472] Step 4:

[0473] The server receives the conversation data at the API endpoint and parses it into the appropriate format.

[0474] Step 5:

[0475] The server inputs the received conversation data into a natural language processing (NLP) model to analyze and extract keywords and needs.

[0476] Step 6:

[0477] The server searches for relevant facilities and stores using an internal database or an external API (e.g., Google Places API) based on the extracted keywords and needs.

[0478] Step 7:

[0479] Based on the search results, the server generates a list of recommended facilities and stores, taking into account ratings and popularity.

[0480] Step 8:

[0481] The server obtains the coordinate information of each facility in the recommendation list and plots it on the map data.

[0482] Step 9:

[0483] The server formats the plotted map data for transmission to the terminal.

[0484] Step 10:

[0485] The terminal displays the received map data within the chat application.

[0486] Step 11:

[0487] The user looks at the displayed map and checks the suggested facilities and stores.

[0488] Step 12:

[0489] When a user selects a specific facility or store, the device displays detailed information about that facility (address, opening hours, ratings, reviews, etc.).

[0490] Step 13:

[0491] The device provides navigation buttons and gestures that allow users to seamlessly switch between the chat screen and the map screen.

[0492] Step 14:

[0493] When the device returns to the chat screen, it continues to display the latest conversation content, and when it returns to the map screen, it redisplays the map information that was previously displayed.

[0494] Example 1

[0495] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0496] In conventional chat applications, it was difficult for users to efficiently obtain information on recommended facilities when planning trips or outings. Furthermore, few systems had the functionality to search for related facilities based on conversation content and display them on a map, resulting in low user convenience. Furthermore, users were unable to seamlessly switch between the chat screen and the map display screen, resulting in a poor user experience.

[0497] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0498] In this invention, the server includes means for acquiring conversation content in a chat application, means for analyzing the acquired conversation content to extract keywords and needs, means for searching for recommended facilities based on the extracted keywords and needs, means for acquiring coordinate information of the searched recommended facilities and plotting it on a map, and means for displaying the plotted map data within the chat application, thereby enabling users to efficiently search for and display recommended facilities based on the conversation content within the chat application and make travel plans.

[0499] A "chat application" is software that allows users to exchange text messages and communicate.

[0500] "Conversation content" refers to the content of text messages exchanged between users within a chat application.

[0501] The "means for acquiring" refers to a function or process for collecting and storing conversation content within a chat application.

[0502] "Means for analysis" refers to techniques or methods for analyzing acquired text data and extracting useful information.

[0503] "Keywords" refer to particularly important words or phrases in the conversation content, and are used as search and recommendation criteria.

[0504] "Needs" refer to the requirements, desires, and needs of users, and are elements that are revealed through analysis.

[0505] "Recommended facilities" refers to places, stores, and services suggested based on the user's conversation content.

[0506] "Search means" refers to methods and techniques for searching the Internet for relevant facilities and information based on specified keywords or needs.

[0507] "Plotting means" refers to the technology or method for displaying the location of the searched facility on a map.

[0508] "Map data" refers to digital map data that includes information indicating locations and areas, and includes location information and display information.

[0509] "Navigation means" refers to functions and UI components that allow users to smoothly move between the chat screen and the map display screen.

[0510] "Detailed Information" refers to additional information about the recommended establishment, including address, opening hours, reviews, etc.

[0511] The present invention provides a system that analyzes conversations when a user is planning a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. Specific embodiments of this system are described below.

[0512] 1. Acquiring chat contents

[0513] The device monitors the text entered by the user in the chat application in real time. The device can be a smartphone or tablet. When the user sends a message, the text is stored in the device's temporary storage and then sent to a server via the Internet. The data is encrypted using the HTTPS protocol.

[0514] 2. Conversation Analysis

[0515] The server receives the text data sent from the device and automatically analyzes it using natural language processing (NLP) technology. Services such as Google Cloud Natural Language API and Amazon Comprehend are used. The server uses an NLP model to analyze the text and extract keywords (e.g., "Fukuoka" and "Mizutaki") and the user's needs.

[0516] 3. Recommended stores and facilities

[0517] Based on the analysis results, the server searches for relevant facilities. Based on the generated search query (e.g., "Fukuoka Mizutaki restaurants with high ratings"), it sends a request to an external API (e.g., Google Places, Yelp API) to obtain information on recommended facilities.

[0518] 4. Mapping and Display

[0519] The server obtains the coordinate information (latitude and longitude) of each store based on the acquired store information. The Google Maps Geocoding API is used for this purpose. The coordinate information is then plotted on map data. The plotted map data is sent to the device and displayed within the chat app. The map display uses the Google Maps or Mapbox API.

[0520] 5. User confirmation and selection

[0521] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[0522] 6. Seamlessly switch between chat and maps

[0523] The device provides a seamless navigation method for switching between chat and map views, using Android and iOS user interface components, allowing users to seamlessly view and share information.

[0524] Specific examples

[0525] For example, suppose user A and user B are planning a trip to Fukuoka. The following conversation takes place:

[0526] User A: "Let's go to Fukuoka. What do you want to eat?"

[0527] User B: "I want to eat mizutaki."

[0528] Once this conversation has taken place, the system will operate as follows:

[0529] 1. The device acquires the conversation content in real time and sends it to the server.

[0530] 2. The server analyzes the conversation and extracts the keywords "Fukuoka" and "Mizutaki."

[0531] 3. The server searches for mizutaki restaurants in Fukuoka based on these keywords and generates a list of recommendations.

[0532] 4. The server retrieves the coordinates of the listed stores and plots them on the map data.

[0533] 5. The plotted map data is sent to the device and displayed within the chat app.

[0534] 6. When User B taps on a store on the map, they can view detailed information on their device.

[0535] This system allows users A and B to efficiently plan their trips, making it extremely easy to check and share information.

[0536] Prompt Sentence Examples

[0537] "Please simulate a situation where User A and User B are planning a trip to Fukuoka and are looking for a Mizutaki restaurant. Analyze the conversation, search for recommended Mizutaki restaurants in Fukuoka, and display them on a map."

[0538] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0539] Step 1: Get chat transcript

[0540] The device monitors and acquires messages entered by the user in the chat application in real time. When the user sends a message, the text is stored in the device's temporary storage. The input is the text message entered by the user, and the output is the text data stored on the device. The stored text data is encrypted using the HTTPS protocol and sent to the server.

[0541] Specific working example:

[0542] When a user types "Let's go to Fukuoka. What do you want to eat?" into the chat and presses the send button, the text is saved on the device and then sent to the server.

[0543] Step 2: Analyzing the conversation

[0544] The server receives the text data sent from the device and analyzes it using the Google Cloud Natural Language API or Amazon Comprehend. The input is the text data received by the server, and the output is the keywords extracted by the analysis and the user's needs. The server then uses this information to identify important keywords (e.g., "Fukuoka" or "Mizutaki") and the user's needs.

[0545] Specific working example:

[0546] The server analyzes the text data "Let's go to Fukuoka. What do you want to eat?" and extracts the keywords "Fukuoka" and "want to eat."

[0547] Step 3: Proposing recommended stores and facilities

[0548] The server searches for related facilities using external APIs (Google Places and Yelp APIs) based on the extracted keywords. The input is the keywords and needs, and the output is a list of recommended facilities. Specifically, a search is performed using the keywords "Fukuoka" and "Mizutaki" to identify highly rated Mizutaki restaurants.

[0549] Specific working example:

[0550] The server calls the Google Places API using the keywords "Fukuoka" and "Mizutaki" and lists recommended Mizutaki restaurants in Fukuoka.

[0551] Step 4: Mapping and display

[0552] The server obtains the coordinate information (latitude and longitude) of the acquired store information and plots it on map data. The input is a list of recommended facilities, and the output is the plotted map data. The coordinate information is obtained using the Google Maps Geocoding API, and plotted on the map using the Google Maps or Mapbox API.

[0553] Specific working example:

[0554] The server calls the Geocoding API using the address information of the recommended mizutaki restaurant, obtains the latitude and longitude, and uses that information to place a marker on the map data.

[0555] Step 5: Confirm and select users

[0556] Users check the plotted map in the chat app on their device and select the store they are interested in. The input is the plotted map data, and the output is detailed information about the store selected by the user. When the user taps a marker on the map, detailed information (address, business hours, reviews) is displayed.

[0557] Specific working example:

[0558] When users tap on a specific mizutaki restaurant displayed on the map, detailed information about the restaurant will be displayed, and they can check its opening hours and reviews.

[0559] Step 6: Seamlessly move between chat and maps

[0560] The device provides a function to seamlessly switch between the chat screen and the map display screen. The input is a user operation, and the output is switching between the chat screen and the map screen. It is implemented using Android and iOS user interface components, and tab bars and swipe gestures are applied.

[0561] Specific working example:

[0562] When a user swipes from the chat screen to the map screen, the screen smoothly switches to the map display, and when they swipe again, they return to the chat screen.

[0563] (Application example 1)

[0564] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0565] In today's information-driven society, when planning trips or outings, users often find it difficult to find the best locations and facilities among the vast amount of information available. Furthermore, there are few systems that effectively extract information about destinations and facilities from conversations within chat applications and suggest them in real time, which hinders the efficiency of planning. Furthermore, the suggested information is not presented in a format that is intuitively understandable, rather than just in text, resulting in a lack of user convenience.

[0566] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0567] In this invention, the server includes means for acquiring conversation content in a chat application, means for analyzing the acquired conversation content to extract keywords and needs, means for searching for recommended facilities based on the extracted keywords and needs, means for acquiring location information of the plotted recommended facilities in a geographic information system, and means for displaying the plotted map data within the chat application, thereby enabling users to efficiently plan trips and outings and to check suggestions and detailed information about recommended facilities in real time.

[0568] A "chat application" is software that allows users to communicate in real time using text messages.

[0569] "Conversation" refers to the text messages sent and received by users within a chat application.

[0570] "Keywords" are important words and phrases extracted from the conversation content.

[0571] "Needs" are the requests and desires that users express through their conversations.

[0572] "Recommended facilities" are places and stores selected based on the user's keywords and needs.

[0573] The "search method" refers to the functions and algorithms used to identify recommended facilities.

[0574] The "means for plotting on a map" is a function for displaying the location information of recommended facilities on map data.

[0575] A "geographic information system" is a system for processing and visualizing location information.

[0576] "Location information of recommended facilities" is information including the latitude and longitude of the location of the recommended facilities.

[0577] "Detailed information" refers to additional information about the recommended facility, such as its name, address, rating, latitude, and longitude.

[0578] "Navigation means" is a function that allows the user to smoothly switch between the chat screen and the map display screen.

[0579] This invention is a system that analyzes conversations when a user is planning a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. This system is realized using a chat application, a natural language processing (NLP) model, a geographic information system (GIS), and a navigation means.

[0580] System configuration and operation overview

[0581] 1. Acquiring chat contents

[0582] When users discuss travel or outing plans within the chat application, the device acquires text data of the conversation in real time, temporarily stores it, and then transmits it to the server.

[0583] 2. Conversation Analysis

[0584] The server analyzes the captured conversation using a natural language processing (NLP) model (e.g., a pre-trained model from the Transformers library) to extract important keywords (e.g., "Shibuya" and "sushi") and user needs. This analysis detects specific requests, such as travel destinations and meals, from the conversation.

[0585] 3. Search for recommended facilities

[0586] The server searches for recommended establishments based on the extracted keywords and user needs. For example, based on the keywords "Shibuya" and "sushi," it searches for highly rated sushi restaurants in the Shibuya area using a geographic information system (e.g., Google Maps API).

[0587] 4. Mapping and Display

[0588] The server obtains the latitude and longitude coordinates of the facilities based on the information in the list of recommended facilities generated from the search results (e.g., store name, address, rating), and plots them on map data. The plotted map data is sent to the device and displayed within the chat application.

[0589] 5. User confirmation and selection

[0590] Users can view the suggested facilities on a map displayed within the chat application, and if they find one that interests them, they can view its details (e.g., address, opening hours, reviews) on their device.

[0591] 6. Seamlessly switch between chat and maps

[0592] The device provides a navigation means for users to seamlessly switch between the chat screen and the map display screen, updating the display in real time according to the content of the conversation.

[0593] Specific examples

[0594] For example, when a user says, "I'm looking for good sushi in Shibuya," the system works as follows:

[0595] The device acquires conversations containing the keywords "Shibuya" and "sushi" and sends them to the server.

[0596] The server analyzes the conversation and extracts the keywords "Shibuya" and "sushi."

[0597] Based on these keywords, the server uses a geographic information system to search for highly rated sushi restaurants.

[0598] The server obtains the coordinate information of the searched sushi restaurant and plots it on the map data.

[0599] The device displays the plotted map data within the chat application, allowing the user to view detailed information.

[0600] Prompt Sentence Examples

[0601] When a user says, "I'm looking for good sushi in Shibuya," the system should extract the keywords "Shibuya" and "sushi," search for related sushi restaurants, and display them on a map.

[0602] Using this system, users can intuitively and efficiently plan outings based on chat conversations. In addition, the information suggested by the system is visualized on a map, allowing users to easily select destinations and stores.

[0603] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0604] Step 1: Get chat transcript

[0605] The device captures text data of conversations in real time when users discuss travel or outing plans within a chat application. The captured text data is temporarily stored within the device and then sent to a server. The input is the user's chat messages, and the output is the text data of these messages.

[0606] Step 2: Analyzing the conversation

[0607] The server receives the text data of the conversation sent from the device and analyzes it using a natural language processing (NLP) model. This process extracts important keywords and user needs from the conversation. The input is text data, and the output is keywords and needs. Specific data processing involves the use of the NLP model for Named Entity Recognition (NER) and keyword extraction.

[0608] Step 3: Find recommended facilities

[0609] The server uses a geographic information system (GIS) to search for relevant facilities based on the extracted keywords and needs. For example, it searches for highly rated sushi restaurants based on the keywords "Shibuya" and "sushi." The input is the keywords and needs, and the output is a list of recommended facilities. As a specific example, it uses the Google Maps API to search for locations and obtain their rating information.

[0610] Step 4: Geographical information acquisition and mapping

[0611] The server obtains location information (latitude and longitude) from a geographic information system based on the address information of the searched recommended facility and plots it on map data. The input is the address information of the recommended facility, and the output is the plotted map data. The location information is processed using libraries such as Geopy.

[0612] Step 5: View detailed information about recommended facilities

[0613] The terminal receives the plotted map data sent from the server and displays it within the chat application. It also allows users to check detailed information (name, address, rating, latitude, longitude) of recommended facilities on the map. The input is the plotted map data, and the output is the facility information and detailed information displayed on the map.

[0614] Step 6: Seamlessly move between chat and maps

[0615] The device provides a navigation mechanism that allows users to seamlessly switch between chat and map views. It also updates the display based on real-time conversation content. Input is user interface manipulation, and output is an updated screen display. Specific operations include smooth switching and real-time updates of the user interface.

[0616] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0617] The present invention is a system that analyzes conversations when a user plans a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. Furthermore, an embodiment of the system will be described, which combines an emotion engine that recognizes the user's emotions and recommends and displays facilities based on the user's emotions.

[0618] System configuration and operation overview

[0619] 1. Acquiring chat contents

[0620] When a user discusses travel plans with a friend in a chat application, the device captures the text data of the conversation in real time, temporarily stores the conversation content, and then transmits it to the server.

[0621] 2. Conversation Analysis

[0622] The server receives the conversation and analyzes it using a pre-trained natural language processing (NLP) model, which extracts important keywords from the conversation (e.g., "Fukuoka" and "Mizutaki") and the user's needs.

[0623] 3. Emotional Recognition

[0624] At the same time, the server uses an emotion engine to analyze and recognize the user's emotions from the conversation content. For example, if the user seems to be having a fun conversation, the emotion of "joy" is recognized.

[0625] 4. Recommended stores and facilities

[0626] The server searches for relevant facilities based on the analyzed keywords, needs, and user emotions. For example, if the emotions "Fukuoka," "Mizutaki," and "Joy" are recognized, the server searches for highly rated Mizutaki restaurants in the Fukuoka area and prioritizes a list of the most suitable restaurants based on the user's emotions.

[0627] 5. Mapping and Display

[0628] Based on the information in the generated list of recommended facilities (e.g., store name, address, rating), the server obtains the coordinates of the facilities and plots them on a map. The plotted map data is sent to the device and displayed within the chat application.

[0629] 6. User confirmation and selection

[0630] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[0631] 7. Seamlessly switch between chat and maps

[0632] The device provides a navigation method that seamlessly switches between the chat screen and the map display screen, allowing users to smoothly check and share information.

[0633] Specific examples

[0634] For example, suppose user A and user B are planning a trip to Fukuoka. The following conversation takes place:

[0635] User A: "Let's go to Fukuoka. What do you want to eat?"

[0636] User B: "I want to eat mizutaki."

[0637] Once this conversation has taken place, the system will operate as follows:

[0638] 1. The device acquires the conversation content in real time and sends it to the server.

[0639] 2. The server analyzes the conversation and extracts the keywords "Fukuoka" and "Mizutaki."

[0640] 3. The server uses an emotion engine to recognize the user's emotions. If the conversation seems enjoyable, the emotion of "joy" is recognized.

[0641] 4. The server searches for mizutaki restaurants in Fukuoka based on the keywords and the recognized emotions, and generates a list of recommendations. At this time, it prioritizes the restaurant that best matches the user's emotions (e.g., "Hakata Kamaitachi").

[0642] 5. The server retrieves the coordinates of the listed stores and plots them on the map data.

[0643] 6. The plotted map data is sent to the device and displayed within the chat app.

[0644] 7. When User B taps on a store on the map, detailed information can be viewed on the device.

[0645] This system allows users A and B to efficiently plan their trips, making it extremely easy to confirm and share information. Furthermore, it improves the user experience by making suggestions based on the user's emotions.

[0646] The processing flow will be explained below.

[0647] Step 1:

[0648] The device monitors text data of conversations in a chat application in real time and acquires the content of the conversation when a sending or receiving event occurs.

[0649] Step 2:

[0650] The device temporarily stores the acquired conversation content in memory and formats it for transmission to the server.

[0651] Step 3:

[0652] The device sends the formatted conversation content to the server via an API endpoint.

[0653] Step 4:

[0654] The server receives the conversation data at the API endpoint and parses it into the appropriate format.

[0655] Step 5:

[0656] The server inputs the received conversation data into a natural language processing (NLP) model to analyze and extract keywords and needs.

[0657] Step 6:

[0658] At the same time, the server uses an emotion engine to analyze and recognize the user's emotions (e.g., "happiness," "surprise," "dissatisfaction," etc.) from the conversation data.

[0659] Step 7:

[0660] The server searches for relevant facilities and stores using an internal database or an external API (e.g., Google Places API) based on the extracted keywords, needs, and recognized user sentiment.

[0661] Step 8:

[0662] Based on the search results, the server generates a list of recommended facilities and stores, taking into account ratings, popularity, and recognized user emotions.

[0663] Step 9:

[0664] The server obtains the coordinate information of each facility in the recommendation list and plots it on the map data.

[0665] Step 10:

[0666] The server formats the plotted map data for transmission to the terminal.

[0667] Step 11:

[0668] The terminal displays the received map data within the chat application.

[0669] Step 12:

[0670] The user looks at the displayed map and checks the suggested facilities and stores.

[0671] Step 13:

[0672] When a user selects a specific facility or store, the device displays detailed information about that facility (address, opening hours, ratings, reviews, etc.).

[0673] Step 14:

[0674] The device provides navigation buttons and gestures that allow users to seamlessly switch between the chat screen and the map screen.

[0675] Step 15:

[0676] When the device returns to the chat screen, it continues to display the latest conversation content, and when it returns to the map screen, it redisplays the map information that was previously displayed.

[0677] Example 2

[0678] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0679] In conventional travel and outing planning, users have to search for information on the Internet and select appropriate facilities, which takes time and effort to obtain and select information.Furthermore, there is also the problem that the user experience is not sufficiently improved because it is not possible to suggest facilities based on the user's emotions and needs.

[0680] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0681] In this invention, the server includes means for acquiring conversation content between users in a chat application, means for the server to analyze the acquired conversation content and extract keywords and needs, and means for the server to search for facilities to recommend based on the extracted keywords and needs and adjust the recommendation ranking of the facilities based on the user's emotions. This enables efficient travel and outing planning by automatically searching for facilities that match the user's emotions and needs and displaying them on a map within the chat application.

[0682] "User" refers to an individual who uses the chat application to plan trips and outings.

[0683] "Server" refers to the central processing unit that analyzes the conversation content of the chat application and searches for and adjusts recommended facilities based on keywords, user needs, and emotions.

[0684] "Terminal" refers to a device (e.g., smartphone, tablet, or PC) that is directly operated by the user, has a chat application installed, acquires conversation content, and sends and receives data to and from the server.

[0685] A "chat application" is software used by users to communicate, and refers to an application that allows users to send and receive text messages in real time.

[0686] "Conversation content" refers to the content of text messages exchanged between users within a chat application, including information about travel and outings.

[0687] "Keywords" are important words and phrases extracted from conversations and used to identify a user's needs and destinations.

[0688] "Needs" refer to the desires and requirements that a user has regarding travel or outings.

[0689] "Emotion engine" refers to software or a module for analyzing and recognizing a user's emotions from the content of a conversation.

[0690] "Recommended facilities" refers to facilities such as travel destinations and restaurants that are suggested based on the user's keywords and needs.

[0691] "Recommendation ranking" refers to the priority of a recommended facility within the list, and is adjusted based on the user's feelings, ratings, etc.

[0692] "Map data" refers to map information on which the location information of recommended facilities is plotted.

[0693] The present invention is a system that analyzes conversations when a user plans a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. Furthermore, an embodiment of the system is described below, which combines an emotion engine that recognizes the user's emotions and recommends and displays facilities based on the user's emotions.

[0694] System configuration and operation

[0695] Get chat transcript

[0696] When a user discusses travel plans with a friend in a chat application, the device captures the text data of the conversation in real time, temporarily stores the conversation content, and then transmits it to the server.

[0697] Conversation analysis

[0698] The server receives the conversation and analyzes it using a pre-trained natural language processing (NLP) model (such as BERT). Through analysis, important keywords (e.g., "city name," "type of cuisine") and user needs are extracted from the conversation.

[0699] Emotion recognition

[0700] At the same time, the server uses an emotion engine (such as Affectiva) to analyze and recognize the user's emotions from the conversation content. For example, if the user is having a fun conversation, the emotion of "joy" is recognized.

[0701] Recommended stores and facilities

[0702] Based on the analyzed keywords, needs, and user sentiment, the server uses the Google Places API to search for related facilities. For example, if the "city name," "type of cuisine," and "pleasure" sentiment are recognized, the server searches for related facilities with high ratings and prioritizes the most suitable stores from the search results.

[0703] Mapping and Display

[0704] Based on the information in the generated list of recommended facilities (e.g., store name, address, rating), the server uses the Google Maps API to obtain the coordinates of the facilities and plots them on a map. The plotted map data is sent to the device and displayed within the chat application.

[0705] User confirmation and selection

[0706] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[0707] Seamlessly switch between chat and maps

[0708] The device provides a navigation method that seamlessly switches between the chat screen and the map display screen, allowing users to smoothly check and share information.

[0709] Specific examples

[0710] For example, suppose user A and user B are planning a trip to a certain city. The following conversation takes place:

[0711] User A: "Let's go to City Name. What do you want to eat?"

[0712] User B: "I want to try a variety of dishes."

[0713] Once this conversation has taken place, the system will operate as follows:

[0714] 1. The device acquires the conversation content in real time and sends it to the server.

[0715] 2. The server analyzes the conversation and extracts keywords such as "city name" and "type of cuisine."

[0716] 3. The server uses an emotion engine to recognize the user's emotions. If the conversation seems enjoyable, the emotion of "joy" is recognized.

[0717] 4. The server searches for establishments in the city that serve the type of cuisine based on the keywords and the recognized emotion, and generates a list of recommendations, prioritizing establishments that best fit the user's emotion.

[0718] 5. The server retrieves the coordinates of the listed stores and plots them on the map data.

[0719] 6. The plotted map data is sent to the device and displayed within the chat app.

[0720] 7. When User B taps on a store on the map, detailed information can be viewed on the device.

[0721] This system allows users A and B to efficiently plan their trips, making it extremely easy to confirm and share information. Furthermore, it improves the user experience by making suggestions based on the user's emotions.

[0722] Example prompts to input to the generative AI model

[0723] By inputting prompts like the following into the generative AI model, we can get an explanation of the system's behavior:

[0724] "Describe a system in a chat application for planning trips that analyzes conversations, suggests recommended facilities, and displays them on a map. Include recommending and displaying facilities based on the user's emotions."

[0725] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0726] Step 1: Get chat transcript

[0727] When a user starts a conversation on a chat application, the device collects the text data of the conversation in real time. Specifically, the user sends a message such as "Let's go to City Name. What do you want to eat?" The device receives this message and temporarily stores it. The input is the text message that the user sends on the application, and the output is the stored text data.

[0728] Step 2: Send the conversation

[0729] The terminal transmits the temporarily saved text data to the server. Specifically, the terminal transmits the temporarily saved text data to the server via the network. The input is the saved text data, and the output is the transmitted text data.

[0730] Step 3: Analyzing the conversation

[0731] The text data received by the server is analyzed using a pre-trained natural language processing (NLP) model (e.g., BERT). Specifically, the received text data is input into the NLP model, which extracts important keywords such as "city name" and "type of cuisine" as well as the user's needs. The input is the received text data, and the output is the extracted keywords and needs.

[0732] Step 4: Recognize emotions

[0733] The server uses an emotion engine (e.g., Affectiva) to analyze the user's emotions from the conversation content. Specifically, the extracted keywords and text data are input into the emotion engine to recognize the user's emotions (e.g., joy). The input is the extracted keywords and text data, and the output is the recognized emotion.

[0734] Step 5: Proposing recommended stores and facilities

[0735] The server uses the Google Places API to search for related facilities based on the extracted keywords and recognized emotions. Specifically, it searches for highly rated facilities based on the criteria of "city name," "type of cuisine," and "pleasure," and generates a list of recommendations. The input is the extracted keywords and recognized emotions, and the output is the generated list of recommended facilities.

[0736] Step 6: Obtain facility coordinates

[0737] The server uses the Google Maps API to obtain coordinate information for facilities from the list of recommended facilities. Specifically, it searches for coordinate information from the facility name or address and obtains coordinate information to plot on map data. The input is the list of recommended facilities, and the output is the obtained coordinate information.

[0738] Step 7: Generate map data

[0739] The server plots the acquired coordinate information onto map data. Specifically, it plots the facility locations on a map to generate visually easy-to-understand map data. The input is the acquired coordinate information, and the output is the plotted map data.

[0740] Step 8: Displaying map data

[0741] The terminal displays the plotted map data received from the server within the chat application. Specifically, the map data is reflected in the user interface, allowing the user to view facilities on the map. The input is the plotted map data, and the output is the displayed map interface.

[0742] Step 9: View facility details

[0743] When a user taps on a facility of interest on the map, the device displays detailed information about that facility (address, business hours, reviews, etc.). Specifically, the device obtains detailed information according to the user's operation and displays it on the interface. The input is the user's tap operation, and the output is the displayed detailed facility information.

[0744] Step 10: Seamlessly move between chat and maps

[0745] The device provides a navigation method that allows seamless switching between the chat screen and the map display screen. Specifically, the user can smoothly move back and forth between the chat screen and the map display screen using the tabs at the top of the screen. The input is the user's navigation operation, and the output is the switched screen.

[0746] (Application example 2)

[0747] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0748] Conventional chat applications lack the functionality to analyze conversation content and suggest suitable facilities when users plan trips or outings, making planning cumbersome and inefficient. Furthermore, suggestions do not reflect the user's emotions, making it difficult to provide a better experience. Furthermore, the inability to effectively present users with location information and detailed information about recommended facilities also delays planning.

[0749] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0750] In this invention, the server includes means for acquiring conversation content in a chat application, means for analyzing the acquired conversation content to extract keywords and needs, means for analyzing and recognizing user emotions from the conversation content using an emotion engine, means for searching for recommended facilities based on the extracted keywords, needs, and emotions, means for plotting the searched recommended facilities on a map, means for displaying the plotted map data within the chat application, means for displaying detailed information about the recommended facilities, and navigation means for seamlessly switching between the chat screen and the map display screen. This makes it possible to suggest optimal facilities based on the user's conversation content and emotions, making planning more efficient and convenient.

[0751] A "chat application" is an application that allows users to send and receive text messages in real time.

[0752] The "means for acquiring conversation content" is a function for acquiring text data of conversations taking place within a chat application in real time.

[0753] "Means for extracting keywords and needs" is a function for recognizing and extracting important phrases and user requests from the acquired conversation content.

[0754] An "emotion engine" is software or a component that analyzes a user's emotional state from the content of a conversation and recognizes specific emotions.

[0755] The "means for searching for recommended facilities" is a function for searching for relevant facilities from a database or the Internet based on the extracted keywords, needs, and emotions.

[0756] "Means for plotting on a map" is a function for obtaining the location information of the searched facility and marking it on a digital map.

[0757] The "means for displaying" is a function for visually displaying the plotted map data within the chat application.

[0758] "Means for displaying detailed information about recommended facilities" is a function for providing users with specific information about recommended facilities (e.g., address, business hours, reviews).

[0759] "Seamless switching navigation means" is a function that provides operability for switching between the chat screen and the map display screen without delay.

[0760] System configuration and operation overview

[0761] The present invention is a travel planning support system for smartphones. This system is composed of the following main components:

[0762] 1. Acquiring chat contents

[0763] When a user is chatting with another user about travel or outing plans within a chat application, the device captures the text data of the conversation in real time. This data is temporarily stored on the device and then sent to a server.

[0764] 2. Conversation Analysis

[0765] The server analyzes the captured conversation using natural language processing (NLP) technology, specifically using the Bidirectional Encoder Representations from Transformers (BERT) model to extract important keywords and user needs from the conversation.

[0766] 3. Emotional Recognition

[0767] The server uses an emotion engine to analyze and recognize the user's emotions from the conversation content. It uses the VADER (Valence Aware Dictionary and sEntiment Reasoner) algorithm to determine the positivity, negativity, and intensity of each emotion.

[0768] 4. Recommended facilities

[0769] The server searches a database or the Internet for relevant facilities based on the extracted keywords, needs, and recognized emotions, and prioritizes and lists the facilities that best fit the user's emotions from the search results.

[0770] 5. Mapping and Display

[0771] The location information of the recommended establishments is retrieved and plotted on a digital map. The map data is generated using the folium library, sent to the device, and displayed visually within the chat application.

[0772] 6. User confirmation and selection

[0773] Users can view suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its details (address, opening hours, reviews) on their device.

[0774] 7. Seamlessly switch between chat and maps

[0775] The device provides a navigation method that allows users to seamlessly switch between the chat screen and the map display screen, allowing them to smoothly check and share information.

[0776] Specific examples

[0777] For example, if a user has the following conversation:

[0778] User A: "Let's go to Kyoto. What do you want to eat?"

[0779] User B: "I want to eat ramen."

[0780] In this case, the system behaves as follows:

[0781] 1. The device acquires the conversation content in real time and sends it to the server.

[0782] 2. The server uses the BERT model to extract the keywords "Kyoto" and "ramen."

[0783] 3. The server uses the VADER algorithm to recognize the emotion "joy" if the conversation is positive.

[0784] 4. The server searches for highly rated ramen restaurants in the Kyoto area based on keywords and sentiment, and lists the most suitable restaurants.

[0785] 5. The server retrieves the coordinates of the listed stores and plots them on a map using the folium library.

[0786] 6. The plotted map data is sent to the device and displayed within the chat app.

[0787] This allows users to efficiently plan their trips, makes it much easier to check and share information, and improves the user experience by providing suggestions based on user sentiment.

[0788] Prompt Sentence Examples

[0789] For example, the following prompt might be used to search for a recommended cafe in Kumamoto:

[0790] text

[0791] Are there any good cafes in Kumamoto?

[0792] Based on this, the system analyzes the keywords "Kumamoto" and "cafe" along with positive emotions, suggests the most suitable cafe, and displays it on a map.

[0793] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0794] Step 1:

[0795] Acquiring conversation content

[0796] The terminal acquires text data of conversations taking place within a chat application in real time. This data is temporarily stored on the terminal. The specific input is a text message entered by the user, and the output is the acquired text data of the conversation.

[0797] Step 2:

[0798] Sending conversation transcripts

[0799] The terminal sends the acquired text data to the server. The input is the temporarily saved text data, and the output is the data to be sent to the server. Specifically, the data is sent to the server via an HTTP request.

[0800] Step 3:

[0801] Conversation analysis

[0802] The server analyzes the received conversation content using natural language processing (NLP) technology. Specifically, it uses the BERT model to extract keywords and needs from the input text data. The input is the text data sent to the server, and the output is the extracted keywords and needs.

[0803] Step 4:

[0804] Emotion recognition

[0805] The server uses an emotion engine (VADER algorithm) to analyze and recognize the user's emotions from the conversation content. The input is the text data sent to the server, and the output is the recognized emotion data. Specific operations include emotion scoring and analysis.

[0806] Step 5:

[0807] Search for recommended facilities

[0808] The server searches for relevant facilities from a database or the Internet based on the extracted keywords, needs, and recognized emotions. The input is the extracted keywords and emotion data, and the output is a list of recommended facilities. Specific operations include executing API requests and database queries.

[0809] Step 6:

[0810] Facility location and mapping

[0811] The server acquires the location information of the recommended facilities and plots it on a digital map. The input is a list of recommended facilities, and the output is the plotted map data. Specific operations include geocoding and generating map data.

[0812] Step 7:

[0813] Sending map data

[0814] The server sends the plotted map data to the device. The input is the plotted map data, and the output is the data to be sent to the device. Specifically, the server sends the map data to the device via an HTTP request.

[0815] Step 8:

[0816] Displaying the map

[0817] The terminal visually displays the received map data within the chat application. The input is the map data sent from the server, and the output is the visually displayed map.

[0818] Step 9:

[0819] Displaying detailed facility information

[0820] The user views the map displayed in the chat app to check the suggested facilities. When the user selects a facility of interest, its details (address, opening hours, reviews) are displayed on the device. The input is the ID of the selected facility, and the output is the facility details.

[0821] Step 10:

[0822] Seamless switching between chat and maps

[0823] The device provides a navigation method that allows seamless switching between the chat screen and the map display screen. The input is the user's operation, and the output is the screen after switching. Specifically, the operation smoothly switches between user interfaces.

[0824] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0825] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0826] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0827] [Third embodiment]

[0828] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0829] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0830] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0831] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0832] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0833] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0834] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0835] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0836] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0837] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0838] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0839] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0840] The present invention is a system that analyzes the content of conversations when a user plans a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map.

[0841] System configuration and operation overview

[0842] 1. Acquiring chat contents

[0843] When a user discusses travel plans with a friend in a chat application, the device captures the text data of the conversation in real time, temporarily stores the conversation content, and then transmits it to the server.

[0844] 2. Conversation Analysis

[0845] The server receives the conversation and analyzes it using a pre-trained natural language processing (NLP) model, which extracts important keywords from the conversation (e.g., "Fukuoka" and "Mizutaki") and the user's needs.

[0846] 3. Recommended stores and facilities

[0847] Based on the analysis results, the server searches for related facilities. For example, based on keywords such as "Fukuoka" and "Mizutaki," it searches for highly rated Mizutaki restaurants in the Fukuoka area. From the search results, it generates a list of recommended facilities.

[0848] 4. Mapping and Display

[0849] Based on the information in the generated list of recommended facilities (e.g., store name, address, rating), the server obtains the coordinates of the facilities and plots them on a map. The plotted map data is sent to the device and displayed within the chat application.

[0850] 5. User confirmation and selection

[0851] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[0852] 6. Seamlessly switch between chat and maps

[0853] The device provides a navigation method that seamlessly switches between the chat screen and the map display screen, allowing users to smoothly check and share information.

[0854] Specific examples

[0855] For example, suppose user A and user B are planning a trip to Fukuoka. The following conversation takes place:

[0856] User A: "Let's go to Fukuoka. What do you want to eat?"

[0857] User B: "I want to eat mizutaki."

[0858] Once this conversation has taken place, the system will operate as follows:

[0859] 1. The device acquires the conversation content in real time and sends it to the server.

[0860] 2. The server analyzes the conversation and extracts the keywords "Fukuoka" and "Mizutaki."

[0861] 3. The server searches for mizutaki restaurants in Fukuoka based on these keywords and generates a list of recommendations.

[0862] 4. The server retrieves the coordinates of the listed stores and plots them on the map data.

[0863] 5. The plotted map data is sent to the device and displayed within the chat app.

[0864] 6. When User B taps on a store on the map, they can view detailed information on their device.

[0865] This system allows users A and B to efficiently plan their trips, making it extremely easy to check and share information.

[0866] The processing flow will be explained below.

[0867] Step 1:

[0868] The device monitors text data of conversations in a chat application in real time and acquires the content of the conversation when a sending or receiving event occurs.

[0869] Step 2:

[0870] The device temporarily stores the acquired conversation content in memory and formats it for transmission to the server.

[0871] Step 3:

[0872] The device sends the formatted conversation content to the server via an API endpoint.

[0873] Step 4:

[0874] The server receives the conversation data at the API endpoint and parses it into the appropriate format.

[0875] Step 5:

[0876] The server inputs the received conversation data into a natural language processing (NLP) model to analyze and extract keywords and needs.

[0877] Step 6:

[0878] The server searches for relevant facilities and stores using an internal database or an external API (e.g., Google Places API) based on the extracted keywords and needs.

[0879] Step 7:

[0880] Based on the search results, the server generates a list of recommended facilities and stores, taking into account ratings and popularity.

[0881] Step 8:

[0882] The server obtains the coordinate information of each facility in the recommendation list and plots it on the map data.

[0883] Step 9:

[0884] The server formats the plotted map data for transmission to the terminal.

[0885] Step 10:

[0886] The terminal displays the received map data within the chat application.

[0887] Step 11:

[0888] The user looks at the displayed map and checks the suggested facilities and stores.

[0889] Step 12:

[0890] When a user selects a specific facility or store, the device displays detailed information about that facility (address, opening hours, ratings, reviews, etc.).

[0891] Step 13:

[0892] The device provides navigation buttons and gestures that allow users to seamlessly switch between the chat screen and the map screen.

[0893] Step 14:

[0894] When the device returns to the chat screen, it continues to display the latest conversation content, and when it returns to the map screen, it redisplays the map information that was previously displayed.

[0895] Example 1

[0896] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0897] In conventional chat applications, it was difficult for users to efficiently obtain information on recommended facilities when planning trips or outings. Furthermore, few systems had the functionality to search for related facilities based on conversation content and display them on a map, resulting in low user convenience. Furthermore, users were unable to seamlessly switch between the chat screen and the map display screen, resulting in a poor user experience.

[0898] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0899] In this invention, the server includes means for acquiring conversation content in a chat application, means for analyzing the acquired conversation content to extract keywords and needs, means for searching for recommended facilities based on the extracted keywords and needs, means for acquiring coordinate information of the searched recommended facilities and plotting it on a map, and means for displaying the plotted map data within the chat application, thereby enabling users to efficiently search for and display recommended facilities based on the conversation content within the chat application and make travel plans.

[0900] A "chat application" is software that allows users to exchange text messages and communicate.

[0901] "Conversation content" refers to the content of text messages exchanged between users within a chat application.

[0902] The "means for acquiring" refers to a function or process for collecting and storing conversation content within a chat application.

[0903] "Means for analysis" refers to techniques or methods for analyzing acquired text data and extracting useful information.

[0904] "Keywords" refer to particularly important words or phrases in the conversation content, and are used as search and recommendation criteria.

[0905] "Needs" refer to the requirements, desires, and needs of users, and are elements that are revealed through analysis.

[0906] "Recommended facilities" refers to places, stores, and services suggested based on the user's conversation content.

[0907] "Search means" refers to methods and techniques for searching the Internet for relevant facilities and information based on specified keywords or needs.

[0908] "Plotting means" refers to the technology or method for displaying the location of the searched facility on a map.

[0909] "Map data" refers to digital map data that includes information indicating locations and areas, and includes location information and display information.

[0910] "Navigation means" refers to functions and UI components that allow users to smoothly move between the chat screen and the map display screen.

[0911] "Detailed Information" refers to additional information about the recommended establishment, including address, opening hours, reviews, etc.

[0912] The present invention provides a system that analyzes conversations when a user is planning a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. Specific embodiments of this system are described below.

[0913] 1. Acquiring chat contents

[0914] The device monitors the text entered by the user in the chat application in real time. The device can be a smartphone or tablet. When the user sends a message, the text is stored in the device's temporary storage and then sent to a server via the Internet. The data is encrypted using the HTTPS protocol.

[0915] 2. Conversation Analysis

[0916] The server receives the text data sent from the device and automatically analyzes it using natural language processing (NLP) technology. Services such as Google Cloud Natural Language API and Amazon Comprehend are used. The server uses an NLP model to analyze the text and extract keywords (e.g., "Fukuoka" and "Mizutaki") and the user's needs.

[0917] 3. Recommended stores and facilities

[0918] Based on the analysis results, the server searches for relevant facilities. Based on the generated search query (e.g., "Fukuoka Mizutaki restaurants with high ratings"), it sends a request to an external API (e.g., Google Places, Yelp API) to obtain information on recommended facilities.

[0919] 4. Mapping and Display

[0920] The server obtains the coordinate information (latitude and longitude) of each store based on the acquired store information. The Google Maps Geocoding API is used for this purpose. The coordinate information is then plotted on map data. The plotted map data is sent to the device and displayed within the chat app. The map display uses the Google Maps or Mapbox API.

[0921] 5. User confirmation and selection

[0922] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[0923] 6. Seamlessly switch between chat and maps

[0924] The device provides a seamless navigation method for switching between chat and map views, using Android and iOS user interface components, allowing users to seamlessly view and share information.

[0925] Specific examples

[0926] For example, suppose user A and user B are planning a trip to Fukuoka. The following conversation takes place:

[0927] User A: "Let's go to Fukuoka. What do you want to eat?"

[0928] User B: "I want to eat mizutaki."

[0929] Once this conversation has taken place, the system will operate as follows:

[0930] 1. The device acquires the conversation content in real time and sends it to the server.

[0931] 2. The server analyzes the conversation and extracts the keywords "Fukuoka" and "Mizutaki."

[0932] 3. The server searches for mizutaki restaurants in Fukuoka based on these keywords and generates a list of recommendations.

[0933] 4. The server retrieves the coordinates of the listed stores and plots them on the map data.

[0934] 5. The plotted map data is sent to the device and displayed within the chat app.

[0935] 6. When User B taps on a store on the map, they can view detailed information on their device.

[0936] This system allows users A and B to efficiently plan their trips, making it extremely easy to check and share information.

[0937] Prompt Sentence Examples

[0938] "Please simulate a situation where User A and User B are planning a trip to Fukuoka and are looking for a Mizutaki restaurant. Analyze the conversation, search for recommended Mizutaki restaurants in Fukuoka, and display them on a map."

[0939] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0940] Step 1: Get chat transcript

[0941] The device monitors and acquires messages entered by the user in the chat application in real time. When the user sends a message, the text is stored in the device's temporary storage. The input is the text message entered by the user, and the output is the text data stored on the device. The stored text data is encrypted using the HTTPS protocol and sent to the server.

[0942] Specific working example:

[0943] When a user types "Let's go to Fukuoka. What do you want to eat?" into the chat and presses the send button, the text is saved on the device and then sent to the server.

[0944] Step 2: Analyzing the conversation

[0945] The server receives the text data sent from the device and analyzes it using the Google Cloud Natural Language API or Amazon Comprehend. The input is the text data received by the server, and the output is the keywords extracted by the analysis and the user's needs. The server then uses this information to identify important keywords (e.g., "Fukuoka" or "Mizutaki") and the user's needs.

[0946] Specific working example:

[0947] The server analyzes the text data "Let's go to Fukuoka. What do you want to eat?" and extracts the keywords "Fukuoka" and "want to eat."

[0948] Step 3: Proposing recommended stores and facilities

[0949] The server searches for related facilities using external APIs (Google Places and Yelp APIs) based on the extracted keywords. The input is the keywords and needs, and the output is a list of recommended facilities. Specifically, a search is performed using the keywords "Fukuoka" and "Mizutaki" to identify highly rated Mizutaki restaurants.

[0950] Specific working example:

[0951] The server calls the Google Places API using the keywords "Fukuoka" and "Mizutaki" and lists recommended Mizutaki restaurants in Fukuoka.

[0952] Step 4: Mapping and display

[0953] The server obtains the coordinate information (latitude and longitude) of the acquired store information and plots it on map data. The input is a list of recommended facilities, and the output is the plotted map data. The coordinate information is obtained using the Google Maps Geocoding API, and plotted on the map using the Google Maps or Mapbox API.

[0954] Specific working example:

[0955] The server calls the Geocoding API using the address information of the recommended mizutaki restaurant, obtains the latitude and longitude, and uses that information to place a marker on the map data.

[0956] Step 5: Confirm and select users

[0957] Users check the plotted map in the chat app on their device and select the store they are interested in. The input is the plotted map data, and the output is detailed information about the store selected by the user. When the user taps a marker on the map, detailed information (address, business hours, reviews) is displayed.

[0958] Specific working example:

[0959] When users tap on a specific mizutaki restaurant displayed on the map, detailed information about the restaurant will be displayed, and they can check its opening hours and reviews.

[0960] Step 6: Seamlessly move between chat and maps

[0961] The device provides a function to seamlessly switch between the chat screen and the map display screen. The input is a user operation, and the output is switching between the chat screen and the map screen. It is implemented using Android and iOS user interface components, and tab bars and swipe gestures are applied.

[0962] Specific working example:

[0963] When a user swipes from the chat screen to the map screen, the screen smoothly switches to the map display, and when they swipe again, they return to the chat screen.

[0964] (Application example 1)

[0965] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0966] In today's information-driven society, when planning trips or outings, users often find it difficult to find the best locations and facilities among the vast amount of information available. Furthermore, there are few systems that effectively extract information about destinations and facilities from conversations within chat applications and suggest them in real time, which hinders the efficiency of planning. Furthermore, the suggested information is not presented in a format that is intuitively understandable, rather than just in text, resulting in a lack of user convenience.

[0967] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0968] In this invention, the server includes means for acquiring conversation content in a chat application, means for analyzing the acquired conversation content to extract keywords and needs, means for searching for recommended facilities based on the extracted keywords and needs, means for acquiring location information of the plotted recommended facilities in a geographic information system, and means for displaying the plotted map data within the chat application, thereby enabling users to efficiently plan trips and outings and to check suggestions and detailed information about recommended facilities in real time.

[0969] A "chat application" is software that allows users to communicate in real time using text messages.

[0970] "Conversation" refers to the text messages sent and received by users within a chat application.

[0971] "Keywords" are important words and phrases extracted from the conversation content.

[0972] "Needs" are the requests and desires that users express through their conversations.

[0973] "Recommended facilities" are places and stores selected based on the user's keywords and needs.

[0974] The "search method" refers to the functions and algorithms used to identify recommended facilities.

[0975] The "means for plotting on a map" is a function for displaying the location information of recommended facilities on map data.

[0976] A "geographic information system" is a system for processing and visualizing location information.

[0977] "Location information of recommended facilities" is information including the latitude and longitude of the location of the recommended facilities.

[0978] "Detailed information" refers to additional information about the recommended facility, such as its name, address, rating, latitude, and longitude.

[0979] "Navigation means" is a function that allows the user to smoothly switch between the chat screen and the map display screen.

[0980] This invention is a system that analyzes conversations when a user is planning a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. This system is realized using a chat application, a natural language processing (NLP) model, a geographic information system (GIS), and a navigation means.

[0981] System configuration and operation overview

[0982] 1. Acquiring chat contents

[0983] When users discuss travel or outing plans within the chat application, the device acquires text data of the conversation in real time, temporarily stores it, and then transmits it to the server.

[0984] 2. Conversation Analysis

[0985] The server analyzes the captured conversation using a natural language processing (NLP) model (e.g., a pre-trained model from the Transformers library) to extract important keywords (e.g., "Shibuya" and "sushi") and user needs. This analysis detects specific requests, such as travel destinations and meals, from the conversation.

[0986] 3. Search for recommended facilities

[0987] The server searches for recommended establishments based on the extracted keywords and user needs. For example, based on the keywords "Shibuya" and "sushi," it searches for highly rated sushi restaurants in the Shibuya area using a geographic information system (e.g., Google Maps API).

[0988] 4. Mapping and Display

[0989] The server obtains the latitude and longitude coordinates of the facilities based on the information in the list of recommended facilities generated from the search results (e.g., store name, address, rating), and plots them on map data. The plotted map data is sent to the device and displayed within the chat application.

[0990] 5. User confirmation and selection

[0991] Users can view the suggested facilities on a map displayed within the chat application, and if they find one that interests them, they can view its details (e.g., address, opening hours, reviews) on their device.

[0992] 6. Seamlessly switch between chat and maps

[0993] The device provides a navigation means for users to seamlessly switch between the chat screen and the map display screen, updating the display in real time according to the content of the conversation.

[0994] Specific examples

[0995] For example, when a user says, "I'm looking for good sushi in Shibuya," the system works as follows:

[0996] The device acquires conversations containing the keywords "Shibuya" and "sushi" and sends them to the server.

[0997] The server analyzes the conversation and extracts the keywords "Shibuya" and "sushi."

[0998] Based on these keywords, the server uses a geographic information system to search for highly rated sushi restaurants.

[0999] The server obtains the coordinate information of the searched sushi restaurant and plots it on the map data.

[1000] The device displays the plotted map data within the chat application, allowing the user to view detailed information.

[1001] Prompt Sentence Examples

[1002] When a user says, "I'm looking for good sushi in Shibuya," the system should extract the keywords "Shibuya" and "sushi," search for related sushi restaurants, and display them on a map.

[1003] Using this system, users can intuitively and efficiently plan outings based on chat conversations. In addition, the information suggested by the system is visualized on a map, allowing users to easily select destinations and stores.

[1004] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1005] Step 1: Get chat transcript

[1006] The device captures text data of conversations in real time when users discuss travel or outing plans within a chat application. The captured text data is temporarily stored within the device and then sent to a server. The input is the user's chat messages, and the output is the text data of these messages.

[1007] Step 2: Analyzing the conversation

[1008] The server receives the text data of the conversation sent from the device and analyzes it using a natural language processing (NLP) model. This process extracts important keywords and user needs from the conversation. The input is text data, and the output is keywords and needs. Specific data processing involves the use of the NLP model for Named Entity Recognition (NER) and keyword extraction.

[1009] Step 3: Find recommended facilities

[1010] The server uses a geographic information system (GIS) to search for relevant facilities based on the extracted keywords and needs. For example, it searches for highly rated sushi restaurants based on the keywords "Shibuya" and "sushi." The input is the keywords and needs, and the output is a list of recommended facilities. As a specific example, it uses the Google Maps API to search for locations and obtain their rating information.

[1011] Step 4: Geographical information acquisition and mapping

[1012] The server obtains location information (latitude and longitude) from a geographic information system based on the address information of the searched recommended facility and plots it on map data. The input is the address information of the recommended facility, and the output is the plotted map data. The location information is processed using libraries such as Geopy.

[1013] Step 5: View detailed information about recommended facilities

[1014] The terminal receives the plotted map data sent from the server and displays it within the chat application. It also allows users to check detailed information (name, address, rating, latitude, longitude) of recommended facilities on the map. The input is the plotted map data, and the output is the facility information and detailed information displayed on the map.

[1015] Step 6: Seamlessly move between chat and maps

[1016] The device provides a navigation mechanism that allows users to seamlessly switch between chat and map views. It also updates the display based on real-time conversation content. Input is user interface manipulation, and output is an updated screen display. Specific operations include smooth switching and real-time updates of the user interface.

[1017] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1018] The present invention is a system that analyzes conversations when a user plans a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. Furthermore, an embodiment of the system will be described, which combines an emotion engine that recognizes the user's emotions and recommends and displays facilities based on the user's emotions.

[1019] System configuration and operation overview

[1020] 1. Acquiring chat contents

[1021] When a user discusses travel plans with a friend in a chat application, the device captures the text data of the conversation in real time, temporarily stores the conversation content, and then transmits it to the server.

[1022] 2. Conversation Analysis

[1023] The server receives the conversation and analyzes it using a pre-trained natural language processing (NLP) model, which extracts important keywords from the conversation (e.g., "Fukuoka" and "Mizutaki") and the user's needs.

[1024] 3. Emotional Recognition

[1025] At the same time, the server uses an emotion engine to analyze and recognize the user's emotions from the conversation content. For example, if the user seems to be having a fun conversation, the emotion of "joy" is recognized.

[1026] 4. Recommended stores and facilities

[1027] The server searches for relevant facilities based on the analyzed keywords, needs, and user emotions. For example, if the emotions "Fukuoka," "Mizutaki," and "Joy" are recognized, the server searches for highly rated Mizutaki restaurants in the Fukuoka area and prioritizes a list of the most suitable restaurants based on the user's emotions.

[1028] 5. Mapping and Display

[1029] Based on the information in the generated list of recommended facilities (e.g., store name, address, rating), the server obtains the coordinates of the facilities and plots them on a map. The plotted map data is sent to the device and displayed within the chat application.

[1030] 6. User confirmation and selection

[1031] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[1032] 7. Seamlessly switch between chat and maps

[1033] The device provides a navigation method that seamlessly switches between the chat screen and the map display screen, allowing users to smoothly check and share information.

[1034] Specific examples

[1035] For example, suppose user A and user B are planning a trip to Fukuoka. The following conversation takes place:

[1036] User A: "Let's go to Fukuoka. What do you want to eat?"

[1037] User B: "I want to eat mizutaki."

[1038] Once this conversation has taken place, the system will operate as follows:

[1039] 1. The device acquires the conversation content in real time and sends it to the server.

[1040] 2. The server analyzes the conversation and extracts the keywords "Fukuoka" and "Mizutaki."

[1041] 3. The server uses an emotion engine to recognize the user's emotions. If the conversation seems enjoyable, the emotion of "joy" is recognized.

[1042] 4. The server searches for mizutaki restaurants in Fukuoka based on the keywords and the recognized emotions, and generates a list of recommendations. At this time, it prioritizes the restaurant that best matches the user's emotions (e.g., "Hakata Kamaitachi").

[1043] 5. The server retrieves the coordinates of the listed stores and plots them on the map data.

[1044] 6. The plotted map data is sent to the device and displayed within the chat app.

[1045] 7. When User B taps on a store on the map, detailed information can be viewed on the device.

[1046] This system allows users A and B to efficiently plan their trips, making it extremely easy to confirm and share information. Furthermore, it improves the user experience by making suggestions based on the user's emotions.

[1047] The processing flow will be explained below.

[1048] Step 1:

[1049] The device monitors text data of conversations in a chat application in real time and acquires the content of the conversation when a sending or receiving event occurs.

[1050] Step 2:

[1051] The device temporarily stores the acquired conversation content in memory and formats it for transmission to the server.

[1052] Step 3:

[1053] The device sends the formatted conversation content to the server via an API endpoint.

[1054] Step 4:

[1055] The server receives the conversation data at the API endpoint and parses it into the appropriate format.

[1056] Step 5:

[1057] The server inputs the received conversation data into a natural language processing (NLP) model to analyze and extract keywords and needs.

[1058] Step 6:

[1059] At the same time, the server uses an emotion engine to analyze and recognize the user's emotions (e.g., "happiness," "surprise," "dissatisfaction," etc.) from the conversation data.

[1060] Step 7:

[1061] The server searches for relevant facilities and stores using an internal database or an external API (e.g., Google Places API) based on the extracted keywords, needs, and recognized user sentiment.

[1062] Step 8:

[1063] Based on the search results, the server generates a list of recommended facilities and stores, taking into account ratings, popularity, and recognized user emotions.

[1064] Step 9:

[1065] The server obtains the coordinate information of each facility in the recommendation list and plots it on the map data.

[1066] Step 10:

[1067] The server formats the plotted map data for transmission to the terminal.

[1068] Step 11:

[1069] The terminal displays the received map data within the chat application.

[1070] Step 12:

[1071] The user looks at the displayed map and checks the suggested facilities and stores.

[1072] Step 13:

[1073] When a user selects a specific facility or store, the device displays detailed information about that facility (address, opening hours, ratings, reviews, etc.).

[1074] Step 14:

[1075] The device provides navigation buttons and gestures that allow users to seamlessly switch between the chat screen and the map screen.

[1076] Step 15:

[1077] When the device returns to the chat screen, it continues to display the latest conversation content, and when it returns to the map screen, it redisplays the map information that was previously displayed.

[1078] Example 2

[1079] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1080] In conventional travel and outing planning, users have to search for information on the Internet and select appropriate facilities, which takes time and effort to obtain and select information.Furthermore, there is also the problem that the user experience is not sufficiently improved because it is not possible to suggest facilities based on the user's emotions and needs.

[1081] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1082] In this invention, the server includes means for acquiring conversation content between users in a chat application, means for the server to analyze the acquired conversation content and extract keywords and needs, and means for the server to search for facilities to recommend based on the extracted keywords and needs and adjust the recommendation ranking of the facilities based on the user's emotions. This enables efficient travel and outing planning by automatically searching for facilities that match the user's emotions and needs and displaying them on a map within the chat application.

[1083] "User" refers to an individual who uses the chat application to plan trips and outings.

[1084] "Server" refers to the central processing unit that analyzes the conversation content of the chat application and searches for and adjusts recommended facilities based on keywords, user needs, and emotions.

[1085] "Terminal" refers to a device (e.g., smartphone, tablet, or PC) that is directly operated by the user, has a chat application installed, acquires conversation content, and sends and receives data to and from the server.

[1086] A "chat application" is software used by users to communicate, and refers to an application that allows users to send and receive text messages in real time.

[1087] "Conversation content" refers to the content of text messages exchanged between users within a chat application, including information about travel and outings.

[1088] "Keywords" are important words and phrases extracted from conversations and used to identify a user's needs and destinations.

[1089] "Needs" refer to the desires and requirements that a user has regarding travel or outings.

[1090] "Emotion engine" refers to software or a module for analyzing and recognizing a user's emotions from the content of a conversation.

[1091] "Recommended facilities" refers to facilities such as travel destinations and restaurants that are suggested based on the user's keywords and needs.

[1092] "Recommendation ranking" refers to the priority of a recommended facility within the list, and is adjusted based on the user's feelings, ratings, etc.

[1093] "Map data" refers to map information on which the location information of recommended facilities is plotted.

[1094] The present invention is a system that analyzes conversations when a user plans a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. Furthermore, an embodiment of the system is described below, which combines an emotion engine that recognizes the user's emotions and recommends and displays facilities based on the user's emotions.

[1095] System configuration and operation

[1096] Get chat transcript

[1097] When a user discusses travel plans with a friend in a chat application, the device captures the text data of the conversation in real time, temporarily stores the conversation content, and then transmits it to the server.

[1098] Conversation analysis

[1099] The server receives the conversation and analyzes it using a pre-trained natural language processing (NLP) model (such as BERT). Through analysis, important keywords (e.g., "city name," "type of cuisine") and user needs are extracted from the conversation.

[1100] Emotion recognition

[1101] At the same time, the server uses an emotion engine (such as Affectiva) to analyze and recognize the user's emotions from the conversation content. For example, if the user is having a fun conversation, the emotion of "joy" is recognized.

[1102] Recommended stores and facilities

[1103] Based on the analyzed keywords, needs, and user sentiment, the server uses the Google Places API to search for related facilities. For example, if the "city name," "type of cuisine," and "pleasure" sentiment are recognized, the server searches for related facilities with high ratings and prioritizes the most suitable stores from the search results.

[1104] Mapping and Display

[1105] Based on the information in the generated list of recommended facilities (e.g., store name, address, rating), the server uses the Google Maps API to obtain the coordinates of the facilities and plots them on a map. The plotted map data is sent to the device and displayed within the chat application.

[1106] User confirmation and selection

[1107] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[1108] Seamlessly switch between chat and maps

[1109] The device provides a navigation method that seamlessly switches between the chat screen and the map display screen, allowing users to smoothly check and share information.

[1110] Specific examples

[1111] For example, suppose user A and user B are planning a trip to a certain city. The following conversation takes place:

[1112] User A: "Let's go to City Name. What do you want to eat?"

[1113] User B: "I want to try a variety of dishes."

[1114] Once this conversation has taken place, the system will operate as follows:

[1115] 1. The device acquires the conversation content in real time and sends it to the server.

[1116] 2. The server analyzes the conversation and extracts keywords such as "city name" and "type of cuisine."

[1117] 3. The server uses an emotion engine to recognize the user's emotions. If the conversation seems enjoyable, the emotion of "joy" is recognized.

[1118] 4. The server searches for establishments in the city that serve the type of cuisine based on the keywords and the recognized emotion, and generates a list of recommendations, prioritizing establishments that best fit the user's emotion.

[1119] 5. The server retrieves the coordinates of the listed stores and plots them on the map data.

[1120] 6. The plotted map data is sent to the device and displayed within the chat app.

[1121] 7. When User B taps on a store on the map, detailed information can be viewed on the device.

[1122] This system allows users A and B to efficiently plan their trips, making it extremely easy to confirm and share information. Furthermore, it improves the user experience by making suggestions based on the user's emotions.

[1123] Example prompts to input to the generative AI model

[1124] By inputting prompts like the following into the generative AI model, we can get an explanation of the system's behavior:

[1125] "Describe a system in a chat application for planning trips that analyzes conversations, suggests recommended facilities, and displays them on a map. Include recommending and displaying facilities based on the user's emotions."

[1126] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1127] Step 1: Get chat transcript

[1128] When a user starts a conversation on a chat application, the device collects the text data of the conversation in real time. Specifically, the user sends a message such as "Let's go to City Name. What do you want to eat?" The device receives this message and temporarily stores it. The input is the text message that the user sends on the application, and the output is the stored text data.

[1129] Step 2: Send the conversation

[1130] The terminal transmits the temporarily saved text data to the server. Specifically, the terminal transmits the temporarily saved text data to the server via the network. The input is the saved text data, and the output is the transmitted text data.

[1131] Step 3: Analyzing the conversation

[1132] The text data received by the server is analyzed using a pre-trained natural language processing (NLP) model (e.g., BERT). Specifically, the received text data is input into the NLP model, which extracts important keywords such as "city name" and "type of cuisine" as well as the user's needs. The input is the received text data, and the output is the extracted keywords and needs.

[1133] Step 4: Recognize emotions

[1134] The server uses an emotion engine (e.g., Affectiva) to analyze the user's emotions from the conversation content. Specifically, the extracted keywords and text data are input into the emotion engine to recognize the user's emotions (e.g., joy). The input is the extracted keywords and text data, and the output is the recognized emotion.

[1135] Step 5: Proposing recommended stores and facilities

[1136] The server uses the Google Places API to search for related facilities based on the extracted keywords and recognized emotions. Specifically, it searches for highly rated facilities based on the criteria of "city name," "type of cuisine," and "pleasure," and generates a list of recommendations. The input is the extracted keywords and recognized emotions, and the output is the generated list of recommended facilities.

[1137] Step 6: Obtain facility coordinates

[1138] The server uses the Google Maps API to obtain coordinate information for facilities from the list of recommended facilities. Specifically, it searches for coordinate information from the facility name or address and obtains coordinate information to plot on map data. The input is the list of recommended facilities, and the output is the obtained coordinate information.

[1139] Step 7: Generate map data

[1140] The server plots the acquired coordinate information onto map data. Specifically, it plots the facility locations on a map to generate visually easy-to-understand map data. The input is the acquired coordinate information, and the output is the plotted map data.

[1141] Step 8: Displaying map data

[1142] The terminal displays the plotted map data received from the server within the chat application. Specifically, the map data is reflected in the user interface, allowing the user to view facilities on the map. The input is the plotted map data, and the output is the displayed map interface.

[1143] Step 9: View facility details

[1144] When a user taps on a facility of interest on the map, the device displays detailed information about that facility (address, business hours, reviews, etc.). Specifically, the device obtains detailed information according to the user's operation and displays it on the interface. The input is the user's tap operation, and the output is the displayed detailed facility information.

[1145] Step 10: Seamlessly move between chat and maps

[1146] The device provides a navigation method that allows seamless switching between the chat screen and the map display screen. Specifically, the user can smoothly move back and forth between the chat screen and the map display screen using the tabs at the top of the screen. The input is the user's navigation operation, and the output is the switched screen.

[1147] (Application example 2)

[1148] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1149] Conventional chat applications lack the functionality to analyze conversation content and suggest suitable facilities when users plan trips or outings, making planning cumbersome and inefficient. Furthermore, suggestions do not reflect the user's emotions, making it difficult to provide a better experience. Furthermore, the inability to effectively present users with location information and detailed information about recommended facilities also delays planning.

[1150] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1151] In this invention, the server includes means for acquiring conversation content in a chat application, means for analyzing the acquired conversation content to extract keywords and needs, means for analyzing and recognizing user emotions from the conversation content using an emotion engine, means for searching for recommended facilities based on the extracted keywords, needs, and emotions, means for plotting the searched recommended facilities on a map, means for displaying the plotted map data within the chat application, means for displaying detailed information about the recommended facilities, and navigation means for seamlessly switching between the chat screen and the map display screen. This makes it possible to suggest optimal facilities based on the user's conversation content and emotions, making planning more efficient and convenient.

[1152] A "chat application" is an application that allows users to send and receive text messages in real time.

[1153] The "means for acquiring conversation content" is a function for acquiring text data of conversations taking place within a chat application in real time.

[1154] "Means for extracting keywords and needs" is a function for recognizing and extracting important phrases and user requests from the acquired conversation content.

[1155] An "emotion engine" is software or a component that analyzes a user's emotional state from the content of a conversation and recognizes specific emotions.

[1156] The "means for searching for recommended facilities" is a function for searching for relevant facilities from a database or the Internet based on the extracted keywords, needs, and emotions.

[1157] "Means for plotting on a map" is a function for obtaining the location information of the searched facility and marking it on a digital map.

[1158] The "means for displaying" is a function for visually displaying the plotted map data within the chat application.

[1159] "Means for displaying detailed information about recommended facilities" is a function for providing users with specific information about recommended facilities (e.g., address, business hours, reviews).

[1160] "Seamless switching navigation means" is a function that provides operability for switching between the chat screen and the map display screen without delay.

[1161] System configuration and operation overview

[1162] The present invention is a travel planning support system for smartphones. This system is composed of the following main components:

[1163] 1. Acquiring chat contents

[1164] When a user is chatting with another user about travel or outing plans within a chat application, the device captures the text data of the conversation in real time. This data is temporarily stored on the device and then sent to a server.

[1165] 2. Conversation Analysis

[1166] The server analyzes the captured conversation using natural language processing (NLP) technology, specifically using the Bidirectional Encoder Representations from Transformers (BERT) model to extract important keywords and user needs from the conversation.

[1167] 3. Emotional Recognition

[1168] The server uses an emotion engine to analyze and recognize the user's emotions from the conversation content. It uses the VADER (Valence Aware Dictionary and sEntiment Reasoner) algorithm to determine the positivity, negativity, and intensity of each emotion.

[1169] 4. Recommended facilities

[1170] The server searches a database or the Internet for relevant facilities based on the extracted keywords, needs, and recognized emotions, and prioritizes and lists the facilities that best fit the user's emotions from the search results.

[1171] 5. Mapping and Display

[1172] The location information of the recommended establishments is retrieved and plotted on a digital map. The map data is generated using the folium library, sent to the device, and displayed visually within the chat application.

[1173] 6. User confirmation and selection

[1174] Users can view suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its details (address, opening hours, reviews) on their device.

[1175] 7. Seamlessly switch between chat and maps

[1176] The device provides a navigation method that allows users to seamlessly switch between the chat screen and the map display screen, allowing them to smoothly check and share information.

[1177] Specific examples

[1178] For example, if a user has the following conversation:

[1179] User A: "Let's go to Kyoto. What do you want to eat?"

[1180] User B: "I want to eat ramen."

[1181] In this case, the system behaves as follows:

[1182] 1. The device acquires the conversation content in real time and sends it to the server.

[1183] 2. The server uses the BERT model to extract the keywords "Kyoto" and "ramen."

[1184] 3. The server uses the VADER algorithm to recognize the emotion "joy" if the conversation is positive.

[1185] 4. The server searches for highly rated ramen restaurants in the Kyoto area based on keywords and sentiment, and lists the most suitable restaurants.

[1186] 5. The server retrieves the coordinates of the listed stores and plots them on a map using the folium library.

[1187] 6. The plotted map data is sent to the device and displayed within the chat app.

[1188] This allows users to efficiently plan their trips, makes it much easier to check and share information, and improves the user experience by providing suggestions based on user sentiment.

[1189] Prompt Sentence Examples

[1190] For example, the following prompt might be used to search for a recommended cafe in Kumamoto:

[1191] text

[1192] Are there any good cafes in Kumamoto?

[1193] Based on this, the system analyzes the keywords "Kumamoto" and "cafe" along with positive emotions, suggests the most suitable cafe, and displays it on a map.

[1194] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1195] Step 1:

[1196] Acquiring conversation content

[1197] The terminal acquires text data of conversations taking place within a chat application in real time. This data is temporarily stored on the terminal. The specific input is a text message entered by the user, and the output is the acquired text data of the conversation.

[1198] Step 2:

[1199] Sending conversation transcripts

[1200] The terminal sends the acquired text data to the server. The input is the temporarily saved text data, and the output is the data to be sent to the server. Specifically, the data is sent to the server via an HTTP request.

[1201] Step 3:

[1202] Conversation analysis

[1203] The server analyzes the received conversation content using natural language processing (NLP) technology. Specifically, it uses the BERT model to extract keywords and needs from the input text data. The input is the text data sent to the server, and the output is the extracted keywords and needs.

[1204] Step 4:

[1205] Emotion recognition

[1206] The server uses an emotion engine (VADER algorithm) to analyze and recognize the user's emotions from the conversation content. The input is the text data sent to the server, and the output is the recognized emotion data. Specific operations include emotion scoring and analysis.

[1207] Step 5:

[1208] Search for recommended facilities

[1209] The server searches for relevant facilities from a database or the Internet based on the extracted keywords, needs, and recognized emotions. The input is the extracted keywords and emotion data, and the output is a list of recommended facilities. Specific operations include executing API requests and database queries.

[1210] Step 6:

[1211] Facility location and mapping

[1212] The server acquires the location information of the recommended facilities and plots it on a digital map. The input is a list of recommended facilities, and the output is the plotted map data. Specific operations include geocoding and generating map data.

[1213] Step 7:

[1214] Sending map data

[1215] The server sends the plotted map data to the device. The input is the plotted map data, and the output is the data to be sent to the device. Specifically, the server sends the map data to the device via an HTTP request.

[1216] Step 8:

[1217] Displaying the map

[1218] The terminal visually displays the received map data within the chat application. The input is the map data sent from the server, and the output is the visually displayed map.

[1219] Step 9:

[1220] Displaying detailed facility information

[1221] The user views the map displayed in the chat app to check the suggested facilities. When the user selects a facility of interest, its details (address, opening hours, reviews) are displayed on the device. The input is the ID of the selected facility, and the output is the facility details.

[1222] Step 10:

[1223] Seamless switching between chat and maps

[1224] The device provides a navigation method that allows seamless switching between the chat screen and the map display screen. The input is the user's operation, and the output is the screen after switching. Specifically, the operation smoothly switches between user interfaces.

[1225] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1226] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1227] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1228] [Fourth embodiment]

[1229] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1230] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1231] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1232] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1233] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1234] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1235] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1236] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1237] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1238] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1239] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1240] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1241] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1242] The present invention is a system that analyzes the content of conversations when a user plans a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map.

[1243] System configuration and operation overview

[1244] 1. Acquiring chat contents

[1245] When a user discusses travel plans with a friend in a chat application, the device captures the text data of the conversation in real time, temporarily stores the conversation content, and then transmits it to the server.

[1246] 2. Conversation Analysis

[1247] The server receives the conversation and analyzes it using a pre-trained natural language processing (NLP) model, which extracts important keywords from the conversation (e.g., "Fukuoka" and "Mizutaki") and the user's needs.

[1248] 3. Recommended stores and facilities

[1249] Based on the analysis results, the server searches for related facilities. For example, based on keywords such as "Fukuoka" and "Mizutaki," it searches for highly rated Mizutaki restaurants in the Fukuoka area. From the search results, it generates a list of recommended facilities.

[1250] 4. Mapping and Display

[1251] Based on the information in the generated list of recommended facilities (e.g., store name, address, rating), the server obtains the coordinates of the facilities and plots them on a map. The plotted map data is sent to the device and displayed within the chat application.

[1252] 5. User confirmation and selection

[1253] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[1254] 6. Seamlessly switch between chat and maps

[1255] The device provides a navigation method that seamlessly switches between the chat screen and the map display screen, allowing users to smoothly check and share information.

[1256] Specific examples

[1257] For example, suppose user A and user B are planning a trip to Fukuoka. The following conversation takes place:

[1258] User A: "Let's go to Fukuoka. What do you want to eat?"

[1259] User B: "I want to eat mizutaki."

[1260] Once this conversation has taken place, the system will operate as follows:

[1261] 1. The device acquires the conversation content in real time and sends it to the server.

[1262] 2. The server analyzes the conversation and extracts the keywords "Fukuoka" and "Mizutaki."

[1263] 3. The server searches for mizutaki restaurants in Fukuoka based on these keywords and generates a list of recommendations.

[1264] 4. The server retrieves the coordinates of the listed stores and plots them on the map data.

[1265] 5. The plotted map data is sent to the device and displayed within the chat app.

[1266] 6. When User B taps on a store on the map, they can view detailed information on their device.

[1267] This system allows users A and B to efficiently plan their trips, making it extremely easy to check and share information.

[1268] The processing flow will be explained below.

[1269] Step 1:

[1270] The device monitors text data of conversations in a chat application in real time and acquires the content of the conversation when a sending or receiving event occurs.

[1271] Step 2:

[1272] The device temporarily stores the acquired conversation content in memory and formats it for transmission to the server.

[1273] Step 3:

[1274] The device sends the formatted conversation content to the server via an API endpoint.

[1275] Step 4:

[1276] The server receives the conversation data at the API endpoint and parses it into the appropriate format.

[1277] Step 5:

[1278] The server inputs the received conversation data into a natural language processing (NLP) model to analyze and extract keywords and needs.

[1279] Step 6:

[1280] The server searches for relevant facilities and stores using an internal database or an external API (e.g., Google Places API) based on the extracted keywords and needs.

[1281] Step 7:

[1282] Based on the search results, the server generates a list of recommended facilities and stores, taking into account ratings and popularity.

[1283] Step 8:

[1284] The server obtains the coordinate information of each facility in the recommendation list and plots it on the map data.

[1285] Step 9:

[1286] The server formats the plotted map data for transmission to the terminal.

[1287] Step 10:

[1288] The terminal displays the received map data within the chat application.

[1289] Step 11:

[1290] The user looks at the displayed map and checks the suggested facilities and stores.

[1291] Step 12:

[1292] When a user selects a specific facility or store, the device displays detailed information about that facility (address, opening hours, ratings, reviews, etc.).

[1293] Step 13:

[1294] The device provides navigation buttons and gestures that allow users to seamlessly switch between the chat screen and the map screen.

[1295] Step 14:

[1296] When the device returns to the chat screen, it continues to display the latest conversation content, and when it returns to the map screen, it redisplays the map information that was previously displayed.

[1297] Example 1

[1298] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1299] In conventional chat applications, it was difficult for users to efficiently obtain information on recommended facilities when planning trips or outings. Furthermore, few systems had the functionality to search for related facilities based on conversation content and display them on a map, resulting in low user convenience. Furthermore, users were unable to seamlessly switch between the chat screen and the map display screen, resulting in a poor user experience.

[1300] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1301] In this invention, the server includes means for acquiring conversation content in a chat application, means for analyzing the acquired conversation content to extract keywords and needs, means for searching for recommended facilities based on the extracted keywords and needs, means for acquiring coordinate information of the searched recommended facilities and plotting it on a map, and means for displaying the plotted map data within the chat application, thereby enabling users to efficiently search for and display recommended facilities based on the conversation content within the chat application and make travel plans.

[1302] A "chat application" is software that allows users to exchange text messages and communicate.

[1303] "Conversation content" refers to the content of text messages exchanged between users within a chat application.

[1304] The "means for acquiring" refers to a function or process for collecting and storing conversation content within a chat application.

[1305] "Means for analysis" refers to techniques or methods for analyzing acquired text data and extracting useful information.

[1306] "Keywords" refer to particularly important words or phrases in the conversation content, and are used as search and recommendation criteria.

[1307] "Needs" refer to the requirements, desires, and needs of users, and are elements that are revealed through analysis.

[1308] "Recommended facilities" refers to places, stores, and services suggested based on the user's conversation content.

[1309] "Search means" refers to methods and techniques for searching the Internet for relevant facilities and information based on specified keywords or needs.

[1310] "Plotting means" refers to the technology or method for displaying the location of the searched facility on a map.

[1311] "Map data" refers to digital map data that includes information indicating locations and areas, and includes location information and display information.

[1312] "Navigation means" refers to functions and UI components that allow users to smoothly move between the chat screen and the map display screen.

[1313] "Detailed Information" refers to additional information about the recommended establishment, including address, opening hours, reviews, etc.

[1314] The present invention provides a system that analyzes conversations when a user is planning a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. Specific embodiments of this system are described below.

[1315] 1. Acquiring chat contents

[1316] The device monitors the text entered by the user in the chat application in real time. The device can be a smartphone or tablet. When the user sends a message, the text is stored in the device's temporary storage and then sent to a server via the Internet. The data is encrypted using the HTTPS protocol.

[1317] 2. Conversation Analysis

[1318] The server receives the text data sent from the device and automatically analyzes it using natural language processing (NLP) technology. Services such as Google Cloud Natural Language API and Amazon Comprehend are used. The server uses an NLP model to analyze the text and extract keywords (e.g., "Fukuoka" and "Mizutaki") and the user's needs.

[1319] 3. Recommended stores and facilities

[1320] Based on the analysis results, the server searches for relevant facilities. Based on the generated search query (e.g., "Fukuoka Mizutaki restaurants with high ratings"), it sends a request to an external API (e.g., Google Places, Yelp API) to obtain information on recommended facilities.

[1321] 4. Mapping and Display

[1322] The server obtains the coordinate information (latitude and longitude) of each store based on the acquired store information. The Google Maps Geocoding API is used for this purpose. The coordinate information is then plotted on map data. The plotted map data is sent to the device and displayed within the chat app. The map display uses the Google Maps or Mapbox API.

[1323] 5. User confirmation and selection

[1324] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[1325] 6. Seamlessly switch between chat and maps

[1326] The device provides a seamless navigation method for switching between chat and map views, using Android and iOS user interface components, allowing users to seamlessly view and share information.

[1327] Specific examples

[1328] For example, suppose user A and user B are planning a trip to Fukuoka. The following conversation takes place:

[1329] User A: "Let's go to Fukuoka. What do you want to eat?"

[1330] User B: "I want to eat mizutaki."

[1331] Once this conversation has taken place, the system will operate as follows:

[1332] 1. The device acquires the conversation content in real time and sends it to the server.

[1333] 2. The server analyzes the conversation and extracts the keywords "Fukuoka" and "Mizutaki."

[1334] 3. The server searches for mizutaki restaurants in Fukuoka based on these keywords and generates a list of recommendations.

[1335] 4. The server retrieves the coordinates of the listed stores and plots them on the map data.

[1336] 5. The plotted map data is sent to the device and displayed within the chat app.

[1337] 6. When User B taps on a store on the map, they can view detailed information on their device.

[1338] This system allows users A and B to efficiently plan their trips, making it extremely easy to check and share information.

[1339] Prompt Sentence Examples

[1340] "Please simulate a situation where User A and User B are planning a trip to Fukuoka and are looking for a Mizutaki restaurant. Analyze the conversation, search for recommended Mizutaki restaurants in Fukuoka, and display them on a map."

[1341] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1342] Step 1: Get chat transcript

[1343] The device monitors and acquires messages entered by the user in the chat application in real time. When the user sends a message, the text is stored in the device's temporary storage. The input is the text message entered by the user, and the output is the text data stored on the device. The stored text data is encrypted using the HTTPS protocol and sent to the server.

[1344] Specific working example:

[1345] When a user types "Let's go to Fukuoka. What do you want to eat?" into the chat and presses the send button, the text is saved on the device and then sent to the server.

[1346] Step 2: Analyzing the conversation

[1347] The server receives the text data sent from the device and analyzes it using the Google Cloud Natural Language API or Amazon Comprehend. The input is the text data received by the server, and the output is the keywords extracted by the analysis and the user's needs. The server then uses this information to identify important keywords (e.g., "Fukuoka" or "Mizutaki") and the user's needs.

[1348] Specific working example:

[1349] The server analyzes the text data "Let's go to Fukuoka. What do you want to eat?" and extracts the keywords "Fukuoka" and "want to eat."

[1350] Step 3: Proposing recommended stores and facilities

[1351] The server searches for related facilities using external APIs (Google Places and Yelp APIs) based on the extracted keywords. The input is the keywords and needs, and the output is a list of recommended facilities. Specifically, a search is performed using the keywords "Fukuoka" and "Mizutaki" to identify highly rated Mizutaki restaurants.

[1352] Specific working example:

[1353] The server calls the Google Places API using the keywords "Fukuoka" and "Mizutaki" and lists recommended Mizutaki restaurants in Fukuoka.

[1354] Step 4: Mapping and display

[1355] The server obtains the coordinate information (latitude and longitude) of the acquired store information and plots it on map data. The input is a list of recommended facilities, and the output is the plotted map data. The coordinate information is obtained using the Google Maps Geocoding API, and plotted on the map using the Google Maps or Mapbox API.

[1356] Specific working example:

[1357] The server calls the Geocoding API using the address information of the recommended mizutaki restaurant, obtains the latitude and longitude, and uses that information to place a marker on the map data.

[1358] Step 5: Confirm and select users

[1359] Users check the plotted map in the chat app on their device and select the store they are interested in. The input is the plotted map data, and the output is detailed information about the store selected by the user. When the user taps a marker on the map, detailed information (address, business hours, reviews) is displayed.

[1360] Specific working example:

[1361] When users tap on a specific mizutaki restaurant displayed on the map, detailed information about the restaurant will be displayed, and they can check its opening hours and reviews.

[1362] Step 6: Seamlessly move between chat and maps

[1363] The device provides a function to seamlessly switch between the chat screen and the map display screen. The input is a user operation, and the output is switching between the chat screen and the map screen. It is implemented using Android and iOS user interface components, and tab bars and swipe gestures are applied.

[1364] Specific working example:

[1365] When a user swipes from the chat screen to the map screen, the screen smoothly switches to the map display, and when they swipe again, they return to the chat screen.

[1366] (Application example 1)

[1367] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1368] In today's information-driven society, when planning trips or outings, users often find it difficult to find the best locations and facilities among the vast amount of information available. Furthermore, there are few systems that effectively extract information about destinations and facilities from conversations within chat applications and suggest them in real time, which hinders the efficiency of planning. Furthermore, the suggested information is not presented in a format that is intuitively understandable, rather than just in text, resulting in a lack of user convenience.

[1369] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1370] In this invention, the server includes means for acquiring conversation content in a chat application, means for analyzing the acquired conversation content to extract keywords and needs, means for searching for recommended facilities based on the extracted keywords and needs, means for acquiring location information of the plotted recommended facilities in a geographic information system, and means for displaying the plotted map data within the chat application, thereby enabling users to efficiently plan trips and outings and to check suggestions and detailed information about recommended facilities in real time.

[1371] A "chat application" is software that allows users to communicate in real time using text messages.

[1372] "Conversation" refers to the text messages sent and received by users within a chat application.

[1373] "Keywords" are important words and phrases extracted from the conversation content.

[1374] "Needs" are the requests and desires that users express through their conversations.

[1375] "Recommended facilities" are places and stores selected based on the user's keywords and needs.

[1376] The "search method" refers to the functions and algorithms used to identify recommended facilities.

[1377] The "means for plotting on a map" is a function for displaying the location information of recommended facilities on map data.

[1378] A "geographic information system" is a system for processing and visualizing location information.

[1379] "Location information of recommended facilities" is information including the latitude and longitude of the location of the recommended facilities.

[1380] "Detailed information" refers to additional information about the recommended facility, such as its name, address, rating, latitude, and longitude.

[1381] "Navigation means" is a function that allows the user to smoothly switch between the chat screen and the map display screen.

[1382] This invention is a system that analyzes conversations when a user is planning a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. This system is realized using a chat application, a natural language processing (NLP) model, a geographic information system (GIS), and a navigation means.

[1383] System configuration and operation overview

[1384] 1. Acquiring chat contents

[1385] When users discuss travel or outing plans within the chat application, the device acquires text data of the conversation in real time, temporarily stores it, and then transmits it to the server.

[1386] 2. Conversation Analysis

[1387] The server analyzes the captured conversation using a natural language processing (NLP) model (e.g., a pre-trained model from the Transformers library) to extract important keywords (e.g., "Shibuya" and "sushi") and user needs. This analysis detects specific requests, such as travel destinations and meals, from the conversation.

[1388] 3. Search for recommended facilities

[1389] The server searches for recommended establishments based on the extracted keywords and user needs. For example, based on the keywords "Shibuya" and "sushi," it searches for highly rated sushi restaurants in the Shibuya area using a geographic information system (e.g., Google Maps API).

[1390] 4. Mapping and Display

[1391] The server obtains the latitude and longitude coordinates of the facilities based on the information in the list of recommended facilities generated from the search results (e.g., store name, address, rating), and plots them on map data. The plotted map data is sent to the device and displayed within the chat application.

[1392] 5. User confirmation and selection

[1393] Users can view the suggested facilities on a map displayed within the chat application, and if they find one that interests them, they can view its details (e.g., address, opening hours, reviews) on their device.

[1394] 6. Seamlessly switch between chat and maps

[1395] The device provides a navigation means for users to seamlessly switch between the chat screen and the map display screen, updating the display in real time according to the content of the conversation.

[1396] Specific examples

[1397] For example, when a user says, "I'm looking for good sushi in Shibuya," the system works as follows:

[1398] The device acquires conversations containing the keywords "Shibuya" and "sushi" and sends them to the server.

[1399] The server analyzes the conversation and extracts the keywords "Shibuya" and "sushi."

[1400] Based on these keywords, the server uses a geographic information system to search for highly rated sushi restaurants.

[1401] The server obtains the coordinate information of the searched sushi restaurant and plots it on the map data.

[1402] The device displays the plotted map data within the chat application, allowing the user to view detailed information.

[1403] Prompt Sentence Examples

[1404] When a user says, "I'm looking for good sushi in Shibuya," the system should extract the keywords "Shibuya" and "sushi," search for related sushi restaurants, and display them on a map.

[1405] Using this system, users can intuitively and efficiently plan outings based on chat conversations. In addition, the information suggested by the system is visualized on a map, allowing users to easily select destinations and stores.

[1406] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1407] Step 1: Get chat transcript

[1408] The device captures text data of conversations in real time when users discuss travel or outing plans within a chat application. The captured text data is temporarily stored within the device and then sent to a server. The input is the user's chat messages, and the output is the text data of these messages.

[1409] Step 2: Analyzing the conversation

[1410] The server receives the text data of the conversation sent from the device and analyzes it using a natural language processing (NLP) model. This process extracts important keywords and user needs from the conversation. The input is text data, and the output is keywords and needs. Specific data processing involves the use of the NLP model for Named Entity Recognition (NER) and keyword extraction.

[1411] Step 3: Find recommended facilities

[1412] The server uses a geographic information system (GIS) to search for relevant facilities based on the extracted keywords and needs. For example, it searches for highly rated sushi restaurants based on the keywords "Shibuya" and "sushi." The input is the keywords and needs, and the output is a list of recommended facilities. As a specific example, it uses the Google Maps API to search for locations and obtain their rating information.

[1413] Step 4: Geographical information acquisition and mapping

[1414] The server obtains location information (latitude and longitude) from a geographic information system based on the address information of the searched recommended facility and plots it on map data. The input is the address information of the recommended facility, and the output is the plotted map data. The location information is processed using libraries such as Geopy.

[1415] Step 5: View detailed information about recommended facilities

[1416] The terminal receives the plotted map data sent from the server and displays it within the chat application. It also allows users to check detailed information (name, address, rating, latitude, longitude) of recommended facilities on the map. The input is the plotted map data, and the output is the facility information and detailed information displayed on the map.

[1417] Step 6: Seamlessly move between chat and maps

[1418] The device provides a navigation mechanism that allows users to seamlessly switch between chat and map views. It also updates the display based on real-time conversation content. Input is user interface manipulation, and output is an updated screen display. Specific operations include smooth switching and real-time updates of the user interface.

[1419] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1420] The present invention is a system that analyzes conversations when a user plans a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. Furthermore, an embodiment of the system will be described, which combines an emotion engine that recognizes the user's emotions and recommends and displays facilities based on the user's emotions.

[1421] System configuration and operation overview

[1422] 1. Acquiring chat contents

[1423] When a user discusses travel plans with a friend in a chat application, the device captures the text data of the conversation in real time, temporarily stores the conversation content, and then transmits it to the server.

[1424] 2. Conversation Analysis

[1425] The server receives the conversation and analyzes it using a pre-trained natural language processing (NLP) model, which extracts important keywords from the conversation (e.g., "Fukuoka" and "Mizutaki") and the user's needs.

[1426] 3. Emotional Recognition

[1427] At the same time, the server uses an emotion engine to analyze and recognize the user's emotions from the conversation content. For example, if the user seems to be having a fun conversation, the emotion of "joy" is recognized.

[1428] 4. Recommended stores and facilities

[1429] The server searches for relevant facilities based on the analyzed keywords, needs, and user emotions. For example, if the emotions "Fukuoka," "Mizutaki," and "Joy" are recognized, the server searches for highly rated Mizutaki restaurants in the Fukuoka area and prioritizes a list of the most suitable restaurants based on the user's emotions.

[1430] 5. Mapping and Display

[1431] Based on the information in the generated list of recommended facilities (e.g., store name, address, rating), the server obtains the coordinates of the facilities and plots them on a map. The plotted map data is sent to the device and displayed within the chat application.

[1432] 6. User confirmation and selection

[1433] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[1434] 7. Seamlessly switch between chat and maps

[1435] The device provides a navigation method that seamlessly switches between the chat screen and the map display screen, allowing users to smoothly check and share information.

[1436] Specific examples

[1437] For example, suppose user A and user B are planning a trip to Fukuoka. The following conversation takes place:

[1438] User A: "Let's go to Fukuoka. What do you want to eat?"

[1439] User B: "I want to eat mizutaki."

[1440] Once this conversation has taken place, the system will operate as follows:

[1441] 1. The device acquires the conversation content in real time and sends it to the server.

[1442] 2. The server analyzes the conversation and extracts the keywords "Fukuoka" and "Mizutaki."

[1443] 3. The server uses an emotion engine to recognize the user's emotions. If the conversation seems enjoyable, the emotion of "joy" is recognized.

[1444] 4. The server searches for mizutaki restaurants in Fukuoka based on the keywords and the recognized emotions, and generates a list of recommendations. At this time, it prioritizes the restaurant that best matches the user's emotions (e.g., "Hakata Kamaitachi").

[1445] 5. The server retrieves the coordinates of the listed stores and plots them on the map data.

[1446] 6. The plotted map data is sent to the device and displayed within the chat app.

[1447] 7. When User B taps on a store on the map, detailed information can be viewed on the device.

[1448] This system allows users A and B to efficiently plan their trips, making it extremely easy to confirm and share information. Furthermore, it improves the user experience by making suggestions based on the user's emotions.

[1449] The processing flow will be explained below.

[1450] Step 1:

[1451] The device monitors text data of conversations in a chat application in real time and acquires the content of the conversation when a sending or receiving event occurs.

[1452] Step 2:

[1453] The device temporarily stores the acquired conversation content in memory and formats it for transmission to the server.

[1454] Step 3:

[1455] The device sends the formatted conversation content to the server via an API endpoint.

[1456] Step 4:

[1457] The server receives the conversation data at the API endpoint and parses it into the appropriate format.

[1458] Step 5:

[1459] The server inputs the received conversation data into a natural language processing (NLP) model to analyze and extract keywords and needs.

[1460] Step 6:

[1461] At the same time, the server uses an emotion engine to analyze and recognize the user's emotions (e.g., "happiness," "surprise," "dissatisfaction," etc.) from the conversation data.

[1462] Step 7:

[1463] The server searches for relevant facilities and stores using an internal database or an external API (e.g., Google Places API) based on the extracted keywords, needs, and recognized user sentiment.

[1464] Step 8:

[1465] Based on the search results, the server generates a list of recommended facilities and stores, taking into account ratings, popularity, and recognized user emotions.

[1466] Step 9:

[1467] The server obtains the coordinate information of each facility in the recommendation list and plots it on the map data.

[1468] Step 10:

[1469] The server formats the plotted map data for transmission to the terminal.

[1470] Step 11:

[1471] The terminal displays the received map data within the chat application.

[1472] Step 12:

[1473] The user looks at the displayed map and checks the suggested facilities and stores.

[1474] Step 13:

[1475] When a user selects a specific facility or store, the device displays detailed information about that facility (address, opening hours, ratings, reviews, etc.).

[1476] Step 14:

[1477] The device provides navigation buttons and gestures that allow users to seamlessly switch between the chat screen and the map screen.

[1478] Step 15:

[1479] When the device returns to the chat screen, it continues to display the latest conversation content, and when it returns to the map screen, it redisplays the map information that was previously displayed.

[1480] Example 2

[1481] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1482] In conventional travel and outing planning, users have to search for information on the Internet and select appropriate facilities, which takes time and effort to obtain and select information.Furthermore, there is also the problem that the user experience is not sufficiently improved because it is not possible to suggest facilities based on the user's emotions and needs.

[1483] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1484] In this invention, the server includes means for acquiring conversation content between users in a chat application, means for the server to analyze the acquired conversation content and extract keywords and needs, and means for the server to search for facilities to recommend based on the extracted keywords and needs and adjust the recommendation ranking of the facilities based on the user's emotions. This enables efficient travel and outing planning by automatically searching for facilities that match the user's emotions and needs and displaying them on a map within the chat application.

[1485] "User" refers to an individual who uses the chat application to plan trips and outings.

[1486] "Server" refers to the central processing unit that analyzes the conversation content of the chat application and searches for and adjusts recommended facilities based on keywords, user needs, and emotions.

[1487] "Terminal" refers to a device (e.g., smartphone, tablet, or PC) that is directly operated by the user, has a chat application installed, acquires conversation content, and sends and receives data to and from the server.

[1488] A "chat application" is software used by users to communicate, and refers to an application that allows users to send and receive text messages in real time.

[1489] "Conversation content" refers to the content of text messages exchanged between users within a chat application, including information about travel and outings.

[1490] "Keywords" are important words and phrases extracted from conversations and used to identify a user's needs and destinations.

[1491] "Needs" refer to the desires and requirements that a user has regarding travel or outings.

[1492] "Emotion engine" refers to software or a module for analyzing and recognizing a user's emotions from the content of a conversation.

[1493] "Recommended facilities" refers to facilities such as travel destinations and restaurants that are suggested based on the user's keywords and needs.

[1494] "Recommendation ranking" refers to the priority of a recommended facility within the list, and is adjusted based on the user's feelings, ratings, etc.

[1495] "Map data" refers to map information on which the location information of recommended facilities is plotted.

[1496] The present invention is a system that analyzes conversations when a user plans a trip or outing within a chat application, automatically suggests recommended facilities, and displays them on a map. Furthermore, an embodiment of the system is described below, which combines an emotion engine that recognizes the user's emotions and recommends and displays facilities based on the user's emotions.

[1497] System configuration and operation

[1498] Get chat transcript

[1499] When a user discusses travel plans with a friend in a chat application, the device captures the text data of the conversation in real time, temporarily stores the conversation content, and then transmits it to the server.

[1500] Conversation analysis

[1501] The server receives the conversation and analyzes it using a pre-trained natural language processing (NLP) model (such as BERT). Through analysis, important keywords (e.g., "city name," "type of cuisine") and user needs are extracted from the conversation.

[1502] Emotion recognition

[1503] At the same time, the server uses an emotion engine (such as Affectiva) to analyze and recognize the user's emotions from the conversation content. For example, if the user is having a fun conversation, the emotion of "joy" is recognized.

[1504] Recommended stores and facilities

[1505] Based on the analyzed keywords, needs, and user sentiment, the server uses the Google Places API to search for related facilities. For example, if the "city name," "type of cuisine," and "pleasure" sentiment are recognized, the server searches for related facilities with high ratings and prioritizes the most suitable stores from the search results.

[1506] Mapping and Display

[1507] Based on the information in the generated list of recommended facilities (e.g., store name, address, rating), the server uses the Google Maps API to obtain the coordinates of the facilities and plots them on a map. The plotted map data is sent to the device and displayed within the chat application.

[1508] User confirmation and selection

[1509] Users can view the suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its detailed information (e.g., address, opening hours, reviews) on their device.

[1510] Seamlessly switch between chat and maps

[1511] The device provides a navigation method that seamlessly switches between the chat screen and the map display screen, allowing users to smoothly check and share information.

[1512] Specific examples

[1513] For example, suppose user A and user B are planning a trip to a certain city. The following conversation takes place:

[1514] User A: "Let's go to City Name. What do you want to eat?"

[1515] User B: "I want to try a variety of dishes."

[1516] Once this conversation has taken place, the system will operate as follows:

[1517] 1. The device acquires the conversation content in real time and sends it to the server.

[1518] 2. The server analyzes the conversation and extracts keywords such as "city name" and "type of cuisine."

[1519] 3. The server uses an emotion engine to recognize the user's emotions. If the conversation seems enjoyable, the emotion of "joy" is recognized.

[1520] 4. The server searches for establishments in the city that serve the type of cuisine based on the keywords and the recognized emotion, and generates a list of recommendations, prioritizing establishments that best fit the user's emotion.

[1521] 5. The server retrieves the coordinates of the listed stores and plots them on the map data.

[1522] 6. The plotted map data is sent to the device and displayed within the chat app.

[1523] 7. When User B taps on a store on the map, detailed information can be viewed on the device.

[1524] This system allows users A and B to efficiently plan their trips, making it extremely easy to confirm and share information. Furthermore, it improves the user experience by making suggestions based on the user's emotions.

[1525] Example prompts to input to the generative AI model

[1526] By inputting prompts like the following into the generative AI model, we can get an explanation of the system's behavior:

[1527] "Describe a system in a chat application for planning trips that analyzes conversations, suggests recommended facilities, and displays them on a map. Include recommending and displaying facilities based on the user's emotions."

[1528] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1529] Step 1: Get chat transcript

[1530] When a user starts a conversation on a chat application, the device collects the text data of the conversation in real time. Specifically, the user sends a message such as "Let's go to City Name. What do you want to eat?" The device receives this message and temporarily stores it. The input is the text message that the user sends on the application, and the output is the stored text data.

[1531] Step 2: Send the conversation

[1532] The terminal transmits the temporarily saved text data to the server. Specifically, the terminal transmits the temporarily saved text data to the server via the network. The input is the saved text data, and the output is the transmitted text data.

[1533] Step 3: Analyzing the conversation

[1534] The text data received by the server is analyzed using a pre-trained natural language processing (NLP) model (e.g., BERT). Specifically, the received text data is input into the NLP model, which extracts important keywords such as "city name" and "type of cuisine" as well as the user's needs. The input is the received text data, and the output is the extracted keywords and needs.

[1535] Step 4: Recognize emotions

[1536] The server uses an emotion engine (e.g., Affectiva) to analyze the user's emotions from the conversation content. Specifically, the extracted keywords and text data are input into the emotion engine to recognize the user's emotions (e.g., joy). The input is the extracted keywords and text data, and the output is the recognized emotion.

[1537] Step 5: Proposing recommended stores and facilities

[1538] The server uses the Google Places API to search for related facilities based on the extracted keywords and recognized emotions. Specifically, it searches for highly rated facilities based on the criteria of "city name," "type of cuisine," and "pleasure," and generates a list of recommendations. The input is the extracted keywords and recognized emotions, and the output is the generated list of recommended facilities.

[1539] Step 6: Obtain facility coordinates

[1540] The server uses the Google Maps API to obtain coordinate information for facilities from the list of recommended facilities. Specifically, it searches for coordinate information from the facility name or address and obtains coordinate information to plot on map data. The input is the list of recommended facilities, and the output is the obtained coordinate information.

[1541] Step 7: Generate map data

[1542] The server plots the acquired coordinate information onto map data. Specifically, it plots the facility locations on a map to generate visually easy-to-understand map data. The input is the acquired coordinate information, and the output is the plotted map data.

[1543] Step 8: Displaying map data

[1544] The terminal displays the plotted map data received from the server within the chat application. Specifically, the map data is reflected in the user interface, allowing the user to view facilities on the map. The input is the plotted map data, and the output is the displayed map interface.

[1545] Step 9: View facility details

[1546] When a user taps on a facility of interest on the map, the device displays detailed information about that facility (address, business hours, reviews, etc.). Specifically, the device obtains detailed information according to the user's operation and displays it on the interface. The input is the user's tap operation, and the output is the displayed detailed facility information.

[1547] Step 10: Seamlessly move between chat and maps

[1548] The device provides a navigation method that allows seamless switching between the chat screen and the map display screen. Specifically, the user can smoothly move back and forth between the chat screen and the map display screen using the tabs at the top of the screen. The input is the user's navigation operation, and the output is the switched screen.

[1549] (Application example 2)

[1550] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1551] Conventional chat applications lack the functionality to analyze conversation content and suggest suitable facilities when users plan trips or outings, making planning cumbersome and inefficient. Furthermore, suggestions do not reflect the user's emotions, making it difficult to provide a better experience. Furthermore, the inability to effectively present users with location information and detailed information about recommended facilities also delays planning.

[1552] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1553] In this invention, the server includes means for acquiring conversation content in a chat application, means for analyzing the acquired conversation content to extract keywords and needs, means for analyzing and recognizing user emotions from the conversation content using an emotion engine, means for searching for recommended facilities based on the extracted keywords, needs, and emotions, means for plotting the searched recommended facilities on a map, means for displaying the plotted map data within the chat application, means for displaying detailed information about the recommended facilities, and navigation means for seamlessly switching between the chat screen and the map display screen. This makes it possible to suggest optimal facilities based on the user's conversation content and emotions, making planning more efficient and convenient.

[1554] A "chat application" is an application that allows users to send and receive text messages in real time.

[1555] The "means for acquiring conversation content" is a function for acquiring text data of conversations taking place within a chat application in real time.

[1556] "Means for extracting keywords and needs" is a function for recognizing and extracting important phrases and user requests from the acquired conversation content.

[1557] An "emotion engine" is software or a component that analyzes a user's emotional state from the content of a conversation and recognizes specific emotions.

[1558] The "means for searching for recommended facilities" is a function for searching for relevant facilities from a database or the Internet based on the extracted keywords, needs, and emotions.

[1559] "Means for plotting on a map" is a function for obtaining the location information of the searched facility and marking it on a digital map.

[1560] The "means for displaying" is a function for visually displaying the plotted map data within the chat application.

[1561] "Means for displaying detailed information about recommended facilities" is a function for providing users with specific information about recommended facilities (e.g., address, business hours, reviews).

[1562] "Seamless switching navigation means" is a function that provides operability for switching between the chat screen and the map display screen without delay.

[1563] System configuration and operation overview

[1564] The present invention is a travel planning support system for smartphones. This system is composed of the following main components:

[1565] 1. Acquiring chat contents

[1566] When a user is chatting with another user about travel or outing plans within a chat application, the device captures the text data of the conversation in real time. This data is temporarily stored on the device and then sent to a server.

[1567] 2. Conversation Analysis

[1568] The server analyzes the captured conversation using natural language processing (NLP) technology, specifically using the Bidirectional Encoder Representations from Transformers (BERT) model to extract important keywords and user needs from the conversation.

[1569] 3. Emotional Recognition

[1570] The server uses an emotion engine to analyze and recognize the user's emotions from the conversation content. It uses the VADER (Valence Aware Dictionary and sEntiment Reasoner) algorithm to determine the positivity, negativity, and intensity of each emotion.

[1571] 4. Recommended facilities

[1572] The server searches a database or the Internet for relevant facilities based on the extracted keywords, needs, and recognized emotions, and prioritizes and lists the facilities that best fit the user's emotions from the search results.

[1573] 5. Mapping and Display

[1574] The location information of the recommended establishments is retrieved and plotted on a digital map. The map data is generated using the folium library, sent to the device, and displayed visually within the chat application.

[1575] 6. User confirmation and selection

[1576] Users can view suggested facilities on a map displayed within the chat app, and if they find one that interests them, they can view its details (address, opening hours, reviews) on their device.

[1577] 7. Seamlessly switch between chat and maps

[1578] The device provides a navigation method that allows users to seamlessly switch between the chat screen and the map display screen, allowing them to smoothly check and share information.

[1579] Specific examples

[1580] For example, if a user has the following conversation:

[1581] User A: "Let's go to Kyoto. What do you want to eat?"

[1582] User B: "I want to eat ramen."

[1583] In this case, the system behaves as follows:

[1584] 1. The device acquires the conversation content in real time and sends it to the server.

[1585] 2. The server uses the BERT model to extract the keywords "Kyoto" and "ramen."

[1586] 3. The server uses the VADER algorithm to recognize the emotion "joy" if the conversation is positive.

[1587] 4. The server searches for highly rated ramen restaurants in the Kyoto area based on keywords and sentiment, and lists the most suitable restaurants.

[1588] 5. The server retrieves the coordinates of the listed stores and plots them on a map using the folium library.

[1589] 6. The plotted map data is sent to the device and displayed within the chat app.

[1590] This allows users to efficiently plan their trips, makes it much easier to check and share information, and improves the user experience by providing suggestions based on user sentiment.

[1591] Prompt Sentence Examples

[1592] For example, the following prompt might be used to search for a recommended cafe in Kumamoto:

[1593] text

[1594] Are there any good cafes in Kumamoto?

[1595] Based on this, the system analyzes the keywords "Kumamoto" and "cafe" along with positive emotions, suggests the most suitable cafe, and displays it on a map.

[1596] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1597] Step 1:

[1598] Acquiring conversation content

[1599] The terminal acquires text data of conversations taking place within a chat application in real time. This data is temporarily stored on the terminal. The specific input is a text message entered by the user, and the output is the acquired text data of the conversation.

[1600] Step 2:

[1601] Sending conversation transcripts

[1602] The terminal sends the acquired text data to the server. The input is the temporarily saved text data, and the output is the data to be sent to the server. Specifically, the data is sent to the server via an HTTP request.

[1603] Step 3:

[1604] Conversation analysis

[1605] The server analyzes the received conversation content using natural language processing (NLP) technology. Specifically, it uses the BERT model to extract keywords and needs from the input text data. The input is the text data sent to the server, and the output is the extracted keywords and needs.

[1606] Step 4:

[1607] Emotion recognition

[1608] The server uses an emotion engine (VADER algorithm) to analyze and recognize the user's emotions from the conversation content. The input is the text data sent to the server, and the output is the recognized emotion data. Specific operations include emotion scoring and analysis.

[1609] Step 5:

[1610] Search for recommended facilities

[1611] The server searches for relevant facilities from a database or the Internet based on the extracted keywords, needs, and recognized emotions. The input is the extracted keywords and emotion data, and the output is a list of recommended facilities. Specific operations include executing API requests and database queries.

[1612] Step 6:

[1613] Facility location and mapping

[1614] The server acquires the location information of the recommended facilities and plots it on a digital map. The input is a list of recommended facilities, and the output is the plotted map data. Specific operations include geocoding and generating map data.

[1615] Step 7:

[1616] Sending map data

[1617] The server sends the plotted map data to the device. The input is the plotted map data, and the output is the data to be sent to the device. Specifically, the server sends the map data to the device via an HTTP request.

[1618] Step 8:

[1619] Displaying the map

[1620] The terminal visually displays the received map data within the chat application. The input is the map data sent from the server, and the output is the visually displayed map.

[1621] Step 9:

[1622] Displaying detailed facility information

[1623] The user views the map displayed in the chat app to check the suggested facilities. When the user selects a facility of interest, its details (address, opening hours, reviews) are displayed on the device. The input is the ID of the selected facility, and the output is the facility details.

[1624] Step 10:

[1625] Seamless switching between chat and maps

[1626] The device provides a navigation method that allows seamless switching between the chat screen and the map display screen. The input is the user's operation, and the output is the screen after switching. Specifically, the operation smoothly switches between user interfaces.

[1627] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1628] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1629] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1630] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1631] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1632] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1633] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1634] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1635] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1636] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1637] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1638] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1639] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1640] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1641] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1642] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1643] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1644] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1645] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1646] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1647] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1648] The following is further disclosed regarding the above embodiment.

[1649] (Claim 1)

[1650] A means for acquiring conversation content in a chat application;

[1651] A means for analyzing the acquired conversation content and extracting keywords and needs;

[1652] A means for searching for recommended facilities based on the extracted keywords and needs;

[1653] A means for plotting the searched recommended facilities on a map;

[1654] means for displaying the plotted map data within the chat application;

[1655] A system including:

[1656] (Claim 2)

[1657] The system of claim 1, further comprising means for displaying detailed information about the recommended facility.

[1658] (Claim 3)

[1659] 10. The system of claim 1, further comprising a navigation means for seamlessly switching between a chat screen and a map display screen.

[1660] "Example 1"

[1661] (Claim 1)

[1662] A means for acquiring conversation content in a chat application;

[1663] A means for analyzing the acquired conversation content and extracting keywords and needs;

[1664] A means for searching for recommended facilities based on the extracted keywords and needs;

[1665] A means for acquiring coordinate information of the searched recommended facilities and plotting it on a map;

[1666] means for displaying the plotted map data within the chat application;

[1667] A system including:

[1668] (Claim 2)

[1669] The system of claim 1, further comprising means for displaying detailed information about the recommended facility.

[1670] (Claim 3)

[1671] 10. The system of claim 1, further comprising a navigation means for seamlessly switching between a chat screen and a map display screen.

[1672] "Application Example 1"

[1673] (Claim 1)

[1674] A means for acquiring conversation content in a chat application;

[1675] A means for analyzing the acquired conversation content and extracting keywords and needs;

[1676] A means for searching for recommended facilities based on the extracted keywords and needs;

[1677] A means for plotting the searched recommended facilities on a map;

[1678] means for displaying the plotted map data within the chat application;

[1679] A means for acquiring location information of the plotted recommended facilities from a geographic information system;

[1680] A system including:

[1681] (Claim 2)

[1682] 10. The system of claim 1, further comprising means for displaying detailed information including the name, address, rating, latitude, and longitude of the recommended facility.

[1683] (Claim 3)

[1684] A navigation method that allows you to seamlessly switch between the chat screen and the map display screen,

[1685] 10. The system of claim 1, further comprising means for updating navigation and display based on conversation content analyzed in real time.

[1686] "Example 2: Combining Emotion Engines"

[1687] (Claim 1)

[1688] A means for a user to acquire conversation content in a chat application;

[1689] A means for the server to analyze the acquired conversation content and extract keywords and needs;

[1690] A means for the server to search for facilities to recommend based on the extracted keywords and needs, and adjust the recommendation ranking of the facilities based on the user's emotions;

[1691] A means for plotting the searched recommended facilities on a map by the server;

[1692] means for the terminal to display the plotted map data within a chat application;

[1693] A system including:

[1694] (Claim 2)

[1695] The system according to claim 1, further comprising means for the terminal to display detailed information about the recommended facility.

[1696] (Claim 3)

[1697] 10. The system of claim 1, further comprising a navigation means for the terminal to seamlessly switch between a chat screen and a map display screen.

[1698] "Application example 2 when combining emotion engines"

[1699] (Claim 1)

[1700] A means for acquiring conversation content in a chat application;

[1701] A means for analyzing the acquired conversation content and extracting keywords and needs;

[1702] A means for searching for recommended facilities based on the extracted keywords and needs;

[1703] A means for plotting the searched recommended facilities on a map;

[1704] means for displaying the plotted map data within the chat application;

[1705] A means of analyzing and recognizing the user's emotions from the content of the conversation using an emotion engine;

[1706] A means to prioritize and list the best facilities based on emotions,

[1707] A system including:

[1708] (Claim 2)

[1709] The system of claim 1, further comprising means for displaying detailed information about the recommended facility.

[1710] (Claim 3)

[1711] 10. The system of claim 1, further comprising a navigation means for seamlessly switching between a chat screen and a map display screen.

[1712] (Claim 4)

[1713] The system of claim 1, further comprising means for displaying a rating for the recommended facility based on the user's sentiment.

[1714] (Claim 5)

[1715] 10. The system according to claim 1, further comprising means for acquiring current location information of the user and narrowing down the facilities based on the location. [Explanation of symbols]

[1716] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring conversation content in a chat application; A means for analyzing the acquired conversation content and extracting keywords and needs; A means for searching for recommended facilities based on the extracted keywords and needs; A means for plotting the searched recommended facilities on a map; means for displaying the plotted map data within the chat application; A system including:

2. The system according to claim 1 , further comprising means for displaying detailed information about the recommended facility.

3. The system according to claim 1 , further comprising a navigation means for seamlessly switching between the chat screen and the map display screen.

Images