System

Abstract

A system is provided.SOLUTION: A system includes a means for inputting a request by a user by text or voice, a means for transmitting the request to a server, a means for analyzing the request in the server, a means for collecting satellite images, aerial images, and open data, a means for generating a digital twin map, a means for providing personalized information, and a means for transmitting the generated information to a terminal and displaying the information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional map services mainly require users to search using specific names or addresses, making it difficult to experience new discoveries and encounters. Furthermore, conventional systems lacked sufficient request analysis, making it difficult to respond to complex user needs. Furthermore, delays in updating satellite images and databases meant that the latest information could not always be provided. There is a need for a system that can solve these issues and allow users to easily enjoy new discoveries and encounters. [Means for solving the problem]

[0005] The present invention solves the above problems by the following means. A system is provided that allows users to input requests via text or voice and transmits the requests to a server. The server is provided with a means for analyzing the requests and combines a means for collecting satellite images, aerial images, and open data with a means for generating a digital twin map. Furthermore, a system is constructed that provides personalized information and transmits the generated information to a terminal for display. This allows users to efficiently discover new information and locations in response to complex requests. The system also has the function of acquiring current location information and transmitting it to the server along with the request, and uses natural language analysis technology in request analysis. Furthermore, by calculating the optimal route and displaying it on a digital twin map, users can easily understand the visual information and always obtain the latest information through satellite and aerial images that are updated in real time. The user interface is optimized and displayed based on the personalized information, improving the user experience. Using these means, the existing problems can be comprehensively solved.

[0006] "User" means an individual or organization that uses the system to search for information or navigate maps.

[0007] A "request" is a question or instruction a user enters into the system via text or voice regarding the information or destination they wish to search for.

[0008] "Server" means a central processing unit that analyzes requests received from Users, collects and processes necessary data, and provides the results to Users.

[0009] "Satellite imagery" means high-resolution images of the Earth's surface taken from a satellite and used to generate maps in geographic information systems.

[0010] "Aerial imagery" refers to high-resolution images of the Earth's surface taken from an aircraft and used to generate detailed maps in geographic information systems.

[0011] "Open data" is data that is made public by public institutions and companies and can be freely used and reused, and is used to improve the accuracy and usefulness of search results in systems.

[0012] A "digital twin map" is a map that reproduces real-world geographic information and objects in a digital space in real time, and is provided in a visually easy-to-understand format.

[0013] "Personalized information" refers to customized information that is optimized based on a user's individual requests and location information.

[0014] "Means of request analysis" refers to the process of understanding the content of the request received from the user and extracting relevant keywords and intent.

[0015] "Means for displaying information" means a method for visually presenting the system-generated digital twin map and other related information to the user. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a system in which a user inputs a request by text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. The present invention is primarily provided via smartphones and car navigation systems.

[0038] Program processing

[0039] 1. The user enters a request

[0040] Subject: User

[0041] Users make requests using an application on their smartphone or car navigation system. For example, they might say, "Tell me about a popular restaurant within five minutes of my current location that would be fun for a family of four for half a day." The user's request includes their current location.

[0042] 2. Submitting a Request

[0043] Subject: Terminal

[0044] The device receives the user's voice or text request and sends it to the server as request data, along with the current location information (GPS data).

[0045] 3. Parsing the Request

[0046] Subject: Server

[0047] The server analyzes the received request data using natural language processing technology to understand the user's request. For example, it extracts keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant."

[0048] 4. Preparation for Data Collection

[0049] Subject: Server

[0050] Based on the analysis results, the server prepares to collect the necessary data. Specifically, it identifies an area within a five-minute radius of the current location and sets up the system to acquire satellite images, aerial images, and open database data within that area.

[0051] 5. Data Acquisition

[0052] Subject: Server

[0053] The server retrieves the latest data from various data sources (satellite imagery API, aerial imagery API, open databases, etc.), including store information and tourist spot information that may be of interest to users.

[0054] 6. Digital Twin Map Generation

[0055] Subject: Server

[0056] The server integrates the collected data to generate a digital twin map that displays real-time geographic information and recommendations based on user requests.

[0057] 7. Providing personalized information

[0058] Subject: Server

[0059] The server generates personalized information based on the digital twin map and sends it to the device as display data, including optimal route guidance, detailed store information, and reviews.

[0060] 8. Display of Information

[0061] Subject: Terminal

[0062] The device receives the display data sent from the server and displays it in a visually easy-to-understand format for the user. Users can check the recommended route and the location of each store on a map. They can also view detailed store information and user reviews as text information.

[0063] Specific examples

[0064] Example 1: Searching for family-friendly restaurants

[0065] User: Type into a smartphone app, "Tell me about a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal."

[0066] Device: Sends a request to the server along with its current location information.

[0067] Server: Parses the request and collects the necessary data.

[0068] Server: Generates a digital twin map and sends optimal routes and recommended restaurants to the device.

[0069] On your device: Displays recommended restaurants and optimal routes on a map, along with detailed information and reviews.

[0070] Example 2: Searching for tourist spots

[0071] User: Voice-input to the car navigation system, "Tell me places I can visit within 10 minutes of my current location."

[0072] Terminal: Sends requests to the server.

[0073] Server: Collects data based on the analysis results and obtains information about tourist spots.

[0074] Server: Generates a digital twin map and sends tourist spot and route information to the device.

[0075] Device: Displays the location of tourist attractions and the best route on a map.

[0076] Through this system, users can easily and intuitively make new discoveries and have new experiences, even when dealing with complex requests.

[0077] The processing flow will be explained below.

[0078] Step 1: User enters request

[0079] Subject: User

[0080] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice. For example, they might input, "Tell me about a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day." The application obtains the user's current location along with this request.

[0081] Step 2: Submitting the request

[0082] Subject: Terminal

[0083] The device sends the request entered by the user and the current location information to the server, where voice input is converted to text and the request is prepared as data.

[0084] Step 3: Parsing the request

[0085] Subject: Server

[0086] The server analyzes the received request data. First, it analyzes the request string using natural language analysis technology to extract keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." This process uses generative AI such as ChatGPT.

[0087] Step 4: Prepare for data collection

[0088] Subject: Server

[0089] Based on the analysis results, the server prepares to collect the necessary data. For example, it identifies an area "within 5 minutes of the current location" and sets up an API call to obtain satellite images, aerial images, and open data for that area.

[0090] Step 5: Get the data

[0091] Subject: Server

[0092] The server collects the necessary data from multiple data sources, including satellite imagery APIs and aerial imagery APIs to obtain geographical information, and store information, user reviews, and other data from open databases.

[0093] Step 6: Generate the digital twin map

[0094] Subject: Server

[0095] The server integrates the collected data to generate a digital twin map, which includes relevant information based on the user's request, such as visually displaying the locations of restaurants and tourist attractions that the user can enjoy, as well as route information.

[0096] Step 7: Provide personalized information

[0097] Subject: Server

[0098] The server then personalizes the generated digital twin map and related information for the user, building data based on individual requests and location information, including optimal routes, recommended stores, detailed information, and user reviews.

[0099] Step 8: Submit your information

[0100] Subject: Server

[0101] The server transmits the personalized information to the terminal, including map data and text information for visual display.

[0102] Step 9: Viewing Information

[0103] Subject: Terminal

[0104] The device displays the information received from the server on a user interface, allowing users to move and make selections while checking recommended routes, recommended places, detailed information, user reviews, and more on the displayed map.

[0105] Through the above processing steps, the system provides users with a highly convenient and innovative map search experience.

[0106] Example 1

[0107] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0108] In modern society, there is a need to provide a wide variety of information quickly and efficiently, but there is a lack of systems that can collect appropriate information in response to user requests and provide it in a personalized format. Another issue is the difficulty of providing personalized information in real time based on the user's current location. Therefore, there is a need to develop a system that allows users to intuitively and quickly obtain the information they need.

[0109] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0110] In this invention, the server includes an analysis means, a data collection means, a digital twin map generation means, a personalized information provision means, a range specification means using current location information, a collection preparation means, and a spot display means, which enable users to efficiently obtain information within a specific range from their current location and use it in a personalized manner.

[0111] "User" means any person or agent who uses the System to request information.

[0112] "Request" means a request for information entered by a User into a System, including text input or voice input.

[0113] "Device" refers to the device used by a user to enter requests and obtain information, including smartphones and car navigation systems.

[0114] "Server" refers to the central computer in the system that receives user requests, analyzes them, collects the necessary data, and provides the results to the user.

[0115] "Data Sources" refers to the various databases and APIs that the server accesses to gather information, including satellite imagery APIs and open databases.

[0116] A "digital twin map" refers to a real-time geographic information system that is generated based on collected data.

[0117] "Location information" refers to location data that indicates the user's current location, primarily GPS data.

[0118] "Analysis Method" refers to the technology used by the server to understand the content of the user's request, including natural language processing technology.

[0119] "Data collection means" refers to the methods and processes by which the server obtains the required data from various data sources.

[0120] "Digital twin map generation means" refers to the method or process for integrating data collected by the server and creating a digital twin map.

[0121] "Personalized information provision means" refers to a method or process for generating and providing personalized information based on a user's request.

[0122] "Area specification means using current location information" refers to a method or process by which a server specifies a specified range based on the user's current location information.

[0123] "Collection preparation means" refers to the method or process by which the server configures and prepares itself to collect the required data.

[0124] "Spot display means" refers to a method or process by which the server displays spots based on a user's request on the digital twin map.

[0125] The present invention is a system in which a user inputs a request, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. Specific embodiments of the system are described below.

[0126] Users can use their smartphone or car navigation system to input requests by voice or text into a dedicated application. For example, a user might request, "Tell me about some popular restaurants within five minutes of my current location that a family of four can enjoy for half a day."

[0127] The device sends the input request and current location information (GPS data) to the server using a data communication protocol such as an HTTP POST request.

[0128] The server uses natural language processing technology to analyze the received request, specifically leveraging a natural language processing API (e.g., Google Cloud Natural Language API) to extract keywords and important information from the request.

[0129] Once the request is analyzed, the server will use the user's current location information to begin collecting data within the specified area. For example, it will use the Google Maps API or OpenStreetMap API to prepare to obtain the necessary information (store information, tourist spot information, etc.).

[0130] The server then uses satellite and aerial imagery APIs to retrieve data, including up-to-date geographic information, and combines this collected data to generate a digital twin map.

[0131] The generated digital twin map displays recommended spots and optimal route guidance based on user requests, and also includes detailed store information and user reviews for personalized information.

[0132] Finally, the server sends the generated information to the device as display data, which is then presented to the user in a visually easy-to-understand format, allowing the user to check the recommended route, the location of each spot, and detailed information on the map.

[0133] Specific examples

[0134] Example 1: Searching for family-friendly restaurants

[0135] User: Type into a smartphone app, "Tell me about a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal."

[0136] Terminal: Sends the request data to the server along with the current location information.

[0137] Server: Analyzes the request using natural language processing technology and collects the necessary data.

[0138] Server: Collects restaurant information from Google Maps API and other data sources and generates a digital twin map.

[0139] Server: Sends optimal route and restaurant information to the device.

[0140] Device: Displays restaurant recommendations and optimal routes on an interactive map, along with detailed information and reviews.

[0141] Prompt Sentence Examples

[0142] "Tell me a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal."

[0143] This system utilizes natural language processing technology and various data collection APIs to provide users with intuitive and personalized information, allowing them to easily and efficiently obtain information that meets their needs.

[0144] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0145] Step 1:

[0146] When a user uses a smartphone or car navigation system to input a request within an application, they can use text or voice. For example, they might say, "Tell me about a popular restaurant within five minutes of my current location that would be fun for a family of four for half a day." This input becomes input data for the device. The device converts the voice data into text data and processes it as request data.

[0147] Step 2:

[0148] The device sends the request data entered by the user and current location information (GPS data) to the server. This is done using an HTTP POST request. The input data is the request data (text request) and current location information (GPS data), and the output data is the request payload sent to the server.

[0149] Step 3:

[0150] The server analyzes the received request data. Natural language processing technology is used for the analysis. Specifically, a natural language processing API (e.g., a natural language processing cloud API) is used to extract keywords and important information from the request data. The input data is the request payload, and the output data is the analysis results (e.g., keywords "within 5 minutes," "family of 4," "half day," "popular restaurant").

[0151] Step 4:

[0152] The server prepares for data collection based on the analysis results. It uses the current location information to identify an area within a specified range (e.g., within 5 minutes) and configures data collection settings within that area. This preparation includes identifying the area using a map API (e.g., Map Cloud API) and configuring data collection parameters. The input data are the analysis results and current location information, and the output data is the configuration information for which collection preparation is complete.

[0153] Step 5:

[0154] The server obtains the necessary data from various data sources (e.g., satellite imagery API, aerial imagery API, open databases, etc.). This includes real-time updated geographic information and information requested by users (e.g., store information, tourist spot information, etc.). The input data is the setup information for collection preparation, and the output data is the various data obtained.

[0155] Step 6:

[0156] The server integrates the acquired data to generate a digital twin map. For example, it uses geographic information system (GIS) technology to draw a map based on satellite and aerial images and plot recommended spots based on user requests. The input data is the acquired data, and the output data is the digital twin map.

[0157] Step 7:

[0158] The server generates personalized information for the user based on the generated digital twin map. This includes optimal route guidance, detailed store information, reviews, and more. The input data is the digital twin map and the user's requested information, and the output data is personalized information. This is then sent to the device as display data.

[0159] Step 8:

[0160] The terminal receives the display data sent from the server and displays it to the user in a visually easy-to-understand format. The user can check the recommended route and the location and detailed information of each spot on an interactive map. Detailed store information and user reviews are also displayed as text information. The input data is the display data, and the output data is the display screen for the user.

[0161] (Application example 1)

[0162] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0163] Conventional map information systems make it difficult for users to obtain detailed information about virtual stores. They also lack the means to provide personalized information based on the user's current location in real time. As a result, it is difficult for users to intuitively obtain specific store or product information, or to navigate within the virtual space.

[0164] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0165] In this invention, the server includes a means for a user to input a request for a virtual store using a smartphone or smart glasses, a means for the server to provide detailed information about the virtual store based on a digital twin map, and a means for navigating the virtual store and viewing product information, allowing the user to obtain detailed, personalized store and product information in real time and achieving intuitive navigation within the virtual store.

[0166] 1. "Means for users to input requests by text or voice" refers to devices or software that allow users to input their requests or questions by text or voice.

[0167] 2. "Means for sending a request to the server" refers to the communications means for transmitting text or voice data entered by the user to the server via the Internet.

[0168] 3. "Means for analyzing requests on the server" refers to the processes and technologies used by the server to analyze the request data received and understand its contents.

[0169] 4. "Means for collecting satellite imagery, aerial imagery, and open data" refers to the API or database access method that the server uses to collect the necessary satellite imagery, aerial imagery, and open data based on the results of request analysis.

[0170] 5. "Means for generating digital twin maps" refers to technologies that generate virtual maps based on collected data, creating digital models that function as twins of the physical world.

[0171] 6. "Means for providing personalized information" refers to methods and technologies for providing information optimized for individual users based on their requests and current location information.

[0172] 7. "Means for transmitting and displaying generated information to a terminal" refers to the technology and devices for transmitting personalized information generated by the server to a user's terminal and visually displaying it.

[0173] 8. "Means for users to input requests about a virtual store using a smartphone or smart glasses" refers to a method and device for users to request information about a virtual store using a smartphone or smart glasses.

[0174] 9. "Means for the server to provide detailed information about the virtual store based on the digital twin map" refers to methods and technologies for utilizing the digital twin map to provide users with detailed information about the virtual store.

[0175] 10. "Means for navigating the virtual store and viewing product information" means the interface or technology that allows a user to virtually navigate the virtual store and view detailed product information.

[0176] The present invention is a system in which a user inputs a request for a virtual store using a smartphone or smart glasses, and detailed information is provided based on that request using a digital twin.

[0177] The system uses the following hardware and software:

[0178] Smartphones and smart glasses: These are devices through which users input requests.

[0179] GPS module: Used to obtain the user's current location information.

[0180] Server: This is the central system that analyzes request data and collects and processes the necessary data. It uses various APIs (satellite imagery API, aerial imagery API, open database), natural language processing engines (e.g., SpaCy), and database management systems (e.g., MySQL).

[0181] Communication module: Transmits data between the device and the server via the Internet.

[0182] The program processing proceeds as follows:

[0183] Using a smartphone or smart glasses, a user can input a request by voice or text, such as "Tell me what fashion items are popular at the nearest department store right now." This request also includes the user's current location information. The device then sends the request data and current location information to the server.

[0184] When the server receives the request data, it analyzes the request using natural language processing technology and extracts keywords. Based on the results of this analysis, the server prepares to collect the necessary data. Specifically, it obtains the latest data from satellite images, aerial images, and open databases.

[0185] The server integrates the collected data and generates a digital twin map, which includes store information and product details that the user may be interested in. The generated information is sent from the server to the user's device and displayed visually.

[0186] For example, if a user requests, "Tell me the popular products on sale at the nearest shopping mall right now," the server analyzes the request and generates a digital twin map of the shopping mall. The generated map includes detailed information about popular products on sale and stores. The user can also navigate within the virtual store and view detailed product information.

[0187] Here are some examples of prompts for generative AI models:

[0188] "Please tell me information about popular fashion items at the department store nearest to my current location. Details should include the item name, image, price, store location, etc."

[0189] This allows users to receive real-time personalized information and intuitive navigation within the virtual store.

[0190] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0191] Step 1:

[0192] The user uses a smartphone or smart glasses to enter a request.

[0193] Specific operation: The user inputs a request via text or voice, for example, "Tell me what fashion items are popular at the nearest department store right now."

[0194] Input: User request voice or text data, current location information.

[0195] Output: Request data and current location information.

[0196] Step 2:

[0197] The device sends the request data and current location information to the server.

[0198] Specific operation: The terminal sends the request data entered by the user and the current location information obtained using the GPS module to the server.

[0199] Input: Request data, current location information.

[0200] Output: Request data sent to the server and current location information.

[0201] Step 3:

[0202] The server parses the request data.

[0203] Specific operation: The server analyzes the received request data using a natural language processing engine (e.g., SpaCy) and extracts keywords such as "department store," "fashion item," and "popular."

[0204] Input: Request data.

[0205] Output: Analysis results (extracted keywords).

[0206] Step 4:

[0207] Prepare the server to collect the required data.

[0208] Specific operation: Based on the analysis results, the server generates a query to obtain information from the required data sources (satellite imagery API, aerial imagery API, open database, etc.).

[0209] Input: Analysis results (extracted keywords).

[0210] Output: Data collection query.

[0211] Step 5:

[0212] The server retrieves data from various data sources.

[0213] Specific operation: The server uses the generated query to collect satellite images, aerial images, store information, and product information from data sources.

[0214] Input: Data collection query.

[0215] Output: Collected data (satellite images, aerial images, open data, store information, product information).

[0216] Step 6:

[0217] The server generates a digital twin map.

[0218] How it works: The server integrates the collected data and generates a personalized digital twin map based on the user's request.

[0219] Input: Collected data (satellite images, aerial images, open data, store information, product information).

[0220] Output: Digital twin map.

[0221] Step 7:

[0222] The server generates personalized information based on the digital twin map and sends it to the device.

[0223] Specific operation: Based on the generated digital twin map, the server generates the optimal route, detailed store information, and product information to suggest to the user, and sends this to the terminal.

[0224] Input: Digital twin map.

[0225] Output: Display data (optimal route, detailed store information, product information).

[0226] Step 8:

[0227] The terminal receives the display data sent from the server and displays it in a visually easy-to-understand format for the user.

[0228] Specific operation: The device processes the received display data and displays it so that the user can navigate through the virtual store and view product information.

[0229] Input: Display data (optimal route, detailed store information, product information).

[0230] Output: Information presented in a visually understandable format.

[0231] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0232] This invention is a system in which a user inputs a request via text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. Furthermore, this system combines an emotion engine that recognizes the user's emotions and optimizes information based on the user's emotions. This invention is mainly provided via smartphones and car navigation systems.

[0233] Program processing

[0234] 1. The user enters a request

[0235] Subject: User

[0236] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice. For example, they might input, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day." The user's emotional state is also recorded as a log.

[0237] 2. Submitting a Request

[0238] Subject: Terminal

[0239] The device sends the user's input request, current location information, and emotional state to the server. If the input is voice, it is converted to text and the request is prepared as data.

[0240] 3. Parsing the Request

[0241] Subject: Server

[0242] The server analyzes the received request data. First, it analyzes the request string using natural language analysis technology to extract keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." This process uses generative AI.

[0243] 4. Emotion analysis

[0244] Subject: Server

[0245] The server analyzes the received emotional state data and uses an emotion engine to determine the user's emotional state (e.g., excited, calm, tired, etc.).

[0246] 5. Preparation for Data Collection

[0247] Subject: Server

[0248] The server then prepares to collect the necessary data based on the analysis results. It also configures the system to recommend more appropriate stores and routes based on the user's emotional state. For example, if the user is tired, it will prioritize places where they can relax and routes that are quick to travel.

[0249] 6. Data Acquisition

[0250] Subject: Server

[0251] The server collects the necessary data from multiple data sources, including satellite imagery APIs and aerial imagery APIs to obtain geographical information, and store information, user reviews, and other data from open databases.

[0252] 7. Digital Twin Map Generation

[0253] Subject: Server

[0254] The server integrates the collected data and generates a digital twin map that includes relevant information based on the user's requests, and displays recommended spots and routes that take into account the user's emotional state, among other things.

[0255] 8. Providing personalized information

[0256] Subject: Server

[0257] The server then personalizes the generated digital twin map and related information for the user, building data including optimal routes, recommended stores, detailed information, and user reviews based on individual requests, location, and emotional state.

[0258] 9. Transmission of Information

[0259] Subject: Server

[0260] The server transmits the personalized information to the terminal, including map data and text information for visual display.

[0261] 10. Display of Information

[0262] Subject: Terminal

[0263] The device displays the information received from the server on a user interface. Users can navigate and make selections on the displayed map while checking recommended routes, recommended locations, detailed information, user reviews, etc. The UI itself is also optimized according to the user's emotional state, providing, for example, color schemes and interactions that promote relaxation.

[0264] Specific examples

[0265] Example 1: Searching for family-friendly restaurants

[0266] User: Enters into a smartphone app, "Tell me about a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal." If the emotion engine determines that the user is tired, it will prioritize restaurants with a relaxed atmosphere.

[0267] Device: Sends a request to the server along with its current location information.

[0268] Server: Analyzes requests and sentiments and collects necessary data.

[0269] Server: Generates a digital twin map and sends optimal routes and recommended restaurants to the device.

[0270] On your device: Displays recommended restaurants and optimal routes on a map, along with detailed information and reviews.

[0271] Example 2: Searching for tourist spots

[0272] User: Say to the car navigation system, "Tell me places I can visit within 10 minutes of my current location." If the emotion engine determines that the user is excited, it will prioritize active tourist spots.

[0273] Terminal: Sends requests to the server.

[0274] Server: Collects data based on analysis results and emotions, and obtains information on tourist spots.

[0275] Server: Generates a digital twin map and sends tourist spot and route information to the device.

[0276] Device: Displays the location of tourist attractions and the best route on a map.

[0277] Through this system, users can easily and intuitively make new discoveries and experiences, even when dealing with complex requests. Furthermore, the use of an emotion engine allows for more personalized services.

[0278] The processing flow will be explained below.

[0279] Step 1: User enters request

[0280] Subject: User

[0281] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice, for example, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day at." At this time, the system automatically records the user's emotional state based on facial recognition data and voice tone.

[0282] Step 2: Submitting the request

[0283] Subject: Terminal

[0284] The device sends the user's input request, current location information, and emotional state to the server. In the case of voice input, the voice is converted into text and the request is prepared as data.

[0285] Step 3: Parsing the request

[0286] Subject: Server

[0287] The server analyzes the received request data. It uses natural language analysis technology to extract keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." Generative AI is used in this process.

[0288] Step 4: Sentiment Analysis

[0289] Subject: Server

[0290] The server analyzes the received emotional state data and uses an emotion engine to analyze the user's emotions, for example, to determine whether the user is in an emotional state of "excited," "calm," or "tired."

[0291] Step 5: Prepare for data collection

[0292] Subject: Server

[0293] The server then prepares to collect the necessary data based on the analysis results. It also configures the system to recommend more appropriate stores and routes based on the user's emotional state. For example, if the user is feeling tired, it will prioritize places where they can relax and routes that are quick to travel.

[0294] Step 6: Get the data

[0295] Subject: Server

[0296] The server collects the necessary data from multiple data sources, including satellite imagery APIs and aerial imagery APIs to obtain geographical information, and store information, user reviews, and other data from open databases.

[0297] Step 7: Generate the digital twin map

[0298] Subject: Server

[0299] The server integrates the collected data and generates a digital twin map that includes relevant information based on the user's requests, and displays recommended spots and routes that take into account the user's emotional state, among other things.

[0300] Step 8: Provide personalized information

[0301] Subject: Server

[0302] The server then personalizes the generated digital twin map and related information for the user, building data including optimal routes, recommended stores, detailed information, user reviews, and more based on individual requests, location information, and emotional state.

[0303] Step 9: Submit your information

[0304] Subject: Server

[0305] The server transmits the personalized information to the terminal, including map data and text information for visual display.

[0306] Step 10: Viewing Information

[0307] Subject: Terminal

[0308] The device displays the information received from the server on a user interface. Users can navigate and make selections on the displayed map while checking recommended routes, recommended locations, detailed information, user reviews, etc. The UI itself is also optimized according to the user's emotional state, providing, for example, color schemes and interactions that promote relaxation.

[0309] Specific examples

[0310] Example 1: Searching for family-friendly restaurants

[0311] Step 1: User: Enters into the smartphone app, "Tell me about a popular restaurant within 5 minutes of my current location where a family of four can enjoy a half-day meal." If the emotion engine determines that the user is tired, restaurants with a relaxed atmosphere will be prioritized.

[0312] Step 2: Device: Sends a request to the server along with its current location and emotional state.

[0313] Step 3: Server: Analyze the request and emotional state and collect the necessary data.

[0314] Step 4: Server: Generates a digital twin map and sends optimal route and restaurant recommendation information to the device.

[0315] Step 5: On your device: Display recommended restaurants and optimal routes on a map, along with detailed information and reviews.

[0316] Example 2: Searching for tourist spots

[0317] Step 1: User: Say to the car navigation system, "Tell me places I can visit within 10 minutes of my current location." If the emotion engine determines that the user is excited, active tourist spots will be prioritized.

[0318] Step 2: Terminal: Sends request to server.

[0319] Step 3: Server: Collect data based on the analysis results and emotional state to obtain tourist attraction information.

[0320] Step 4: Server: Generates a digital twin map and sends tourist spot and route information to the device.

[0321] Step 5: Device: Display the location of tourist attractions and the best route on the map.

[0322] This system allows users to easily and intuitively discover new things and enjoy new experiences, even when dealing with complex requests. The introduction of an emotion engine also enables the provision of more personalized services suited to the user's emotional state.

[0323] Example 2

[0324] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0325] Conventional information provision systems provide information taking into account the user's location and requests, but do not support personalization that takes into account the user's emotional state. As a result, user satisfaction can be low, and appropriate information cannot be provided in response to requests. Furthermore, there are issues with the lack of real-timeness and accuracy when generating digital twin maps.

[0326] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing emotional state data and determining the user's emotion, a means for optimizing collected data based on the user's emotional state, and a means for collecting necessary data from multiple data sources. This makes it possible to provide personalized information that takes the user's emotional state into consideration.

[0327] "User" means any person who uses the System to enter requests and obtain information.

[0328] A "request" refers to the information or service a user requests from a system.

[0329] "Server" refers to a computer system that analyzes received requests and collects, processes, and provides the necessary information.

[0330] "Emotional state data" refers to information about a user's emotional state or psychological state.

[0331] An "emotion engine" refers to an algorithm or system that analyzes the emotional state data provided by the user and recognizes specific emotions (excitement, calmness, fatigue, etc.).

[0332] "Natural language processing technology" refers to technology that analyzes text and voice information entered by users and converts it into a form that can be understood by humans.

[0333] A "digital twin map" refers to a map that digitally reproduces real-world geographic information and related data.

[0334] "Multiple data sources" refers to different databases and APIs (application programming interfaces) used to obtain information.

[0335] "Personalization" refers to optimizing and providing information according to each user's individual characteristics, requests, and emotions.

[0336] "Terminal" refers to a device (smartphone, car navigation system, etc.) through which a user inputs requests and displays information provided by the server.

[0337] The present invention is a system in which a user inputs a request via text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. Furthermore, this system incorporates an emotion engine that recognizes the user's emotions and optimizes information based on the user's emotions. The present invention is primarily provided via smartphones and car navigation systems.

[0338] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice. For example, they might input a request like, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day at." At this time, the user's emotional state is also recorded as a log.

[0339] The device sends the user's input request, location information, and emotional state to the server. In the case of voice input, it converts the voice data into text using speech recognition software (e.g., a common speech recognition API) and sends the prepared text and additional data to the server.

[0340] The server analyzes the received request data. It uses natural language processing technology (for example, a general natural language processing API) to analyze the request string and extract keywords, which clarify the information the user is looking for.

[0341] Furthermore, the server uses an emotion engine to analyze the emotional state data and determine the user's emotions (excitement, calmness, fatigue, etc.). Based on the analysis results, the server then prepares to collect necessary data. In particular, it configures the system to recommend more appropriate stores and routes based on the user's emotional state.

[0342] The server collects the necessary data from multiple data sources (e.g., satellite imagery APIs and open databases). It calls the satellite imagery API to obtain geographic information and retrieves store information, user reviews, etc. from the open database.

[0343] The collected data is integrated to generate a digital twin map that includes relevant information based on the user's requests and displays recommended spots and routes that take into account the user's emotional state, among other things.

[0344] The server then creates a digital twin map and provides related information personalized to the user, such as optimal routes, recommended stores, detailed information, and user reviews based on individual requests, location information, and emotional state.

[0345] Finally, the server transmits the personalized information to the terminal, including map data and text information for visual display.

[0346] The device displays the information received from the server on a user interface. Users can navigate and make selections on the displayed map while checking recommended routes, recommended locations, detailed information, user reviews, etc. The UI itself is also optimized according to the user's emotional state, providing, for example, color schemes and interactions that promote relaxation.

[0347] A concrete example is searching for a restaurant that the whole family can enjoy. If a user types into a smartphone app, "Tell me a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day," and the emotion engine determines that the user is tired, the server will prioritize recommending restaurants with a relaxed atmosphere. The device sends the request and current location information to the server, which then analyzes, collects data, creates a map, and sends the final information to the device. The user can then view the restaurants and the optimal route displayed on the map.

[0348] Examples of prompt sentences include "Tell me about a popular restaurant within five minutes of our current location where a family of four can spend half a day," or "Tell me about a tourist spot within ten minutes that I can go to when I'm feeling excited." In response to such specific requests, the system of the present invention can quickly and appropriately provide personalized information.

[0349] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0350] Step 1: User enters request

[0351] Subject: User

[0352] The user opens a smartphone or car navigation application and inputs a request by text or voice. For example, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day." Along with this, the user also records their emotional state. The input data includes the content of the request and the user's emotional state.

[0353] Step 2: Submitting the request

[0354] Subject: Terminal

[0355] The device sends the input request, location information, and the user's emotional state to the server. In the case of voice input, speech recognition software is used to convert the speech to text. The input data includes the text of the request, location information, and emotional state. The output is the prepared data sent to the server.

[0356] Step 3: Parsing the request

[0357] Subject: Server

[0358] The server analyzes the received request data and uses natural language processing technology to extract keywords from the request, such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." The input is the text-converted request data, and the output is a list of extracted keywords.

[0359] Step 4: Sentiment Analysis

[0360] Subject: Server

[0361] The server uses an emotion engine to analyze the received emotional state data and determine what emotional state the user is in (e.g., excited, calm, tired, etc.). The input is the user's emotional state data, and the output is the interpreted emotional state (e.g., tired, excited, etc.).

[0362] Step 5: Prepare for data collection

[0363] Subject: Server

[0364] The server prepares to collect the necessary data based on the analysis results. It optimizes recommended stores and routes based on the user's emotional state. For example, if the user is tired, it prioritizes places where they can relax and routes that are quick to travel. The input is the extracted keywords and the interpretation of the user's emotional state, and the output is the data collection conditions.

[0365] Step 6: Get the data

[0366] Subject: Server

[0367] The server collects the necessary data from multiple data sources. Coordinate information is obtained using a satellite imagery API, and store information and user reviews are obtained from an open database. The input is the data collection conditions, and the output is the collected geographic information and related data.

[0368] Step 7: Generate the digital twin map

[0369] Subject: Server

[0370] The server integrates the collected data and generates a digital twin map. The map visually displays relevant information based on the user's request, including recommended spots and routes that take into account the user's emotional state. The input is the collected geographic information and related data, and the output is the generated digital twin map.

[0371] Step 8: Provide personalized information

[0372] Subject: Server

[0373] The server personalizes the generated digital twin map and related information for the user, building optimal routes, recommended stores, detailed information, user reviews, etc. based on individual requests, location information, and emotional state. The input is the generated digital twin map and related information, and the output is personalized information.

[0374] Step 9: Submit your information

[0375] Subject: Server

[0376] The server sends personalized information to the terminal, including map data and text information for visual display. The input is the personalized information, and the output is the data sent to the terminal.

[0377] Step 10: Viewing Information

[0378] Subject: Terminal

[0379] The device displays the information received from the server in a user interface. Users can view recommended routes, recommended places, detailed information, and user reviews on a map. The UI also provides optimal color schemes and interactions depending on the user's emotional state. The input is the received data, and the output is the displayed map and related information.

[0380] (Application example 2)

[0381] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0382] Conventional systems that provide information based on requests do not take into account the user's emotional state, making it difficult to fully improve user satisfaction.Furthermore, in food delivery services, the quality of service declines because the system does not select the store or menu that best suits the user's mood or emotion at the time.

[0383] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0384] In this invention, the server includes means for analyzing user emotions, means for providing personalized information, means for generating a digital twin map, and means for recommending optimal stores and menus to users based on data collected from multiple stores, thereby enabling optimal information provision and delivery services that take into account the user's requests and emotional state.

[0385] "User" means any person or entity that uses the System to enter a Request.

[0386] A "request" is a request or question that a user enters into a system.

[0387] "Analysis" refers to the process by which the server uses natural language processing and other technologies to understand the request and emotional state it receives and extract the necessary information.

[0388] A "digital twin map" is a map that digitally reproduces information from the physical world, and includes real-time data and its analysis results.

[0389] "Personalized information" is information that is individually optimized based on the user's requests and emotional state.

[0390] An "emotion engine" refers to software or algorithms that analyze a user's voice, text, or other input to determine their emotional state.

[0391] "Recommendation" is the process of presenting the best options to the user based on the analysis results.

[0392] "Store" refers to a place or business that offers goods or services.

[0393] A "menu" is a list of products offered by a store and their details.

[0394] "Collection" refers to the operation or process by which a server obtains the necessary data in response to a request.

[0395] This invention is a system in which a user inputs a request via text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information according to the user's emotional state.

[0396] System Program Overview

[0397] 1. The user enters a request

[0398] Users open the smartphone application and input their request by text or voice. For example, they can input a request such as, "I've been feeling stressed lately, so please recommend some sweets that will help me relax." At this time, the user's emotional state is also recorded.

[0399] 2. Submitting a Request

[0400] The device sends the user's input request, current location, and emotional state to the server, and if the input is voice, it is converted into text.

[0401] 3. Parsing the Request

[0402] The server analyzes the received request data and uses natural language analysis technology to extract keywords such as "stress," "relaxation," and "sweets."

[0403] 4. Emotion analysis

[0404] The server analyzes the user's emotional state using an emotion engine, which uses the IBM Watson Tone Analyzer API.

[0405] 5. Preparation for Data Collection

[0406] The server prepares to collect the necessary data based on the analysis results and the user's emotional state, and prioritizes settings such as locations and menus suitable for relaxation.

[0407] 6. Data Acquisition

[0408] The server uses the Google Maps API and open databases to obtain information about stores that offer relaxing sweets.

[0409] 7. Digital Twin Map Generation

[0410] The server generates a digital twin map based on the collected data, which contains relevant information based on the user's request.

[0411] 8. Providing personalized information

[0412] The server then personalizes the generated digital twin map and related information for the user, including information on the best stores, menus, routes, etc. based on the user's request, emotional state, and location.

[0413] 9. Transmission of Information

[0414] The server sends the personalized information to the terminal.

[0415] 10. Display of Information

[0416] The device displays the received information on the user interface, where users can check the recommended route, detailed store information, and menu information on the displayed map.

[0417] Hardware and software used

[0418] Smartphone app: iOS or Android

[0419] Server: AWS Lambda, Amazon RDS

[0420] Data analysis engine: Python, TensorFlow

[0421] Sentiment analysis engine: IBM Watson Tone Analyzer API

[0422] Map data: Google Maps API

[0423] Specific examples

[0424] User: "I've been feeling stressed lately, so please recommend a relaxing dessert."

[0425] App: "Here are some relaxing sweets in the neighborhood. Recommended stores are XX Cafe and △△ Restaurant."

[0426] Prompt Sentence Examples

[0427] "Generate a Python program that provides optimal store information based on user requests and sentiment data."

[0428] In this way, the present invention provides a system that provides personalized information that takes into account the user's requests and emotional state.

[0429] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0430] Step 1:

[0431] The user enters a request

[0432] Input: A user types a request into a smartphone app by text or voice. For example, "I've been feeling stressed lately, so please recommend some sweets that will help me relax."

[0433] How it works: The application records the user's emotional state in parallel, where the input data are text data and emotional data.

[0434] Output: Text and emotion data are generated and ready to be sent to the next step.

[0435] Step 2:

[0436] Submitting a Request

[0437] Input: Request text data and emotion data entered by the user.

[0438] Operation: The device sends the request text data, emotion data, and current location information to the server. If the input is voice, it is converted into text data.

[0439] Output: The request text data, emotion data, and current location information are sent to the server.

[0440] Step 3:

[0441] Parsing the request

[0442] Input: The request text data sent to the server.

[0443] How it works: The server uses natural language analysis techniques to parse the request text and extract key keywords (e.g., "stress," "relax," "sweets"), using a generative AI model for this process.

[0444] Output: The extracted keyword data is generated and ready to be sent to the next step.

[0445] Step 4:

[0446] Emotion analysis

[0447] Input: Emotion data sent to the server.

[0448] How it works: The server uses an emotion engine (e.g., IBM Watson Tone Analyzer API) to analyze the user's emotional state. In this case, it analyzes that the user wants to relax.

[0449] Output: The parsed emotion data is generated and ready to be sent to the next step.

[0450] Step 5:

[0451] Preparing for data collection

[0452] Input: Extracted keyword data and parsed sentiment data.

[0453] How it works: Based on the analysis results, the server prepares data collection tailored to the user's needs (sweets with a relaxing effect).

[0454] Output: Your data collection setup is complete and you are ready to move on to the next step.

[0455] Step 6:

[0456] Data Acquisition

[0457] Input: Collection settings based on data the server is ready to collect.

[0458] How it works: The server calls the Google Maps API and open databases to retrieve information about stores that offer relaxing sweets.

[0459] Output: The retrieved store and menu information is generated and ready to be sent to the next step.

[0460] Step 7:

[0461] Digital twin map generation

[0462] Input: Store information and menu information obtained.

[0463] How it works: The server aggregates the collected data and generates a digital twin map that includes relevant information based on the user's request.

[0464] Output: The generated digital twin map data is prepared and sent to the next step.

[0465] Step 8:

[0466] Providing personalized information

[0467] Input: Generated digital twin map data, request, emotion data, and location information.

[0468] How it works: Based on the digital twin map and related information generated by the server, it creates information optimized for the user, including the best stores, menus, and route information.

[0469] Output: Personalized user information data is generated and sent to the next step.

[0470] Step 9:

[0471] Sending information

[0472] Input: Personalized user information data.

[0473] Operation: The server generates information and sends it to the device.

[0474] Output: The appropriate information data is sent to the terminal.

[0475] Step 10:

[0476] Displaying Information

[0477] Input: Personalized information data sent from the server.

[0478] How it works: The device displays the information it receives in a user interface, allowing users to view recommended routes, detailed store and menu information on the displayed map.

[0479] Output: A visual display is provided for the user to take information and order food delivery.

[0480] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0481] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0482] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0483] [Second embodiment]

[0484] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0485] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0486] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0487] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0488] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0489] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0490] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0491] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0492] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0493] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0494] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0495] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0496] The present invention relates to a system in which a user inputs a request by text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. The present invention is primarily provided via smartphones and car navigation systems.

[0497] Program processing

[0498] 1. The user enters a request

[0499] Subject: User

[0500] Users make requests using an application on their smartphone or car navigation system. For example, they might say, "Tell me about a popular restaurant within five minutes of my current location that would be fun for a family of four for half a day." The user's request includes their current location.

[0501] 2. Submitting a Request

[0502] Subject: Terminal

[0503] The device receives the user's voice or text request and sends it to the server as request data, along with the current location information (GPS data).

[0504] 3. Parsing the Request

[0505] Subject: Server

[0506] The server analyzes the received request data using natural language processing technology to understand the user's request. For example, it extracts keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant."

[0507] 4. Preparation for Data Collection

[0508] Subject: Server

[0509] Based on the analysis results, the server prepares to collect the necessary data. Specifically, it identifies an area within a five-minute radius of the current location and sets up the system to acquire satellite images, aerial images, and open database data within that area.

[0510] 5. Data Acquisition

[0511] Subject: Server

[0512] The server retrieves the latest data from various data sources (satellite imagery API, aerial imagery API, open databases, etc.), including store information and tourist spot information that may be of interest to users.

[0513] 6. Digital Twin Map Generation

[0514] Subject: Server

[0515] The server integrates the collected data to generate a digital twin map that displays real-time geographic information and recommendations based on user requests.

[0516] 7. Providing personalized information

[0517] Subject: Server

[0518] The server generates personalized information based on the digital twin map and sends it to the device as display data, including optimal route guidance, detailed store information, and reviews.

[0519] 8. Display of Information

[0520] Subject: Terminal

[0521] The device receives the display data sent from the server and displays it in a visually easy-to-understand format for the user. Users can check the recommended route and the location of each store on a map. They can also view detailed store information and user reviews as text information.

[0522] Specific examples

[0523] Example 1: Searching for family-friendly restaurants

[0524] User: Type into a smartphone app, "Tell me about a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal."

[0525] Device: Sends a request to the server along with its current location information.

[0526] Server: Parses the request and collects the necessary data.

[0527] Server: Generates a digital twin map and sends optimal routes and recommended restaurants to the device.

[0528] On your device: Displays recommended restaurants and optimal routes on a map, along with detailed information and reviews.

[0529] Example 2: Searching for tourist spots

[0530] User: Voice-input to the car navigation system, "Tell me places I can visit within 10 minutes of my current location."

[0531] Terminal: Sends requests to the server.

[0532] Server: Collects data based on the analysis results and obtains information about tourist spots.

[0533] Server: Generates a digital twin map and sends tourist spot and route information to the device.

[0534] Device: Displays the location of tourist attractions and the best route on a map.

[0535] Through this system, users can easily and intuitively make new discoveries and have new experiences, even when dealing with complex requests.

[0536] The processing flow will be explained below.

[0537] Step 1: User enters request

[0538] Subject: User

[0539] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice. For example, they might input, "Tell me about a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day." The application obtains the user's current location along with this request.

[0540] Step 2: Submitting the request

[0541] Subject: Terminal

[0542] The device sends the request entered by the user and the current location information to the server, where voice input is converted to text and the request is prepared as data.

[0543] Step 3: Parsing the request

[0544] Subject: Server

[0545] The server analyzes the received request data. First, it analyzes the request string using natural language analysis technology to extract keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." This process uses generative AI such as ChatGPT.

[0546] Step 4: Prepare for data collection

[0547] Subject: Server

[0548] Based on the analysis results, the server prepares to collect the necessary data. For example, it identifies an area "within 5 minutes of the current location" and sets up an API call to obtain satellite images, aerial images, and open data for that area.

[0549] Step 5: Get the data

[0550] Subject: Server

[0551] The server collects the necessary data from multiple data sources, including satellite imagery APIs and aerial imagery APIs to obtain geographical information, and store information, user reviews, and other data from open databases.

[0552] Step 6: Generate the digital twin map

[0553] Subject: Server

[0554] The server integrates the collected data to generate a digital twin map, which includes relevant information based on the user's request, such as visually displaying the locations of restaurants and tourist attractions that the user can enjoy, as well as route information.

[0555] Step 7: Provide personalized information

[0556] Subject: Server

[0557] The server then personalizes the generated digital twin map and related information for the user, building data based on individual requests and location information, including optimal routes, recommended stores, detailed information, and user reviews.

[0558] Step 8: Submit your information

[0559] Subject: Server

[0560] The server transmits the personalized information to the terminal, including map data and text information for visual display.

[0561] Step 9: Viewing Information

[0562] Subject: Terminal

[0563] The device displays the information received from the server on a user interface, allowing users to move and make selections while checking recommended routes, recommended places, detailed information, user reviews, and more on the displayed map.

[0564] Through the above processing steps, the system provides users with a highly convenient and innovative map search experience.

[0565] Example 1

[0566] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0567] In modern society, there is a need to provide a wide variety of information quickly and efficiently, but there is a lack of systems that can collect appropriate information in response to user requests and provide it in a personalized format. Another issue is the difficulty of providing personalized information in real time based on the user's current location. Therefore, there is a need to develop a system that allows users to intuitively and quickly obtain the information they need.

[0568] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0569] In this invention, the server includes an analysis means, a data collection means, a digital twin map generation means, a personalized information provision means, a range specification means using current location information, a collection preparation means, and a spot display means, which enable users to efficiently obtain information within a specific range from their current location and use it in a personalized manner.

[0570] "User" means any person or agent who uses the System to request information.

[0571] "Request" means a request for information entered by a User into a System, including text input or voice input.

[0572] "Device" refers to the device used by a user to enter requests and obtain information, including smartphones and car navigation systems.

[0573] "Server" refers to the central computer in the system that receives user requests, analyzes them, collects the necessary data, and provides the results to the user.

[0574] "Data Sources" refers to the various databases and APIs that the server accesses to gather information, including satellite imagery APIs and open databases.

[0575] A "digital twin map" refers to a real-time geographic information system that is generated based on collected data.

[0576] "Location information" refers to location data that indicates the user's current location, primarily GPS data.

[0577] "Analysis Method" refers to the technology used by the server to understand the content of the user's request, including natural language processing technology.

[0578] "Data collection means" refers to the methods and processes by which the server obtains the required data from various data sources.

[0579] "Digital twin map generation means" refers to the method or process for integrating data collected by the server and creating a digital twin map.

[0580] "Personalized information provision means" refers to a method or process for generating and providing personalized information based on a user's request.

[0581] "Area specification means using current location information" refers to a method or process by which a server specifies a specified range based on the user's current location information.

[0582] "Collection preparation means" refers to the method or process by which the server configures and prepares itself to collect the required data.

[0583] "Spot display means" refers to a method or process by which the server displays spots based on a user's request on the digital twin map.

[0584] The present invention is a system in which a user inputs a request, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. Specific embodiments of the system are described below.

[0585] Users can use their smartphone or car navigation system to input requests by voice or text into a dedicated application. For example, a user might request, "Tell me about some popular restaurants within five minutes of my current location that a family of four can enjoy for half a day."

[0586] The device sends the input request and current location information (GPS data) to the server using a data communication protocol such as an HTTP POST request.

[0587] The server uses natural language processing technology to analyze the received request, specifically leveraging a natural language processing API (e.g., Google Cloud Natural Language API) to extract keywords and important information from the request.

[0588] Once the request is analyzed, the server will use the user's current location information to begin collecting data within the specified area. For example, it will use the Google Maps API or OpenStreetMap API to prepare to obtain the necessary information (store information, tourist spot information, etc.).

[0589] The server then uses satellite and aerial imagery APIs to retrieve data, including up-to-date geographic information, and combines this collected data to generate a digital twin map.

[0590] The generated digital twin map displays recommended spots and optimal route guidance based on user requests, and also includes detailed store information and user reviews for personalized information.

[0591] Finally, the server sends the generated information to the device as display data, which is then presented to the user in a visually easy-to-understand format, allowing the user to check the recommended route, the location of each spot, and detailed information on the map.

[0592] Specific examples

[0593] Example 1: Searching for family-friendly restaurants

[0594] User: Type into a smartphone app, "Tell me about a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal."

[0595] Terminal: Sends the request data to the server along with the current location information.

[0596] Server: Analyzes the request using natural language processing technology and collects the necessary data.

[0597] Server: Collects restaurant information from Google Maps API and other data sources and generates a digital twin map.

[0598] Server: Sends optimal route and restaurant information to the device.

[0599] Device: Displays restaurant recommendations and optimal routes on an interactive map, along with detailed information and reviews.

[0600] Prompt Sentence Examples

[0601] "Tell me a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal."

[0602] This system utilizes natural language processing technology and various data collection APIs to provide users with intuitive and personalized information, allowing them to easily and efficiently obtain information that meets their needs.

[0603] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0604] Step 1:

[0605] When a user uses a smartphone or car navigation system to input a request within an application, they can use text or voice. For example, they might say, "Tell me about a popular restaurant within five minutes of my current location that would be fun for a family of four for half a day." This input becomes input data for the device. The device converts the voice data into text data and processes it as request data.

[0606] Step 2:

[0607] The device sends the request data entered by the user and current location information (GPS data) to the server. This is done using an HTTP POST request. The input data is the request data (text request) and current location information (GPS data), and the output data is the request payload sent to the server.

[0608] Step 3:

[0609] The server analyzes the received request data. Natural language processing technology is used for the analysis. Specifically, a natural language processing API (e.g., a natural language processing cloud API) is used to extract keywords and important information from the request data. The input data is the request payload, and the output data is the analysis results (e.g., keywords "within 5 minutes," "family of 4," "half day," "popular restaurant").

[0610] Step 4:

[0611] The server prepares for data collection based on the analysis results. It uses the current location information to identify an area within a specified range (e.g., within 5 minutes) and configures data collection settings within that area. This preparation includes identifying the area using a map API (e.g., Map Cloud API) and configuring data collection parameters. The input data are the analysis results and current location information, and the output data is the configuration information for which collection preparation is complete.

[0612] Step 5:

[0613] The server obtains the necessary data from various data sources (e.g., satellite imagery API, aerial imagery API, open databases, etc.). This includes real-time updated geographic information and information requested by users (e.g., store information, tourist spot information, etc.). The input data is the setup information for collection preparation, and the output data is the various data obtained.

[0614] Step 6:

[0615] The server integrates the acquired data to generate a digital twin map. For example, it uses geographic information system (GIS) technology to draw a map based on satellite and aerial images and plot recommended spots based on user requests. The input data is the acquired data, and the output data is the digital twin map.

[0616] Step 7:

[0617] The server generates personalized information for the user based on the generated digital twin map. This includes optimal route guidance, detailed store information, reviews, and more. The input data is the digital twin map and the user's requested information, and the output data is personalized information. This is then sent to the device as display data.

[0618] Step 8:

[0619] The terminal receives the display data sent from the server and displays it to the user in a visually easy-to-understand format. The user can check the recommended route and the location and detailed information of each spot on an interactive map. Detailed store information and user reviews are also displayed as text information. The input data is the display data, and the output data is the display screen for the user.

[0620] (Application example 1)

[0621] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0622] Conventional map information systems make it difficult for users to obtain detailed information about virtual stores. They also lack the means to provide personalized information based on the user's current location in real time. As a result, it is difficult for users to intuitively obtain specific store or product information, or to navigate within the virtual space.

[0623] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0624] In this invention, the server includes a means for a user to input a request for a virtual store using a smartphone or smart glasses, a means for the server to provide detailed information about the virtual store based on a digital twin map, and a means for navigating the virtual store and viewing product information, allowing the user to obtain detailed, personalized store and product information in real time and achieving intuitive navigation within the virtual store.

[0625] 1. "Means for users to input requests by text or voice" refers to devices or software that allow users to input their requests or questions by text or voice.

[0626] 2. "Means for sending a request to the server" refers to the communications means for transmitting text or voice data entered by the user to the server via the Internet.

[0627] 3. "Means for analyzing requests on the server" refers to the processes and technologies used by the server to analyze the request data received and understand its contents.

[0628] 4. "Means for collecting satellite imagery, aerial imagery, and open data" refers to the API or database access method that the server uses to collect the necessary satellite imagery, aerial imagery, and open data based on the results of request analysis.

[0629] 5. "Means for generating digital twin maps" refers to technologies that generate virtual maps based on collected data, creating digital models that function as twins of the physical world.

[0630] 6. "Means for providing personalized information" refers to methods and technologies for providing information optimized for individual users based on their requests and current location information.

[0631] 7. "Means for transmitting and displaying generated information to a terminal" refers to the technology and devices for transmitting personalized information generated by the server to a user's terminal and visually displaying it.

[0632] 8. "Means for users to input requests about a virtual store using a smartphone or smart glasses" refers to a method and device for users to request information about a virtual store using a smartphone or smart glasses.

[0633] 9. "Means for the server to provide detailed information about the virtual store based on the digital twin map" refers to methods and technologies for utilizing the digital twin map to provide users with detailed information about the virtual store.

[0634] 10. "Means for navigating the virtual store and viewing product information" means the interface or technology that allows a user to virtually navigate the virtual store and view detailed product information.

[0635] The present invention is a system in which a user inputs a request for a virtual store using a smartphone or smart glasses, and detailed information is provided based on that request using a digital twin.

[0636] The system uses the following hardware and software:

[0637] Smartphones and smart glasses: These are devices through which users input requests.

[0638] GPS module: Used to obtain the user's current location information.

[0639] Server: This is the central system that analyzes request data and collects and processes the necessary data. It uses various APIs (satellite imagery API, aerial imagery API, open database), natural language processing engines (e.g., SpaCy), and database management systems (e.g., MySQL).

[0640] Communication module: Transmits data between the device and the server via the Internet.

[0641] The program processing proceeds as follows:

[0642] Using a smartphone or smart glasses, a user can input a request by voice or text, such as "Tell me what fashion items are popular at the nearest department store right now." This request also includes the user's current location information. The device then sends the request data and current location information to the server.

[0643] When the server receives the request data, it analyzes the request using natural language processing technology and extracts keywords. Based on the results of this analysis, the server prepares to collect the necessary data. Specifically, it obtains the latest data from satellite images, aerial images, and open databases.

[0644] The server integrates the collected data and generates a digital twin map, which includes store information and product details that the user may be interested in. The generated information is sent from the server to the user's device and displayed visually.

[0645] For example, if a user requests, "Tell me the popular products on sale at the nearest shopping mall right now," the server analyzes the request and generates a digital twin map of the shopping mall. The generated map includes detailed information about popular products on sale and stores. The user can also navigate within the virtual store and view detailed product information.

[0646] Here are some examples of prompts for generative AI models:

[0647] "Please tell me information about popular fashion items at the department store nearest to my current location. Details should include the item name, image, price, store location, etc."

[0648] This allows users to receive real-time personalized information and intuitive navigation within the virtual store.

[0649] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0650] Step 1:

[0651] The user uses a smartphone or smart glasses to enter a request.

[0652] Specific operation: The user inputs a request via text or voice, for example, "Tell me what fashion items are popular at the nearest department store right now."

[0653] Input: User request voice or text data, current location information.

[0654] Output: Request data and current location information.

[0655] Step 2:

[0656] The device sends the request data and current location information to the server.

[0657] Specific operation: The terminal sends the request data entered by the user and the current location information obtained using the GPS module to the server.

[0658] Input: Request data, current location information.

[0659] Output: Request data sent to the server and current location information.

[0660] Step 3:

[0661] The server parses the request data.

[0662] Specific operation: The server analyzes the received request data using a natural language processing engine (e.g., SpaCy) and extracts keywords such as "department store," "fashion item," and "popular."

[0663] Input: Request data.

[0664] Output: Analysis results (extracted keywords).

[0665] Step 4:

[0666] Prepare the server to collect the required data.

[0667] Specific operation: Based on the analysis results, the server generates a query to obtain information from the required data sources (satellite imagery API, aerial imagery API, open database, etc.).

[0668] Input: Analysis results (extracted keywords).

[0669] Output: Data collection query.

[0670] Step 5:

[0671] The server retrieves data from various data sources.

[0672] Specific operation: The server uses the generated query to collect satellite images, aerial images, store information, and product information from data sources.

[0673] Input: Data collection query.

[0674] Output: Collected data (satellite images, aerial images, open data, store information, product information).

[0675] Step 6:

[0676] The server generates a digital twin map.

[0677] How it works: The server integrates the collected data and generates a personalized digital twin map based on the user's request.

[0678] Input: Collected data (satellite images, aerial images, open data, store information, product information).

[0679] Output: Digital twin map.

[0680] Step 7:

[0681] The server generates personalized information based on the digital twin map and sends it to the device.

[0682] Specific operation: Based on the generated digital twin map, the server generates the optimal route, detailed store information, and product information to suggest to the user, and sends this to the terminal.

[0683] Input: Digital twin map.

[0684] Output: Display data (optimal route, detailed store information, product information).

[0685] Step 8:

[0686] The terminal receives the display data sent from the server and displays it in a visually easy-to-understand format for the user.

[0687] Specific operation: The device processes the received display data and displays it so that the user can navigate through the virtual store and view product information.

[0688] Input: Display data (optimal route, detailed store information, product information).

[0689] Output: Information presented in a visually understandable format.

[0690] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0691] This invention is a system in which a user inputs a request via text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. Furthermore, this system combines an emotion engine that recognizes the user's emotions and optimizes information based on the user's emotions. This invention is mainly provided via smartphones and car navigation systems.

[0692] Program processing

[0693] 1. The user enters a request

[0694] Subject: User

[0695] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice. For example, they might input, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day." The user's emotional state is also recorded as a log.

[0696] 2. Submitting a Request

[0697] Subject: Terminal

[0698] The device sends the user's input request, current location information, and emotional state to the server. If the input is voice, it is converted to text and the request is prepared as data.

[0699] 3. Parsing the Request

[0700] Subject: Server

[0701] The server analyzes the received request data. First, it analyzes the request string using natural language analysis technology to extract keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." This process uses generative AI.

[0702] 4. Emotion analysis

[0703] Subject: Server

[0704] The server analyzes the received emotional state data and uses an emotion engine to determine the user's emotional state (e.g., excited, calm, tired, etc.).

[0705] 5. Preparation for Data Collection

[0706] Subject: Server

[0707] The server then prepares to collect the necessary data based on the analysis results. It also configures the system to recommend more appropriate stores and routes based on the user's emotional state. For example, if the user is tired, it will prioritize places where they can relax and routes that are quick to travel.

[0708] 6. Data Acquisition

[0709] Subject: Server

[0710] The server collects the necessary data from multiple data sources, including satellite imagery APIs and aerial imagery APIs to obtain geographical information, and store information, user reviews, and other data from open databases.

[0711] 7. Digital Twin Map Generation

[0712] Subject: Server

[0713] The server integrates the collected data and generates a digital twin map that includes relevant information based on the user's requests, and displays recommended spots and routes that take into account the user's emotional state, among other things.

[0714] 8. Providing personalized information

[0715] Subject: Server

[0716] The server then personalizes the generated digital twin map and related information for the user, building data including optimal routes, recommended stores, detailed information, and user reviews based on individual requests, location, and emotional state.

[0717] 9. Transmission of Information

[0718] Subject: Server

[0719] The server transmits the personalized information to the terminal, including map data and text information for visual display.

[0720] 10. Display of Information

[0721] Subject: Terminal

[0722] The device displays the information received from the server on a user interface. Users can navigate and make selections on the displayed map while checking recommended routes, recommended locations, detailed information, user reviews, etc. The UI itself is also optimized according to the user's emotional state, providing, for example, color schemes and interactions that promote relaxation.

[0723] Specific examples

[0724] Example 1: Searching for family-friendly restaurants

[0725] User: Enters into a smartphone app, "Tell me about a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal." If the emotion engine determines that the user is tired, it will prioritize restaurants with a relaxed atmosphere.

[0726] Device: Sends a request to the server along with its current location information.

[0727] Server: Analyzes requests and sentiments and collects necessary data.

[0728] Server: Generates a digital twin map and sends optimal routes and recommended restaurants to the device.

[0729] On your device: Displays recommended restaurants and optimal routes on a map, along with detailed information and reviews.

[0730] Example 2: Searching for tourist spots

[0731] User: Say to the car navigation system, "Tell me places I can visit within 10 minutes of my current location." If the emotion engine determines that the user is excited, it will prioritize active tourist spots.

[0732] Terminal: Sends requests to the server.

[0733] Server: Collects data based on analysis results and emotions, and obtains information on tourist spots.

[0734] Server: Generates a digital twin map and sends tourist spot and route information to the device.

[0735] Device: Displays the location of tourist attractions and the best route on a map.

[0736] Through this system, users can easily and intuitively make new discoveries and experiences, even when dealing with complex requests. Furthermore, the use of an emotion engine allows for more personalized services.

[0737] The processing flow will be explained below.

[0738] Step 1: User enters request

[0739] Subject: User

[0740] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice, for example, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day at." At this time, the system automatically records the user's emotional state based on facial recognition data and voice tone.

[0741] Step 2: Submitting the request

[0742] Subject: Terminal

[0743] The device sends the user's input request, current location information, and emotional state to the server. In the case of voice input, the voice is converted into text and the request is prepared as data.

[0744] Step 3: Parsing the request

[0745] Subject: Server

[0746] The server analyzes the received request data. It uses natural language analysis technology to extract keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." Generative AI is used in this process.

[0747] Step 4: Sentiment Analysis

[0748] Subject: Server

[0749] The server analyzes the received emotional state data and uses an emotion engine to analyze the user's emotions, for example, to determine whether the user is in an emotional state of "excited," "calm," or "tired."

[0750] Step 5: Prepare for data collection

[0751] Subject: Server

[0752] The server then prepares to collect the necessary data based on the analysis results. It also configures the system to recommend more appropriate stores and routes based on the user's emotional state. For example, if the user is feeling tired, it will prioritize places where they can relax and routes that are quick to travel.

[0753] Step 6: Get the data

[0754] Subject: Server

[0755] The server collects the necessary data from multiple data sources, including satellite imagery APIs and aerial imagery APIs to obtain geographical information, and store information, user reviews, and other data from open databases.

[0756] Step 7: Generate the digital twin map

[0757] Subject: Server

[0758] The server integrates the collected data and generates a digital twin map that includes relevant information based on the user's requests, and displays recommended spots and routes that take into account the user's emotional state, among other things.

[0759] Step 8: Provide personalized information

[0760] Subject: Server

[0761] The server then personalizes the generated digital twin map and related information for the user, building data including optimal routes, recommended stores, detailed information, user reviews, and more based on individual requests, location information, and emotional state.

[0762] Step 9: Submit your information

[0763] Subject: Server

[0764] The server transmits the personalized information to the terminal, including map data and text information for visual display.

[0765] Step 10: Viewing Information

[0766] Subject: Terminal

[0767] The device displays the information received from the server on a user interface. Users can navigate and make selections on the displayed map while checking recommended routes, recommended locations, detailed information, user reviews, etc. The UI itself is also optimized according to the user's emotional state, providing, for example, color schemes and interactions that promote relaxation.

[0768] Specific examples

[0769] Example 1: Searching for family-friendly restaurants

[0770] Step 1: User: Enters into the smartphone app, "Tell me about a popular restaurant within 5 minutes of my current location where a family of four can enjoy a half-day meal." If the emotion engine determines that the user is tired, restaurants with a relaxed atmosphere will be prioritized.

[0771] Step 2: Device: Sends a request to the server along with its current location and emotional state.

[0772] Step 3: Server: Analyze the request and emotional state and collect the necessary data.

[0773] Step 4: Server: Generates a digital twin map and sends optimal route and restaurant recommendation information to the device.

[0774] Step 5: On your device: Display recommended restaurants and optimal routes on a map, along with detailed information and reviews.

[0775] Example 2: Searching for tourist spots

[0776] Step 1: User: Say to the car navigation system, "Tell me places I can visit within 10 minutes of my current location." If the emotion engine determines that the user is excited, active tourist spots will be prioritized.

[0777] Step 2: Terminal: Sends request to server.

[0778] Step 3: Server: Collect data based on the analysis results and emotional state to obtain tourist attraction information.

[0779] Step 4: Server: Generates a digital twin map and sends tourist spot and route information to the device.

[0780] Step 5: Device: Display the location of tourist attractions and the best route on the map.

[0781] This system allows users to easily and intuitively discover new things and enjoy new experiences, even when dealing with complex requests. The introduction of an emotion engine also enables the provision of more personalized services suited to the user's emotional state.

[0782] Example 2

[0783] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0784] Conventional information provision systems provide information taking into account the user's location and requests, but do not support personalization that takes into account the user's emotional state. As a result, user satisfaction can be low, and appropriate information cannot be provided in response to requests. Furthermore, there are issues with the lack of real-timeness and accuracy when generating digital twin maps.

[0785] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing emotional state data and determining the user's emotion, a means for optimizing collected data based on the user's emotional state, and a means for collecting necessary data from multiple data sources. This makes it possible to provide personalized information that takes the user's emotional state into consideration.

[0786] "User" means any person who uses the System to enter requests and obtain information.

[0787] A "request" refers to the information or service a user requests from a system.

[0788] "Server" refers to a computer system that analyzes received requests and collects, processes, and provides the necessary information.

[0789] "Emotional state data" refers to information about a user's emotional state or psychological state.

[0790] An "emotion engine" refers to an algorithm or system that analyzes the emotional state data provided by the user and recognizes specific emotions (excitement, calmness, fatigue, etc.).

[0791] "Natural language processing technology" refers to technology that analyzes text and voice information entered by users and converts it into a form that can be understood by humans.

[0792] A "digital twin map" refers to a map that digitally reproduces real-world geographic information and related data.

[0793] "Multiple data sources" refers to different databases and APIs (application programming interfaces) used to obtain information.

[0794] "Personalization" refers to optimizing and providing information according to each user's individual characteristics, requests, and emotions.

[0795] "Terminal" refers to a device (smartphone, car navigation system, etc.) through which a user inputs requests and displays information provided by the server.

[0796] The present invention is a system in which a user inputs a request via text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. Furthermore, this system incorporates an emotion engine that recognizes the user's emotions and optimizes information based on the user's emotions. The present invention is primarily provided via smartphones and car navigation systems.

[0797] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice. For example, they might input a request like, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day at." At this time, the user's emotional state is also recorded as a log.

[0798] The device sends the user's input request, location information, and emotional state to the server. In the case of voice input, it converts the voice data into text using speech recognition software (e.g., a common speech recognition API) and sends the prepared text and additional data to the server.

[0799] The server analyzes the received request data. It uses natural language processing technology (for example, a general natural language processing API) to analyze the request string and extract keywords, which clarify the information the user is looking for.

[0800] Furthermore, the server uses an emotion engine to analyze the emotional state data and determine the user's emotions (excitement, calmness, fatigue, etc.). Based on the analysis results, the server then prepares to collect necessary data. In particular, it configures the system to recommend more appropriate stores and routes based on the user's emotional state.

[0801] The server collects the necessary data from multiple data sources (e.g., satellite imagery APIs and open databases). It calls the satellite imagery API to obtain geographic information and retrieves store information, user reviews, etc. from the open database.

[0802] The collected data is integrated to generate a digital twin map that includes relevant information based on the user's requests and displays recommended spots and routes that take into account the user's emotional state, among other things.

[0803] The server then creates a digital twin map and provides related information personalized to the user, such as optimal routes, recommended stores, detailed information, and user reviews based on individual requests, location information, and emotional state.

[0804] Finally, the server transmits the personalized information to the terminal, including map data and text information for visual display.

[0805] The device displays the information received from the server on a user interface. Users can navigate and make selections on the displayed map while checking recommended routes, recommended locations, detailed information, user reviews, etc. The UI itself is also optimized according to the user's emotional state, providing, for example, color schemes and interactions that promote relaxation.

[0806] A concrete example is searching for a restaurant that the whole family can enjoy. If a user types into a smartphone app, "Tell me a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day," and the emotion engine determines that the user is tired, the server will prioritize recommending restaurants with a relaxed atmosphere. The device sends the request and current location information to the server, which then analyzes, collects data, creates a map, and sends the final information to the device. The user can then view the restaurants and the optimal route displayed on the map.

[0807] Examples of prompt sentences include "Tell me about a popular restaurant within five minutes of our current location where a family of four can spend half a day," or "Tell me about a tourist spot within ten minutes that I can go to when I'm feeling excited." In response to such specific requests, the system of the present invention can quickly and appropriately provide personalized information.

[0808] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0809] Step 1: User enters request

[0810] Subject: User

[0811] The user opens a smartphone or car navigation application and inputs a request by text or voice. For example, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day." Along with this, the user also records their emotional state. The input data includes the content of the request and the user's emotional state.

[0812] Step 2: Submitting the request

[0813] Subject: Terminal

[0814] The device sends the input request, location information, and the user's emotional state to the server. In the case of voice input, speech recognition software is used to convert the speech to text. The input data includes the text of the request, location information, and emotional state. The output is the prepared data sent to the server.

[0815] Step 3: Parsing the request

[0816] Subject: Server

[0817] The server analyzes the received request data and uses natural language processing technology to extract keywords from the request, such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." The input is the text-converted request data, and the output is a list of extracted keywords.

[0818] Step 4: Sentiment Analysis

[0819] Subject: Server

[0820] The server uses an emotion engine to analyze the received emotional state data and determine what emotional state the user is in (e.g., excited, calm, tired, etc.). The input is the user's emotional state data, and the output is the interpreted emotional state (e.g., tired, excited, etc.).

[0821] Step 5: Prepare for data collection

[0822] Subject: Server

[0823] The server prepares to collect the necessary data based on the analysis results. It optimizes recommended stores and routes based on the user's emotional state. For example, if the user is tired, it prioritizes places where they can relax and routes that are quick to travel. The input is the extracted keywords and the interpretation of the user's emotional state, and the output is the data collection conditions.

[0824] Step 6: Get the data

[0825] Subject: Server

[0826] The server collects the necessary data from multiple data sources. Coordinate information is obtained using a satellite imagery API, and store information and user reviews are obtained from an open database. The input is the data collection conditions, and the output is the collected geographic information and related data.

[0827] Step 7: Generate the digital twin map

[0828] Subject: Server

[0829] The server integrates the collected data and generates a digital twin map. The map visually displays relevant information based on the user's request, including recommended spots and routes that take into account the user's emotional state. The input is the collected geographic information and related data, and the output is the generated digital twin map.

[0830] Step 8: Provide personalized information

[0831] Subject: Server

[0832] The server personalizes the generated digital twin map and related information for the user, building optimal routes, recommended stores, detailed information, user reviews, etc. based on individual requests, location information, and emotional state. The input is the generated digital twin map and related information, and the output is personalized information.

[0833] Step 9: Submit your information

[0834] Subject: Server

[0835] The server sends personalized information to the terminal, including map data and text information for visual display. The input is the personalized information, and the output is the data sent to the terminal.

[0836] Step 10: Viewing Information

[0837] Subject: Terminal

[0838] The device displays the information received from the server in a user interface. Users can view recommended routes, recommended places, detailed information, and user reviews on a map. The UI also provides optimal color schemes and interactions depending on the user's emotional state. The input is the received data, and the output is the displayed map and related information.

[0839] (Application example 2)

[0840] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0841] Conventional systems that provide information based on requests do not take into account the user's emotional state, making it difficult to fully improve user satisfaction.Furthermore, in food delivery services, the quality of service declines because the system does not select the store or menu that best suits the user's mood or emotion at the time.

[0842] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0843] In this invention, the server includes means for analyzing user emotions, means for providing personalized information, means for generating a digital twin map, and means for recommending optimal stores and menus to users based on data collected from multiple stores, thereby enabling optimal information provision and delivery services that take into account the user's requests and emotional state.

[0844] "User" means any person or entity that uses the System to enter a Request.

[0845] A "request" is a request or question that a user enters into a system.

[0846] "Analysis" refers to the process by which the server uses natural language processing and other technologies to understand the request and emotional state it receives and extract the necessary information.

[0847] A "digital twin map" is a map that digitally reproduces information from the physical world, and includes real-time data and its analysis results.

[0848] "Personalized information" is information that is individually optimized based on the user's requests and emotional state.

[0849] An "emotion engine" refers to software or algorithms that analyze a user's voice, text, or other input to determine their emotional state.

[0850] "Recommendation" is the process of presenting the best options to the user based on the analysis results.

[0851] "Store" refers to a place or business that offers goods or services.

[0852] A "menu" is a list of products offered by a store and their details.

[0853] "Collection" refers to the operation or process by which a server obtains the necessary data in response to a request.

[0854] This invention is a system in which a user inputs a request via text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information according to the user's emotional state.

[0855] System Program Overview

[0856] 1. The user enters a request

[0857] Users open the smartphone application and input their request by text or voice. For example, they can input a request such as, "I've been feeling stressed lately, so please recommend some sweets that will help me relax." At this time, the user's emotional state is also recorded.

[0858] 2. Submitting a Request

[0859] The device sends the user's input request, current location, and emotional state to the server, and if the input is voice, it is converted into text.

[0860] 3. Parsing the Request

[0861] The server analyzes the received request data and uses natural language analysis technology to extract keywords such as "stress," "relaxation," and "sweets."

[0862] 4. Emotion analysis

[0863] The server analyzes the user's emotional state using an emotion engine, which uses the IBM Watson Tone Analyzer API.

[0864] 5. Preparation for Data Collection

[0865] The server prepares to collect the necessary data based on the analysis results and the user's emotional state, and prioritizes settings such as locations and menus suitable for relaxation.

[0866] 6. Data Acquisition

[0867] The server uses the Google Maps API and open databases to obtain information about stores that offer relaxing sweets.

[0868] 7. Digital Twin Map Generation

[0869] The server generates a digital twin map based on the collected data, which contains relevant information based on the user's request.

[0870] 8. Providing personalized information

[0871] The server then personalizes the generated digital twin map and related information for the user, including information on the best stores, menus, routes, etc. based on the user's request, emotional state, and location.

[0872] 9. Transmission of Information

[0873] The server sends the personalized information to the terminal.

[0874] 10. Display of Information

[0875] The device displays the received information on the user interface, where users can check the recommended route, detailed store information, and menu information on the displayed map.

[0876] Hardware and software used

[0877] Smartphone app: iOS or Android

[0878] Server: AWS Lambda, Amazon RDS

[0879] Data analysis engine: Python, TensorFlow

[0880] Sentiment analysis engine: IBM Watson Tone Analyzer API

[0881] Map data: Google Maps API

[0882] Specific examples

[0883] User: "I've been feeling stressed lately, so please recommend a relaxing dessert."

[0884] App: "Here are some relaxing sweets in the neighborhood. Recommended stores are XX Cafe and △△ Restaurant."

[0885] Prompt Sentence Examples

[0886] "Generate a Python program that provides optimal store information based on user requests and sentiment data."

[0887] In this way, the present invention provides a system that provides personalized information that takes into account the user's requests and emotional state.

[0888] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0889] Step 1:

[0890] The user enters a request

[0891] Input: A user types a request into a smartphone app by text or voice. For example, "I've been feeling stressed lately, so please recommend some sweets that will help me relax."

[0892] How it works: The application records the user's emotional state in parallel, where the input data are text data and emotional data.

[0893] Output: Text and emotion data are generated and ready to be sent to the next step.

[0894] Step 2:

[0895] Submitting a Request

[0896] Input: Request text data and emotion data entered by the user.

[0897] Operation: The device sends the request text data, emotion data, and current location information to the server. If the input is voice, it is converted into text data.

[0898] Output: The request text data, emotion data, and current location information are sent to the server.

[0899] Step 3:

[0900] Parsing the request

[0901] Input: The request text data sent to the server.

[0902] How it works: The server uses natural language analysis techniques to parse the request text and extract key keywords (e.g., "stress," "relax," "sweets"), using a generative AI model for this process.

[0903] Output: The extracted keyword data is generated and ready to be sent to the next step.

[0904] Step 4:

[0905] Emotion analysis

[0906] Input: Emotion data sent to the server.

[0907] How it works: The server uses an emotion engine (e.g., IBM Watson Tone Analyzer API) to analyze the user's emotional state. In this case, it analyzes that the user wants to relax.

[0908] Output: The parsed emotion data is generated and ready to be sent to the next step.

[0909] Step 5:

[0910] Preparing for data collection

[0911] Input: Extracted keyword data and parsed sentiment data.

[0912] How it works: Based on the analysis results, the server prepares data collection tailored to the user's needs (sweets with a relaxing effect).

[0913] Output: Your data collection setup is complete and you are ready to move on to the next step.

[0914] Step 6:

[0915] Data Acquisition

[0916] Input: Collection settings based on data the server is ready to collect.

[0917] How it works: The server calls the Google Maps API and open databases to retrieve information about stores that offer relaxing sweets.

[0918] Output: The retrieved store and menu information is generated and ready to be sent to the next step.

[0919] Step 7:

[0920] Digital twin map generation

[0921] Input: Store information and menu information obtained.

[0922] How it works: The server aggregates the collected data and generates a digital twin map that includes relevant information based on the user's request.

[0923] Output: The generated digital twin map data is prepared and sent to the next step.

[0924] Step 8:

[0925] Providing personalized information

[0926] Input: Generated digital twin map data, request, emotion data, and location information.

[0927] How it works: Based on the digital twin map and related information generated by the server, it creates information optimized for the user, including the best stores, menus, and route information.

[0928] Output: Personalized user information data is generated and sent to the next step.

[0929] Step 9:

[0930] Sending information

[0931] Input: Personalized user information data.

[0932] Operation: The server generates information and sends it to the device.

[0933] Output: The appropriate information data is sent to the terminal.

[0934] Step 10:

[0935] Displaying Information

[0936] Input: Personalized information data sent from the server.

[0937] How it works: The device displays the information it receives in a user interface, allowing users to view recommended routes, detailed store and menu information on the displayed map.

[0938] Output: A visual display is provided for the user to take information and order food delivery.

[0939] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0940] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0941] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0942] [Third embodiment]

[0943] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0944] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0945] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0946] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0947] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0948] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0949] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0950] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0951] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0952] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0953] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0954] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0955] The present invention relates to a system in which a user inputs a request by text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. The present invention is primarily provided via smartphones and car navigation systems.

[0956] Program processing

[0957] 1. The user enters a request

[0958] Subject: User

[0959] Users make requests using an application on their smartphone or car navigation system. For example, they might say, "Tell me about a popular restaurant within five minutes of my current location that would be fun for a family of four for half a day." The user's request includes their current location.

[0960] 2. Submitting a Request

[0961] Subject: Terminal

[0962] The device receives the user's voice or text request and sends it to the server as request data, along with the current location information (GPS data).

[0963] 3. Parsing the Request

[0964] Subject: Server

[0965] The server analyzes the received request data using natural language processing technology to understand the user's request. For example, it extracts keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant."

[0966] 4. Preparation for Data Collection

[0967] Subject: Server

[0968] Based on the analysis results, the server prepares to collect the necessary data. Specifically, it identifies an area within a five-minute radius of the current location and sets up the system to acquire satellite images, aerial images, and open database data within that area.

[0969] 5. Data Acquisition

[0970] Subject: Server

[0971] The server retrieves the latest data from various data sources (satellite imagery API, aerial imagery API, open databases, etc.), including store information and tourist spot information that may be of interest to users.

[0972] 6. Digital Twin Map Generation

[0973] Subject: Server

[0974] The server integrates the collected data to generate a digital twin map that displays real-time geographic information and recommendations based on user requests.

[0975] 7. Providing personalized information

[0976] Subject: Server

[0977] The server generates personalized information based on the digital twin map and sends it to the device as display data, including optimal route guidance, detailed store information, and reviews.

[0978] 8. Display of Information

[0979] Subject: Terminal

[0980] The device receives the display data sent from the server and displays it in a visually easy-to-understand format for the user. Users can check the recommended route and the location of each store on a map. They can also view detailed store information and user reviews as text information.

[0981] Specific examples

[0982] Example 1: Searching for family-friendly restaurants

[0983] User: Type into a smartphone app, "Tell me about a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal."

[0984] Device: Sends a request to the server along with its current location information.

[0985] Server: Parses the request and collects the necessary data.

[0986] Server: Generates a digital twin map and sends optimal routes and recommended restaurants to the device.

[0987] On your device: Displays recommended restaurants and optimal routes on a map, along with detailed information and reviews.

[0988] Example 2: Searching for tourist spots

[0989] User: Voice-input to the car navigation system, "Tell me places I can visit within 10 minutes of my current location."

[0990] Terminal: Sends requests to the server.

[0991] Server: Collects data based on the analysis results and obtains information about tourist spots.

[0992] Server: Generates a digital twin map and sends tourist spot and route information to the device.

[0993] Device: Displays the location of tourist attractions and the best route on a map.

[0994] Through this system, users can easily and intuitively make new discoveries and have new experiences, even when dealing with complex requests.

[0995] The processing flow will be explained below.

[0996] Step 1: User enters request

[0997] Subject: User

[0998] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice. For example, they might input, "Tell me about a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day." The application obtains the user's current location along with this request.

[0999] Step 2: Submitting the request

[1000] Subject: Terminal

[1001] The device sends the request entered by the user and the current location information to the server, where voice input is converted to text and the request is prepared as data.

[1002] Step 3: Parsing the request

[1003] Subject: Server

[1004] The server analyzes the received request data. First, it analyzes the request string using natural language analysis technology to extract keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." This process uses generative AI such as ChatGPT.

[1005] Step 4: Prepare for data collection

[1006] Subject: Server

[1007] Based on the analysis results, the server prepares to collect the necessary data. For example, it identifies an area "within 5 minutes of the current location" and sets up an API call to obtain satellite images, aerial images, and open data for that area.

[1008] Step 5: Get the data

[1009] Subject: Server

[1010] The server collects the necessary data from multiple data sources, including satellite imagery APIs and aerial imagery APIs to obtain geographical information, and store information, user reviews, and other data from open databases.

[1011] Step 6: Generate the digital twin map

[1012] Subject: Server

[1013] The server integrates the collected data to generate a digital twin map, which includes relevant information based on the user's request, such as visually displaying the locations of restaurants and tourist attractions that the user can enjoy, as well as route information.

[1014] Step 7: Provide personalized information

[1015] Subject: Server

[1016] The server then personalizes the generated digital twin map and related information for the user, building data based on individual requests and location information, including optimal routes, recommended stores, detailed information, and user reviews.

[1017] Step 8: Submit your information

[1018] Subject: Server

[1019] The server transmits the personalized information to the terminal, including map data and text information for visual display.

[1020] Step 9: Viewing Information

[1021] Subject: Terminal

[1022] The device displays the information received from the server on a user interface, allowing users to move and make selections while checking recommended routes, recommended places, detailed information, user reviews, and more on the displayed map.

[1023] Through the above processing steps, the system provides users with a highly convenient and innovative map search experience.

[1024] Example 1

[1025] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1026] In modern society, there is a need to provide a wide variety of information quickly and efficiently, but there is a lack of systems that can collect appropriate information in response to user requests and provide it in a personalized format. Another issue is the difficulty of providing personalized information in real time based on the user's current location. Therefore, there is a need to develop a system that allows users to intuitively and quickly obtain the information they need.

[1027] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1028] In this invention, the server includes an analysis means, a data collection means, a digital twin map generation means, a personalized information provision means, a range specification means using current location information, a collection preparation means, and a spot display means, which enable users to efficiently obtain information within a specific range from their current location and use it in a personalized manner.

[1029] "User" means any person or agent who uses the System to request information.

[1030] "Request" means a request for information entered by a User into a System, including text input or voice input.

[1031] "Device" refers to the device used by a user to enter requests and obtain information, including smartphones and car navigation systems.

[1032] "Server" refers to the central computer in the system that receives user requests, analyzes them, collects the necessary data, and provides the results to the user.

[1033] "Data Sources" refers to the various databases and APIs that the server accesses to gather information, including satellite imagery APIs and open databases.

[1034] A "digital twin map" refers to a real-time geographic information system that is generated based on collected data.

[1035] "Location information" refers to location data that indicates the user's current location, primarily GPS data.

[1036] "Analysis Method" refers to the technology used by the server to understand the content of the user's request, including natural language processing technology.

[1037] "Data collection means" refers to the methods and processes by which the server obtains the required data from various data sources.

[1038] "Digital twin map generation means" refers to the method or process for integrating data collected by the server and creating a digital twin map.

[1039] "Personalized information provision means" refers to a method or process for generating and providing personalized information based on a user's request.

[1040] "Area specification means using current location information" refers to a method or process by which a server specifies a specified range based on the user's current location information.

[1041] "Collection preparation means" refers to the method or process by which the server configures and prepares itself to collect the required data.

[1042] "Spot display means" refers to a method or process by which the server displays spots based on a user's request on the digital twin map.

[1043] The present invention is a system in which a user inputs a request, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. Specific embodiments of the system are described below.

[1044] Users can use their smartphone or car navigation system to input requests by voice or text into a dedicated application. For example, a user might request, "Tell me about some popular restaurants within five minutes of my current location that a family of four can enjoy for half a day."

[1045] The device sends the input request and current location information (GPS data) to the server using a data communication protocol such as an HTTP POST request.

[1046] The server uses natural language processing technology to analyze the received request, specifically leveraging a natural language processing API (e.g., Google Cloud Natural Language API) to extract keywords and important information from the request.

[1047] Once the request is analyzed, the server will use the user's current location information to begin collecting data within the specified area. For example, it will use the Google Maps API or OpenStreetMap API to prepare to obtain the necessary information (store information, tourist spot information, etc.).

[1048] The server then uses satellite and aerial imagery APIs to retrieve data, including up-to-date geographic information, and combines this collected data to generate a digital twin map.

[1049] The generated digital twin map displays recommended spots and optimal route guidance based on user requests, and also includes detailed store information and user reviews for personalized information.

[1050] Finally, the server sends the generated information to the device as display data, which is then presented to the user in a visually easy-to-understand format, allowing the user to check the recommended route, the location of each spot, and detailed information on the map.

[1051] Specific examples

[1052] Example 1: Searching for family-friendly restaurants

[1053] User: Type into a smartphone app, "Tell me about a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal."

[1054] Terminal: Sends the request data to the server along with the current location information.

[1055] Server: Analyzes the request using natural language processing technology and collects the necessary data.

[1056] Server: Collects restaurant information from Google Maps API and other data sources and generates a digital twin map.

[1057] Server: Sends optimal route and restaurant information to the device.

[1058] Device: Displays restaurant recommendations and optimal routes on an interactive map, along with detailed information and reviews.

[1059] Prompt Sentence Examples

[1060] "Tell me a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal."

[1061] This system utilizes natural language processing technology and various data collection APIs to provide users with intuitive and personalized information, allowing them to easily and efficiently obtain information that meets their needs.

[1062] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1063] Step 1:

[1064] When a user uses a smartphone or car navigation system to input a request within an application, they can use text or voice. For example, they might say, "Tell me about a popular restaurant within five minutes of my current location that would be fun for a family of four for half a day." This input becomes input data for the device. The device converts the voice data into text data and processes it as request data.

[1065] Step 2:

[1066] The device sends the request data entered by the user and current location information (GPS data) to the server. This is done using an HTTP POST request. The input data is the request data (text request) and current location information (GPS data), and the output data is the request payload sent to the server.

[1067] Step 3:

[1068] The server analyzes the received request data. Natural language processing technology is used for the analysis. Specifically, a natural language processing API (e.g., a natural language processing cloud API) is used to extract keywords and important information from the request data. The input data is the request payload, and the output data is the analysis results (e.g., keywords "within 5 minutes," "family of 4," "half day," "popular restaurant").

[1069] Step 4:

[1070] The server prepares for data collection based on the analysis results. It uses the current location information to identify an area within a specified range (e.g., within 5 minutes) and configures data collection settings within that area. This preparation includes identifying the area using a map API (e.g., Map Cloud API) and configuring data collection parameters. The input data are the analysis results and current location information, and the output data is the configuration information for which collection preparation is complete.

[1071] Step 5:

[1072] The server obtains the necessary data from various data sources (e.g., satellite imagery API, aerial imagery API, open databases, etc.). This includes real-time updated geographic information and information requested by users (e.g., store information, tourist spot information, etc.). The input data is the setup information for collection preparation, and the output data is the various data obtained.

[1073] Step 6:

[1074] The server integrates the acquired data to generate a digital twin map. For example, it uses geographic information system (GIS) technology to draw a map based on satellite and aerial images and plot recommended spots based on user requests. The input data is the acquired data, and the output data is the digital twin map.

[1075] Step 7:

[1076] The server generates personalized information for the user based on the generated digital twin map. This includes optimal route guidance, detailed store information, reviews, and more. The input data is the digital twin map and the user's requested information, and the output data is personalized information. This is then sent to the device as display data.

[1077] Step 8:

[1078] The terminal receives the display data sent from the server and displays it to the user in a visually easy-to-understand format. The user can check the recommended route and the location and detailed information of each spot on an interactive map. Detailed store information and user reviews are also displayed as text information. The input data is the display data, and the output data is the display screen for the user.

[1079] (Application example 1)

[1080] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1081] Conventional map information systems make it difficult for users to obtain detailed information about virtual stores. They also lack the means to provide personalized information based on the user's current location in real time. As a result, it is difficult for users to intuitively obtain specific store or product information, or to navigate within the virtual space.

[1082] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1083] In this invention, the server includes a means for a user to input a request for a virtual store using a smartphone or smart glasses, a means for the server to provide detailed information about the virtual store based on a digital twin map, and a means for navigating the virtual store and viewing product information, allowing the user to obtain detailed, personalized store and product information in real time and achieving intuitive navigation within the virtual store.

[1084] 1. "Means for users to input requests by text or voice" refers to devices or software that allow users to input their requests or questions by text or voice.

[1085] 2. "Means for sending a request to the server" refers to the communications means for transmitting text or voice data entered by the user to the server via the Internet.

[1086] 3. "Means for analyzing requests on the server" refers to the processes and technologies used by the server to analyze the request data received and understand its contents.

[1087] 4. "Means for collecting satellite imagery, aerial imagery, and open data" refers to the API or database access method that the server uses to collect the necessary satellite imagery, aerial imagery, and open data based on the results of request analysis.

[1088] 5. "Means for generating digital twin maps" refers to technologies that generate virtual maps based on collected data, creating digital models that function as twins of the physical world.

[1089] 6. "Means for providing personalized information" refers to methods and technologies for providing information optimized for individual users based on their requests and current location information.

[1090] 7. "Means for transmitting and displaying generated information to a terminal" refers to the technology and devices for transmitting personalized information generated by the server to a user's terminal and visually displaying it.

[1091] 8. "Means for users to input requests about a virtual store using a smartphone or smart glasses" refers to a method and device for users to request information about a virtual store using a smartphone or smart glasses.

[1092] 9. "Means for the server to provide detailed information about the virtual store based on the digital twin map" refers to methods and technologies for utilizing the digital twin map to provide users with detailed information about the virtual store.

[1093] 10. "Means for navigating the virtual store and viewing product information" means the interface or technology that allows a user to virtually navigate the virtual store and view detailed product information.

[1094] The present invention is a system in which a user inputs a request for a virtual store using a smartphone or smart glasses, and detailed information is provided based on that request using a digital twin.

[1095] The system uses the following hardware and software:

[1096] Smartphones and smart glasses: These are devices through which users input requests.

[1097] GPS module: Used to obtain the user's current location information.

[1098] Server: This is the central system that analyzes request data and collects and processes the necessary data. It uses various APIs (satellite imagery API, aerial imagery API, open database), natural language processing engines (e.g., SpaCy), and database management systems (e.g., MySQL).

[1099] Communication module: Transmits data between the device and the server via the Internet.

[1100] The program processing proceeds as follows:

[1101] Using a smartphone or smart glasses, a user can input a request by voice or text, such as "Tell me what fashion items are popular at the nearest department store right now." This request also includes the user's current location information. The device then sends the request data and current location information to the server.

[1102] When the server receives the request data, it analyzes the request using natural language processing technology and extracts keywords. Based on the results of this analysis, the server prepares to collect the necessary data. Specifically, it obtains the latest data from satellite images, aerial images, and open databases.

[1103] The server integrates the collected data and generates a digital twin map, which includes store information and product details that the user may be interested in. The generated information is sent from the server to the user's device and displayed visually.

[1104] For example, if a user requests, "Tell me the popular products on sale at the nearest shopping mall right now," the server analyzes the request and generates a digital twin map of the shopping mall. The generated map includes detailed information about popular products on sale and stores. The user can also navigate within the virtual store and view detailed product information.

[1105] Here are some examples of prompts for generative AI models:

[1106] "Please tell me information about popular fashion items at the department store nearest to my current location. Details should include the item name, image, price, store location, etc."

[1107] This allows users to receive real-time personalized information and intuitive navigation within the virtual store.

[1108] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1109] Step 1:

[1110] The user uses a smartphone or smart glasses to enter a request.

[1111] Specific operation: The user inputs a request via text or voice, for example, "Tell me what fashion items are popular at the nearest department store right now."

[1112] Input: User request voice or text data, current location information.

[1113] Output: Request data and current location information.

[1114] Step 2:

[1115] The device sends the request data and current location information to the server.

[1116] Specific operation: The terminal sends the request data entered by the user and the current location information obtained using the GPS module to the server.

[1117] Input: Request data, current location information.

[1118] Output: Request data sent to the server and current location information.

[1119] Step 3:

[1120] The server parses the request data.

[1121] Specific operation: The server analyzes the received request data using a natural language processing engine (e.g., SpaCy) and extracts keywords such as "department store," "fashion item," and "popular."

[1122] Input: Request data.

[1123] Output: Analysis results (extracted keywords).

[1124] Step 4:

[1125] Prepare the server to collect the required data.

[1126] Specific operation: Based on the analysis results, the server generates a query to obtain information from the required data sources (satellite imagery API, aerial imagery API, open database, etc.).

[1127] Input: Analysis results (extracted keywords).

[1128] Output: Data collection query.

[1129] Step 5:

[1130] The server retrieves data from various data sources.

[1131] Specific operation: The server uses the generated query to collect satellite images, aerial images, store information, and product information from data sources.

[1132] Input: Data collection query.

[1133] Output: Collected data (satellite images, aerial images, open data, store information, product information).

[1134] Step 6:

[1135] The server generates a digital twin map.

[1136] How it works: The server integrates the collected data and generates a personalized digital twin map based on the user's request.

[1137] Input: Collected data (satellite images, aerial images, open data, store information, product information).

[1138] Output: Digital twin map.

[1139] Step 7:

[1140] The server generates personalized information based on the digital twin map and sends it to the device.

[1141] Specific operation: Based on the generated digital twin map, the server generates the optimal route, detailed store information, and product information to suggest to the user, and sends this to the terminal.

[1142] Input: Digital twin map.

[1143] Output: Display data (optimal route, detailed store information, product information).

[1144] Step 8:

[1145] The terminal receives the display data sent from the server and displays it in a visually easy-to-understand format for the user.

[1146] Specific operation: The device processes the received display data and displays it so that the user can navigate through the virtual store and view product information.

[1147] Input: Display data (optimal route, detailed store information, product information).

[1148] Output: Information presented in a visually understandable format.

[1149] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1150] This invention is a system in which a user inputs a request via text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. Furthermore, this system combines an emotion engine that recognizes the user's emotions and optimizes information based on the user's emotions. This invention is mainly provided via smartphones and car navigation systems.

[1151] Program processing

[1152] 1. The user enters a request

[1153] Subject: User

[1154] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice. For example, they might input, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day." The user's emotional state is also recorded as a log.

[1155] 2. Submitting a Request

[1156] Subject: Terminal

[1157] The device sends the user's input request, current location information, and emotional state to the server. If the input is voice, it is converted to text and the request is prepared as data.

[1158] 3. Parsing the Request

[1159] Subject: Server

[1160] The server analyzes the received request data. First, it analyzes the request string using natural language analysis technology to extract keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." This process uses generative AI.

[1161] 4. Emotion analysis

[1162] Subject: Server

[1163] The server analyzes the received emotional state data and uses an emotion engine to determine the user's emotional state (e.g., excited, calm, tired, etc.).

[1164] 5. Preparation for Data Collection

[1165] Subject: Server

[1166] The server then prepares to collect the necessary data based on the analysis results. It also configures the system to recommend more appropriate stores and routes based on the user's emotional state. For example, if the user is tired, it will prioritize places where they can relax and routes that are quick to travel.

[1167] 6. Data Acquisition

[1168] Subject: Server

[1169] The server collects the necessary data from multiple data sources, including satellite imagery APIs and aerial imagery APIs to obtain geographical information, and store information, user reviews, and other data from open databases.

[1170] 7. Digital Twin Map Generation

[1171] Subject: Server

[1172] The server integrates the collected data and generates a digital twin map that includes relevant information based on the user's requests, and displays recommended spots and routes that take into account the user's emotional state, among other things.

[1173] 8. Providing personalized information

[1174] Subject: Server

[1175] The server then personalizes the generated digital twin map and related information for the user, building data including optimal routes, recommended stores, detailed information, and user reviews based on individual requests, location, and emotional state.

[1176] 9. Transmission of Information

[1177] Subject: Server

[1178] The server transmits the personalized information to the terminal, including map data and text information for visual display.

[1179] 10. Display of Information

[1180] Subject: Terminal

[1181] The device displays the information received from the server on a user interface. Users can navigate and make selections on the displayed map while checking recommended routes, recommended locations, detailed information, user reviews, etc. The UI itself is also optimized according to the user's emotional state, providing, for example, color schemes and interactions that promote relaxation.

[1182] Specific examples

[1183] Example 1: Searching for family-friendly restaurants

[1184] User: Enters into a smartphone app, "Tell me about a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal." If the emotion engine determines that the user is tired, it will prioritize restaurants with a relaxed atmosphere.

[1185] Device: Sends a request to the server along with its current location information.

[1186] Server: Analyzes requests and sentiments and collects necessary data.

[1187] Server: Generates a digital twin map and sends optimal routes and recommended restaurants to the device.

[1188] On your device: Displays recommended restaurants and optimal routes on a map, along with detailed information and reviews.

[1189] Example 2: Searching for tourist spots

[1190] User: Say to the car navigation system, "Tell me places I can visit within 10 minutes of my current location." If the emotion engine determines that the user is excited, it will prioritize active tourist spots.

[1191] Terminal: Sends requests to the server.

[1192] Server: Collects data based on analysis results and emotions, and obtains information on tourist spots.

[1193] Server: Generates a digital twin map and sends tourist spot and route information to the device.

[1194] Device: Displays the location of tourist attractions and the best route on a map.

[1195] Through this system, users can easily and intuitively make new discoveries and experiences, even when dealing with complex requests. Furthermore, the use of an emotion engine allows for more personalized services.

[1196] The processing flow will be explained below.

[1197] Step 1: User enters request

[1198] Subject: User

[1199] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice, for example, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day at." At this time, the system automatically records the user's emotional state based on facial recognition data and voice tone.

[1200] Step 2: Submitting the request

[1201] Subject: Terminal

[1202] The device sends the user's input request, current location information, and emotional state to the server. In the case of voice input, the voice is converted into text and the request is prepared as data.

[1203] Step 3: Parsing the request

[1204] Subject: Server

[1205] The server analyzes the received request data. It uses natural language analysis technology to extract keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." Generative AI is used in this process.

[1206] Step 4: Sentiment Analysis

[1207] Subject: Server

[1208] The server analyzes the received emotional state data and uses an emotion engine to analyze the user's emotions, for example, to determine whether the user is in an emotional state of "excited," "calm," or "tired."

[1209] Step 5: Prepare for data collection

[1210] Subject: Server

[1211] The server then prepares to collect the necessary data based on the analysis results. It also configures the system to recommend more appropriate stores and routes based on the user's emotional state. For example, if the user is feeling tired, it will prioritize places where they can relax and routes that are quick to travel.

[1212] Step 6: Get the data

[1213] Subject: Server

[1214] The server collects the necessary data from multiple data sources, including satellite imagery APIs and aerial imagery APIs to obtain geographical information, and store information, user reviews, and other data from open databases.

[1215] Step 7: Generate the digital twin map

[1216] Subject: Server

[1217] The server integrates the collected data and generates a digital twin map that includes relevant information based on the user's requests, and displays recommended spots and routes that take into account the user's emotional state, among other things.

[1218] Step 8: Provide personalized information

[1219] Subject: Server

[1220] The server then personalizes the generated digital twin map and related information for the user, building data including optimal routes, recommended stores, detailed information, user reviews, and more based on individual requests, location information, and emotional state.

[1221] Step 9: Submit your information

[1222] Subject: Server

[1223] The server transmits the personalized information to the terminal, including map data and text information for visual display.

[1224] Step 10: Viewing Information

[1225] Subject: Terminal

[1226] The device displays the information received from the server on a user interface. Users can navigate and make selections on the displayed map while checking recommended routes, recommended locations, detailed information, user reviews, etc. The UI itself is also optimized according to the user's emotional state, providing, for example, color schemes and interactions that promote relaxation.

[1227] Specific examples

[1228] Example 1: Searching for family-friendly restaurants

[1229] Step 1: User: Enters into the smartphone app, "Tell me about a popular restaurant within 5 minutes of my current location where a family of four can enjoy a half-day meal." If the emotion engine determines that the user is tired, restaurants with a relaxed atmosphere will be prioritized.

[1230] Step 2: Device: Sends a request to the server along with its current location and emotional state.

[1231] Step 3: Server: Analyze the request and emotional state and collect the necessary data.

[1232] Step 4: Server: Generates a digital twin map and sends optimal route and restaurant recommendation information to the device.

[1233] Step 5: On your device: Display recommended restaurants and optimal routes on a map, along with detailed information and reviews.

[1234] Example 2: Searching for tourist spots

[1235] Step 1: User: Say to the car navigation system, "Tell me places I can visit within 10 minutes of my current location." If the emotion engine determines that the user is excited, active tourist spots will be prioritized.

[1236] Step 2: Terminal: Sends request to server.

[1237] Step 3: Server: Collect data based on the analysis results and emotional state to obtain tourist attraction information.

[1238] Step 4: Server: Generates a digital twin map and sends tourist spot and route information to the device.

[1239] Step 5: Device: Display the location of tourist attractions and the best route on the map.

[1240] This system allows users to easily and intuitively discover new things and enjoy new experiences, even when dealing with complex requests. The introduction of an emotion engine also enables the provision of more personalized services suited to the user's emotional state.

[1241] Example 2

[1242] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1243] Conventional information provision systems provide information taking into account the user's location and requests, but do not support personalization that takes into account the user's emotional state. As a result, user satisfaction can be low, and appropriate information cannot be provided in response to requests. Furthermore, there are issues with the lack of real-timeness and accuracy when generating digital twin maps.

[1244] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing emotional state data and determining the user's emotion, a means for optimizing collected data based on the user's emotional state, and a means for collecting necessary data from multiple data sources. This makes it possible to provide personalized information that takes the user's emotional state into consideration.

[1245] "User" means any person who uses the System to enter requests and obtain information.

[1246] A "request" refers to the information or service a user requests from a system.

[1247] "Server" refers to a computer system that analyzes received requests and collects, processes, and provides the necessary information.

[1248] "Emotional state data" refers to information about a user's emotional state or psychological state.

[1249] An "emotion engine" refers to an algorithm or system that analyzes the emotional state data provided by the user and recognizes specific emotions (excitement, calmness, fatigue, etc.).

[1250] "Natural language processing technology" refers to technology that analyzes text and voice information entered by users and converts it into a form that can be understood by humans.

[1251] A "digital twin map" refers to a map that digitally reproduces real-world geographic information and related data.

[1252] "Multiple data sources" refers to different databases and APIs (application programming interfaces) used to obtain information.

[1253] "Personalization" refers to optimizing and providing information according to each user's individual characteristics, requests, and emotions.

[1254] "Terminal" refers to a device (smartphone, car navigation system, etc.) through which a user inputs requests and displays information provided by the server.

[1255] The present invention is a system in which a user inputs a request via text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. Furthermore, this system incorporates an emotion engine that recognizes the user's emotions and optimizes information based on the user's emotions. The present invention is primarily provided via smartphones and car navigation systems.

[1256] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice. For example, they might input a request like, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day at." At this time, the user's emotional state is also recorded as a log.

[1257] The device sends the user's input request, location information, and emotional state to the server. In the case of voice input, it converts the voice data into text using speech recognition software (e.g., a common speech recognition API) and sends the prepared text and additional data to the server.

[1258] The server analyzes the received request data. It uses natural language processing technology (for example, a general natural language processing API) to analyze the request string and extract keywords, which clarify the information the user is looking for.

[1259] Furthermore, the server uses an emotion engine to analyze the emotional state data and determine the user's emotions (excitement, calmness, fatigue, etc.). Based on the analysis results, the server then prepares to collect necessary data. In particular, it configures the system to recommend more appropriate stores and routes based on the user's emotional state.

[1260] The server collects the necessary data from multiple data sources (e.g., satellite imagery APIs and open databases). It calls the satellite imagery API to obtain geographic information and retrieves store information, user reviews, etc. from the open database.

[1261] The collected data is integrated to generate a digital twin map that includes relevant information based on the user's requests and displays recommended spots and routes that take into account the user's emotional state, among other things.

[1262] The server then creates a digital twin map and provides related information personalized to the user, such as optimal routes, recommended stores, detailed information, and user reviews based on individual requests, location information, and emotional state.

[1263] Finally, the server transmits the personalized information to the terminal, including map data and text information for visual display.

[1264] The device displays the information received from the server on a user interface. Users can navigate and make selections on the displayed map while checking recommended routes, recommended locations, detailed information, user reviews, etc. The UI itself is also optimized according to the user's emotional state, providing, for example, color schemes and interactions that promote relaxation.

[1265] A concrete example is searching for a restaurant that the whole family can enjoy. If a user types into a smartphone app, "Tell me a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day," and the emotion engine determines that the user is tired, the server will prioritize recommending restaurants with a relaxed atmosphere. The device sends the request and current location information to the server, which then analyzes, collects data, creates a map, and sends the final information to the device. The user can then view the restaurants and the optimal route displayed on the map.

[1266] Examples of prompt sentences include "Tell me about a popular restaurant within five minutes of our current location where a family of four can spend half a day," or "Tell me about a tourist spot within ten minutes that I can go to when I'm feeling excited." In response to such specific requests, the system of the present invention can quickly and appropriately provide personalized information.

[1267] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1268] Step 1: User enters request

[1269] Subject: User

[1270] The user opens a smartphone or car navigation application and inputs a request by text or voice. For example, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day." Along with this, the user also records their emotional state. The input data includes the content of the request and the user's emotional state.

[1271] Step 2: Submitting the request

[1272] Subject: Terminal

[1273] The device sends the input request, location information, and the user's emotional state to the server. In the case of voice input, speech recognition software is used to convert the speech to text. The input data includes the text of the request, location information, and emotional state. The output is the prepared data sent to the server.

[1274] Step 3: Parsing the request

[1275] Subject: Server

[1276] The server analyzes the received request data and uses natural language processing technology to extract keywords from the request, such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." The input is the text-converted request data, and the output is a list of extracted keywords.

[1277] Step 4: Sentiment Analysis

[1278] Subject: Server

[1279] The server uses an emotion engine to analyze the received emotional state data and determine what emotional state the user is in (e.g., excited, calm, tired, etc.). The input is the user's emotional state data, and the output is the interpreted emotional state (e.g., tired, excited, etc.).

[1280] Step 5: Prepare for data collection

[1281] Subject: Server

[1282] The server prepares to collect the necessary data based on the analysis results. It optimizes recommended stores and routes based on the user's emotional state. For example, if the user is tired, it prioritizes places where they can relax and routes that are quick to travel. The input is the extracted keywords and the interpretation of the user's emotional state, and the output is the data collection conditions.

[1283] Step 6: Get the data

[1284] Subject: Server

[1285] The server collects the necessary data from multiple data sources. Coordinate information is obtained using a satellite imagery API, and store information and user reviews are obtained from an open database. The input is the data collection conditions, and the output is the collected geographic information and related data.

[1286] Step 7: Generate the digital twin map

[1287] Subject: Server

[1288] The server integrates the collected data and generates a digital twin map. The map visually displays relevant information based on the user's request, including recommended spots and routes that take into account the user's emotional state. The input is the collected geographic information and related data, and the output is the generated digital twin map.

[1289] Step 8: Provide personalized information

[1290] Subject: Server

[1291] The server personalizes the generated digital twin map and related information for the user, building optimal routes, recommended stores, detailed information, user reviews, etc. based on individual requests, location information, and emotional state. The input is the generated digital twin map and related information, and the output is personalized information.

[1292] Step 9: Submit your information

[1293] Subject: Server

[1294] The server sends personalized information to the terminal, including map data and text information for visual display. The input is the personalized information, and the output is the data sent to the terminal.

[1295] Step 10: Viewing Information

[1296] Subject: Terminal

[1297] The device displays the information received from the server in a user interface. Users can view recommended routes, recommended places, detailed information, and user reviews on a map. The UI also provides optimal color schemes and interactions depending on the user's emotional state. The input is the received data, and the output is the displayed map and related information.

[1298] (Application example 2)

[1299] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1300] Conventional systems that provide information based on requests do not take into account the user's emotional state, making it difficult to fully improve user satisfaction.Furthermore, in food delivery services, the quality of service declines because the system does not select the store or menu that best suits the user's mood or emotion at the time.

[1301] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1302] In this invention, the server includes means for analyzing user emotions, means for providing personalized information, means for generating a digital twin map, and means for recommending optimal stores and menus to users based on data collected from multiple stores, thereby enabling optimal information provision and delivery services that take into account the user's requests and emotional state.

[1303] "User" means any person or entity that uses the System to enter a Request.

[1304] A "request" is a request or question that a user enters into a system.

[1305] "Analysis" refers to the process by which the server uses natural language processing and other technologies to understand the request and emotional state it receives and extract the necessary information.

[1306] A "digital twin map" is a map that digitally reproduces information from the physical world, and includes real-time data and its analysis results.

[1307] "Personalized information" is information that is individually optimized based on the user's requests and emotional state.

[1308] An "emotion engine" refers to software or algorithms that analyze a user's voice, text, or other input to determine their emotional state.

[1309] "Recommendation" is the process of presenting the best options to the user based on the analysis results.

[1310] "Store" refers to a place or business that offers goods or services.

[1311] A "menu" is a list of products offered by a store and their details.

[1312] "Collection" refers to the operation or process by which a server obtains the necessary data in response to a request.

[1313] This invention is a system in which a user inputs a request via text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information according to the user's emotional state.

[1314] System Program Overview

[1315] 1. The user enters a request

[1316] Users open the smartphone application and input their request by text or voice. For example, they can input a request such as, "I've been feeling stressed lately, so please recommend some sweets that will help me relax." At this time, the user's emotional state is also recorded.

[1317] 2. Submitting a Request

[1318] The device sends the user's input request, current location, and emotional state to the server, and if the input is voice, it is converted into text.

[1319] 3. Parsing the Request

[1320] The server analyzes the received request data and uses natural language analysis technology to extract keywords such as "stress," "relaxation," and "sweets."

[1321] 4. Emotion analysis

[1322] The server analyzes the user's emotional state using an emotion engine, which uses the IBM Watson Tone Analyzer API.

[1323] 5. Preparation for Data Collection

[1324] The server prepares to collect the necessary data based on the analysis results and the user's emotional state, and prioritizes settings such as locations and menus suitable for relaxation.

[1325] 6. Data Acquisition

[1326] The server uses the Google Maps API and open databases to obtain information about stores that offer relaxing sweets.

[1327] 7. Digital Twin Map Generation

[1328] The server generates a digital twin map based on the collected data, which contains relevant information based on the user's request.

[1329] 8. Providing personalized information

[1330] The server then personalizes the generated digital twin map and related information for the user, including information on the best stores, menus, routes, etc. based on the user's request, emotional state, and location.

[1331] 9. Transmission of Information

[1332] The server sends the personalized information to the terminal.

[1333] 10. Display of Information

[1334] The device displays the received information on the user interface, where users can check the recommended route, detailed store information, and menu information on the displayed map.

[1335] Hardware and software used

[1336] Smartphone app: iOS or Android

[1337] Server: AWS Lambda, Amazon RDS

[1338] Data analysis engine: Python, TensorFlow

[1339] Sentiment analysis engine: IBM Watson Tone Analyzer API

[1340] Map data: Google Maps API

[1341] Specific examples

[1342] User: "I've been feeling stressed lately, so please recommend a relaxing dessert."

[1343] App: "Here are some relaxing sweets in the neighborhood. Recommended stores are XX Cafe and △△ Restaurant."

[1344] Prompt Sentence Examples

[1345] "Generate a Python program that provides optimal store information based on user requests and sentiment data."

[1346] In this way, the present invention provides a system that provides personalized information that takes into account the user's requests and emotional state.

[1347] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1348] Step 1:

[1349] The user enters a request

[1350] Input: A user types a request into a smartphone app by text or voice. For example, "I've been feeling stressed lately, so please recommend some sweets that will help me relax."

[1351] How it works: The application records the user's emotional state in parallel, where the input data are text data and emotional data.

[1352] Output: Text and emotion data are generated and ready to be sent to the next step.

[1353] Step 2:

[1354] Submitting a Request

[1355] Input: Request text data and emotion data entered by the user.

[1356] Operation: The device sends the request text data, emotion data, and current location information to the server. If the input is voice, it is converted into text data.

[1357] Output: The request text data, emotion data, and current location information are sent to the server.

[1358] Step 3:

[1359] Parsing the request

[1360] Input: The request text data sent to the server.

[1361] How it works: The server uses natural language analysis techniques to parse the request text and extract key keywords (e.g., "stress," "relax," "sweets"), using a generative AI model for this process.

[1362] Output: The extracted keyword data is generated and ready to be sent to the next step.

[1363] Step 4:

[1364] Emotion analysis

[1365] Input: Emotion data sent to the server.

[1366] How it works: The server uses an emotion engine (e.g., IBM Watson Tone Analyzer API) to analyze the user's emotional state. In this case, it analyzes that the user wants to relax.

[1367] Output: The parsed emotion data is generated and ready to be sent to the next step.

[1368] Step 5:

[1369] Preparing for data collection

[1370] Input: Extracted keyword data and parsed sentiment data.

[1371] How it works: Based on the analysis results, the server prepares data collection tailored to the user's needs (sweets with a relaxing effect).

[1372] Output: Your data collection setup is complete and you are ready to move on to the next step.

[1373] Step 6:

[1374] Data Acquisition

[1375] Input: Collection settings based on data the server is ready to collect.

[1376] How it works: The server calls the Google Maps API and open databases to retrieve information about stores that offer relaxing sweets.

[1377] Output: The retrieved store and menu information is generated and ready to be sent to the next step.

[1378] Step 7:

[1379] Digital twin map generation

[1380] Input: Store information and menu information obtained.

[1381] How it works: The server aggregates the collected data and generates a digital twin map that includes relevant information based on the user's request.

[1382] Output: The generated digital twin map data is prepared and sent to the next step.

[1383] Step 8:

[1384] Providing personalized information

[1385] Input: Generated digital twin map data, request, emotion data, and location information.

[1386] How it works: Based on the digital twin map and related information generated by the server, it creates information optimized for the user, including the best stores, menus, and route information.

[1387] Output: Personalized user information data is generated and sent to the next step.

[1388] Step 9:

[1389] Sending information

[1390] Input: Personalized user information data.

[1391] Operation: The server generates information and sends it to the device.

[1392] Output: The appropriate information data is sent to the terminal.

[1393] Step 10:

[1394] Displaying Information

[1395] Input: Personalized information data sent from the server.

[1396] How it works: The device displays the information it receives in a user interface, allowing users to view recommended routes, detailed store and menu information on the displayed map.

[1397] Output: A visual display is provided for the user to take information and order food delivery.

[1398] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1399] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1400] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1401] [Fourth embodiment]

[1402] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1403] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1404] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1405] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1406] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1407] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1408] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1409] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1410] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1411] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1412] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1413] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1414] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1415] The present invention relates to a system in which a user inputs a request by text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. The present invention is primarily provided via smartphones and car navigation systems.

[1416] Program processing

[1417] 1. The user enters a request

[1418] Subject: User

[1419] Users make requests using an application on their smartphone or car navigation system. For example, they might say, "Tell me about a popular restaurant within five minutes of my current location that would be fun for a family of four for half a day." The user's request includes their current location.

[1420] 2. Submitting a Request

[1421] Subject: Terminal

[1422] The device receives the user's voice or text request and sends it to the server as request data, along with the current location information (GPS data).

[1423] 3. Parsing the Request

[1424] Subject: Server

[1425] The server analyzes the received request data using natural language processing technology to understand the user's request. For example, it extracts keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant."

[1426] 4. Preparation for Data Collection

[1427] Subject: Server

[1428] Based on the analysis results, the server prepares to collect the necessary data. Specifically, it identifies an area within a five-minute radius of the current location and sets up the system to acquire satellite images, aerial images, and open database data within that area.

[1429] 5. Data Acquisition

[1430] Subject: Server

[1431] The server retrieves the latest data from various data sources (satellite imagery API, aerial imagery API, open databases, etc.), including store information and tourist spot information that may be of interest to users.

[1432] 6. Digital Twin Map Generation

[1433] Subject: Server

[1434] The server integrates the collected data to generate a digital twin map that displays real-time geographic information and recommendations based on user requests.

[1435] 7. Providing personalized information

[1436] Subject: Server

[1437] The server generates personalized information based on the digital twin map and sends it to the device as display data, including optimal route guidance, detailed store information, and reviews.

[1438] 8. Display of Information

[1439] Subject: Terminal

[1440] The device receives the display data sent from the server and displays it in a visually easy-to-understand format for the user. Users can check the recommended route and the location of each store on a map. They can also view detailed store information and user reviews as text information.

[1441] Specific examples

[1442] Example 1: Searching for family-friendly restaurants

[1443] User: Type into a smartphone app, "Tell me about a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal."

[1444] Device: Sends a request to the server along with its current location information.

[1445] Server: Parses the request and collects the necessary data.

[1446] Server: Generates a digital twin map and sends optimal routes and recommended restaurants to the device.

[1447] On your device: Displays recommended restaurants and optimal routes on a map, along with detailed information and reviews.

[1448] Example 2: Searching for tourist spots

[1449] User: Voice-input to the car navigation system, "Tell me places I can visit within 10 minutes of my current location."

[1450] Terminal: Sends requests to the server.

[1451] Server: Collects data based on the analysis results and obtains information about tourist spots.

[1452] Server: Generates a digital twin map and sends tourist spot and route information to the device.

[1453] Device: Displays the location of tourist attractions and the best route on a map.

[1454] Through this system, users can easily and intuitively make new discoveries and have new experiences, even when dealing with complex requests.

[1455] The processing flow will be explained below.

[1456] Step 1: User enters request

[1457] Subject: User

[1458] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice. For example, they might input, "Tell me about a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day." The application obtains the user's current location along with this request.

[1459] Step 2: Submitting the request

[1460] Subject: Terminal

[1461] The device sends the request entered by the user and the current location information to the server, where voice input is converted to text and the request is prepared as data.

[1462] Step 3: Parsing the request

[1463] Subject: Server

[1464] The server analyzes the received request data. First, it analyzes the request string using natural language analysis technology to extract keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." This process uses generative AI such as ChatGPT.

[1465] Step 4: Prepare for data collection

[1466] Subject: Server

[1467] Based on the analysis results, the server prepares to collect the necessary data. For example, it identifies an area "within 5 minutes of the current location" and sets up an API call to obtain satellite images, aerial images, and open data for that area.

[1468] Step 5: Get the data

[1469] Subject: Server

[1470] The server collects the necessary data from multiple data sources, including satellite imagery APIs and aerial imagery APIs to obtain geographical information, and store information, user reviews, and other data from open databases.

[1471] Step 6: Generate the digital twin map

[1472] Subject: Server

[1473] The server integrates the collected data to generate a digital twin map, which includes relevant information based on the user's request, such as visually displaying the locations of restaurants and tourist attractions that the user can enjoy, as well as route information.

[1474] Step 7: Provide personalized information

[1475] Subject: Server

[1476] The server then personalizes the generated digital twin map and related information for the user, building data based on individual requests and location information, including optimal routes, recommended stores, detailed information, and user reviews.

[1477] Step 8: Submit your information

[1478] Subject: Server

[1479] The server transmits the personalized information to the terminal, including map data and text information for visual display.

[1480] Step 9: Viewing Information

[1481] Subject: Terminal

[1482] The device displays the information received from the server on a user interface, allowing users to move and make selections while checking recommended routes, recommended places, detailed information, user reviews, and more on the displayed map.

[1483] Through the above processing steps, the system provides users with a highly convenient and innovative map search experience.

[1484] Example 1

[1485] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1486] In modern society, there is a need to provide a wide variety of information quickly and efficiently, but there is a lack of systems that can collect appropriate information in response to user requests and provide it in a personalized format. Another issue is the difficulty of providing personalized information in real time based on the user's current location. Therefore, there is a need to develop a system that allows users to intuitively and quickly obtain the information they need.

[1487] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1488] In this invention, the server includes an analysis means, a data collection means, a digital twin map generation means, a personalized information provision means, a range specification means using current location information, a collection preparation means, and a spot display means, which enable users to efficiently obtain information within a specific range from their current location and use it in a personalized manner.

[1489] "User" means any person or agent who uses the System to request information.

[1490] "Request" means a request for information entered by a User into a System, including text input or voice input.

[1491] "Device" refers to the device used by a user to enter requests and obtain information, including smartphones and car navigation systems.

[1492] "Server" refers to the central computer in the system that receives user requests, analyzes them, collects the necessary data, and provides the results to the user.

[1493] "Data Sources" refers to the various databases and APIs that the server accesses to gather information, including satellite imagery APIs and open databases.

[1494] A "digital twin map" refers to a real-time geographic information system that is generated based on collected data.

[1495] "Location information" refers to location data that indicates the user's current location, primarily GPS data.

[1496] "Analysis Method" refers to the technology used by the server to understand the content of the user's request, including natural language processing technology.

[1497] "Data collection means" refers to the methods and processes by which the server obtains the required data from various data sources.

[1498] "Digital twin map generation means" refers to the method or process for integrating data collected by the server and creating a digital twin map.

[1499] "Personalized information provision means" refers to a method or process for generating and providing personalized information based on a user's request.

[1500] "Area specification means using current location information" refers to a method or process by which a server specifies a specified range based on the user's current location information.

[1501] "Collection preparation means" refers to the method or process by which the server configures and prepares itself to collect the required data.

[1502] "Spot display means" refers to a method or process by which the server displays spots based on a user's request on the digital twin map.

[1503] The present invention is a system in which a user inputs a request, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. Specific embodiments of the system are described below.

[1504] Users can use their smartphone or car navigation system to input requests by voice or text into a dedicated application. For example, a user might request, "Tell me about some popular restaurants within five minutes of my current location that a family of four can enjoy for half a day."

[1505] The device sends the input request and current location information (GPS data) to the server using a data communication protocol such as an HTTP POST request.

[1506] The server uses natural language processing technology to analyze the received request, specifically leveraging a natural language processing API (e.g., Google Cloud Natural Language API) to extract keywords and important information from the request.

[1507] Once the request is analyzed, the server will use the user's current location information to begin collecting data within the specified area. For example, it will use the Google Maps API or OpenStreetMap API to prepare to obtain the necessary information (store information, tourist spot information, etc.).

[1508] The server then uses satellite and aerial imagery APIs to retrieve data, including up-to-date geographic information, and combines this collected data to generate a digital twin map.

[1509] The generated digital twin map displays recommended spots and optimal route guidance based on user requests, and also includes detailed store information and user reviews for personalized information.

[1510] Finally, the server sends the generated information to the device as display data, which is then presented to the user in a visually easy-to-understand format, allowing the user to check the recommended route, the location of each spot, and detailed information on the map.

[1511] Specific examples

[1512] Example 1: Searching for family-friendly restaurants

[1513] User: Type into a smartphone app, "Tell me about a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal."

[1514] Terminal: Sends the request data to the server along with the current location information.

[1515] Server: Analyzes the request using natural language processing technology and collects the necessary data.

[1516] Server: Collects restaurant information from Google Maps API and other data sources and generates a digital twin map.

[1517] Server: Sends optimal route and restaurant information to the device.

[1518] Device: Displays restaurant recommendations and optimal routes on an interactive map, along with detailed information and reviews.

[1519] Prompt Sentence Examples

[1520] "Tell me a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal."

[1521] This system utilizes natural language processing technology and various data collection APIs to provide users with intuitive and personalized information, allowing them to easily and efficiently obtain information that meets their needs.

[1522] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1523] Step 1:

[1524] When a user uses a smartphone or car navigation system to input a request within an application, they can use text or voice. For example, they might say, "Tell me about a popular restaurant within five minutes of my current location that would be fun for a family of four for half a day." This input becomes input data for the device. The device converts the voice data into text data and processes it as request data.

[1525] Step 2:

[1526] The device sends the request data entered by the user and current location information (GPS data) to the server. This is done using an HTTP POST request. The input data is the request data (text request) and current location information (GPS data), and the output data is the request payload sent to the server.

[1527] Step 3:

[1528] The server analyzes the received request data. Natural language processing technology is used for the analysis. Specifically, a natural language processing API (e.g., a natural language processing cloud API) is used to extract keywords and important information from the request data. The input data is the request payload, and the output data is the analysis results (e.g., keywords "within 5 minutes," "family of 4," "half day," "popular restaurant").

[1529] Step 4:

[1530] The server prepares for data collection based on the analysis results. It uses the current location information to identify an area within a specified range (e.g., within 5 minutes) and configures data collection settings within that area. This preparation includes identifying the area using a map API (e.g., Map Cloud API) and configuring data collection parameters. The input data are the analysis results and current location information, and the output data is the configuration information for which collection preparation is complete.

[1531] Step 5:

[1532] The server obtains the necessary data from various data sources (e.g., satellite imagery API, aerial imagery API, open databases, etc.). This includes real-time updated geographic information and information requested by users (e.g., store information, tourist spot information, etc.). The input data is the setup information for collection preparation, and the output data is the various data obtained.

[1533] Step 6:

[1534] The server integrates the acquired data to generate a digital twin map. For example, it uses geographic information system (GIS) technology to draw a map based on satellite and aerial images and plot recommended spots based on user requests. The input data is the acquired data, and the output data is the digital twin map.

[1535] Step 7:

[1536] The server generates personalized information for the user based on the generated digital twin map. This includes optimal route guidance, detailed store information, reviews, and more. The input data is the digital twin map and the user's requested information, and the output data is personalized information. This is then sent to the device as display data.

[1537] Step 8:

[1538] The terminal receives the display data sent from the server and displays it to the user in a visually easy-to-understand format. The user can check the recommended route and the location and detailed information of each spot on an interactive map. Detailed store information and user reviews are also displayed as text information. The input data is the display data, and the output data is the display screen for the user.

[1539] (Application example 1)

[1540] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1541] Conventional map information systems make it difficult for users to obtain detailed information about virtual stores. They also lack the means to provide personalized information based on the user's current location in real time. As a result, it is difficult for users to intuitively obtain specific store or product information, or to navigate within the virtual space.

[1542] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1543] In this invention, the server includes a means for a user to input a request for a virtual store using a smartphone or smart glasses, a means for the server to provide detailed information about the virtual store based on a digital twin map, and a means for navigating the virtual store and viewing product information, allowing the user to obtain detailed, personalized store and product information in real time and achieving intuitive navigation within the virtual store.

[1544] 1. "Means for users to input requests by text or voice" refers to devices or software that allow users to input their requests or questions by text or voice.

[1545] 2. "Means for sending a request to the server" refers to the communications means for transmitting text or voice data entered by the user to the server via the Internet.

[1546] 3. "Means for analyzing requests on the server" refers to the processes and technologies used by the server to analyze the request data received and understand its contents.

[1547] 4. "Means for collecting satellite imagery, aerial imagery, and open data" refers to the API or database access method that the server uses to collect the necessary satellite imagery, aerial imagery, and open data based on the results of request analysis.

[1548] 5. "Means for generating digital twin maps" refers to technologies that generate virtual maps based on collected data, creating digital models that function as twins of the physical world.

[1549] 6. "Means for providing personalized information" refers to methods and technologies for providing information optimized for individual users based on their requests and current location information.

[1550] 7. "Means for transmitting and displaying generated information to a terminal" refers to the technology and devices for transmitting personalized information generated by the server to a user's terminal and visually displaying it.

[1551] 8. "Means for users to input requests about a virtual store using a smartphone or smart glasses" refers to a method and device for users to request information about a virtual store using a smartphone or smart glasses.

[1552] 9. "Means for the server to provide detailed information about the virtual store based on the digital twin map" refers to methods and technologies for utilizing the digital twin map to provide users with detailed information about the virtual store.

[1553] 10. "Means for navigating the virtual store and viewing product information" means the interface or technology that allows a user to virtually navigate the virtual store and view detailed product information.

[1554] The present invention is a system in which a user inputs a request for a virtual store using a smartphone or smart glasses, and detailed information is provided based on that request using a digital twin.

[1555] The system uses the following hardware and software:

[1556] Smartphones and smart glasses: These are devices through which users input requests.

[1557] GPS module: Used to obtain the user's current location information.

[1558] Server: This is the central system that analyzes request data and collects and processes the necessary data. It uses various APIs (satellite imagery API, aerial imagery API, open database), natural language processing engines (e.g., SpaCy), and database management systems (e.g., MySQL).

[1559] Communication module: Transmits data between the device and the server via the Internet.

[1560] The program processing proceeds as follows:

[1561] Using a smartphone or smart glasses, a user can input a request by voice or text, such as "Tell me what fashion items are popular at the nearest department store right now." This request also includes the user's current location information. The device then sends the request data and current location information to the server.

[1562] When the server receives the request data, it analyzes the request using natural language processing technology and extracts keywords. Based on the results of this analysis, the server prepares to collect the necessary data. Specifically, it obtains the latest data from satellite images, aerial images, and open databases.

[1563] The server integrates the collected data and generates a digital twin map, which includes store information and product details that the user may be interested in. The generated information is sent from the server to the user's device and displayed visually.

[1564] For example, if a user requests, "Tell me the popular products on sale at the nearest shopping mall right now," the server analyzes the request and generates a digital twin map of the shopping mall. The generated map includes detailed information about popular products on sale and stores. The user can also navigate within the virtual store and view detailed product information.

[1565] Here are some examples of prompts for generative AI models:

[1566] "Please tell me information about popular fashion items at the department store nearest to my current location. Details should include the item name, image, price, store location, etc."

[1567] This allows users to receive real-time personalized information and intuitive navigation within the virtual store.

[1568] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1569] Step 1:

[1570] The user uses a smartphone or smart glasses to enter a request.

[1571] Specific operation: The user inputs a request via text or voice, for example, "Tell me what fashion items are popular at the nearest department store right now."

[1572] Input: User request voice or text data, current location information.

[1573] Output: Request data and current location information.

[1574] Step 2:

[1575] The device sends the request data and current location information to the server.

[1576] Specific operation: The terminal sends the request data entered by the user and the current location information obtained using the GPS module to the server.

[1577] Input: Request data, current location information.

[1578] Output: Request data sent to the server and current location information.

[1579] Step 3:

[1580] The server parses the request data.

[1581] Specific operation: The server analyzes the received request data using a natural language processing engine (e.g., SpaCy) and extracts keywords such as "department store," "fashion item," and "popular."

[1582] Input: Request data.

[1583] Output: Analysis results (extracted keywords).

[1584] Step 4:

[1585] Prepare the server to collect the required data.

[1586] Specific operation: Based on the analysis results, the server generates a query to obtain information from the required data sources (satellite imagery API, aerial imagery API, open database, etc.).

[1587] Input: Analysis results (extracted keywords).

[1588] Output: Data collection query.

[1589] Step 5:

[1590] The server retrieves data from various data sources.

[1591] Specific operation: The server uses the generated query to collect satellite images, aerial images, store information, and product information from data sources.

[1592] Input: Data collection query.

[1593] Output: Collected data (satellite images, aerial images, open data, store information, product information).

[1594] Step 6:

[1595] The server generates a digital twin map.

[1596] How it works: The server integrates the collected data and generates a personalized digital twin map based on the user's request.

[1597] Input: Collected data (satellite images, aerial images, open data, store information, product information).

[1598] Output: Digital twin map.

[1599] Step 7:

[1600] The server generates personalized information based on the digital twin map and sends it to the device.

[1601] Specific operation: Based on the generated digital twin map, the server generates the optimal route, detailed store information, and product information to suggest to the user, and sends this to the terminal.

[1602] Input: Digital twin map.

[1603] Output: Display data (optimal route, detailed store information, product information).

[1604] Step 8:

[1605] The terminal receives the display data sent from the server and displays it in a visually easy-to-understand format for the user.

[1606] Specific operation: The device processes the received display data and displays it so that the user can navigate through the virtual store and view product information.

[1607] Input: Display data (optimal route, detailed store information, product information).

[1608] Output: Information presented in a visually understandable format.

[1609] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1610] This invention is a system in which a user inputs a request via text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. Furthermore, this system combines an emotion engine that recognizes the user's emotions and optimizes information based on the user's emotions. This invention is mainly provided via smartphones and car navigation systems.

[1611] Program processing

[1612] 1. The user enters a request

[1613] Subject: User

[1614] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice. For example, they might input, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day." The user's emotional state is also recorded as a log.

[1615] 2. Submitting a Request

[1616] Subject: Terminal

[1617] The device sends the user's input request, current location information, and emotional state to the server. If the input is voice, it is converted to text and the request is prepared as data.

[1618] 3. Parsing the Request

[1619] Subject: Server

[1620] The server analyzes the received request data. First, it analyzes the request string using natural language analysis technology to extract keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." This process uses generative AI.

[1621] 4. Emotion analysis

[1622] Subject: Server

[1623] The server analyzes the received emotional state data and uses an emotion engine to determine the user's emotional state (e.g., excited, calm, tired, etc.).

[1624] 5. Preparation for Data Collection

[1625] Subject: Server

[1626] The server then prepares to collect the necessary data based on the analysis results. It also configures the system to recommend more appropriate stores and routes based on the user's emotional state. For example, if the user is tired, it will prioritize places where they can relax and routes that are quick to travel.

[1627] 6. Data Acquisition

[1628] Subject: Server

[1629] The server collects the necessary data from multiple data sources, including satellite imagery APIs and aerial imagery APIs to obtain geographical information, and store information, user reviews, and other data from open databases.

[1630] 7. Digital Twin Map Generation

[1631] Subject: Server

[1632] The server integrates the collected data and generates a digital twin map that includes relevant information based on the user's requests, and displays recommended spots and routes that take into account the user's emotional state, among other things.

[1633] 8. Providing personalized information

[1634] Subject: Server

[1635] The server then personalizes the generated digital twin map and related information for the user, building data including optimal routes, recommended stores, detailed information, and user reviews based on individual requests, location, and emotional state.

[1636] 9. Transmission of Information

[1637] Subject: Server

[1638] The server transmits the personalized information to the terminal, including map data and text information for visual display.

[1639] 10. Display of Information

[1640] Subject: Terminal

[1641] The device displays the information received from the server on a user interface. Users can navigate and make selections on the displayed map while checking recommended routes, recommended locations, detailed information, user reviews, etc. The UI itself is also optimized according to the user's emotional state, providing, for example, color schemes and interactions that promote relaxation.

[1642] Specific examples

[1643] Example 1: Searching for family-friendly restaurants

[1644] User: Enters into a smartphone app, "Tell me about a popular restaurant within five minutes of my current location where a family of four can enjoy a half-day meal." If the emotion engine determines that the user is tired, it will prioritize restaurants with a relaxed atmosphere.

[1645] Device: Sends a request to the server along with its current location information.

[1646] Server: Analyzes requests and sentiments and collects necessary data.

[1647] Server: Generates a digital twin map and sends optimal routes and recommended restaurants to the device.

[1648] On your device: Displays recommended restaurants and optimal routes on a map, along with detailed information and reviews.

[1649] Example 2: Searching for tourist spots

[1650] User: Say to the car navigation system, "Tell me places I can visit within 10 minutes of my current location." If the emotion engine determines that the user is excited, it will prioritize active tourist spots.

[1651] Terminal: Sends requests to the server.

[1652] Server: Collects data based on analysis results and emotions, and obtains information on tourist spots.

[1653] Server: Generates a digital twin map and sends tourist spot and route information to the device.

[1654] Device: Displays the location of tourist attractions and the best route on a map.

[1655] Through this system, users can easily and intuitively make new discoveries and experiences, even when dealing with complex requests. Furthermore, the use of an emotion engine allows for more personalized services.

[1656] The processing flow will be explained below.

[1657] Step 1: User enters request

[1658] Subject: User

[1659] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice, for example, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day at." At this time, the system automatically records the user's emotional state based on facial recognition data and voice tone.

[1660] Step 2: Submitting the request

[1661] Subject: Terminal

[1662] The device sends the user's input request, current location information, and emotional state to the server. In the case of voice input, the voice is converted into text and the request is prepared as data.

[1663] Step 3: Parsing the request

[1664] Subject: Server

[1665] The server analyzes the received request data. It uses natural language analysis technology to extract keywords such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." Generative AI is used in this process.

[1666] Step 4: Sentiment Analysis

[1667] Subject: Server

[1668] The server analyzes the received emotional state data and uses an emotion engine to analyze the user's emotions, for example, to determine whether the user is in an emotional state of "excited," "calm," or "tired."

[1669] Step 5: Prepare for data collection

[1670] Subject: Server

[1671] The server then prepares to collect the necessary data based on the analysis results. It also configures the system to recommend more appropriate stores and routes based on the user's emotional state. For example, if the user is feeling tired, it will prioritize places where they can relax and routes that are quick to travel.

[1672] Step 6: Get the data

[1673] Subject: Server

[1674] The server collects the necessary data from multiple data sources, including satellite imagery APIs and aerial imagery APIs to obtain geographical information, and store information, user reviews, and other data from open databases.

[1675] Step 7: Generate the digital twin map

[1676] Subject: Server

[1677] The server integrates the collected data and generates a digital twin map that includes relevant information based on the user's requests, and displays recommended spots and routes that take into account the user's emotional state, among other things.

[1678] Step 8: Provide personalized information

[1679] Subject: Server

[1680] The server then personalizes the generated digital twin map and related information for the user, building data including optimal routes, recommended stores, detailed information, user reviews, and more based on individual requests, location information, and emotional state.

[1681] Step 9: Submit your information

[1682] Subject: Server

[1683] The server transmits the personalized information to the terminal, including map data and text information for visual display.

[1684] Step 10: Viewing Information

[1685] Subject: Terminal

[1686] The device displays the information received from the server on a user interface. Users can navigate and make selections on the displayed map while checking recommended routes, recommended locations, detailed information, user reviews, etc. The UI itself is also optimized according to the user's emotional state, providing, for example, color schemes and interactions that promote relaxation.

[1687] Specific examples

[1688] Example 1: Searching for family-friendly restaurants

[1689] Step 1: User: Enters into the smartphone app, "Tell me about a popular restaurant within 5 minutes of my current location where a family of four can enjoy a half-day meal." If the emotion engine determines that the user is tired, restaurants with a relaxed atmosphere will be prioritized.

[1690] Step 2: Device: Sends a request to the server along with its current location and emotional state.

[1691] Step 3: Server: Analyze the request and emotional state and collect the necessary data.

[1692] Step 4: Server: Generates a digital twin map and sends optimal route and restaurant recommendation information to the device.

[1693] Step 5: On your device: Display recommended restaurants and optimal routes on a map, along with detailed information and reviews.

[1694] Example 2: Searching for tourist spots

[1695] Step 1: User: Say to the car navigation system, "Tell me places I can visit within 10 minutes of my current location." If the emotion engine determines that the user is excited, active tourist spots will be prioritized.

[1696] Step 2: Terminal: Sends request to server.

[1697] Step 3: Server: Collect data based on the analysis results and emotional state to obtain tourist attraction information.

[1698] Step 4: Server: Generates a digital twin map and sends tourist spot and route information to the device.

[1699] Step 5: Device: Display the location of tourist attractions and the best route on the map.

[1700] This system allows users to easily and intuitively discover new things and enjoy new experiences, even when dealing with complex requests. The introduction of an emotion engine also enables the provision of more personalized services suited to the user's emotional state.

[1701] Example 2

[1702] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1703] Conventional information provision systems provide information taking into account the user's location and requests, but do not support personalization that takes into account the user's emotional state. As a result, user satisfaction can be low, and appropriate information cannot be provided in response to requests. Furthermore, there are issues with the lack of real-timeness and accuracy when generating digital twin maps.

[1704] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for analyzing emotional state data and determining the user's emotion, a means for optimizing collected data based on the user's emotional state, and a means for collecting necessary data from multiple data sources. This makes it possible to provide personalized information that takes the user's emotional state into consideration.

[1705] "User" means any person who uses the System to enter requests and obtain information.

[1706] A "request" refers to the information or service a user requests from a system.

[1707] "Server" refers to a computer system that analyzes received requests and collects, processes, and provides the necessary information.

[1708] "Emotional state data" refers to information about a user's emotional state or psychological state.

[1709] An "emotion engine" refers to an algorithm or system that analyzes the emotional state data provided by the user and recognizes specific emotions (excitement, calmness, fatigue, etc.).

[1710] "Natural language processing technology" refers to technology that analyzes text and voice information entered by users and converts it into a form that can be understood by humans.

[1711] A "digital twin map" refers to a map that digitally reproduces real-world geographic information and related data.

[1712] "Multiple data sources" refers to different databases and APIs (application programming interfaces) used to obtain information.

[1713] "Personalization" refers to optimizing and providing information according to each user's individual characteristics, requests, and emotions.

[1714] "Terminal" refers to a device (smartphone, car navigation system, etc.) through which a user inputs requests and displays information provided by the server.

[1715] The present invention is a system in which a user inputs a request via text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information. Furthermore, this system incorporates an emotion engine that recognizes the user's emotions and optimizes information based on the user's emotions. The present invention is primarily provided via smartphones and car navigation systems.

[1716] The user opens the application on their smartphone or car navigation system and inputs a request by text or voice. For example, they might input a request like, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day at." At this time, the user's emotional state is also recorded as a log.

[1717] The device sends the user's input request, location information, and emotional state to the server. In the case of voice input, it converts the voice data into text using speech recognition software (e.g., a common speech recognition API) and sends the prepared text and additional data to the server.

[1718] The server analyzes the received request data. It uses natural language processing technology (for example, a general natural language processing API) to analyze the request string and extract keywords, which clarify the information the user is looking for.

[1719] Furthermore, the server uses an emotion engine to analyze the emotional state data and determine the user's emotions (excitement, calmness, fatigue, etc.). Based on the analysis results, the server then prepares to collect necessary data. In particular, it configures the system to recommend more appropriate stores and routes based on the user's emotional state.

[1720] The server collects the necessary data from multiple data sources (e.g., satellite imagery APIs and open databases). It calls the satellite imagery API to obtain geographic information and retrieves store information, user reviews, etc. from the open database.

[1721] The collected data is integrated to generate a digital twin map that includes relevant information based on the user's requests and displays recommended spots and routes that take into account the user's emotional state, among other things.

[1722] The server then creates a digital twin map and provides related information personalized to the user, such as optimal routes, recommended stores, detailed information, and user reviews based on individual requests, location information, and emotional state.

[1723] Finally, the server transmits the personalized information to the terminal, including map data and text information for visual display.

[1724] The device displays the information received from the server on a user interface. Users can navigate and make selections on the displayed map while checking recommended routes, recommended locations, detailed information, user reviews, etc. The UI itself is also optimized according to the user's emotional state, providing, for example, color schemes and interactions that promote relaxation.

[1725] A concrete example is searching for a restaurant that the whole family can enjoy. If a user types into a smartphone app, "Tell me a popular restaurant within five minutes of my current location that a family of four can enjoy for half a day," and the emotion engine determines that the user is tired, the server will prioritize recommending restaurants with a relaxed atmosphere. The device sends the request and current location information to the server, which then analyzes, collects data, creates a map, and sends the final information to the device. The user can then view the restaurants and the optimal route displayed on the map.

[1726] Examples of prompt sentences include "Tell me about a popular restaurant within five minutes of our current location where a family of four can spend half a day," or "Tell me about a tourist spot within ten minutes that I can go to when I'm feeling excited." In response to such specific requests, the system of the present invention can quickly and appropriately provide personalized information.

[1727] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1728] Step 1: User enters request

[1729] Subject: User

[1730] The user opens a smartphone or car navigation application and inputs a request by text or voice. For example, "Tell me about a popular restaurant within five minutes of my current location that would be good for a family of four to spend half a day." Along with this, the user also records their emotional state. The input data includes the content of the request and the user's emotional state.

[1731] Step 2: Submitting the request

[1732] Subject: Terminal

[1733] The device sends the input request, location information, and the user's emotional state to the server. In the case of voice input, speech recognition software is used to convert the speech to text. The input data includes the text of the request, location information, and emotional state. The output is the prepared data sent to the server.

[1734] Step 3: Parsing the request

[1735] Subject: Server

[1736] The server analyzes the received request data and uses natural language processing technology to extract keywords from the request, such as "within 5 minutes," "family of 4," "half day," and "popular restaurant." The input is the text-converted request data, and the output is a list of extracted keywords.

[1737] Step 4: Sentiment Analysis

[1738] Subject: Server

[1739] The server uses an emotion engine to analyze the received emotional state data and determine what emotional state the user is in (e.g., excited, calm, tired, etc.). The input is the user's emotional state data, and the output is the interpreted emotional state (e.g., tired, excited, etc.).

[1740] Step 5: Prepare for data collection

[1741] Subject: Server

[1742] The server prepares to collect the necessary data based on the analysis results. It optimizes recommended stores and routes based on the user's emotional state. For example, if the user is tired, it prioritizes places where they can relax and routes that are quick to travel. The input is the extracted keywords and the interpretation of the user's emotional state, and the output is the data collection conditions.

[1743] Step 6: Get the data

[1744] Subject: Server

[1745] The server collects the necessary data from multiple data sources. Coordinate information is obtained using a satellite imagery API, and store information and user reviews are obtained from an open database. The input is the data collection conditions, and the output is the collected geographic information and related data.

[1746] Step 7: Generate the digital twin map

[1747] Subject: Server

[1748] The server integrates the collected data and generates a digital twin map. The map visually displays relevant information based on the user's request, including recommended spots and routes that take into account the user's emotional state. The input is the collected geographic information and related data, and the output is the generated digital twin map.

[1749] Step 8: Provide personalized information

[1750] Subject: Server

[1751] The server personalizes the generated digital twin map and related information for the user, building optimal routes, recommended stores, detailed information, user reviews, etc. based on individual requests, location information, and emotional state. The input is the generated digital twin map and related information, and the output is personalized information.

[1752] Step 9: Submit your information

[1753] Subject: Server

[1754] The server sends personalized information to the terminal, including map data and text information for visual display. The input is the personalized information, and the output is the data sent to the terminal.

[1755] Step 10: Viewing Information

[1756] Subject: Terminal

[1757] The device displays the information received from the server in a user interface. Users can view recommended routes, recommended places, detailed information, and user reviews on a map. The UI also provides optimal color schemes and interactions depending on the user's emotional state. The input is the received data, and the output is the displayed map and related information.

[1758] (Application example 2)

[1759] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1760] Conventional systems that provide information based on requests do not take into account the user's emotional state, making it difficult to fully improve user satisfaction.Furthermore, in food delivery services, the quality of service declines because the system does not select the store or menu that best suits the user's mood or emotion at the time.

[1761] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1762] In this invention, the server includes means for analyzing user emotions, means for providing personalized information, means for generating a digital twin map, and means for recommending optimal stores and menus to users based on data collected from multiple stores, thereby enabling optimal information provision and delivery services that take into account the user's requests and emotional state.

[1763] "User" means any person or entity that uses the System to enter a Request.

[1764] A "request" is a request or question that a user enters into a system.

[1765] "Analysis" refers to the process by which the server uses natural language processing and other technologies to understand the request and emotional state it receives and extract the necessary information.

[1766] A "digital twin map" is a map that digitally reproduces information from the physical world, and includes real-time data and its analysis results.

[1767] "Personalized information" is information that is individually optimized based on the user's requests and emotional state.

[1768] An "emotion engine" refers to software or algorithms that analyze a user's voice, text, or other input to determine their emotional state.

[1769] "Recommendation" is the process of presenting the best options to the user based on the analysis results.

[1770] "Store" refers to a place or business that offers goods or services.

[1771] A "menu" is a list of products offered by a store and their details.

[1772] "Collection" refers to the operation or process by which a server obtains the necessary data in response to a request.

[1773] This invention is a system in which a user inputs a request via text or voice, a server analyzes the request, generates a digital twin map based on the collected data, and provides personalized information according to the user's emotional state.

[1774] System Program Overview

[1775] 1. The user enters a request

[1776] Users open the smartphone application and input their request by text or voice. For example, they can input a request such as, "I've been feeling stressed lately, so please recommend some sweets that will help me relax." At this time, the user's emotional state is also recorded.

[1777] 2. Submitting a Request

[1778] The device sends the user's input request, current location, and emotional state to the server, and if the input is voice, it is converted into text.

[1779] 3. Parsing the Request

[1780] The server analyzes the received request data and uses natural language analysis technology to extract keywords such as "stress," "relaxation," and "sweets."

[1781] 4. Emotion analysis

[1782] The server analyzes the user's emotional state using an emotion engine, which uses the IBM Watson Tone Analyzer API.

[1783] 5. Preparation for Data Collection

[1784] The server prepares to collect the necessary data based on the analysis results and the user's emotional state, and prioritizes settings such as locations and menus suitable for relaxation.

[1785] 6. Data Acquisition

[1786] The server uses the Google Maps API and open databases to obtain information about stores that offer relaxing sweets.

[1787] 7. Digital Twin Map Generation

[1788] The server generates a digital twin map based on the collected data, which contains relevant information based on the user's request.

[1789] 8. Providing personalized information

[1790] The server then personalizes the generated digital twin map and related information for the user, including information on the best stores, menus, routes, etc. based on the user's request, emotional state, and location.

[1791] 9. Transmission of Information

[1792] The server sends the personalized information to the terminal.

[1793] 10. Display of Information

[1794] The device displays the received information on the user interface, where users can check the recommended route, detailed store information, and menu information on the displayed map.

[1795] Hardware and software used

[1796] Smartphone app: iOS or Android

[1797] Server: AWS Lambda, Amazon RDS

[1798] Data analysis engine: Python, TensorFlow

[1799] Sentiment analysis engine: IBM Watson Tone Analyzer API

[1800] Map data: Google Maps API

[1801] Specific examples

[1802] User: "I've been feeling stressed lately, so please recommend a relaxing dessert."

[1803] App: "Here are some relaxing sweets in the neighborhood. Recommended stores are XX Cafe and △△ Restaurant."

[1804] Prompt Sentence Examples

[1805] "Generate a Python program that provides optimal store information based on user requests and sentiment data."

[1806] In this way, the present invention provides a system that provides personalized information that takes into account the user's requests and emotional state.

[1807] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1808] Step 1:

[1809] The user enters a request

[1810] Input: A user types a request into a smartphone app by text or voice. For example, "I've been feeling stressed lately, so please recommend some sweets that will help me relax."

[1811] How it works: The application records the user's emotional state in parallel, where the input data are text data and emotional data.

[1812] Output: Text and emotion data are generated and ready to be sent to the next step.

[1813] Step 2:

[1814] Submitting a Request

[1815] Input: Request text data and emotion data entered by the user.

[1816] Operation: The device sends the request text data, emotion data, and current location information to the server. If the input is voice, it is converted into text data.

[1817] Output: The request text data, emotion data, and current location information are sent to the server.

[1818] Step 3:

[1819] Parsing the request

[1820] Input: The request text data sent to the server.

[1821] How it works: The server uses natural language analysis techniques to parse the request text and extract key keywords (e.g., "stress," "relax," "sweets"), using a generative AI model for this process.

[1822] Output: The extracted keyword data is generated and ready to be sent to the next step.

[1823] Step 4:

[1824] Emotion analysis

[1825] Input: Emotion data sent to the server.

[1826] How it works: The server uses an emotion engine (e.g., IBM Watson Tone Analyzer API) to analyze the user's emotional state. In this case, it analyzes that the user wants to relax.

[1827] Output: The parsed emotion data is generated and ready to be sent to the next step.

[1828] Step 5:

[1829] Preparing for data collection

[1830] Input: Extracted keyword data and parsed sentiment data.

[1831] How it works: Based on the analysis results, the server prepares data collection tailored to the user's needs (sweets with a relaxing effect).

[1832] Output: Your data collection setup is complete and you are ready to move on to the next step.

[1833] Step 6:

[1834] Data Acquisition

[1835] Input: Collection settings based on data the server is ready to collect.

[1836] How it works: The server calls the Google Maps API and open databases to retrieve information about stores that offer relaxing sweets.

[1837] Output: The retrieved store and menu information is generated and ready to be sent to the next step.

[1838] Step 7:

[1839] Digital twin map generation

[1840] Input: Store information and menu information obtained.

[1841] How it works: The server aggregates the collected data and generates a digital twin map that includes relevant information based on the user's request.

[1842] Output: The generated digital twin map data is prepared and sent to the next step.

[1843] Step 8:

[1844] Providing personalized information

[1845] Input: Generated digital twin map data, request, emotion data, and location information.

[1846] How it works: Based on the digital twin map and related information generated by the server, it creates information optimized for the user, including the best stores, menus, and route information.

[1847] Output: Personalized user information data is generated and sent to the next step.

[1848] Step 9:

[1849] Sending information

[1850] Input: Personalized user information data.

[1851] Operation: The server generates information and sends it to the device.

[1852] Output: The appropriate information data is sent to the terminal.

[1853] Step 10:

[1854] Displaying Information

[1855] Input: Personalized information data sent from the server.

[1856] How it works: The device displays the information it receives in a user interface, allowing users to view recommended routes, detailed store and menu information on the displayed map.

[1857] Output: A visual display is provided for the user to take information and order food delivery.

[1858] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1859] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1860] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1861] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1862] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1863] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1864] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1865] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1866] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1867] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1868] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1869] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1870] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1871] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1872] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1873] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1874] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1875] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1876] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1877] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1878] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1879] The following is further disclosed regarding the above embodiment.

[1880] (Claim 1)

[1881] A means for users to input requests via text or voice;

[1882] a means for sending a request to a server;

[1883] means for parsing the request at the server;

[1884] Means of collecting satellite images, aerial images, and open data;

[1885] a means for generating a digital twin map;

[1886] a means of providing personalized information;

[1887] The system includes a means for transmitting the generated information to a terminal and displaying it.

[1888] (Claim 2)

[1889] 10. The system of claim 1, further comprising means for obtaining and transmitting current location information along with the request to the server.

[1890] (Claim 3)

[1891] 10. The system of claim 1, further comprising means for using natural language parsing techniques in parsing the request.

[1892] (Claim 4)

[1893] 10. The system of claim 1, further comprising means for calculating an optimal route based on a user request and displaying it on the digital twin map.

[1894] (Claim 5)

[1895] 10. The system of claim 1, further comprising means for updating satellite imagery, aerial imagery, and open data in real time.

[1896] (Claim 6)

[1897] 10. The system of claim 1, further comprising means for optimizing and displaying a user interface based on the personalized information.

[1898] "Example 1"

[1899] (Claim 1)

[1900] A means for users to input requests via text or voice;

[1901] a means for sending a request to a server;

[1902] means for parsing the request at the server;

[1903] a means of collecting information from various data sources;

[1904] A means for generating a digital twin map;

[1905] a means of providing personalized information;

[1906] means for transmitting the generated information to a terminal and displaying it;

[1907] A means for the server to use the user's current location information to identify an area within a 5-minute range and prepare to collect data within that area;

[1908] The system includes a means for displaying spots based on user requests on a map based on data acquired by the server.

[1909] (Claim 2)

[1910] 10. The system of claim 1, further comprising means for obtaining and transmitting current location information along with the request to the server.

[1911] (Claim 3)

[1912] 10. The system of claim 1, further comprising means for using natural language parsing techniques in parsing the request.

[1913] "Application Example 1"

[1914] (Claim 1)

[1915] A means for users to input requests via text or voice;

[1916] a means for sending a request to a server;

[1917] means for parsing the request at the server;

[1918] Means of collecting satellite images, aerial images, and open data;

[1919] a means for generating a digital twin map;

[1920] a means of providing personalized information;

[1921] means for transmitting the generated information to a terminal and displaying it;

[1922] A means for a user to input a request for a virtual store using a smartphone or smart glasses;

[1923] A means for the server to provide detailed information about the virtual store based on the digital twin map;

[1924] A system including means for navigating a virtual store and viewing product information.

[1925] (Claim 2)

[1926] 10. The system of claim 1, further comprising means for obtaining and transmitting current location information along with the request to the server.

[1927] (Claim 3)

[1928] 10. The system of claim 1, further comprising means for using natural language parsing techniques in parsing the request.

[1929] "Example 2: Combining Emotion Engines"

[1930] (Claim 1)

[1931] A means for users to input requests via text or voice;

[1932] a means for sending a request to a server;

[1933] means for parsing the request at the server;

[1934] means for analyzing the emotional state data to determine the user's emotion;

[1935] a means for optimizing the data collected based on the user's emotional state;

[1936] A means of collecting the necessary data from multiple data sources;

[1937] a means for generating a digital twin map;

[1938] a means of providing personalized information;

[1939] The system includes a means for transmitting the generated information to a terminal and displaying it.

[1940] (Claim 2)

[1941] 10. The system of claim 1, further comprising means for obtaining and transmitting current location information along with the request to the server.

[1942] (Claim 3)

[1943] 10. The system of claim 1, further comprising means for using natural language parsing techniques in parsing the request.

[1944] "Application example 2 when combining emotion engines"

[1945] (Claim 1)

[1946] A means for users to input requests via text or voice;

[1947] a means for sending a request to a server;

[1948] means for parsing the request at the server;

[1949] Means of collecting satellite images, aerial images, and open data;

[1950] a means for generating a digital twin map;

[1951] a means of providing personalized information;

[1952] means for transmitting the generated information to a terminal and displaying it;

[1953] A means of analyzing the user's emotions using an emotion engine and providing information that is optimal for those emotions;

[1954] A method to recommend the best restaurant and menu to users based on data collected from multiple restaurants,

[1955] The system includes a means for integrating the data collected by the server and generating a map for displaying personalized store and menu information to the user.

[1956] (Claim 2)

[1957] 10. The system of claim 1, further comprising means for obtaining and transmitting current location information along with the request to the server.

[1958] (Claim 3)

[1959] 10. The system of claim 1, further comprising means for using natural language parsing techniques in parsing the request. [Explanation of symbols]

[1960] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for users to input requests via text or voice; a means for sending a request to a server; means for parsing the request at the server; Means of collecting satellite images, aerial images, and open data; a means for generating a digital twin map; a means of providing personalized information; The system includes a means for transmitting the generated information to a terminal and displaying it.

2. 10. The system of claim 1, further comprising means for obtaining and transmitting current location information along with the request to the server.

3. The system of claim 1 further comprising means for using natural language analysis techniques in analyzing the request.

4. The system of claim 1 , further comprising means for calculating an optimal route based on a user request and displaying it on the digital twin map.

5. The system of claim 1 , further comprising means for updating satellite imagery, aerial imagery, and open data in real time.

6. The system of claim 1 , further comprising means for optimizing and displaying a user interface based on the personalized information.

Images