System

Abstract

A system is provided.SOLUTION: A system including means for collecting text data related to taste from a gourmet site or a social networking service, means for cleaning the collected text data and extracting a keyword or phrase related to taste, means for assigning a tag using generative artificial intelligence based on the extracted keyword or phrase, means for storing the tagged information in a database, means for providing an interface through which a user can perform a search by designating an area or a detailed picky-about condition, means for querying the database based on a designated search condition and extracting a corresponding store or product, and means for displaying an extracted search result to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] On conventional gourmet websites and social networking services, when users search for a specific genre or restaurant, it is difficult to search based on detailed tastes and flavors. This requires users to manually search through numerous reviews and social media comments to find a restaurant that suits their tastes, which is extremely time-consuming. Furthermore, the wide variety of food and beverage genres makes it difficult to meet the needs of users with specific tastes. The goal of this project is to solve this problem and enable users to easily and efficiently find restaurants and products that suit their preferences. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. Specifically, the system includes a means for collecting text data related to food taste from gourmet websites and social networking services, and a means for cleaning the collected text data to extract keywords and phrases related to food taste. The system also includes a means for tagging the extracted keywords and phrases using generative artificial intelligence, and a means for storing the tagged information in a database. The system also includes a means for providing an interface that allows users to search by specifying an area or detailed preferences, and a means for querying the database based on the specified search criteria to extract relevant stores and products. Finally, the system includes a means for displaying the extracted search results to the user. This allows users to easily find restaurants and products that meet their specific preferences.

[0006] A "gourmet site" is a website that lists restaurant information, reviews, and ratings, and allows users to search and compare restaurants.

[0007] A "social networking service" is an online platform that enables people to interact and share information over the Internet.

[0008] "Text data" refers to information expressed in text, including information in the form of word-of-mouth reviews and comments.

[0009] "Cleaning" is the process of removing unnecessary special characters and noise, and organizing and shaping the data.

[0010] "Flavor" refers to the characteristics of food related to its taste and texture.

[0011] "Keywords and phrases" are important words or sequences of words that can be used to extract specific information from text data.

[0012] "Generative AI" refers to AI that automatically generates and analyzes information using machine learning and natural language processing technologies.

[0013] A "tag" is a specific label or keyword used to classify and identify data.

[0014] A "database" is a system that organizes and stores digital information so that it can be efficiently searched and accessed.

[0015] An "interface" is the screen and input methods that allow a user to interact with a system.

[0016] A "query" is an inquiry made to a database to search for and retrieve information.

[0017] "Extraction" is the process of selecting data based on specific criteria.

[0018] "Search results" refers to a list of relevant data and information provided by the system based on the conditions specified by the user. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6]FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention provides a system that allows users to search for restaurants and products based on area and specific criteria. The system includes a server, a user terminal, and an interface. Specific embodiments of the system are described below.

[0041] 1. System Overview

[0042] In this system, the server collects text data related to food taste from gourmet websites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. It also uses generative AI to assign tags based on the extracted keywords and phrases, and stores the tagged information in a database. Furthermore, it provides an interface that allows users to search by specifying areas and detailed preferences, and displays the search results to the user.

[0043] 2. Information gathering

[0044] The server periodically runs a crawler program to collect text data about food taste from gourmet sites and social networking services. The collected data is temporarily stored and filtered to remove duplication and noise.

[0045] As a concrete example, consider a case where a server collects data from multiple social networking services that have comments about "bagels." From the collected comments, it extracts sentences such as "This bagel is chewy and has plenty of sesame paste."

[0046] 3. Data cleaning and analysis

[0047] The server cleans the collected text data, removing special characters and noise, and then uses natural language processing technology to extract taste-related keywords and phrases from the text data.

[0048] Based on the above specific example, phrases such as "chewy" and "full of sesame paste" are extracted.

[0049] 4. Tagging

[0050] The server uses generative AI to automatically assign specific tags based on the extracted keywords and phrases. For example, the phrase "chewy" is tagged with "chewy texture," and the phrase "plenty of sesame paste" is tagged with "sesame paste."

[0051] 5. Saving to the database

[0052] The server stores the tagged information in a database, which also includes the URL of the original text data, the store name, and detailed product information.

[0053] 6. Providing a search interface

[0054] Users can use a search interface to specify areas and detailed search criteria via a web browser or smartphone application. For example, a user can search by specifying the criteria "Shinjuku," "chewy texture," "sesame paste," and "yumechikara."

[0055] 7. Search processing and result display

[0056] The terminal sends the search criteria entered by the user to the server. The server queries the database based on the received criteria and extracts stores and products that match the criteria. The extracted results are then sent to the terminal, which displays them to the user. The user can then check detailed information from the search results and select stores and products that suit their preferences.

[0057] For example, search results will display stores in Shinjuku that offer bagels with a chewy texture, sesame paste, and Yumechikara. Users can select the most suitable store from the list and check detailed information (such as opening hours and menu).

[0058] In this way, the system of the present invention allows users to easily and efficiently find restaurants and products that suit their preferences based on detailed tastes and flavors.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] The server runs a crawler program based on a list of URLs specified by gourmet sites and social networking services, and the crawler program retrieves text data (word of mouth reviews and comments) related to food taste from each URL.

[0062] Step 2:

[0063] The server temporarily stores the text data acquired by the crawler program, filtering out duplicate data and noise (such as advertisements and unnecessary text).

[0064] Step 3:

[0065] The server cleans the stored text data by removing special characters and unnecessary blank lines and formatting the data.

[0066] Step 4:

[0067] The server uses natural language processing (NLP) technology on the cleaned text data to extract keywords and phrases related to taste. For example, from a sentence such as "This bagel is chewy and filled with sesame paste," the server extracts the keywords "chewy" and "sesame paste."

[0068] Step 5:

[0069] The server uses a generation AI based on the extracted keywords and phrases. The generation AI assigns specific tags to each keyword or phrase. For example, "mochiri" is tagged as "mochiri texture," and "sesame paste" is tagged as "sesame paste."

[0070] Step 6:

[0071] The server stores the tagged information in a database, which also includes the tag, the URL of the original text data, the store name, and detailed product information.

[0072] Step 7:

[0073] The server provides a search interface where users can enter detailed search criteria. Users can access this interface through a web browser or a smartphone application.

[0074] Step 8:

[0075] The user uses the interface to input search criteria, such as "area: Shinjuku," "texture: chewy," "flavor: sesame paste," and "ingredients: dream power."

[0076] Step 9:

[0077] The terminal transmits the search conditions entered by the user to the server.

[0078] Step 10:

[0079] The server queries the database based on the received search criteria and extracts the relevant stores and products.

[0080] Step 11:

[0081] The server transmits the extracted search results to the terminal.

[0082] Step 12:

[0083] The terminal displays the received search results to the user, who can then check detailed information about stores and products that interest them from the displayed results.

[0084] Example 1

[0085] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0086] Conventional restaurant and product search systems have made it difficult for users to efficiently search based on detailed, specific criteria. Furthermore, the technology for accurately extracting and appropriately tagging information related to taste from collected text data is insufficient. Furthermore, the accuracy of displaying search results often fails to meet user expectations.

[0087] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0088] In this invention, the server includes means for collecting text data related to food taste from gourmet websites and social networking services, means for cleaning the collected text data to extract keywords and phrases related to food taste, means for tagging the extracted keywords and phrases using generative AI, means for saving the tagged information in a database, means for providing an interface that allows users to search by specifying an area or detailed preferences, means for querying the database based on the specified search criteria and extracting relevant stores and products, means for displaying the extracted search results to the user, means for periodically running a crawler program, means for removing special characters and noise from the cleaned data, means for analyzing the text data using a natural language processing tool, and means for automatically tagging the data using a generative AI model. This allows users to efficiently and accurately find restaurants and products that suit their preferences based on detailed tastes and flavors.

[0089] A "gourmet site" is a website that provides information about restaurants and cuisine.

[0090] A "social networking service" is an online platform that facilitates communication between users.

[0091] "Text data related to food taste" refers to data that includes descriptions and comments about the taste, texture, aroma, etc. of food and drinks.

[0092] "Means of collection" refers to the software or hardware functions used to obtain the target data.

[0093] "Cleaning means" is a function that removes unnecessary information and noise from text data.

[0094] "Keywords and phrases" are important words or sequences of words extracted from text data.

[0095] "Extraction methods" are software and algorithms that find the necessary information from the data.

[0096] "Generative AI" is AI that has the ability to generate new information and patterns from data.

[0097] A "tag" is a label that indicates an attribute or category that represents the content of data.

[0098] "Means for adding" is a function for adding information such as tags to data.

[0099] A "database" is a system that allows a collection of information to be efficiently managed and searched.

[0100] "Means of storage" refers to a function for retaining data for a certain period of time.

[0101] An "interface" is the screen and input method that a user uses to access and operate a system.

[0102] A "querying means" is a function for searching a database under specific conditions.

[0103] "Stores and products to be extracted" are restaurants and products that match the search conditions.

[0104] The "display means" is a function for visually showing the search results to the user.

[0105] A "crawler program" is software for automatically collecting information on the web.

[0106] "Special characters and noise" refers to unnecessary symbols and irrelevant information contained in text data.

[0107] "Natural language processing tools" are software and libraries for analyzing and processing text data.

[0108] "Means of analysis" are functions and algorithms for extracting useful information from data.

[0109] A "generative AI model" is an artificial intelligence model that can learn using large amounts of data and generate new information.

[0110] "Automatic attachment" refers to the ability of the system to automatically add tags and other information to data without human intervention.

[0111] MODE FOR CARRYING OUT THE INVENTION

[0112] This invention is a system that allows users to search for restaurants and products based on area and specific requirements. The system is composed of a server, a user terminal, and an interface. Specific embodiments of the system are described below.

[0113] System Overview

[0114] In this system, the server collects text data related to food taste from gourmet sites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. It also uses generative artificial intelligence (generative AI model) to assign tags based on the extracted keywords and phrases, and stores the tagged information in a database. It also provides an interface that allows users to search by specifying areas and detailed preferences, and displays the search results to the user.

[0115] Information gathering

[0116] The server periodically runs a crawler program (using, for example, Scrapy or BeautifulSoup) to collect text data related to food taste from gourmet sites and social networking services (SNS). For example, the server targets data related to "bagels" and collects comments from multiple SNSs. The collected comments include sentences such as "This bagel is chewy and has plenty of sesame paste."

[0117] Data Cleaning and Analysis

[0118] The server uses natural language processing (NLP) tools (such as NLTK or SpaCy) to clean the collected text data. This removes special characters and noise. After cleaning, natural language processing techniques are used to extract keywords and phrases related to taste from the data. Specifically, phrases such as "chewy" and "plenty of sesame paste" are extracted.

[0119] Tagging

[0120] The server automatically assigns tags based on keywords and phrases extracted using a generative AI model (e.g., BERT or GPT). For example, the phrase "chewy" is tagged with "chewy texture," and the phrase "plenty of sesame paste" is tagged with "sesame paste."

[0121] Saving to a database

[0122] The server stores the tagged information in a database (e.g., MySQL or PostgreSQL), which also includes the URL of the original text data, the store name, and detailed product information.

[0123] Providing a search interface

[0124] Users access the system using a web browser or smartphone application. The interface provides fields for entering area and detailed search criteria. For example, a user might search for "Shinjuku," "chewy texture," "sesame paste," and "yumechikara."

[0125] Search processing and result display

[0126] The device sends the search criteria entered by the user to the server. The server queries the database based on the received criteria and extracts stores and products that match the criteria. The extracted results are then sent to the device, which displays them to the user. For example, the search results show stores in Shinjuku that offer bagels with a chewy texture, sesame paste, and Yumechikara. The user can then select the most suitable store from the list and check detailed information (such as opening hours and menu).

[0127] Prompt Sentence Examples

[0128] "Find a store in Shinjuku that serves chewy bagels with sesame paste."

[0129] This system allows users to easily and efficiently find restaurants and products that suit their tastes based on detailed taste and flavor information.

[0130] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0131] Step 1:

[0132] The server periodically runs a crawler program to collect text data about food taste from gourmet sites and social networking services (SNS). The collected data consists of comments and reviews based on specific keywords. For example, data about "bagels" is collected. The input at this point is the keywords to be collected, and the output is the initial collected data.

[0133] Step 2:

[0134] The server temporarily stores the collected text data in a storage system such as a database. This data may contain duplicates and preliminary noise. The input is the collected initial data, and the output is the temporarily stored raw data.

[0135] Step 3:

[0136] The server cleans the stored text data. During this process, it uses natural language processing (NLP) tools (e.g., NLTK or SpaCy) to remove special characters and noise. The input is the temporarily stored raw data, and the output is the cleaned data.

[0137] Step 4:

[0138] The server extracts taste-related keywords and phrases from the cleaned text data. Specifically, it uses NLP technology to apply morphological analysis, TF-IDF, and other techniques. For example, it extracts phrases such as "chewy" and "full of sesame paste." The input is the cleaned data, and the output is the extracted keywords and phrases.

[0139] Step 5:

[0140] The server assigns tags based on extracted keywords and phrases using a generative AI model (e.g., BERT or GPT). For example, the phrase "chewy" is tagged with "chewy texture," and the phrase "plenty of sesame paste" is tagged with "sesame paste." The input is the extracted keywords and phrases, and the output is the tagged data.

[0141] Step 6:

[0142] The server stores the tagged information in a database (e.g., MySQL or PostgreSQL). This data includes the URL of the original text data, the store name, and detailed product information. The input is the tagged data, and the output is the information stored in the database.

[0143] Step 7:

[0144] Users access the system using a web browser or smartphone application. The interface provides fields for entering area and detailed search criteria. For example, users can enter criteria such as "Shinjuku," "chewy texture," "sesame paste," and "yumechikara." The input at this point is the user's search criteria.

[0145] Step 8:

[0146] The terminal sends the search criteria entered by the user to the server. This input is the criteria specified by the user in step 7. The output is the search query sent to the server.

[0147] Step 9:

[0148] The server queries the database based on the received search criteria and extracts stores and products that match the criteria. For example, search for stores in Shinjuku that offer bagels with a chewy texture, sesame paste, and Yumechikara. The input is the search query, and the output is the search results.

[0149] Step 10:

[0150] The server sends the extracted search results to the terminal. For example, it contains a list of stores that match "Shinjuku." At this point, the input is the search results, and the output is the transmitted data.

[0151] Step 11:

[0152] The terminal displays the received search results to the user. The user can select a store from the displayed results and check further details (such as opening hours and menu). The input is the search results sent, and the output is the information displayed on the user's screen.

[0153] In this way, a series of processes are performed, allowing the user to efficiently search for restaurants and products based on detailed, specific conditions.

[0154] (Application example 1)

[0155] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0156] Existing systems that allow users to search for restaurants and products based on area or specific preferences only display search results on a screen, making it difficult to provide detailed information in real time when selecting specific products in a physical store. In addition, there is a lack of a way for users to quickly obtain detailed product information in a store using a smart device.

[0157] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0158] In this invention, the server includes: means for collecting text data related to food taste from gourmet sites and social networking services; means for cleaning the collected text data to extract keywords and phrases related to food taste; means for tagging the extracted keywords and phrases using generative artificial intelligence; means for saving the tagged information in a database; means for providing an interface that allows a user to search by specifying an area or detailed preferences; means for querying the database based on the specified search conditions and extracting relevant stores and products; means for the smart glasses to display detailed information about products and stores based on eye tracking; and means for displaying the extracted search results to the user. This enables detailed information about a specific product to be displayed in real time on the smart glasses when the user looks at the product in a physical store.

[0159] A "gourmet site" is a website that provides information about restaurants and food.

[0160] A "social networking service" is a platform that allows users to exchange information and communicate with each other online.

[0161] "Text data" refers to data that is structured in the form of written words, sentences, comments, etc.

[0162] "Cleaning" is the process of filtering out noise and unnecessary information from collected data and extracting only the necessary parts.

[0163] A "keyword" is an important word that has a specific meaning in text data.

[0164] A "phrase" is a series of words that combine multiple keywords to give them meaning.

[0165] "Generative AI" is a machine learning technique that learns patterns from collected data and automatically tags and categorizes it.

[0166] A "tag" is a label that is attached to information to make it easier to classify or search.

[0167] A "database" is a system for systematically storing and managing data.

[0168] An "interface" is the means or screen through which a user interacts with a system.

[0169] A "query" is an inquiry issued to a database to retrieve specific information.

[0170] "Smart glasses" are devices equipped with displays and sensors that provide information based on the user's line of sight.

[0171] "Eye tracking" is a technology that detects the direction of a user's gaze and collects that data.

[0172] The present invention is a system that allows users to search for restaurants and products based on area and detailed preferences, and provides related information in real time. Specific embodiments of the system will be described below.

[0173] System Overview

[0174] The system consists of the following main components:

[0175] 1. Server: Responsible for data collection, cleaning, tagging, database management, and processing user queries.

[0176] 2. User terminal: Provides a search interface and displays search results. Includes smart glasses.

[0177] 3. Smart glasses: Provides real-time information display based on eye tracking.

[0178] Information gathering

[0179] The server automatically collects text data about food taste from gourmet sites and social networking services using information gathering software such as a web crawler. The collected data is temporarily stored on the server.

[0180] Data cleaning and tagging

[0181] The server cleans the collected text data, removing unnecessary noise and duplication, then uses natural language processing (NLP) to extract taste-related keywords and phrases from the text data, and then uses a generative AI model to automatically assign tags based on these keywords and phrases.

[0182] Database Management

[0183] The server stores the tagged information in a database, which also includes the URL of the original text data, the store name, and detailed product information.

[0184] Providing a search interface

[0185] Users can use a web browser or smartphone application to search by specifying an area and specific search criteria, such as "Shinjuku," "chewy texture," and "sesame paste."

[0186] Query processing and result display

[0187] The user terminal sends the entered search criteria to the server. The server queries the database based on the received criteria and extracts stores and products that match the criteria. The extracted results are then sent to the user terminal, allowing the user to view the search results.

[0188] Information display using smart glasses

[0189] The smart glasses use eye-tracking to detect which products or menu items the user is looking at. When the user's gaze is directed toward a specific product, the relevant information is received from the server and displayed on the smart glasses' display in real time.

[0190] Specific examples

[0191] For example, if a user wants to find a store in Shinjuku that serves bagels with a chewy texture, sesame paste, and Yumechikara, they can enter the search criteria into the interface, and the server will extract matching data. Then, when the user puts on the smart glasses and looks at a bagel in the store, detailed information about the product will be displayed on the glasses' display in real time.

[0192] Example prompts to be input to the generative AI model

[0193] "Imagine a user is searching for a bagel in the Shinjuku area that meets the criteria of 'chewy texture' and 'sesame paste'. Imagine a smart glasses application that displays detailed information about the bagel in real time."

[0194] In this way, the system of the present invention allows users to search for restaurants and products based on detailed preferences and obtain product information in real time within physical stores.

[0195] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0196] Step 1: Gather information

[0197] The server collects text data related to food taste from gourmet websites and social networking services. A crawler program crawls through web pages and extracts text data such as comments and reviews. The input is the website URL or API endpoint, and the output is the collected text data.

[0198] Step 2: Data cleaning

[0199] The server cleans the collected text data. Specifically, it removes unnecessary noise and redundant data and prepares it in a form that can be analyzed. The input is the collected text data, and the output is the cleaned text data.

[0200] Step 3: Extract keywords and phrases

[0201] The server uses natural language processing (NLP) technology to extract taste-related keywords and phrases from the cleaned text data. The input is the cleaned text data, and the output is taste-related keywords and phrases. Specifically, it performs morphological analysis.

[0202] Step 4: Tagging

[0203] The server automatically assigns tags using a generative AI model based on the extracted keywords and phrases. The input is keywords and phrases, and the output is tags assigned to each keyword or phrase. The generative AI understands the context and assigns tags.

[0204] Step 5: Saving to the Database

[0205] The server stores the tagged information in a database. The stored data includes the URL of the original text data, the store name, and detailed product information. The input is the tagged information, and the output is the data stored in the database.

[0206] Step 6: Provide a search interface

[0207] The user terminal provides an interface for searching by specifying an area or detailed specific conditions through a web browser or smartphone application. The input is the search conditions specified by the user, and the output is the transmission of the search conditions to the server.

[0208] Step 7: Query Processing

[0209] The server queries the database based on the search criteria received from the user and extracts stores and products that match the criteria. The input is the search criteria and the output is the extracted search results. Specifically, a database query language (such as SQL) is used.

[0210] Step 8: Viewing search results

[0211] The user terminal displays the search results received from the server to the user, with the input being the search result data from the server and the output being the search results displayed on the user interface.

[0212] Step 9: Eye tracking with smart glasses

[0213] The smart glasses use an eye-tracking function to detect the product or menu item that the user is looking at. The input is the user's gaze data, and the output is the ID of the detected product or menu item. Specifically, an eye-tracking sensor is used.

[0214] Step 10: Real-time information display

[0215] The server sends detailed information about the detected product or menu item to the smart glasses based on its ID. The smart glasses then display this information in real time. The input is the product or menu item ID, and the output is the detailed information displayed on the smart glasses' display. Specifically, wireless communication technology (such as Wi-Fi or Bluetooth) is used.

[0216] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0217] The present invention provides a system that allows users to search for restaurants and products based on area and specific preferences, and also includes a function that recognizes the user's emotions and optimizes search results and suggestions based on those emotions. The system includes a server, a user terminal, a search interface, and an emotion engine. Specific embodiments of the system are described below.

[0218] 1. System Overview

[0219] In this system, the server collects text data related to food taste from gourmet websites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. Generative AI is then used to assign tags based on the extracted keywords and phrases, and the tagged information is stored in a database. Users can use an interface to search by area or specific, detailed criteria, and the system displays search results. An emotion engine is also incorporated to optimize search results and suggestions based on the user's emotional data.

[0220] 2. Information gathering

[0221] The server periodically runs a crawler program that collects text data about food taste from gourmet websites and social networking services. The collected data is temporarily stored and filtered to remove duplication and noise.

[0222] As a concrete example, consider a case where a server collects data from a large number of social networking services that contain comments about "ramen." From the collected comments, it extracts sentences such as "This ramen has a rich soup and is very delicious."

[0223] 3. Data cleaning and analysis

[0224] The server cleans the collected text data, removing special characters and noise, and then uses natural language processing (NLP) technology to extract taste-related keywords and phrases from the text data.

[0225] Based on the above specific example, phrases such as "rich" and "delicious" are extracted.

[0226] 4. Tagging

[0227] The server uses a generation AI based on the extracted keywords and phrases. The generation AI automatically assigns specific tags to each keyword or phrase. For example, "rich" is assigned a tag such as "rich soup" and "delicious" is assigned a tag such as "highly rated."

[0228] 5. Saving to the database

[0229] The server stores the tagged information in a database, which also includes the tags, the URL of the original text data, the store name, and detailed product information.

[0230] 6. Introducing the Emotion Engine

[0231] The server is equipped with an emotion engine that analyzes the user's emotions. The emotion engine collects and analyzes emotion data from the user's facial expressions, voice, text input, etc., and estimates the user's emotional state.

[0232] 7. Providing a search interface

[0233] Users can use a web browser or smartphone application to search by area and specific search criteria. For example, a user can search for "Shinjuku" with the criteria "rich soup" and "highly rated."

[0234] 8. Search processing and result display

[0235] The device sends the search criteria entered by the user to the server. The server queries the database based on the received criteria and extracts relevant stores and products. The emotion engine uses the user's emotional data to adjust the display order and content of the search results.

[0236] For example, if the emotion engine recognizes that the user has a "very satisfied" expression, it will prioritize displaying stores that have received high ratings in the past. Also, if the user has a "highly anticipated" expression, it will recommend new or popular stores.

[0237] 9. Displaying the results

[0238] The server sends the optimized search results to the terminal, which then displays the received search results to the user. The user can check the detailed information of the stores and products that interest them from the displayed results and make the best selection according to their emotional state.

[0239] This embodiment allows the system of the present invention to provide personalized search results based on the user's emotional state in addition to the ability to search based on detailed user preferences.

[0240] The processing flow will be explained below.

[0241] Step 1:

[0242] The server runs a crawler program based on a list of URLs specified by gourmet sites and social networking services, and automatically retrieves text data (reviews and comments) related to food taste from each URL.

[0243] Step 2:

[0244] The server temporarily stores the text data acquired by the crawler program, filtering out duplicate data and noise (such as advertisements and unnecessary text).

[0245] Step 3:

[0246] The server cleans the stored text data, removing special characters and unnecessary blank lines and formatting the data.

[0247] Step 4:

[0248] The server uses natural language processing (NLP) technology on the cleaned text data to extract keywords and phrases related to taste. For example, "rich soup" and "delicious" are extracted from the text "This ramen has a rich soup and is very delicious."

[0249] Step 5:

[0250] The server then uses generative AI to tag the extracted keywords and phrases. For example, it might tag "rich" for "rich soup" and "highly rated" for "delicious."

[0251] Step 6:

[0252] The server stores the tagged information in a database, which also includes the tags, the URL of the original text data, the store name, and detailed product information.

[0253] Step 7:

[0254] The server includes an emotion engine that analyzes the user's emotions by collecting and analyzing emotion data from the user's facial expressions, voice, and text input.

[0255] Step 8:

[0256] Users access the search interface through a web browser or smartphone app and enter the area and detailed criteria they prefer, such as "Shinjuku," "rich soup," and "highly rated."

[0257] Step 9:

[0258] The terminal transmits the input search conditions to the server.

[0259] Step 10:

[0260] The server queries the database based on the received search criteria and extracts relevant stores and products. The emotion engine optimizes the display order and content of search results based on the user's emotional data.

[0261] Step 11:

[0262] The server transmits the optimized search results to the terminal.

[0263] Step 12:

[0264] The terminal displays the received search results to the user. For example, it displays a list of ramen restaurants in "Shinjuku" that meet the conditions of "rich soup" and "highly rated."

[0265] Step 13:

[0266] Users can check the detailed information of stores and products they are interested in from the displayed search results. Recommended stores and products are displayed preferentially based on the user's emotional data.

[0267] In this way, the system of the present invention can provide optimal restaurants and products based on the user's detailed preferences and feelings.

[0268] Example 2

[0269] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0270] Conventional restaurant and product search systems have the problem that it is difficult to search based on the user's detailed preferences, and the search results are not optimized according to the user's emotional state, which limits the user experience. A system that solves these problems is needed.

[0271] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting information about restaurants and products from websites and social media, means for cleaning the collected information and extracting keywords and phrases related to specific attributes, means for tagging the extracted keywords and phrases using a machine learning model, means for saving the tagged tags in a database, means for providing a user interface that allows a user to search by specifying an area or detailed specific conditions, means for querying the database based on the specified search conditions and extracting relevant information, means for displaying the extracted search results to the user, and means for analyzing the user's emotional data and optimizing the search results. This enables advanced searches based on the user's detailed specific conditions and personalized search results based on the user's emotional state.

[0272] "Information about restaurants and products" refers to data about ingredients, dishes, drinks, menus, services, store locations, business hours, ratings, reviews, etc.

[0273] "Websites and social media" refers to media and platforms on the Internet that provide information and enable users to interact with each other, and specifically includes gourmet sites and social networking sites.

[0274] "Means of collection" refers to the use of web crawlers or APIs to automatically obtain data from designated websites and social media.

[0275] "Cleaning means" refers to the process of removing duplicate data, invalid data, and noise from collected data, preparing it for analysis.

[0276] "Means for extracting keywords and phrases" refers to a method of using natural language processing techniques to identify important words and phrases from text data.

[0277] "Means of tagging using a machine learning model" refers to the process of automatically adding relevant tags to extracted keywords and phrases using a trained AI model.

[0278] "Means of saving to a database" refers to the method of storing the processed data in a relational database, NoSQL database, etc.

[0279] "Means for providing a user interface" refers to the method of providing a web page or mobile application that allows a user to enter search criteria and view results.

[0280] "Means of querying a database and extracting relevant information" refers to a method of searching for and retrieving the required information from a database using a query language such as SQL.

[0281] The "means for displaying search results to the user" refers to a method for visually presenting the acquired information to the user in the form of a list, graph, or the like.

[0282] "Means for analyzing emotional data and optimizing search results" refers to a process that analyzes emotions from the user's facial expressions, voice, text input, etc., and displays search results in the most optimal order for the user based on the results.

[0283] The present invention provides a system for efficiently collecting information about restaurants and products and providing optimal search results based on the user's detailed preferences and emotional state. Specific embodiments of this system are described in detail below.

[0284] The server runs a crawler program that collects information about restaurants and products from websites and social media. The collected data is temporarily stored and then cleaned. Cleaning involves analyzing HTML pages using Python's BeautifulSoup library and using regular expressions to remove special characters and noise. For example, if review comments about "ramen" are collected, keywords such as "rich" and "delicious" are extracted from those comments using regular expressions.

[0285] The server then uses natural language processing (NLP) techniques to extract specific keywords and phrases from the collected data. Specifically, it uses Python's NLTK (Natural Language Toolkit) library to extract phrases such as "rich" and "delicious." The extracted data is tagged using a generative AI model, such as GPT-3. For tagging, instructions are given to the generative AI using prompt sentences. The following prompt sentences are used as examples:

[0286] "Analyze the following sentence and generate keywords and appropriate tags related to taste.

[0287] Sentence: "This ramen has a rich soup and is very tasty."

[0288] Once tagged, the data is stored in a database. This is done using a database system such as PostgreSQL, and the information stored includes the tag, the URL of the original text data, the store name, detailed product information, etc. For example, the database contains the tag "rich soup" along with information about "Store A" and "Product B."

[0289] Users can use a search interface to specify areas and detailed search criteria through a web browser or smartphone application. The criteria specified by the user are the area name "Shinjuku" and keywords such as "rich soup" and "highly rated." These criteria are sent from the device to the server via an HTTP POST request. The server executes a query to the database based on the received criteria and extracts search results. SQL is used for the query.

[0290] The extracted search results are optimized using an emotion engine. The emotion engine analyzes emotional data from the user's facial expressions, voice, text input, etc. to determine the user's emotional state in real time. For example, if the user's facial expression indicates "satisfaction," the server will place stores that have received high ratings in the past at the top of the search results. Conversely, if the user's facial expression indicates "expectation," the server will recommend new or popular stores.

[0291] Finally, the device displays the optimized search results to the user. This involves parsing the JSON data and converting it into a user-friendly format. For example, the smartphone screen displays a list of store names, product names, ratings, and summaries.

[0292] This embodiment allows users to perform advanced searches based on detailed preferences and also provides personalized search results based on emotional state, improving the user experience and enabling optimal choices.

[0293] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0294] Step 1: Gather information

[0295] The server runs a crawler program to collect information about restaurants and products from specified websites and social media. The input is the specified query or URL, and the output is the collected raw data.

[0296] Specific operation: The server uses Python's BeautifulSoup library to parse HTML pages from specific gourmet sites and social media sites and extract review comments and ratings about restaurants and products.

[0297] Step 2: Data cleaning

[0298] The server removes duplicates and noise from the collected data. The input is the raw data collected in step 1, and the output is the clean data.

[0299] Specific operation: The server uses regular expressions to remove unnecessary special characters and noise components from the text data and filters out duplicate comments.

[0300] Step 3: Extract keywords and phrases

[0301] The server then uses natural language processing (NLP) techniques to extract specific keywords and phrases from the cleaned data. The input is the clean text data, and the output is the extracted keywords and phrases.

[0302] How it works: The server uses Python's NLTK library to extract keywords and phrases such as "rich" and "delicious" from the text data.

[0303] Step 4: Tagging

[0304] The server then assigns tags using a generative AI model based on the extracted keywords and phrases. The input is the extracted keywords and phrases, and the output is the tagged data.

[0305] How it works: The server uses a generative AI model like GPT-3 to generate appropriate tags for keywords (e.g., "rich soup" for "rich") based on a prompt.

[0306] Example prompt: "Analyze the following sentence and generate taste-related keywords and appropriate tags. Sentence: 'This ramen has a rich soup and is very delicious.'"

[0307] Step 5: Saving to the Database

[0308] The server stores the tagged data in a database. The input is the tagged data and the output is the information stored in the database.

[0309] Specific operation: The server executes an INSERT query to a database system such as PostgreSQL, and saves the tag, the URL of the original text data, the store name, and detailed product information in the database.

[0310] Step 6: Collect emotion data

[0311] The server collects and analyzes emotional data from the user's facial expressions, voice, text input, etc. The input is the user's emotional data, and the output is the analyzed emotional state.

[0312] Specific operation: The server inputs data obtained from the user's webcam and microphone into an emotion analysis algorithm to determine the user's emotional state in real time.

[0313] Step 7: Provide a search interface

[0314] Users use an interface to search by specifying an area and detailed preferences. The input is the user's search query, and the output is a search request based on that query.

[0315] Specific operation: The user enters the area name and keywords into a search form via a web browser or smartphone application.

[0316] Step 8: Search process

[0317] The terminal sends the user's search criteria to the server, which then executes a search query against the database. The input is the user's search query, and the output is the search results for the relevant information.

[0318] Specific operation: Upon receiving a search query sent from the terminal, the server generates an SQL query and searches the database.

[0319] Step 9: Optimize search results

[0320] The server uses an emotion engine to optimize search results based on the user's emotional state, where the inputs are search results and the user's emotional state, and the output is the optimized search results.

[0321] Specific operation: The emotion engine analyzes the user's emotional data and, if it recognizes the emotion of "satisfaction," places stores that have received high ratings in the past at the top of the search results.

[0322] Step 10: View the results

[0323] The server sends the optimized search results to the terminal, and the terminal displays the results to the user. The input is the optimized search results, and the output is the search results presented to the user.

[0324] Specific operation: The server sends optimized search results in JSON format, and the device displays the data in a user-friendly format.

[0325] (Application example 2)

[0326] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0327] Conventional Internet search systems only provide search results based on the area or specific conditions specified by the user, and have limitations in optimizing search results in response to the user's emotional state. Furthermore, to obtain highly satisfying results, it is necessary to dynamically adjust the results based on the user's emotions, but no such system exists yet. The objective of this invention is to solve this problem.

[0328] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0329] In this invention, the server includes means for collecting text data related to food taste from gourmet sites and social networking services, means for cleaning the collected text data to extract keywords and phrases related to food taste, means for tagging the extracted keywords and phrases using generative artificial intelligence, means for saving the tagged information in a database, means for providing an interface that allows users to specify areas and detailed search criteria to perform searches, means for querying the database based on the specified search criteria and extracting relevant stores and products, means for displaying the extracted search results to the user, means for analyzing a user's facial expression image to recognize the user's emotional state, and means for optimizing the search results based on the recognized emotional state, thereby enabling the provision of personalized search results according to the user's emotional state.

[0330] A "gourmet site" is a website that provides information about restaurants and allows users to search for, browse, and rate restaurants.

[0331] A "social networking service" is a web service that allows users to interact and share information online.

[0332] "Text data" is data that mainly consists of characters and symbols, and is information expressed in the form of sentences, comments, etc.

[0333] "Cleaning" refers to the process of removing unnecessary parts from data and making it accurate and usable.

[0334] A "keyword" is an important word or phrase for extracting specific information from text data.

[0335] A "phrase" is a meaningful clause or phrase within text data, and is a collection of words and phrases related to a specific purpose.

[0336] "Generative AI" is a type of machine learning technology that is artificial intelligence that recognizes patterns from given data and generates new data and information.

[0337] A "tag" is a label or identifier assigned to data, and is used to improve the efficiency of information classification and search.

[0338] A "database" is a system for efficiently organizing, storing, searching, and managing large amounts of data.

[0339] An "interface" is a means or screen through which a user operates a system or inputs information.

[0340] A "query" is a question or command used to search for information or to instruct a database to perform an operation.

[0341] "Emotional state" represents the user's psychological state and refers to emotions estimated from facial expressions, voice, text input, etc.

[0342] "Optimization" is the process of adjusting or improving something to maximize its efficiency or effectiveness for a specific purpose.

[0343] An "expression image" is image data capturing a user's facial expression.

[0344] "Personalization" means providing experiences and services that are customized according to the characteristics and preferences of individual users.

[0345] The present invention is a system that allows users to search for restaurants and products based on area and detailed preferences, and provides optimized search results by recognizing the user's emotional state. The system includes a server, a user terminal, a search interface, and an emotion engine.

[0346] System configuration

[0347] The server collects text data related to food taste from gourmet sites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. Based on the extracted keywords and phrases, generative artificial intelligence (generative AI) is used to assign specific tags, and the tagged information is stored in a database. Natural language processing (NLP) technology is used for collection, data cleaning, and keyword extraction.

[0348] The search interface is designed to allow users to search by specifying an area or specific detailed criteria. Users use this interface through a web browser or smartphone application to enter search criteria. The entered criteria are sent to the server, which queries the database to extract relevant stores and products.

[0349] The emotion engine analyzes the user's facial expressions and recognizes their emotional state. The analysis uses the image processing library OpenCV and a custom emotion analysis model (EmotionRecognizer). The analysis results are used as an index to optimize the user's search results.

[0350] Hardware and software used

[0351] Hardware:

[0352] Smartphone: The device on which the user accesses the application.

[0353] Camera: Captures the user's facial expressions using the smartphone's built-in camera.

[0354] software:

[0355] Flask: A Python web framework that acts as the backend for your application.

[0356] OpenCV: An image processing library used to analyze the user's facial expressions.

[0357] EmotionRecognizer: A custom emotion analysis model, used to recognize user emotions.

[0358] Requests: An HTTP library used to communicate with external APIs.

[0359] Specific examples of operation procedures

[0360] 1. The user launches the application, enters detailed preferences such as "I want to eat ramen in the Shinjuku area," and takes a photo of their own facial expression using their smartphone camera.

[0361] 2. The device uploads the captured facial expression image to the application and sends the area and preference conditions to the server.

[0362] 3. The server queries the database based on the received search criteria and generates a list of relevant stores.

[0363] 4. In parallel, the emotion engine analyzes the facial expression image and recognizes it as a "satisfied expression."

[0364] 5. The server optimizes the search results based on the results of sentiment analysis, prioritizing the display of highly rated ramen restaurants.

[0365] 6. The optimized search results are sent to the device, where the user can view the final results.

[0366] Prompt Sentence Examples

[0367] "The system uses images of the user's facial expressions as input to recognize the user's emotions. Based on the recognized emotions, it optimizes search results for restaurants.

[0368] Area: Shinjuku

[0369] Special requirements: Ramen

[0370] Facial image: user_image.jpg

[0371] Output: List of highly rated ramen restaurants"

[0372] In this way, the system of the present invention not only provides a search function based on the user's detailed preferences, but also provides personalized search results based on the user's emotional state.

[0373] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0374] Step 1:

[0375] The user launches the application, inputs the area and detailed preferences, and takes a photo of their facial expression using their smartphone camera. In this case, the user inputs preferences such as "I want to eat ramen in the Shinjuku area" and uploads an image of their facial expression. The input area information, preferences, and facial expression image are then acquired.

[0376] Step 2:

[0377] The device sends the search criteria entered by the user and the captured facial image to the server. Specifically, area information, preference criteria, and facial image data are transferred from the smartphone to the server. The input is area information, preference criteria, and facial image, and the output is data sent to the server.

[0378] Step 3:

[0379] The server queries the database based on the received area information and preferences, and generates a list of relevant stores. Here, it searches the tagged information stored in the database and extracts stores that match the conditions. The input is the area information and preferences, and the output is a list of relevant stores. Specifically, it searches for "ramen restaurants in the Shinjuku area."

[0380] Step 4:

[0381] In parallel, the server analyzes the facial expression image and recognizes the user's emotional state. This analysis is performed using OpenCV and EmotionRecognizer. The input is the facial expression image, and the output is the recognized emotional state. Specifically, it is analyzed as a "satisfied expression."

[0382] Step 5:

[0383] The server optimizes search results based on the emotional state obtained by the emotion engine. Specifically, if the recognized emotion is a "satisfied expression," it prioritizes displaying stores that have received high ratings in the past. The input is a list of relevant stores and the emotional state, and the output is an optimized list of stores.

[0384] Step 6:

[0385] The server sends the optimized search results to the terminal. The input is the optimized store list, and the output is the data transmission to the terminal.

[0386] Step 7:

[0387] The device displays the optimized search results to the user. The user can check the detailed information of the stores and products that interest them from the displayed results and make a selection according to their emotional state. Specifically, "highly rated ramen restaurants in Shinjuku" will be displayed.

[0388] In this way, the system of the present invention can dynamically optimize search results and provide personalized restaurant recommendations based on the user's detailed preferences and emotional state.

[0389] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0390] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0391] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0392] [Second embodiment]

[0393] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0394] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0395] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0396] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0397] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0398] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0399] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0400] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0401] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0402] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0403] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0404] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0405] The present invention provides a system that allows users to search for restaurants and products based on area and specific criteria. The system includes a server, a user terminal, and an interface. Specific embodiments of the system are described below.

[0406] 1. System Overview

[0407] In this system, the server collects text data related to food taste from gourmet websites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. It also uses generative AI to assign tags based on the extracted keywords and phrases, and stores the tagged information in a database. Furthermore, it provides an interface that allows users to search by specifying areas and detailed preferences, and displays the search results to the user.

[0408] 2. Information gathering

[0409] The server periodically runs a crawler program to collect text data about food taste from gourmet sites and social networking services. The collected data is temporarily stored and filtered to remove duplication and noise.

[0410] As a concrete example, consider a case where a server collects data from multiple social networking services that have comments about "bagels." From the collected comments, it extracts sentences such as "This bagel is chewy and has plenty of sesame paste."

[0411] 3. Data cleaning and analysis

[0412] The server cleans the collected text data, removing special characters and noise, and then uses natural language processing technology to extract taste-related keywords and phrases from the text data.

[0413] Based on the above specific example, phrases such as "chewy" and "full of sesame paste" are extracted.

[0414] 4. Tagging

[0415] The server uses generative AI to automatically assign specific tags based on the extracted keywords and phrases. For example, the phrase "chewy" is tagged with "chewy texture," and the phrase "plenty of sesame paste" is tagged with "sesame paste."

[0416] 5. Saving to the database

[0417] The server stores the tagged information in a database, which also includes the URL of the original text data, the store name, and detailed product information.

[0418] 6. Providing a search interface

[0419] Users can use a search interface to specify areas and detailed search criteria via a web browser or smartphone application. For example, a user can search by specifying the criteria "Shinjuku," "chewy texture," "sesame paste," and "yumechikara."

[0420] 7. Search processing and result display

[0421] The terminal sends the search criteria entered by the user to the server. The server queries the database based on the received criteria and extracts stores and products that match the criteria. The extracted results are then sent to the terminal, which displays them to the user. The user can then check detailed information from the search results and select stores and products that suit their preferences.

[0422] For example, search results will display stores in Shinjuku that offer bagels with a chewy texture, sesame paste, and Yumechikara. Users can select the most suitable store from the list and check detailed information (such as opening hours and menu).

[0423] In this way, the system of the present invention allows users to easily and efficiently find restaurants and products that suit their preferences based on detailed tastes and flavors.

[0424] The processing flow will be explained below.

[0425] Step 1:

[0426] The server runs a crawler program based on a list of URLs specified by gourmet sites and social networking services, and the crawler program retrieves text data (word of mouth reviews and comments) related to food taste from each URL.

[0427] Step 2:

[0428] The server temporarily stores the text data acquired by the crawler program, filtering out duplicate data and noise (such as advertisements and unnecessary text).

[0429] Step 3:

[0430] The server cleans the stored text data by removing special characters and unnecessary blank lines and formatting the data.

[0431] Step 4:

[0432] The server uses natural language processing (NLP) technology on the cleaned text data to extract keywords and phrases related to taste. For example, from a sentence such as "This bagel is chewy and filled with sesame paste," the server extracts the keywords "chewy" and "sesame paste."

[0433] Step 5:

[0434] The server uses a generation AI based on the extracted keywords and phrases. The generation AI assigns specific tags to each keyword or phrase. For example, "mochiri" is tagged as "mochiri texture," and "sesame paste" is tagged as "sesame paste."

[0435] Step 6:

[0436] The server stores the tagged information in a database, which also includes the tag, the URL of the original text data, the store name, and detailed product information.

[0437] Step 7:

[0438] The server provides a search interface where users can enter detailed search criteria. Users can access this interface through a web browser or a smartphone application.

[0439] Step 8:

[0440] The user uses the interface to input search criteria, such as "area: Shinjuku," "texture: chewy," "flavor: sesame paste," and "ingredients: dream power."

[0441] Step 9:

[0442] The terminal transmits the search conditions entered by the user to the server.

[0443] Step 10:

[0444] The server queries the database based on the received search criteria and extracts the relevant stores and products.

[0445] Step 11:

[0446] The server transmits the extracted search results to the terminal.

[0447] Step 12:

[0448] The terminal displays the received search results to the user, who can then check detailed information about stores and products that interest them from the displayed results.

[0449] Example 1

[0450] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0451] Conventional restaurant and product search systems have made it difficult for users to efficiently search based on detailed, specific criteria. Furthermore, the technology for accurately extracting and appropriately tagging information related to taste from collected text data is insufficient. Furthermore, the accuracy of displaying search results often fails to meet user expectations.

[0452] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0453] In this invention, the server includes means for collecting text data related to food taste from gourmet websites and social networking services, means for cleaning the collected text data to extract keywords and phrases related to food taste, means for tagging the extracted keywords and phrases using generative AI, means for saving the tagged information in a database, means for providing an interface that allows users to search by specifying an area or detailed preferences, means for querying the database based on the specified search criteria and extracting relevant stores and products, means for displaying the extracted search results to the user, means for periodically running a crawler program, means for removing special characters and noise from the cleaned data, means for analyzing the text data using a natural language processing tool, and means for automatically tagging the data using a generative AI model. This allows users to efficiently and accurately find restaurants and products that suit their preferences based on detailed tastes and flavors.

[0454] A "gourmet site" is a website that provides information about restaurants and cuisine.

[0455] A "social networking service" is an online platform that facilitates communication between users.

[0456] "Text data related to food taste" refers to data that includes descriptions and comments about the taste, texture, aroma, etc. of food and drinks.

[0457] "Means of collection" refers to the software or hardware functions used to obtain the target data.

[0458] "Cleaning means" is a function that removes unnecessary information and noise from text data.

[0459] "Keywords and phrases" are important words or sequences of words extracted from text data.

[0460] "Extraction methods" are software and algorithms that find the necessary information from the data.

[0461] "Generative AI" is AI that has the ability to generate new information and patterns from data.

[0462] A "tag" is a label that indicates an attribute or category that represents the content of data.

[0463] "Means for adding" is a function for adding information such as tags to data.

[0464] A "database" is a system that allows a collection of information to be efficiently managed and searched.

[0465] "Means of storage" refers to a function for retaining data for a certain period of time.

[0466] An "interface" is the screen and input method that a user uses to access and operate a system.

[0467] A "querying means" is a function for searching a database under specific conditions.

[0468] "Stores and products to be extracted" are restaurants and products that match the search conditions.

[0469] The "display means" is a function for visually showing the search results to the user.

[0470] A "crawler program" is software for automatically collecting information on the web.

[0471] "Special characters and noise" refers to unnecessary symbols and irrelevant information contained in text data.

[0472] "Natural language processing tools" are software and libraries for analyzing and processing text data.

[0473] "Means of analysis" are functions and algorithms for extracting useful information from data.

[0474] A "generative AI model" is an artificial intelligence model that can learn using large amounts of data and generate new information.

[0475] "Automatic attachment" refers to the ability of the system to automatically add tags and other information to data without human intervention.

[0476] MODE FOR CARRYING OUT THE INVENTION

[0477] This invention is a system that allows users to search for restaurants and products based on area and specific requirements. The system is composed of a server, a user terminal, and an interface. Specific embodiments of the system are described below.

[0478] System Overview

[0479] In this system, the server collects text data related to food taste from gourmet sites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. It also uses generative artificial intelligence (generative AI model) to assign tags based on the extracted keywords and phrases, and stores the tagged information in a database. It also provides an interface that allows users to search by specifying areas and detailed preferences, and displays the search results to the user.

[0480] Information gathering

[0481] The server periodically runs a crawler program (using, for example, Scrapy or BeautifulSoup) to collect text data related to food taste from gourmet sites and social networking services (SNS). For example, the server targets data related to "bagels" and collects comments from multiple SNSs. The collected comments include sentences such as "This bagel is chewy and has plenty of sesame paste."

[0482] Data Cleaning and Analysis

[0483] The server uses natural language processing (NLP) tools (such as NLTK or SpaCy) to clean the collected text data. This removes special characters and noise. After cleaning, natural language processing techniques are used to extract keywords and phrases related to taste from the data. Specifically, phrases such as "chewy" and "plenty of sesame paste" are extracted.

[0484] Tagging

[0485] The server automatically assigns tags based on keywords and phrases extracted using a generative AI model (e.g., BERT or GPT). For example, the phrase "chewy" is tagged with "chewy texture," and the phrase "plenty of sesame paste" is tagged with "sesame paste."

[0486] Saving to a database

[0487] The server stores the tagged information in a database (e.g., MySQL or PostgreSQL), which also includes the URL of the original text data, the store name, and detailed product information.

[0488] Providing a search interface

[0489] Users access the system using a web browser or smartphone application. The interface provides fields for entering area and detailed search criteria. For example, a user might search for "Shinjuku," "chewy texture," "sesame paste," and "yumechikara."

[0490] Search processing and result display

[0491] The device sends the search criteria entered by the user to the server. The server queries the database based on the received criteria and extracts stores and products that match the criteria. The extracted results are then sent to the device, which displays them to the user. For example, the search results show stores in Shinjuku that offer bagels with a chewy texture, sesame paste, and Yumechikara. The user can then select the most suitable store from the list and check detailed information (such as opening hours and menu).

[0492] Prompt Sentence Examples

[0493] "Find a store in Shinjuku that serves chewy bagels with sesame paste."

[0494] This system allows users to easily and efficiently find restaurants and products that suit their tastes based on detailed taste and flavor information.

[0495] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0496] Step 1:

[0497] The server periodically runs a crawler program to collect text data about food taste from gourmet sites and social networking services (SNS). The collected data consists of comments and reviews based on specific keywords. For example, data about "bagels" is collected. The input at this point is the keywords to be collected, and the output is the initial collected data.

[0498] Step 2:

[0499] The server temporarily stores the collected text data in a storage system such as a database. This data may contain duplicates and preliminary noise. The input is the collected initial data, and the output is the temporarily stored raw data.

[0500] Step 3:

[0501] The server cleans the stored text data. During this process, it uses natural language processing (NLP) tools (e.g., NLTK or SpaCy) to remove special characters and noise. The input is the temporarily stored raw data, and the output is the cleaned data.

[0502] Step 4:

[0503] The server extracts taste-related keywords and phrases from the cleaned text data. Specifically, it uses NLP technology to apply morphological analysis, TF-IDF, and other techniques. For example, it extracts phrases such as "chewy" and "full of sesame paste." The input is the cleaned data, and the output is the extracted keywords and phrases.

[0504] Step 5:

[0505] The server assigns tags based on extracted keywords and phrases using a generative AI model (e.g., BERT or GPT). For example, the phrase "chewy" is tagged with "chewy texture," and the phrase "plenty of sesame paste" is tagged with "sesame paste." The input is the extracted keywords and phrases, and the output is the tagged data.

[0506] Step 6:

[0507] The server stores the tagged information in a database (e.g., MySQL or PostgreSQL). This data includes the URL of the original text data, the store name, and detailed product information. The input is the tagged data, and the output is the information stored in the database.

[0508] Step 7:

[0509] Users access the system using a web browser or smartphone application. The interface provides fields for entering area and detailed search criteria. For example, users can enter criteria such as "Shinjuku," "chewy texture," "sesame paste," and "yumechikara." The input at this point is the user's search criteria.

[0510] Step 8:

[0511] The terminal sends the search criteria entered by the user to the server. This input is the criteria specified by the user in step 7. The output is the search query sent to the server.

[0512] Step 9:

[0513] The server queries the database based on the received search criteria and extracts stores and products that match the criteria. For example, search for stores in Shinjuku that offer bagels with a chewy texture, sesame paste, and Yumechikara. The input is the search query, and the output is the search results.

[0514] Step 10:

[0515] The server sends the extracted search results to the terminal. For example, it contains a list of stores that match "Shinjuku." At this point, the input is the search results, and the output is the transmitted data.

[0516] Step 11:

[0517] The terminal displays the received search results to the user. The user can select a store from the displayed results and check further details (such as opening hours and menu). The input is the search results sent, and the output is the information displayed on the user's screen.

[0518] In this way, a series of processes are performed, allowing the user to efficiently search for restaurants and products based on detailed, specific conditions.

[0519] (Application example 1)

[0520] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0521] Existing systems that allow users to search for restaurants and products based on area or specific preferences only display search results on a screen, making it difficult to provide detailed information in real time when selecting specific products in a physical store. In addition, there is a lack of a way for users to quickly obtain detailed product information in a store using a smart device.

[0522] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0523] In this invention, the server includes: means for collecting text data related to food taste from gourmet sites and social networking services; means for cleaning the collected text data to extract keywords and phrases related to food taste; means for tagging the extracted keywords and phrases using generative artificial intelligence; means for saving the tagged information in a database; means for providing an interface that allows a user to search by specifying an area or detailed preferences; means for querying the database based on the specified search conditions and extracting relevant stores and products; means for the smart glasses to display detailed information about products and stores based on eye tracking; and means for displaying the extracted search results to the user. This enables detailed information about a specific product to be displayed in real time on the smart glasses when the user looks at the product in a physical store.

[0524] A "gourmet site" is a website that provides information about restaurants and food.

[0525] A "social networking service" is a platform that allows users to exchange information and communicate with each other online.

[0526] "Text data" refers to data that is structured in the form of written words, sentences, comments, etc.

[0527] "Cleaning" is the process of filtering out noise and unnecessary information from collected data and extracting only the necessary parts.

[0528] A "keyword" is an important word that has a specific meaning in text data.

[0529] A "phrase" is a series of words that combine multiple keywords to give them meaning.

[0530] "Generative AI" is a machine learning technique that learns patterns from collected data and automatically tags and categorizes it.

[0531] A "tag" is a label that is attached to information to make it easier to classify or search.

[0532] A "database" is a system for systematically storing and managing data.

[0533] An "interface" is the means or screen through which a user interacts with a system.

[0534] A "query" is an inquiry issued to a database to retrieve specific information.

[0535] "Smart glasses" are devices equipped with displays and sensors that provide information based on the user's line of sight.

[0536] "Eye tracking" is a technology that detects the direction of a user's gaze and collects that data.

[0537] The present invention is a system that allows users to search for restaurants and products based on area and detailed preferences, and provides related information in real time. Specific embodiments of the system will be described below.

[0538] System Overview

[0539] The system consists of the following main components:

[0540] 1. Server: Responsible for data collection, cleaning, tagging, database management, and processing user queries.

[0541] 2. User terminal: Provides a search interface and displays search results. Includes smart glasses.

[0542] 3. Smart glasses: Provides real-time information display based on eye tracking.

[0543] Information gathering

[0544] The server automatically collects text data about food taste from gourmet sites and social networking services using information gathering software such as a web crawler. The collected data is temporarily stored on the server.

[0545] Data cleaning and tagging

[0546] The server cleans the collected text data, removing unnecessary noise and duplication, then uses natural language processing (NLP) to extract taste-related keywords and phrases from the text data, and then uses a generative AI model to automatically assign tags based on these keywords and phrases.

[0547] Database Management

[0548] The server stores the tagged information in a database, which also includes the URL of the original text data, the store name, and detailed product information.

[0549] Providing a search interface

[0550] Users can use a web browser or smartphone application to search by specifying an area and specific search criteria, such as "Shinjuku," "chewy texture," and "sesame paste."

[0551] Query processing and result display

[0552] The user terminal sends the entered search criteria to the server. The server queries the database based on the received criteria and extracts stores and products that match the criteria. The extracted results are then sent to the user terminal, allowing the user to view the search results.

[0553] Information display using smart glasses

[0554] The smart glasses use eye-tracking to detect which products or menu items the user is looking at. When the user's gaze is directed toward a specific product, the relevant information is received from the server and displayed on the smart glasses' display in real time.

[0555] Specific examples

[0556] For example, if a user wants to find a store in Shinjuku that serves bagels with a chewy texture, sesame paste, and Yumechikara, they can enter the search criteria into the interface, and the server will extract matching data. Then, when the user puts on the smart glasses and looks at a bagel in the store, detailed information about the product will be displayed on the glasses' display in real time.

[0557] Example prompts to be input to the generative AI model

[0558] "Imagine a user is searching for a bagel in the Shinjuku area that meets the criteria of 'chewy texture' and 'sesame paste'. Imagine a smart glasses application that displays detailed information about the bagel in real time."

[0559] In this way, the system of the present invention allows users to search for restaurants and products based on detailed preferences and obtain product information in real time within physical stores.

[0560] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0561] Step 1: Gather information

[0562] The server collects text data related to food taste from gourmet websites and social networking services. A crawler program crawls through web pages and extracts text data such as comments and reviews. The input is the website URL or API endpoint, and the output is the collected text data.

[0563] Step 2: Data cleaning

[0564] The server cleans the collected text data. Specifically, it removes unnecessary noise and redundant data and prepares it in a form that can be analyzed. The input is the collected text data, and the output is the cleaned text data.

[0565] Step 3: Extract keywords and phrases

[0566] The server uses natural language processing (NLP) technology to extract taste-related keywords and phrases from the cleaned text data. The input is the cleaned text data, and the output is taste-related keywords and phrases. Specifically, it performs morphological analysis.

[0567] Step 4: Tagging

[0568] The server automatically assigns tags using a generative AI model based on the extracted keywords and phrases. The input is keywords and phrases, and the output is tags assigned to each keyword or phrase. The generative AI understands the context and assigns tags.

[0569] Step 5: Saving to the Database

[0570] The server stores the tagged information in a database. The stored data includes the URL of the original text data, the store name, and detailed product information. The input is the tagged information, and the output is the data stored in the database.

[0571] Step 6: Provide a search interface

[0572] The user terminal provides an interface for searching by specifying an area or detailed specific conditions through a web browser or smartphone application. The input is the search conditions specified by the user, and the output is the transmission of the search conditions to the server.

[0573] Step 7: Query Processing

[0574] The server queries the database based on the search criteria received from the user and extracts stores and products that match the criteria. The input is the search criteria and the output is the extracted search results. Specifically, a database query language (such as SQL) is used.

[0575] Step 8: Viewing search results

[0576] The user terminal displays the search results received from the server to the user, with the input being the search result data from the server and the output being the search results displayed on the user interface.

[0577] Step 9: Eye tracking with smart glasses

[0578] The smart glasses use an eye-tracking function to detect the product or menu item that the user is looking at. The input is the user's gaze data, and the output is the ID of the detected product or menu item. Specifically, an eye-tracking sensor is used.

[0579] Step 10: Real-time information display

[0580] The server sends detailed information about the detected product or menu item to the smart glasses based on its ID. The smart glasses then display this information in real time. The input is the product or menu item ID, and the output is the detailed information displayed on the smart glasses' display. Specifically, wireless communication technology (such as Wi-Fi or Bluetooth) is used.

[0581] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0582] The present invention provides a system that allows users to search for restaurants and products based on area and specific preferences, and also includes a function that recognizes the user's emotions and optimizes search results and suggestions based on those emotions. The system includes a server, a user terminal, a search interface, and an emotion engine. Specific embodiments of the system are described below.

[0583] 1. System Overview

[0584] In this system, the server collects text data related to food taste from gourmet websites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. Generative AI is then used to assign tags based on the extracted keywords and phrases, and the tagged information is stored in a database. Users can use an interface to search by area or specific, detailed criteria, and the system displays search results. An emotion engine is also incorporated to optimize search results and suggestions based on the user's emotional data.

[0585] 2. Information gathering

[0586] The server periodically runs a crawler program that collects text data about food taste from gourmet websites and social networking services. The collected data is temporarily stored and filtered to remove duplication and noise.

[0587] As a concrete example, consider a case where a server collects data from a large number of social networking services that contain comments about "ramen." From the collected comments, it extracts sentences such as "This ramen has a rich soup and is very delicious."

[0588] 3. Data cleaning and analysis

[0589] The server cleans the collected text data, removing special characters and noise, and then uses natural language processing (NLP) technology to extract taste-related keywords and phrases from the text data.

[0590] Based on the above specific example, phrases such as "rich" and "delicious" are extracted.

[0591] 4. Tagging

[0592] The server uses a generation AI based on the extracted keywords and phrases. The generation AI automatically assigns specific tags to each keyword or phrase. For example, "rich" is assigned a tag such as "rich soup" and "delicious" is assigned a tag such as "highly rated."

[0593] 5. Saving to the database

[0594] The server stores the tagged information in a database, which also includes the tags, the URL of the original text data, the store name, and detailed product information.

[0595] 6. Introducing the Emotion Engine

[0596] The server is equipped with an emotion engine that analyzes the user's emotions. The emotion engine collects and analyzes emotion data from the user's facial expressions, voice, text input, etc., and estimates the user's emotional state.

[0597] 7. Providing a search interface

[0598] Users can use a web browser or smartphone application to search by area and specific search criteria. For example, a user can search for "Shinjuku" with the criteria "rich soup" and "highly rated."

[0599] 8. Search processing and result display

[0600] The device sends the search criteria entered by the user to the server. The server queries the database based on the received criteria and extracts relevant stores and products. The emotion engine uses the user's emotional data to adjust the display order and content of the search results.

[0601] For example, if the emotion engine recognizes that the user has a "very satisfied" expression, it will prioritize displaying stores that have received high ratings in the past. Also, if the user has a "highly anticipated" expression, it will recommend new or popular stores.

[0602] 9. Displaying the results

[0603] The server sends the optimized search results to the terminal, which then displays the received search results to the user. The user can check the detailed information of the stores and products that interest them from the displayed results and make the best selection according to their emotional state.

[0604] This embodiment allows the system of the present invention to provide personalized search results based on the user's emotional state in addition to the ability to search based on detailed user preferences.

[0605] The processing flow will be explained below.

[0606] Step 1:

[0607] The server runs a crawler program based on a list of URLs specified by gourmet sites and social networking services, and automatically retrieves text data (reviews and comments) related to food taste from each URL.

[0608] Step 2:

[0609] The server temporarily stores the text data acquired by the crawler program, filtering out duplicate data and noise (such as advertisements and unnecessary text).

[0610] Step 3:

[0611] The server cleans the stored text data, removing special characters and unnecessary blank lines and formatting the data.

[0612] Step 4:

[0613] The server uses natural language processing (NLP) technology on the cleaned text data to extract keywords and phrases related to taste. For example, "rich soup" and "delicious" are extracted from the text "This ramen has a rich soup and is very delicious."

[0614] Step 5:

[0615] The server then uses generative AI to tag the extracted keywords and phrases. For example, it might tag "rich" for "rich soup" and "highly rated" for "delicious."

[0616] Step 6:

[0617] The server stores the tagged information in a database, which also includes the tags, the URL of the original text data, the store name, and detailed product information.

[0618] Step 7:

[0619] The server includes an emotion engine that analyzes the user's emotions by collecting and analyzing emotion data from the user's facial expressions, voice, and text input.

[0620] Step 8:

[0621] Users access the search interface through a web browser or smartphone app and enter the area and detailed criteria they prefer, such as "Shinjuku," "rich soup," and "highly rated."

[0622] Step 9:

[0623] The terminal transmits the input search conditions to the server.

[0624] Step 10:

[0625] The server queries the database based on the received search criteria and extracts relevant stores and products. The emotion engine optimizes the display order and content of search results based on the user's emotional data.

[0626] Step 11:

[0627] The server transmits the optimized search results to the terminal.

[0628] Step 12:

[0629] The terminal displays the received search results to the user. For example, it displays a list of ramen restaurants in "Shinjuku" that meet the conditions of "rich soup" and "highly rated."

[0630] Step 13:

[0631] Users can check the detailed information of stores and products they are interested in from the displayed search results. Recommended stores and products are displayed preferentially based on the user's emotional data.

[0632] In this way, the system of the present invention can provide optimal restaurants and products based on the user's detailed preferences and feelings.

[0633] Example 2

[0634] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0635] Conventional restaurant and product search systems have the problem that it is difficult to search based on the user's detailed preferences, and the search results are not optimized according to the user's emotional state, which limits the user experience. A system that solves these problems is needed.

[0636] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting information about restaurants and products from websites and social media, means for cleaning the collected information and extracting keywords and phrases related to specific attributes, means for tagging the extracted keywords and phrases using a machine learning model, means for saving the tagged tags in a database, means for providing a user interface that allows a user to search by specifying an area or detailed specific conditions, means for querying the database based on the specified search conditions and extracting relevant information, means for displaying the extracted search results to the user, and means for analyzing the user's emotional data and optimizing the search results. This enables advanced searches based on the user's detailed specific conditions and personalized search results based on the user's emotional state.

[0637] "Information about restaurants and products" refers to data about ingredients, dishes, drinks, menus, services, store locations, business hours, ratings, reviews, etc.

[0638] "Websites and social media" refers to media and platforms on the Internet that provide information and enable users to interact with each other, and specifically includes gourmet sites and social networking sites.

[0639] "Means of collection" refers to the use of web crawlers or APIs to automatically obtain data from designated websites and social media.

[0640] "Cleaning means" refers to the process of removing duplicate data, invalid data, and noise from collected data, preparing it for analysis.

[0641] "Means for extracting keywords and phrases" refers to a method of using natural language processing techniques to identify important words and phrases from text data.

[0642] "Means of tagging using a machine learning model" refers to the process of automatically adding relevant tags to extracted keywords and phrases using a trained AI model.

[0643] "Means of saving to a database" refers to the method of storing the processed data in a relational database, NoSQL database, etc.

[0644] "Means for providing a user interface" refers to the method of providing a web page or mobile application that allows a user to enter search criteria and view results.

[0645] "Means of querying a database and extracting relevant information" refers to a method of searching for and retrieving the required information from a database using a query language such as SQL.

[0646] The "means for displaying search results to the user" refers to a method for visually presenting the acquired information to the user in the form of a list, graph, or the like.

[0647] "Means for analyzing emotional data and optimizing search results" refers to a process that analyzes emotions from the user's facial expressions, voice, text input, etc., and displays search results in the most optimal order for the user based on the results.

[0648] The present invention provides a system for efficiently collecting information about restaurants and products and providing optimal search results based on the user's detailed preferences and emotional state. Specific embodiments of this system are described in detail below.

[0649] The server runs a crawler program that collects information about restaurants and products from websites and social media. The collected data is temporarily stored and then cleaned. Cleaning involves analyzing HTML pages using Python's BeautifulSoup library and using regular expressions to remove special characters and noise. For example, if review comments about "ramen" are collected, keywords such as "rich" and "delicious" are extracted from those comments using regular expressions.

[0650] The server then uses natural language processing (NLP) techniques to extract specific keywords and phrases from the collected data. Specifically, it uses Python's NLTK (Natural Language Toolkit) library to extract phrases such as "rich" and "delicious." The extracted data is tagged using a generative AI model, such as GPT-3. For tagging, instructions are given to the generative AI using prompt sentences. The following prompt sentences are used as examples:

[0651] "Analyze the following sentence and generate keywords and appropriate tags related to taste.

[0652] Sentence: "This ramen has a rich soup and is very tasty."

[0653] Once tagged, the data is stored in a database. This is done using a database system such as PostgreSQL, and the information stored includes the tag, the URL of the original text data, the store name, detailed product information, etc. For example, the database contains the tag "rich soup" along with information about "Store A" and "Product B."

[0654] Users can use a search interface to specify areas and detailed search criteria through a web browser or smartphone application. The criteria specified by the user are the area name "Shinjuku" and keywords such as "rich soup" and "highly rated." These criteria are sent from the device to the server via an HTTP POST request. The server executes a query to the database based on the received criteria and extracts search results. SQL is used for the query.

[0655] The extracted search results are optimized using an emotion engine. The emotion engine analyzes emotional data from the user's facial expressions, voice, text input, etc. to determine the user's emotional state in real time. For example, if the user's facial expression indicates "satisfaction," the server will place stores that have received high ratings in the past at the top of the search results. Conversely, if the user's facial expression indicates "expectation," the server will recommend new or popular stores.

[0656] Finally, the device displays the optimized search results to the user. This involves parsing the JSON data and converting it into a user-friendly format. For example, the smartphone screen displays a list of store names, product names, ratings, and summaries.

[0657] This embodiment allows users to perform advanced searches based on detailed preferences and also provides personalized search results based on emotional state, improving the user experience and enabling optimal choices.

[0658] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0659] Step 1: Gather information

[0660] The server runs a crawler program to collect information about restaurants and products from specified websites and social media. The input is the specified query or URL, and the output is the collected raw data.

[0661] Specific operation: The server uses Python's BeautifulSoup library to parse HTML pages from specific gourmet sites and social media sites and extract review comments and ratings about restaurants and products.

[0662] Step 2: Data cleaning

[0663] The server removes duplicates and noise from the collected data. The input is the raw data collected in step 1, and the output is the clean data.

[0664] Specific operation: The server uses regular expressions to remove unnecessary special characters and noise components from the text data and filters out duplicate comments.

[0665] Step 3: Extract keywords and phrases

[0666] The server then uses natural language processing (NLP) techniques to extract specific keywords and phrases from the cleaned data. The input is the clean text data, and the output is the extracted keywords and phrases.

[0667] How it works: The server uses Python's NLTK library to extract keywords and phrases such as "rich" and "delicious" from the text data.

[0668] Step 4: Tagging

[0669] The server then assigns tags using a generative AI model based on the extracted keywords and phrases. The input is the extracted keywords and phrases, and the output is the tagged data.

[0670] How it works: The server uses a generative AI model like GPT-3 to generate appropriate tags for keywords (e.g., "rich soup" for "rich") based on a prompt.

[0671] Example prompt: "Analyze the following sentence and generate taste-related keywords and appropriate tags. Sentence: 'This ramen has a rich soup and is very delicious.'"

[0672] Step 5: Saving to the Database

[0673] The server stores the tagged data in a database. The input is the tagged data and the output is the information stored in the database.

[0674] Specific operation: The server executes an INSERT query to a database system such as PostgreSQL, and saves the tag, the URL of the original text data, the store name, and detailed product information in the database.

[0675] Step 6: Collect emotion data

[0676] The server collects and analyzes emotional data from the user's facial expressions, voice, text input, etc. The input is the user's emotional data, and the output is the analyzed emotional state.

[0677] Specific operation: The server inputs data obtained from the user's webcam and microphone into an emotion analysis algorithm to determine the user's emotional state in real time.

[0678] Step 7: Provide a search interface

[0679] Users use an interface to search by specifying an area and detailed preferences. The input is the user's search query, and the output is a search request based on that query.

[0680] Specific operation: The user enters the area name and keywords into a search form via a web browser or smartphone application.

[0681] Step 8: Search process

[0682] The terminal sends the user's search criteria to the server, which then executes a search query against the database. The input is the user's search query, and the output is the search results for the relevant information.

[0683] Specific operation: Upon receiving a search query sent from the terminal, the server generates an SQL query and searches the database.

[0684] Step 9: Optimize search results

[0685] The server uses an emotion engine to optimize search results based on the user's emotional state, where the inputs are search results and the user's emotional state, and the output is the optimized search results.

[0686] Specific operation: The emotion engine analyzes the user's emotional data and, if it recognizes the emotion of "satisfaction," places stores that have received high ratings in the past at the top of the search results.

[0687] Step 10: View the results

[0688] The server sends the optimized search results to the terminal, and the terminal displays the results to the user. The input is the optimized search results, and the output is the search results presented to the user.

[0689] Specific operation: The server sends optimized search results in JSON format, and the device displays the data in a user-friendly format.

[0690] (Application example 2)

[0691] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0692] Conventional Internet search systems only provide search results based on the area or specific conditions specified by the user, and have limitations in optimizing search results in response to the user's emotional state. Furthermore, to obtain highly satisfying results, it is necessary to dynamically adjust the results based on the user's emotions, but no such system exists yet. The objective of this invention is to solve this problem.

[0693] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0694] In this invention, the server includes means for collecting text data related to food taste from gourmet sites and social networking services, means for cleaning the collected text data to extract keywords and phrases related to food taste, means for tagging the extracted keywords and phrases using generative artificial intelligence, means for saving the tagged information in a database, means for providing an interface that allows users to specify areas and detailed search criteria to perform searches, means for querying the database based on the specified search criteria and extracting relevant stores and products, means for displaying the extracted search results to the user, means for analyzing a user's facial expression image to recognize the user's emotional state, and means for optimizing the search results based on the recognized emotional state, thereby enabling the provision of personalized search results according to the user's emotional state.

[0695] A "gourmet site" is a website that provides information about restaurants and allows users to search for, browse, and rate restaurants.

[0696] A "social networking service" is a web service that allows users to interact and share information online.

[0697] "Text data" is data that mainly consists of characters and symbols, and is information expressed in the form of sentences, comments, etc.

[0698] "Cleaning" refers to the process of removing unnecessary parts from data and making it accurate and usable.

[0699] A "keyword" is an important word or phrase for extracting specific information from text data.

[0700] A "phrase" is a meaningful clause or phrase within text data, and is a collection of words and phrases related to a specific purpose.

[0701] "Generative AI" is a type of machine learning technology that is artificial intelligence that recognizes patterns from given data and generates new data and information.

[0702] A "tag" is a label or identifier assigned to data, and is used to improve the efficiency of information classification and search.

[0703] A "database" is a system for efficiently organizing, storing, searching, and managing large amounts of data.

[0704] An "interface" is a means or screen through which a user operates a system or inputs information.

[0705] A "query" is a question or command used to search for information or to instruct a database to perform an operation.

[0706] "Emotional state" represents the user's psychological state and refers to emotions estimated from facial expressions, voice, text input, etc.

[0707] "Optimization" is the process of adjusting or improving something to maximize its efficiency or effectiveness for a specific purpose.

[0708] An "expression image" is image data capturing a user's facial expression.

[0709] "Personalization" means providing experiences and services that are customized according to the characteristics and preferences of individual users.

[0710] The present invention is a system that allows users to search for restaurants and products based on area and detailed preferences, and provides optimized search results by recognizing the user's emotional state. The system includes a server, a user terminal, a search interface, and an emotion engine.

[0711] System configuration

[0712] The server collects text data related to food taste from gourmet sites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. Based on the extracted keywords and phrases, generative artificial intelligence (generative AI) is used to assign specific tags, and the tagged information is stored in a database. Natural language processing (NLP) technology is used for collection, data cleaning, and keyword extraction.

[0713] The search interface is designed to allow users to search by specifying an area or specific detailed criteria. Users use this interface through a web browser or smartphone application to enter search criteria. The entered criteria are sent to the server, which queries the database to extract relevant stores and products.

[0714] The emotion engine analyzes the user's facial expressions and recognizes their emotional state. The analysis uses the image processing library OpenCV and a custom emotion analysis model (EmotionRecognizer). The analysis results are used as an index to optimize the user's search results.

[0715] Hardware and software used

[0716] Hardware:

[0717] Smartphone: The device on which the user accesses the application.

[0718] Camera: Captures the user's facial expressions using the smartphone's built-in camera.

[0719] software:

[0720] Flask: A Python web framework that acts as the backend for your application.

[0721] OpenCV: An image processing library used to analyze the user's facial expressions.

[0722] EmotionRecognizer: A custom emotion analysis model, used to recognize user emotions.

[0723] Requests: An HTTP library used to communicate with external APIs.

[0724] Specific examples of operation procedures

[0725] 1. The user launches the application, enters detailed preferences such as "I want to eat ramen in the Shinjuku area," and takes a photo of their own facial expression using their smartphone camera.

[0726] 2. The device uploads the captured facial expression image to the application and sends the area and preference conditions to the server.

[0727] 3. The server queries the database based on the received search criteria and generates a list of relevant stores.

[0728] 4. In parallel, the emotion engine analyzes the facial expression image and recognizes it as a "satisfied expression."

[0729] 5. The server optimizes the search results based on the results of sentiment analysis, prioritizing the display of highly rated ramen restaurants.

[0730] 6. The optimized search results are sent to the device, where the user can view the final results.

[0731] Prompt Sentence Examples

[0732] "The system uses images of the user's facial expressions as input to recognize the user's emotions. Based on the recognized emotions, it optimizes search results for restaurants.

[0733] Area: Shinjuku

[0734] Special requirements: Ramen

[0735] Facial image: user_image.jpg

[0736] Output: List of highly rated ramen restaurants"

[0737] In this way, the system of the present invention not only provides a search function based on the user's detailed preferences, but also provides personalized search results based on the user's emotional state.

[0738] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0739] Step 1:

[0740] The user launches the application, inputs the area and detailed preferences, and takes a photo of their facial expression using their smartphone camera. In this case, the user inputs preferences such as "I want to eat ramen in the Shinjuku area" and uploads an image of their facial expression. The input area information, preferences, and facial expression image are then acquired.

[0741] Step 2:

[0742] The device sends the search criteria entered by the user and the captured facial image to the server. Specifically, area information, preference criteria, and facial image data are transferred from the smartphone to the server. The input is area information, preference criteria, and facial image, and the output is data sent to the server.

[0743] Step 3:

[0744] The server queries the database based on the received area information and preferences, and generates a list of relevant stores. Here, it searches the tagged information stored in the database and extracts stores that match the conditions. The input is the area information and preferences, and the output is a list of relevant stores. Specifically, it searches for "ramen restaurants in the Shinjuku area."

[0745] Step 4:

[0746] In parallel, the server analyzes the facial expression image and recognizes the user's emotional state. This analysis is performed using OpenCV and EmotionRecognizer. The input is the facial expression image, and the output is the recognized emotional state. Specifically, it is analyzed as a "satisfied expression."

[0747] Step 5:

[0748] The server optimizes search results based on the emotional state obtained by the emotion engine. Specifically, if the recognized emotion is a "satisfied expression," it prioritizes displaying stores that have received high ratings in the past. The input is a list of relevant stores and the emotional state, and the output is an optimized list of stores.

[0749] Step 6:

[0750] The server sends the optimized search results to the terminal. The input is the optimized store list, and the output is the data transmission to the terminal.

[0751] Step 7:

[0752] The device displays the optimized search results to the user. The user can check the detailed information of the stores and products that interest them from the displayed results and make a selection according to their emotional state. Specifically, "highly rated ramen restaurants in Shinjuku" will be displayed.

[0753] In this way, the system of the present invention can dynamically optimize search results and provide personalized restaurant recommendations based on the user's detailed preferences and emotional state.

[0754] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0755] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0756] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0757] [Third embodiment]

[0758] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0759] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0760] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0761] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0762] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0763] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0764] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0765] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0766] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0767] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0768] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0769] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0770] The present invention provides a system that allows users to search for restaurants and products based on area and specific criteria. The system includes a server, a user terminal, and an interface. Specific embodiments of the system are described below.

[0771] 1. System Overview

[0772] In this system, the server collects text data related to food taste from gourmet websites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. It also uses generative AI to assign tags based on the extracted keywords and phrases, and stores the tagged information in a database. Furthermore, it provides an interface that allows users to search by specifying areas and detailed preferences, and displays the search results to the user.

[0773] 2. Information gathering

[0774] The server periodically runs a crawler program to collect text data about food taste from gourmet sites and social networking services. The collected data is temporarily stored and filtered to remove duplication and noise.

[0775] As a concrete example, consider a case where a server collects data from multiple social networking services that have comments about "bagels." From the collected comments, it extracts sentences such as "This bagel is chewy and has plenty of sesame paste."

[0776] 3. Data cleaning and analysis

[0777] The server cleans the collected text data, removing special characters and noise, and then uses natural language processing technology to extract taste-related keywords and phrases from the text data.

[0778] Based on the above specific example, phrases such as "chewy" and "full of sesame paste" are extracted.

[0779] 4. Tagging

[0780] The server uses generative AI to automatically assign specific tags based on the extracted keywords and phrases. For example, the phrase "chewy" is tagged with "chewy texture," and the phrase "plenty of sesame paste" is tagged with "sesame paste."

[0781] 5. Saving to the database

[0782] The server stores the tagged information in a database, which also includes the URL of the original text data, the store name, and detailed product information.

[0783] 6. Providing a search interface

[0784] Users can use a search interface to specify areas and detailed search criteria via a web browser or smartphone application. For example, a user can search by specifying the criteria "Shinjuku," "chewy texture," "sesame paste," and "yumechikara."

[0785] 7. Search processing and result display

[0786] The terminal sends the search criteria entered by the user to the server. The server queries the database based on the received criteria and extracts stores and products that match the criteria. The extracted results are then sent to the terminal, which displays them to the user. The user can then check detailed information from the search results and select stores and products that suit their preferences.

[0787] For example, search results will display stores in Shinjuku that offer bagels with a chewy texture, sesame paste, and Yumechikara. Users can select the most suitable store from the list and check detailed information (such as opening hours and menu).

[0788] In this way, the system of the present invention allows users to easily and efficiently find restaurants and products that suit their preferences based on detailed tastes and flavors.

[0789] The processing flow will be explained below.

[0790] Step 1:

[0791] The server runs a crawler program based on a list of URLs specified by gourmet sites and social networking services, and the crawler program retrieves text data (word of mouth reviews and comments) related to food taste from each URL.

[0792] Step 2:

[0793] The server temporarily stores the text data acquired by the crawler program, filtering out duplicate data and noise (such as advertisements and unnecessary text).

[0794] Step 3:

[0795] The server cleans the stored text data by removing special characters and unnecessary blank lines and formatting the data.

[0796] Step 4:

[0797] The server uses natural language processing (NLP) technology on the cleaned text data to extract keywords and phrases related to taste. For example, from a sentence such as "This bagel is chewy and filled with sesame paste," the server extracts the keywords "chewy" and "sesame paste."

[0798] Step 5:

[0799] The server uses a generation AI based on the extracted keywords and phrases. The generation AI assigns specific tags to each keyword or phrase. For example, "mochiri" is tagged as "mochiri texture," and "sesame paste" is tagged as "sesame paste."

[0800] Step 6:

[0801] The server stores the tagged information in a database, which also includes the tag, the URL of the original text data, the store name, and detailed product information.

[0802] Step 7:

[0803] The server provides a search interface where users can enter detailed search criteria. Users can access this interface through a web browser or a smartphone application.

[0804] Step 8:

[0805] The user uses the interface to input search criteria, such as "area: Shinjuku," "texture: chewy," "flavor: sesame paste," and "ingredients: dream power."

[0806] Step 9:

[0807] The terminal transmits the search conditions entered by the user to the server.

[0808] Step 10:

[0809] The server queries the database based on the received search criteria and extracts the relevant stores and products.

[0810] Step 11:

[0811] The server transmits the extracted search results to the terminal.

[0812] Step 12:

[0813] The terminal displays the received search results to the user, who can then check detailed information about stores and products that interest them from the displayed results.

[0814] Example 1

[0815] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0816] Conventional restaurant and product search systems have made it difficult for users to efficiently search based on detailed, specific criteria. Furthermore, the technology for accurately extracting and appropriately tagging information related to taste from collected text data is insufficient. Furthermore, the accuracy of displaying search results often fails to meet user expectations.

[0817] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0818] In this invention, the server includes means for collecting text data related to food taste from gourmet websites and social networking services, means for cleaning the collected text data to extract keywords and phrases related to food taste, means for tagging the extracted keywords and phrases using generative AI, means for saving the tagged information in a database, means for providing an interface that allows users to search by specifying an area or detailed preferences, means for querying the database based on the specified search criteria and extracting relevant stores and products, means for displaying the extracted search results to the user, means for periodically running a crawler program, means for removing special characters and noise from the cleaned data, means for analyzing the text data using a natural language processing tool, and means for automatically tagging the data using a generative AI model. This allows users to efficiently and accurately find restaurants and products that suit their preferences based on detailed tastes and flavors.

[0819] A "gourmet site" is a website that provides information about restaurants and cuisine.

[0820] A "social networking service" is an online platform that facilitates communication between users.

[0821] "Text data related to food taste" refers to data that includes descriptions and comments about the taste, texture, aroma, etc. of food and drinks.

[0822] "Means of collection" refers to the software or hardware functions used to obtain the target data.

[0823] "Cleaning means" is a function that removes unnecessary information and noise from text data.

[0824] "Keywords and phrases" are important words or sequences of words extracted from text data.

[0825] "Extraction methods" are software and algorithms that find the necessary information from the data.

[0826] "Generative AI" is AI that has the ability to generate new information and patterns from data.

[0827] A "tag" is a label that indicates an attribute or category that represents the content of data.

[0828] "Means for adding" is a function for adding information such as tags to data.

[0829] A "database" is a system that allows a collection of information to be efficiently managed and searched.

[0830] "Means of storage" refers to a function for retaining data for a certain period of time.

[0831] An "interface" is the screen and input method that a user uses to access and operate a system.

[0832] A "querying means" is a function for searching a database under specific conditions.

[0833] "Stores and products to be extracted" are restaurants and products that match the search conditions.

[0834] The "display means" is a function for visually showing the search results to the user.

[0835] A "crawler program" is software for automatically collecting information on the web.

[0836] "Special characters and noise" refers to unnecessary symbols and irrelevant information contained in text data.

[0837] "Natural language processing tools" are software and libraries for analyzing and processing text data.

[0838] "Means of analysis" are functions and algorithms for extracting useful information from data.

[0839] A "generative AI model" is an artificial intelligence model that can learn using large amounts of data and generate new information.

[0840] "Automatic attachment" refers to the ability of the system to automatically add tags and other information to data without human intervention.

[0841] MODE FOR CARRYING OUT THE INVENTION

[0842] This invention is a system that allows users to search for restaurants and products based on area and specific requirements. The system is composed of a server, a user terminal, and an interface. Specific embodiments of the system are described below.

[0843] System Overview

[0844] In this system, the server collects text data related to food taste from gourmet sites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. It also uses generative artificial intelligence (generative AI model) to assign tags based on the extracted keywords and phrases, and stores the tagged information in a database. It also provides an interface that allows users to search by specifying areas and detailed preferences, and displays the search results to the user.

[0845] Information gathering

[0846] The server periodically runs a crawler program (using, for example, Scrapy or BeautifulSoup) to collect text data related to food taste from gourmet sites and social networking services (SNS). For example, the server targets data related to "bagels" and collects comments from multiple SNSs. The collected comments include sentences such as "This bagel is chewy and has plenty of sesame paste."

[0847] Data Cleaning and Analysis

[0848] The server uses natural language processing (NLP) tools (such as NLTK or SpaCy) to clean the collected text data. This removes special characters and noise. After cleaning, natural language processing techniques are used to extract keywords and phrases related to taste from the data. Specifically, phrases such as "chewy" and "plenty of sesame paste" are extracted.

[0849] Tagging

[0850] The server automatically assigns tags based on keywords and phrases extracted using a generative AI model (e.g., BERT or GPT). For example, the phrase "chewy" is tagged with "chewy texture," and the phrase "plenty of sesame paste" is tagged with "sesame paste."

[0851] Saving to a database

[0852] The server stores the tagged information in a database (e.g., MySQL or PostgreSQL), which also includes the URL of the original text data, the store name, and detailed product information.

[0853] Providing a search interface

[0854] Users access the system using a web browser or smartphone application. The interface provides fields for entering area and detailed search criteria. For example, a user might search for "Shinjuku," "chewy texture," "sesame paste," and "yumechikara."

[0855] Search processing and result display

[0856] The device sends the search criteria entered by the user to the server. The server queries the database based on the received criteria and extracts stores and products that match the criteria. The extracted results are then sent to the device, which displays them to the user. For example, the search results show stores in Shinjuku that offer bagels with a chewy texture, sesame paste, and Yumechikara. The user can then select the most suitable store from the list and check detailed information (such as opening hours and menu).

[0857] Prompt Sentence Examples

[0858] "Find a store in Shinjuku that serves chewy bagels with sesame paste."

[0859] This system allows users to easily and efficiently find restaurants and products that suit their tastes based on detailed taste and flavor information.

[0860] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0861] Step 1:

[0862] The server periodically runs a crawler program to collect text data about food taste from gourmet sites and social networking services (SNS). The collected data consists of comments and reviews based on specific keywords. For example, data about "bagels" is collected. The input at this point is the keywords to be collected, and the output is the initial collected data.

[0863] Step 2:

[0864] The server temporarily stores the collected text data in a storage system such as a database. This data may contain duplicates and preliminary noise. The input is the collected initial data, and the output is the temporarily stored raw data.

[0865] Step 3:

[0866] The server cleans the stored text data. During this process, it uses natural language processing (NLP) tools (e.g., NLTK or SpaCy) to remove special characters and noise. The input is the temporarily stored raw data, and the output is the cleaned data.

[0867] Step 4:

[0868] The server extracts taste-related keywords and phrases from the cleaned text data. Specifically, it uses NLP technology to apply morphological analysis, TF-IDF, and other techniques. For example, it extracts phrases such as "chewy" and "full of sesame paste." The input is the cleaned data, and the output is the extracted keywords and phrases.

[0869] Step 5:

[0870] The server assigns tags based on extracted keywords and phrases using a generative AI model (e.g., BERT or GPT). For example, the phrase "chewy" is tagged with "chewy texture," and the phrase "plenty of sesame paste" is tagged with "sesame paste." The input is the extracted keywords and phrases, and the output is the tagged data.

[0871] Step 6:

[0872] The server stores the tagged information in a database (e.g., MySQL or PostgreSQL). This data includes the URL of the original text data, the store name, and detailed product information. The input is the tagged data, and the output is the information stored in the database.

[0873] Step 7:

[0874] Users access the system using a web browser or smartphone application. The interface provides fields for entering area and detailed search criteria. For example, users can enter criteria such as "Shinjuku," "chewy texture," "sesame paste," and "yumechikara." The input at this point is the user's search criteria.

[0875] Step 8:

[0876] The terminal sends the search criteria entered by the user to the server. This input is the criteria specified by the user in step 7. The output is the search query sent to the server.

[0877] Step 9:

[0878] The server queries the database based on the received search criteria and extracts stores and products that match the criteria. For example, search for stores in Shinjuku that offer bagels with a chewy texture, sesame paste, and Yumechikara. The input is the search query, and the output is the search results.

[0879] Step 10:

[0880] The server sends the extracted search results to the terminal. For example, it contains a list of stores that match "Shinjuku." At this point, the input is the search results, and the output is the transmitted data.

[0881] Step 11:

[0882] The terminal displays the received search results to the user. The user can select a store from the displayed results and check further details (such as opening hours and menu). The input is the search results sent, and the output is the information displayed on the user's screen.

[0883] In this way, a series of processes are performed, allowing the user to efficiently search for restaurants and products based on detailed, specific conditions.

[0884] (Application example 1)

[0885] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0886] Existing systems that allow users to search for restaurants and products based on area or specific preferences only display search results on a screen, making it difficult to provide detailed information in real time when selecting specific products in a physical store. In addition, there is a lack of a way for users to quickly obtain detailed product information in a store using a smart device.

[0887] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0888] In this invention, the server includes: means for collecting text data related to food taste from gourmet sites and social networking services; means for cleaning the collected text data to extract keywords and phrases related to food taste; means for tagging the extracted keywords and phrases using generative artificial intelligence; means for saving the tagged information in a database; means for providing an interface that allows a user to search by specifying an area or detailed preferences; means for querying the database based on the specified search conditions and extracting relevant stores and products; means for the smart glasses to display detailed information about products and stores based on eye tracking; and means for displaying the extracted search results to the user. This enables detailed information about a specific product to be displayed in real time on the smart glasses when the user looks at the product in a physical store.

[0889] A "gourmet site" is a website that provides information about restaurants and food.

[0890] A "social networking service" is a platform that allows users to exchange information and communicate with each other online.

[0891] "Text data" refers to data that is structured in the form of written words, sentences, comments, etc.

[0892] "Cleaning" is the process of filtering out noise and unnecessary information from collected data and extracting only the necessary parts.

[0893] A "keyword" is an important word that has a specific meaning in text data.

[0894] A "phrase" is a series of words that combine multiple keywords to give them meaning.

[0895] "Generative AI" is a machine learning technique that learns patterns from collected data and automatically tags and categorizes it.

[0896] A "tag" is a label that is attached to information to make it easier to classify or search.

[0897] A "database" is a system for systematically storing and managing data.

[0898] An "interface" is the means or screen through which a user interacts with a system.

[0899] A "query" is an inquiry issued to a database to retrieve specific information.

[0900] "Smart glasses" are devices equipped with displays and sensors that provide information based on the user's line of sight.

[0901] "Eye tracking" is a technology that detects the direction of a user's gaze and collects that data.

[0902] The present invention is a system that allows users to search for restaurants and products based on area and detailed preferences, and provides related information in real time. Specific embodiments of the system will be described below.

[0903] System Overview

[0904] The system consists of the following main components:

[0905] 1. Server: Responsible for data collection, cleaning, tagging, database management, and processing user queries.

[0906] 2. User terminal: Provides a search interface and displays search results. Includes smart glasses.

[0907] 3. Smart glasses: Provides real-time information display based on eye tracking.

[0908] Information gathering

[0909] The server automatically collects text data about food taste from gourmet sites and social networking services using information gathering software such as a web crawler. The collected data is temporarily stored on the server.

[0910] Data cleaning and tagging

[0911] The server cleans the collected text data, removing unnecessary noise and duplication, then uses natural language processing (NLP) to extract taste-related keywords and phrases from the text data, and then uses a generative AI model to automatically assign tags based on these keywords and phrases.

[0912] Database Management

[0913] The server stores the tagged information in a database, which also includes the URL of the original text data, the store name, and detailed product information.

[0914] Providing a search interface

[0915] Users can use a web browser or smartphone application to search by specifying an area and specific search criteria, such as "Shinjuku," "chewy texture," and "sesame paste."

[0916] Query processing and result display

[0917] The user terminal sends the entered search criteria to the server. The server queries the database based on the received criteria and extracts stores and products that match the criteria. The extracted results are then sent to the user terminal, allowing the user to view the search results.

[0918] Information display using smart glasses

[0919] The smart glasses use eye-tracking to detect which products or menu items the user is looking at. When the user's gaze is directed toward a specific product, the relevant information is received from the server and displayed on the smart glasses' display in real time.

[0920] Specific examples

[0921] For example, if a user wants to find a store in Shinjuku that serves bagels with a chewy texture, sesame paste, and Yumechikara, they can enter the search criteria into the interface, and the server will extract matching data. Then, when the user puts on the smart glasses and looks at a bagel in the store, detailed information about the product will be displayed on the glasses' display in real time.

[0922] Example prompts to be input to the generative AI model

[0923] "Imagine a user is searching for a bagel in the Shinjuku area that meets the criteria of 'chewy texture' and 'sesame paste'. Imagine a smart glasses application that displays detailed information about the bagel in real time."

[0924] In this way, the system of the present invention allows users to search for restaurants and products based on detailed preferences and obtain product information in real time within physical stores.

[0925] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0926] Step 1: Gather information

[0927] The server collects text data related to food taste from gourmet websites and social networking services. A crawler program crawls through web pages and extracts text data such as comments and reviews. The input is the website URL or API endpoint, and the output is the collected text data.

[0928] Step 2: Data cleaning

[0929] The server cleans the collected text data. Specifically, it removes unnecessary noise and redundant data and prepares it in a form that can be analyzed. The input is the collected text data, and the output is the cleaned text data.

[0930] Step 3: Extract keywords and phrases

[0931] The server uses natural language processing (NLP) technology to extract taste-related keywords and phrases from the cleaned text data. The input is the cleaned text data, and the output is taste-related keywords and phrases. Specifically, it performs morphological analysis.

[0932] Step 4: Tagging

[0933] The server automatically assigns tags using a generative AI model based on the extracted keywords and phrases. The input is keywords and phrases, and the output is tags assigned to each keyword or phrase. The generative AI understands the context and assigns tags.

[0934] Step 5: Saving to the Database

[0935] The server stores the tagged information in a database. The stored data includes the URL of the original text data, the store name, and detailed product information. The input is the tagged information, and the output is the data stored in the database.

[0936] Step 6: Provide a search interface

[0937] The user terminal provides an interface for searching by specifying an area or detailed specific conditions through a web browser or smartphone application. The input is the search conditions specified by the user, and the output is the transmission of the search conditions to the server.

[0938] Step 7: Query Processing

[0939] The server queries the database based on the search criteria received from the user and extracts stores and products that match the criteria. The input is the search criteria and the output is the extracted search results. Specifically, a database query language (such as SQL) is used.

[0940] Step 8: Viewing search results

[0941] The user terminal displays the search results received from the server to the user, with the input being the search result data from the server and the output being the search results displayed on the user interface.

[0942] Step 9: Eye tracking with smart glasses

[0943] The smart glasses use an eye-tracking function to detect the product or menu item that the user is looking at. The input is the user's gaze data, and the output is the ID of the detected product or menu item. Specifically, an eye-tracking sensor is used.

[0944] Step 10: Real-time information display

[0945] The server sends detailed information about the detected product or menu item to the smart glasses based on its ID. The smart glasses then display this information in real time. The input is the product or menu item ID, and the output is the detailed information displayed on the smart glasses' display. Specifically, wireless communication technology (such as Wi-Fi or Bluetooth) is used.

[0946] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0947] The present invention provides a system that allows users to search for restaurants and products based on area and specific preferences, and also includes a function that recognizes the user's emotions and optimizes search results and suggestions based on those emotions. The system includes a server, a user terminal, a search interface, and an emotion engine. Specific embodiments of the system are described below.

[0948] 1. System Overview

[0949] In this system, the server collects text data related to food taste from gourmet websites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. Generative AI is then used to assign tags based on the extracted keywords and phrases, and the tagged information is stored in a database. Users can use an interface to search by area or specific, detailed criteria, and the system displays search results. An emotion engine is also incorporated to optimize search results and suggestions based on the user's emotional data.

[0950] 2. Information gathering

[0951] The server periodically runs a crawler program that collects text data about food taste from gourmet websites and social networking services. The collected data is temporarily stored and filtered to remove duplication and noise.

[0952] As a concrete example, consider a case where a server collects data from a large number of social networking services that contain comments about "ramen." From the collected comments, it extracts sentences such as "This ramen has a rich soup and is very delicious."

[0953] 3. Data cleaning and analysis

[0954] The server cleans the collected text data, removing special characters and noise, and then uses natural language processing (NLP) technology to extract taste-related keywords and phrases from the text data.

[0955] Based on the above specific example, phrases such as "rich" and "delicious" are extracted.

[0956] 4. Tagging

[0957] The server uses a generation AI based on the extracted keywords and phrases. The generation AI automatically assigns specific tags to each keyword or phrase. For example, "rich" is assigned a tag such as "rich soup" and "delicious" is assigned a tag such as "highly rated."

[0958] 5. Saving to the database

[0959] The server stores the tagged information in a database, which also includes the tags, the URL of the original text data, the store name, and detailed product information.

[0960] 6. Introducing the Emotion Engine

[0961] The server is equipped with an emotion engine that analyzes the user's emotions. The emotion engine collects and analyzes emotion data from the user's facial expressions, voice, text input, etc., and estimates the user's emotional state.

[0962] 7. Providing a search interface

[0963] Users can use a web browser or smartphone application to search by area and specific search criteria. For example, a user can search for "Shinjuku" with the criteria "rich soup" and "highly rated."

[0964] 8. Search processing and result display

[0965] The device sends the search criteria entered by the user to the server. The server queries the database based on the received criteria and extracts relevant stores and products. The emotion engine uses the user's emotional data to adjust the display order and content of the search results.

[0966] For example, if the emotion engine recognizes that the user has a "very satisfied" expression, it will prioritize displaying stores that have received high ratings in the past. Also, if the user has a "highly anticipated" expression, it will recommend new or popular stores.

[0967] 9. Displaying the results

[0968] The server sends the optimized search results to the terminal, which then displays the received search results to the user. The user can check the detailed information of the stores and products that interest them from the displayed results and make the best selection according to their emotional state.

[0969] This embodiment allows the system of the present invention to provide personalized search results based on the user's emotional state in addition to the ability to search based on detailed user preferences.

[0970] The processing flow will be explained below.

[0971] Step 1:

[0972] The server runs a crawler program based on a list of URLs specified by gourmet sites and social networking services, and automatically retrieves text data (reviews and comments) related to food taste from each URL.

[0973] Step 2:

[0974] The server temporarily stores the text data acquired by the crawler program, filtering out duplicate data and noise (such as advertisements and unnecessary text).

[0975] Step 3:

[0976] The server cleans the stored text data, removing special characters and unnecessary blank lines and formatting the data.

[0977] Step 4:

[0978] The server uses natural language processing (NLP) technology on the cleaned text data to extract keywords and phrases related to taste. For example, "rich soup" and "delicious" are extracted from the text "This ramen has a rich soup and is very delicious."

[0979] Step 5:

[0980] The server then uses generative AI to tag the extracted keywords and phrases. For example, it might tag "rich" for "rich soup" and "highly rated" for "delicious."

[0981] Step 6:

[0982] The server stores the tagged information in a database, which also includes the tags, the URL of the original text data, the store name, and detailed product information.

[0983] Step 7:

[0984] The server includes an emotion engine that analyzes the user's emotions by collecting and analyzing emotion data from the user's facial expressions, voice, and text input.

[0985] Step 8:

[0986] Users access the search interface through a web browser or smartphone app and enter the area and detailed criteria they prefer, such as "Shinjuku," "rich soup," and "highly rated."

[0987] Step 9:

[0988] The terminal transmits the input search conditions to the server.

[0989] Step 10:

[0990] The server queries the database based on the received search criteria and extracts relevant stores and products. The emotion engine optimizes the display order and content of search results based on the user's emotional data.

[0991] Step 11:

[0992] The server transmits the optimized search results to the terminal.

[0993] Step 12:

[0994] The terminal displays the received search results to the user. For example, it displays a list of ramen restaurants in "Shinjuku" that meet the conditions of "rich soup" and "highly rated."

[0995] Step 13:

[0996] Users can check the detailed information of stores and products they are interested in from the displayed search results. Recommended stores and products are displayed preferentially based on the user's emotional data.

[0997] In this way, the system of the present invention can provide optimal restaurants and products based on the user's detailed preferences and feelings.

[0998] Example 2

[0999] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1000] Conventional restaurant and product search systems have the problem that it is difficult to search based on the user's detailed preferences, and the search results are not optimized according to the user's emotional state, which limits the user experience. A system that solves these problems is needed.

[1001] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting information about restaurants and products from websites and social media, means for cleaning the collected information and extracting keywords and phrases related to specific attributes, means for tagging the extracted keywords and phrases using a machine learning model, means for saving the tagged tags in a database, means for providing a user interface that allows a user to search by specifying an area or detailed specific conditions, means for querying the database based on the specified search conditions and extracting relevant information, means for displaying the extracted search results to the user, and means for analyzing the user's emotional data and optimizing the search results. This enables advanced searches based on the user's detailed specific conditions and personalized search results based on the user's emotional state.

[1002] "Information about restaurants and products" refers to data about ingredients, dishes, drinks, menus, services, store locations, business hours, ratings, reviews, etc.

[1003] "Websites and social media" refers to media and platforms on the Internet that provide information and enable users to interact with each other, and specifically includes gourmet sites and social networking sites.

[1004] "Means of collection" refers to the use of web crawlers or APIs to automatically obtain data from designated websites and social media.

[1005] "Cleaning means" refers to the process of removing duplicate data, invalid data, and noise from collected data, preparing it for analysis.

[1006] "Means for extracting keywords and phrases" refers to a method of using natural language processing techniques to identify important words and phrases from text data.

[1007] "Means of tagging using a machine learning model" refers to the process of automatically adding relevant tags to extracted keywords and phrases using a trained AI model.

[1008] "Means of saving to a database" refers to the method of storing the processed data in a relational database, NoSQL database, etc.

[1009] "Means for providing a user interface" refers to the method of providing a web page or mobile application that allows a user to enter search criteria and view results.

[1010] "Means of querying a database and extracting relevant information" refers to a method of searching for and retrieving the required information from a database using a query language such as SQL.

[1011] The "means for displaying search results to the user" refers to a method for visually presenting the acquired information to the user in the form of a list, graph, or the like.

[1012] "Means for analyzing emotional data and optimizing search results" refers to a process that analyzes emotions from the user's facial expressions, voice, text input, etc., and displays search results in the most optimal order for the user based on the results.

[1013] The present invention provides a system for efficiently collecting information about restaurants and products and providing optimal search results based on the user's detailed preferences and emotional state. Specific embodiments of this system are described in detail below.

[1014] The server runs a crawler program that collects information about restaurants and products from websites and social media. The collected data is temporarily stored and then cleaned. Cleaning involves analyzing HTML pages using Python's BeautifulSoup library and using regular expressions to remove special characters and noise. For example, if review comments about "ramen" are collected, keywords such as "rich" and "delicious" are extracted from those comments using regular expressions.

[1015] The server then uses natural language processing (NLP) techniques to extract specific keywords and phrases from the collected data. Specifically, it uses Python's NLTK (Natural Language Toolkit) library to extract phrases such as "rich" and "delicious." The extracted data is tagged using a generative AI model, such as GPT-3. For tagging, instructions are given to the generative AI using prompt sentences. The following prompt sentences are used as examples:

[1016] "Analyze the following sentence and generate keywords and appropriate tags related to taste.

[1017] Sentence: "This ramen has a rich soup and is very tasty."

[1018] Once tagged, the data is stored in a database. This is done using a database system such as PostgreSQL, and the information stored includes the tag, the URL of the original text data, the store name, detailed product information, etc. For example, the database contains the tag "rich soup" along with information about "Store A" and "Product B."

[1019] Users can use a search interface to specify areas and detailed search criteria through a web browser or smartphone application. The criteria specified by the user are the area name "Shinjuku" and keywords such as "rich soup" and "highly rated." These criteria are sent from the device to the server via an HTTP POST request. The server executes a query to the database based on the received criteria and extracts search results. SQL is used for the query.

[1020] The extracted search results are optimized using an emotion engine. The emotion engine analyzes emotional data from the user's facial expressions, voice, text input, etc. to determine the user's emotional state in real time. For example, if the user's facial expression indicates "satisfaction," the server will place stores that have received high ratings in the past at the top of the search results. Conversely, if the user's facial expression indicates "expectation," the server will recommend new or popular stores.

[1021] Finally, the device displays the optimized search results to the user. This involves parsing the JSON data and converting it into a user-friendly format. For example, the smartphone screen displays a list of store names, product names, ratings, and summaries.

[1022] This embodiment allows users to perform advanced searches based on detailed preferences and also provides personalized search results based on emotional state, improving the user experience and enabling optimal choices.

[1023] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1024] Step 1: Gather information

[1025] The server runs a crawler program to collect information about restaurants and products from specified websites and social media. The input is the specified query or URL, and the output is the collected raw data.

[1026] Specific operation: The server uses Python's BeautifulSoup library to parse HTML pages from specific gourmet sites and social media sites and extract review comments and ratings about restaurants and products.

[1027] Step 2: Data cleaning

[1028] The server removes duplicates and noise from the collected data. The input is the raw data collected in step 1, and the output is the clean data.

[1029] Specific operation: The server uses regular expressions to remove unnecessary special characters and noise components from the text data and filters out duplicate comments.

[1030] Step 3: Extract keywords and phrases

[1031] The server then uses natural language processing (NLP) techniques to extract specific keywords and phrases from the cleaned data. The input is the clean text data, and the output is the extracted keywords and phrases.

[1032] How it works: The server uses Python's NLTK library to extract keywords and phrases such as "rich" and "delicious" from the text data.

[1033] Step 4: Tagging

[1034] The server then assigns tags using a generative AI model based on the extracted keywords and phrases. The input is the extracted keywords and phrases, and the output is the tagged data.

[1035] How it works: The server uses a generative AI model like GPT-3 to generate appropriate tags for keywords (e.g., "rich soup" for "rich") based on a prompt.

[1036] Example prompt: "Analyze the following sentence and generate taste-related keywords and appropriate tags. Sentence: 'This ramen has a rich soup and is very delicious.'"

[1037] Step 5: Saving to the Database

[1038] The server stores the tagged data in a database. The input is the tagged data and the output is the information stored in the database.

[1039] Specific operation: The server executes an INSERT query to a database system such as PostgreSQL, and saves the tag, the URL of the original text data, the store name, and detailed product information in the database.

[1040] Step 6: Collect emotion data

[1041] The server collects and analyzes emotional data from the user's facial expressions, voice, text input, etc. The input is the user's emotional data, and the output is the analyzed emotional state.

[1042] Specific operation: The server inputs data obtained from the user's webcam and microphone into an emotion analysis algorithm to determine the user's emotional state in real time.

[1043] Step 7: Provide a search interface

[1044] Users use an interface to search by specifying an area and detailed preferences. The input is the user's search query, and the output is a search request based on that query.

[1045] Specific operation: The user enters the area name and keywords into a search form via a web browser or smartphone application.

[1046] Step 8: Search process

[1047] The terminal sends the user's search criteria to the server, which then executes a search query against the database. The input is the user's search query, and the output is the search results for the relevant information.

[1048] Specific operation: Upon receiving a search query sent from the terminal, the server generates an SQL query and searches the database.

[1049] Step 9: Optimize search results

[1050] The server uses an emotion engine to optimize search results based on the user's emotional state, where the inputs are search results and the user's emotional state, and the output is the optimized search results.

[1051] Specific operation: The emotion engine analyzes the user's emotional data and, if it recognizes the emotion of "satisfaction," places stores that have received high ratings in the past at the top of the search results.

[1052] Step 10: View the results

[1053] The server sends the optimized search results to the terminal, and the terminal displays the results to the user. The input is the optimized search results, and the output is the search results presented to the user.

[1054] Specific operation: The server sends optimized search results in JSON format, and the device displays the data in a user-friendly format.

[1055] (Application example 2)

[1056] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1057] Conventional Internet search systems only provide search results based on the area or specific conditions specified by the user, and have limitations in optimizing search results in response to the user's emotional state. Furthermore, to obtain highly satisfying results, it is necessary to dynamically adjust the results based on the user's emotions, but no such system exists yet. The objective of this invention is to solve this problem.

[1058] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1059] In this invention, the server includes means for collecting text data related to food taste from gourmet sites and social networking services, means for cleaning the collected text data to extract keywords and phrases related to food taste, means for tagging the extracted keywords and phrases using generative artificial intelligence, means for saving the tagged information in a database, means for providing an interface that allows users to specify areas and detailed search criteria to perform searches, means for querying the database based on the specified search criteria and extracting relevant stores and products, means for displaying the extracted search results to the user, means for analyzing a user's facial expression image to recognize the user's emotional state, and means for optimizing the search results based on the recognized emotional state, thereby enabling the provision of personalized search results according to the user's emotional state.

[1060] A "gourmet site" is a website that provides information about restaurants and allows users to search for, browse, and rate restaurants.

[1061] A "social networking service" is a web service that allows users to interact and share information online.

[1062] "Text data" is data that mainly consists of characters and symbols, and is information expressed in the form of sentences, comments, etc.

[1063] "Cleaning" refers to the process of removing unnecessary parts from data and making it accurate and usable.

[1064] A "keyword" is an important word or phrase for extracting specific information from text data.

[1065] A "phrase" is a meaningful clause or phrase within text data, and is a collection of words and phrases related to a specific purpose.

[1066] "Generative AI" is a type of machine learning technology that is artificial intelligence that recognizes patterns from given data and generates new data and information.

[1067] A "tag" is a label or identifier assigned to data, and is used to improve the efficiency of information classification and search.

[1068] A "database" is a system for efficiently organizing, storing, searching, and managing large amounts of data.

[1069] An "interface" is a means or screen through which a user operates a system or inputs information.

[1070] A "query" is a question or command used to search for information or to instruct a database to perform an operation.

[1071] "Emotional state" represents the user's psychological state and refers to emotions estimated from facial expressions, voice, text input, etc.

[1072] "Optimization" is the process of adjusting or improving something to maximize its efficiency or effectiveness for a specific purpose.

[1073] An "expression image" is image data capturing a user's facial expression.

[1074] "Personalization" means providing experiences and services that are customized according to the characteristics and preferences of individual users.

[1075] The present invention is a system that allows users to search for restaurants and products based on area and detailed preferences, and provides optimized search results by recognizing the user's emotional state. The system includes a server, a user terminal, a search interface, and an emotion engine.

[1076] System configuration

[1077] The server collects text data related to food taste from gourmet sites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. Based on the extracted keywords and phrases, generative artificial intelligence (generative AI) is used to assign specific tags, and the tagged information is stored in a database. Natural language processing (NLP) technology is used for collection, data cleaning, and keyword extraction.

[1078] The search interface is designed to allow users to search by specifying an area or specific detailed criteria. Users use this interface through a web browser or smartphone application to enter search criteria. The entered criteria are sent to the server, which queries the database to extract relevant stores and products.

[1079] The emotion engine analyzes the user's facial expressions and recognizes their emotional state. The analysis uses the image processing library OpenCV and a custom emotion analysis model (EmotionRecognizer). The analysis results are used as an index to optimize the user's search results.

[1080] Hardware and software used

[1081] Hardware:

[1082] Smartphone: The device on which the user accesses the application.

[1083] Camera: Captures the user's facial expressions using the smartphone's built-in camera.

[1084] software:

[1085] Flask: A Python web framework that acts as the backend for your application.

[1086] OpenCV: An image processing library used to analyze the user's facial expressions.

[1087] EmotionRecognizer: A custom emotion analysis model, used to recognize user emotions.

[1088] Requests: An HTTP library used to communicate with external APIs.

[1089] Specific examples of operation procedures

[1090] 1. The user launches the application, enters detailed preferences such as "I want to eat ramen in the Shinjuku area," and takes a photo of their own facial expression using their smartphone camera.

[1091] 2. The device uploads the captured facial expression image to the application and sends the area and preference conditions to the server.

[1092] 3. The server queries the database based on the received search criteria and generates a list of relevant stores.

[1093] 4. In parallel, the emotion engine analyzes the facial expression image and recognizes it as a "satisfied expression."

[1094] 5. The server optimizes the search results based on the results of sentiment analysis, prioritizing the display of highly rated ramen restaurants.

[1095] 6. The optimized search results are sent to the device, where the user can view the final results.

[1096] Prompt Sentence Examples

[1097] "The system uses images of the user's facial expressions as input to recognize the user's emotions. Based on the recognized emotions, it optimizes search results for restaurants.

[1098] Area: Shinjuku

[1099] Special requirements: Ramen

[1100] Facial image: user_image.jpg

[1101] Output: List of highly rated ramen restaurants"

[1102] In this way, the system of the present invention not only provides a search function based on the user's detailed preferences, but also provides personalized search results based on the user's emotional state.

[1103] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1104] Step 1:

[1105] The user launches the application, inputs the area and detailed preferences, and takes a photo of their facial expression using their smartphone camera. In this case, the user inputs preferences such as "I want to eat ramen in the Shinjuku area" and uploads an image of their facial expression. The input area information, preferences, and facial expression image are then acquired.

[1106] Step 2:

[1107] The device sends the search criteria entered by the user and the captured facial image to the server. Specifically, area information, preference criteria, and facial image data are transferred from the smartphone to the server. The input is area information, preference criteria, and facial image, and the output is data sent to the server.

[1108] Step 3:

[1109] The server queries the database based on the received area information and preferences, and generates a list of relevant stores. Here, it searches the tagged information stored in the database and extracts stores that match the conditions. The input is the area information and preferences, and the output is a list of relevant stores. Specifically, it searches for "ramen restaurants in the Shinjuku area."

[1110] Step 4:

[1111] In parallel, the server analyzes the facial expression image and recognizes the user's emotional state. This analysis is performed using OpenCV and EmotionRecognizer. The input is the facial expression image, and the output is the recognized emotional state. Specifically, it is analyzed as a "satisfied expression."

[1112] Step 5:

[1113] The server optimizes search results based on the emotional state obtained by the emotion engine. Specifically, if the recognized emotion is a "satisfied expression," it prioritizes displaying stores that have received high ratings in the past. The input is a list of relevant stores and the emotional state, and the output is an optimized list of stores.

[1114] Step 6:

[1115] The server sends the optimized search results to the terminal. The input is the optimized store list, and the output is the data transmission to the terminal.

[1116] Step 7:

[1117] The device displays the optimized search results to the user. The user can check the detailed information of the stores and products that interest them from the displayed results and make a selection according to their emotional state. Specifically, "highly rated ramen restaurants in Shinjuku" will be displayed.

[1118] In this way, the system of the present invention can dynamically optimize search results and provide personalized restaurant recommendations based on the user's detailed preferences and emotional state.

[1119] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1120] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1121] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1122] [Fourth embodiment]

[1123] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1124] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1125] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1126] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1127] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1128] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1129] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1130] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1131] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1132] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1133] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1134] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1135] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1136] The present invention provides a system that allows users to search for restaurants and products based on area and specific criteria. The system includes a server, a user terminal, and an interface. Specific embodiments of the system are described below.

[1137] 1. System Overview

[1138] In this system, the server collects text data related to food taste from gourmet websites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. It also uses generative AI to assign tags based on the extracted keywords and phrases, and stores the tagged information in a database. Furthermore, it provides an interface that allows users to search by specifying areas and detailed preferences, and displays the search results to the user.

[1139] 2. Information gathering

[1140] The server periodically runs a crawler program to collect text data about food taste from gourmet sites and social networking services. The collected data is temporarily stored and filtered to remove duplication and noise.

[1141] As a concrete example, consider a case where a server collects data from multiple social networking services that have comments about "bagels." From the collected comments, it extracts sentences such as "This bagel is chewy and has plenty of sesame paste."

[1142] 3. Data cleaning and analysis

[1143] The server cleans the collected text data, removing special characters and noise, and then uses natural language processing technology to extract taste-related keywords and phrases from the text data.

[1144] Based on the above specific example, phrases such as "chewy" and "full of sesame paste" are extracted.

[1145] 4. Tagging

[1146] The server uses generative AI to automatically assign specific tags based on the extracted keywords and phrases. For example, the phrase "chewy" is tagged with "chewy texture," and the phrase "plenty of sesame paste" is tagged with "sesame paste."

[1147] 5. Saving to the database

[1148] The server stores the tagged information in a database, which also includes the URL of the original text data, the store name, and detailed product information.

[1149] 6. Providing a search interface

[1150] Users can use a search interface to specify areas and detailed search criteria via a web browser or smartphone application. For example, a user can search by specifying the criteria "Shinjuku," "chewy texture," "sesame paste," and "yumechikara."

[1151] 7. Search processing and result display

[1152] The terminal sends the search criteria entered by the user to the server. The server queries the database based on the received criteria and extracts stores and products that match the criteria. The extracted results are then sent to the terminal, which displays them to the user. The user can then check detailed information from the search results and select stores and products that suit their preferences.

[1153] For example, search results will display stores in Shinjuku that offer bagels with a chewy texture, sesame paste, and Yumechikara. Users can select the most suitable store from the list and check detailed information (such as opening hours and menu).

[1154] In this way, the system of the present invention allows users to easily and efficiently find restaurants and products that suit their preferences based on detailed tastes and flavors.

[1155] The processing flow will be explained below.

[1156] Step 1:

[1157] The server runs a crawler program based on a list of URLs specified by gourmet sites and social networking services, and the crawler program retrieves text data (word of mouth reviews and comments) related to food taste from each URL.

[1158] Step 2:

[1159] The server temporarily stores the text data acquired by the crawler program, filtering out duplicate data and noise (such as advertisements and unnecessary text).

[1160] Step 3:

[1161] The server cleans the stored text data by removing special characters and unnecessary blank lines and formatting the data.

[1162] Step 4:

[1163] The server uses natural language processing (NLP) technology on the cleaned text data to extract keywords and phrases related to taste. For example, from a sentence such as "This bagel is chewy and filled with sesame paste," the server extracts the keywords "chewy" and "sesame paste."

[1164] Step 5:

[1165] The server uses a generation AI based on the extracted keywords and phrases. The generation AI assigns specific tags to each keyword or phrase. For example, "mochiri" is tagged as "mochiri texture," and "sesame paste" is tagged as "sesame paste."

[1166] Step 6:

[1167] The server stores the tagged information in a database, which also includes the tag, the URL of the original text data, the store name, and detailed product information.

[1168] Step 7:

[1169] The server provides a search interface where users can enter detailed search criteria. Users can access this interface through a web browser or a smartphone application.

[1170] Step 8:

[1171] The user uses the interface to input search criteria, such as "area: Shinjuku," "texture: chewy," "flavor: sesame paste," and "ingredients: dream power."

[1172] Step 9:

[1173] The terminal transmits the search conditions entered by the user to the server.

[1174] Step 10:

[1175] The server queries the database based on the received search criteria and extracts the relevant stores and products.

[1176] Step 11:

[1177] The server transmits the extracted search results to the terminal.

[1178] Step 12:

[1179] The terminal displays the received search results to the user, who can then check detailed information about stores and products that interest them from the displayed results.

[1180] Example 1

[1181] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1182] Conventional restaurant and product search systems have made it difficult for users to efficiently search based on detailed, specific criteria. Furthermore, the technology for accurately extracting and appropriately tagging information related to taste from collected text data is insufficient. Furthermore, the accuracy of displaying search results often fails to meet user expectations.

[1183] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1184] In this invention, the server includes means for collecting text data related to food taste from gourmet websites and social networking services, means for cleaning the collected text data to extract keywords and phrases related to food taste, means for tagging the extracted keywords and phrases using generative AI, means for saving the tagged information in a database, means for providing an interface that allows users to search by specifying an area or detailed preferences, means for querying the database based on the specified search criteria and extracting relevant stores and products, means for displaying the extracted search results to the user, means for periodically running a crawler program, means for removing special characters and noise from the cleaned data, means for analyzing the text data using a natural language processing tool, and means for automatically tagging the data using a generative AI model. This allows users to efficiently and accurately find restaurants and products that suit their preferences based on detailed tastes and flavors.

[1185] A "gourmet site" is a website that provides information about restaurants and cuisine.

[1186] A "social networking service" is an online platform that facilitates communication between users.

[1187] "Text data related to food taste" refers to data that includes descriptions and comments about the taste, texture, aroma, etc. of food and drinks.

[1188] "Means of collection" refers to the software or hardware functions used to obtain the target data.

[1189] "Cleaning means" is a function that removes unnecessary information and noise from text data.

[1190] "Keywords and phrases" are important words or sequences of words extracted from text data.

[1191] "Extraction methods" are software and algorithms that find the necessary information from the data.

[1192] "Generative AI" is AI that has the ability to generate new information and patterns from data.

[1193] A "tag" is a label that indicates an attribute or category that represents the content of data.

[1194] "Means for adding" is a function for adding information such as tags to data.

[1195] A "database" is a system that allows a collection of information to be efficiently managed and searched.

[1196] "Means of storage" refers to a function for retaining data for a certain period of time.

[1197] An "interface" is the screen and input method that a user uses to access and operate a system.

[1198] A "querying means" is a function for searching a database under specific conditions.

[1199] "Stores and products to be extracted" are restaurants and products that match the search conditions.

[1200] The "display means" is a function for visually showing the search results to the user.

[1201] A "crawler program" is software for automatically collecting information on the web.

[1202] "Special characters and noise" refers to unnecessary symbols and irrelevant information contained in text data.

[1203] "Natural language processing tools" are software and libraries for analyzing and processing text data.

[1204] "Means of analysis" are functions and algorithms for extracting useful information from data.

[1205] A "generative AI model" is an artificial intelligence model that can learn using large amounts of data and generate new information.

[1206] "Automatic attachment" refers to the ability of the system to automatically add tags and other information to data without human intervention.

[1207] MODE FOR CARRYING OUT THE INVENTION

[1208] This invention is a system that allows users to search for restaurants and products based on area and specific requirements. The system is composed of a server, a user terminal, and an interface. Specific embodiments of the system are described below.

[1209] System Overview

[1210] In this system, the server collects text data related to food taste from gourmet sites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. It also uses generative artificial intelligence (generative AI model) to assign tags based on the extracted keywords and phrases, and stores the tagged information in a database. It also provides an interface that allows users to search by specifying areas and detailed preferences, and displays the search results to the user.

[1211] Information gathering

[1212] The server periodically runs a crawler program (using, for example, Scrapy or BeautifulSoup) to collect text data related to food taste from gourmet sites and social networking services (SNS). For example, the server targets data related to "bagels" and collects comments from multiple SNSs. The collected comments include sentences such as "This bagel is chewy and has plenty of sesame paste."

[1213] Data Cleaning and Analysis

[1214] The server uses natural language processing (NLP) tools (such as NLTK or SpaCy) to clean the collected text data. This removes special characters and noise. After cleaning, natural language processing techniques are used to extract keywords and phrases related to taste from the data. Specifically, phrases such as "chewy" and "plenty of sesame paste" are extracted.

[1215] Tagging

[1216] The server automatically assigns tags based on keywords and phrases extracted using a generative AI model (e.g., BERT or GPT). For example, the phrase "chewy" is tagged with "chewy texture," and the phrase "plenty of sesame paste" is tagged with "sesame paste."

[1217] Saving to a database

[1218] The server stores the tagged information in a database (e.g., MySQL or PostgreSQL), which also includes the URL of the original text data, the store name, and detailed product information.

[1219] Providing a search interface

[1220] Users access the system using a web browser or smartphone application. The interface provides fields for entering area and detailed search criteria. For example, a user might search for "Shinjuku," "chewy texture," "sesame paste," and "yumechikara."

[1221] Search processing and result display

[1222] The device sends the search criteria entered by the user to the server. The server queries the database based on the received criteria and extracts stores and products that match the criteria. The extracted results are then sent to the device, which displays them to the user. For example, the search results show stores in Shinjuku that offer bagels with a chewy texture, sesame paste, and Yumechikara. The user can then select the most suitable store from the list and check detailed information (such as opening hours and menu).

[1223] Prompt Sentence Examples

[1224] "Find a store in Shinjuku that serves chewy bagels with sesame paste."

[1225] This system allows users to easily and efficiently find restaurants and products that suit their tastes based on detailed taste and flavor information.

[1226] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1227] Step 1:

[1228] The server periodically runs a crawler program to collect text data about food taste from gourmet sites and social networking services (SNS). The collected data consists of comments and reviews based on specific keywords. For example, data about "bagels" is collected. The input at this point is the keywords to be collected, and the output is the initial collected data.

[1229] Step 2:

[1230] The server temporarily stores the collected text data in a storage system such as a database. This data may contain duplicates and preliminary noise. The input is the collected initial data, and the output is the temporarily stored raw data.

[1231] Step 3:

[1232] The server cleans the stored text data. During this process, it uses natural language processing (NLP) tools (e.g., NLTK or SpaCy) to remove special characters and noise. The input is the temporarily stored raw data, and the output is the cleaned data.

[1233] Step 4:

[1234] The server extracts taste-related keywords and phrases from the cleaned text data. Specifically, it uses NLP technology to apply morphological analysis, TF-IDF, and other techniques. For example, it extracts phrases such as "chewy" and "full of sesame paste." The input is the cleaned data, and the output is the extracted keywords and phrases.

[1235] Step 5:

[1236] The server assigns tags based on extracted keywords and phrases using a generative AI model (e.g., BERT or GPT). For example, the phrase "chewy" is tagged with "chewy texture," and the phrase "plenty of sesame paste" is tagged with "sesame paste." The input is the extracted keywords and phrases, and the output is the tagged data.

[1237] Step 6:

[1238] The server stores the tagged information in a database (e.g., MySQL or PostgreSQL). This data includes the URL of the original text data, the store name, and detailed product information. The input is the tagged data, and the output is the information stored in the database.

[1239] Step 7:

[1240] Users access the system using a web browser or smartphone application. The interface provides fields for entering area and detailed search criteria. For example, users can enter criteria such as "Shinjuku," "chewy texture," "sesame paste," and "yumechikara." The input at this point is the user's search criteria.

[1241] Step 8:

[1242] The terminal sends the search criteria entered by the user to the server. This input is the criteria specified by the user in step 7. The output is the search query sent to the server.

[1243] Step 9:

[1244] The server queries the database based on the received search criteria and extracts stores and products that match the criteria. For example, search for stores in Shinjuku that offer bagels with a chewy texture, sesame paste, and Yumechikara. The input is the search query, and the output is the search results.

[1245] Step 10:

[1246] The server sends the extracted search results to the terminal. For example, it contains a list of stores that match "Shinjuku." At this point, the input is the search results, and the output is the transmitted data.

[1247] Step 11:

[1248] The terminal displays the received search results to the user. The user can select a store from the displayed results and check further details (such as opening hours and menu). The input is the search results sent, and the output is the information displayed on the user's screen.

[1249] In this way, a series of processes are performed, allowing the user to efficiently search for restaurants and products based on detailed, specific conditions.

[1250] (Application example 1)

[1251] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1252] Existing systems that allow users to search for restaurants and products based on area or specific preferences only display search results on a screen, making it difficult to provide detailed information in real time when selecting specific products in a physical store. In addition, there is a lack of a way for users to quickly obtain detailed product information in a store using a smart device.

[1253] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1254] In this invention, the server includes: means for collecting text data related to food taste from gourmet sites and social networking services; means for cleaning the collected text data to extract keywords and phrases related to food taste; means for tagging the extracted keywords and phrases using generative artificial intelligence; means for saving the tagged information in a database; means for providing an interface that allows a user to search by specifying an area or detailed preferences; means for querying the database based on the specified search conditions and extracting relevant stores and products; means for the smart glasses to display detailed information about products and stores based on eye tracking; and means for displaying the extracted search results to the user. This enables detailed information about a specific product to be displayed in real time on the smart glasses when the user looks at the product in a physical store.

[1255] A "gourmet site" is a website that provides information about restaurants and food.

[1256] A "social networking service" is a platform that allows users to exchange information and communicate with each other online.

[1257] "Text data" refers to data that is structured in the form of written words, sentences, comments, etc.

[1258] "Cleaning" is the process of filtering out noise and unnecessary information from collected data and extracting only the necessary parts.

[1259] A "keyword" is an important word that has a specific meaning in text data.

[1260] A "phrase" is a series of words that combine multiple keywords to give them meaning.

[1261] "Generative AI" is a machine learning technique that learns patterns from collected data and automatically tags and categorizes it.

[1262] A "tag" is a label that is attached to information to make it easier to classify or search.

[1263] A "database" is a system for systematically storing and managing data.

[1264] An "interface" is the means or screen through which a user interacts with a system.

[1265] A "query" is an inquiry issued to a database to retrieve specific information.

[1266] "Smart glasses" are devices equipped with displays and sensors that provide information based on the user's line of sight.

[1267] "Eye tracking" is a technology that detects the direction of a user's gaze and collects that data.

[1268] The present invention is a system that allows users to search for restaurants and products based on area and detailed preferences, and provides related information in real time. Specific embodiments of the system will be described below.

[1269] System Overview

[1270] The system consists of the following main components:

[1271] 1. Server: Responsible for data collection, cleaning, tagging, database management, and processing user queries.

[1272] 2. User terminal: Provides a search interface and displays search results. Includes smart glasses.

[1273] 3. Smart glasses: Provides real-time information display based on eye tracking.

[1274] Information gathering

[1275] The server automatically collects text data about food taste from gourmet sites and social networking services using information gathering software such as a web crawler. The collected data is temporarily stored on the server.

[1276] Data cleaning and tagging

[1277] The server cleans the collected text data, removing unnecessary noise and duplication, then uses natural language processing (NLP) to extract taste-related keywords and phrases from the text data, and then uses a generative AI model to automatically assign tags based on these keywords and phrases.

[1278] Database Management

[1279] The server stores the tagged information in a database, which also includes the URL of the original text data, the store name, and detailed product information.

[1280] Providing a search interface

[1281] Users can use a web browser or smartphone application to search by specifying an area and specific search criteria, such as "Shinjuku," "chewy texture," and "sesame paste."

[1282] Query processing and result display

[1283] The user terminal sends the entered search criteria to the server. The server queries the database based on the received criteria and extracts stores and products that match the criteria. The extracted results are then sent to the user terminal, allowing the user to view the search results.

[1284] Information display using smart glasses

[1285] The smart glasses use eye-tracking to detect which products or menu items the user is looking at. When the user's gaze is directed toward a specific product, the relevant information is received from the server and displayed on the smart glasses' display in real time.

[1286] Specific examples

[1287] For example, if a user wants to find a store in Shinjuku that serves bagels with a chewy texture, sesame paste, and Yumechikara, they can enter the search criteria into the interface, and the server will extract matching data. Then, when the user puts on the smart glasses and looks at a bagel in the store, detailed information about the product will be displayed on the glasses' display in real time.

[1288] Example prompts to be input to the generative AI model

[1289] "Imagine a user is searching for a bagel in the Shinjuku area that meets the criteria of 'chewy texture' and 'sesame paste'. Imagine a smart glasses application that displays detailed information about the bagel in real time."

[1290] In this way, the system of the present invention allows users to search for restaurants and products based on detailed preferences and obtain product information in real time within physical stores.

[1291] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1292] Step 1: Gather information

[1293] The server collects text data related to food taste from gourmet websites and social networking services. A crawler program crawls through web pages and extracts text data such as comments and reviews. The input is the website URL or API endpoint, and the output is the collected text data.

[1294] Step 2: Data cleaning

[1295] The server cleans the collected text data. Specifically, it removes unnecessary noise and redundant data and prepares it in a form that can be analyzed. The input is the collected text data, and the output is the cleaned text data.

[1296] Step 3: Extract keywords and phrases

[1297] The server uses natural language processing (NLP) technology to extract taste-related keywords and phrases from the cleaned text data. The input is the cleaned text data, and the output is taste-related keywords and phrases. Specifically, it performs morphological analysis.

[1298] Step 4: Tagging

[1299] The server automatically assigns tags using a generative AI model based on the extracted keywords and phrases. The input is keywords and phrases, and the output is tags assigned to each keyword or phrase. The generative AI understands the context and assigns tags.

[1300] Step 5: Saving to the Database

[1301] The server stores the tagged information in a database. The stored data includes the URL of the original text data, the store name, and detailed product information. The input is the tagged information, and the output is the data stored in the database.

[1302] Step 6: Provide a search interface

[1303] The user terminal provides an interface for searching by specifying an area or detailed specific conditions through a web browser or smartphone application. The input is the search conditions specified by the user, and the output is the transmission of the search conditions to the server.

[1304] Step 7: Query Processing

[1305] The server queries the database based on the search criteria received from the user and extracts stores and products that match the criteria. The input is the search criteria and the output is the extracted search results. Specifically, a database query language (such as SQL) is used.

[1306] Step 8: Viewing search results

[1307] The user terminal displays the search results received from the server to the user, with the input being the search result data from the server and the output being the search results displayed on the user interface.

[1308] Step 9: Eye tracking with smart glasses

[1309] The smart glasses use an eye-tracking function to detect the product or menu item that the user is looking at. The input is the user's gaze data, and the output is the ID of the detected product or menu item. Specifically, an eye-tracking sensor is used.

[1310] Step 10: Real-time information display

[1311] The server sends detailed information about the detected product or menu item to the smart glasses based on its ID. The smart glasses then display this information in real time. The input is the product or menu item ID, and the output is the detailed information displayed on the smart glasses' display. Specifically, wireless communication technology (such as Wi-Fi or Bluetooth) is used.

[1312] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1313] The present invention provides a system that allows users to search for restaurants and products based on area and specific preferences, and also includes a function that recognizes the user's emotions and optimizes search results and suggestions based on those emotions. The system includes a server, a user terminal, a search interface, and an emotion engine. Specific embodiments of the system are described below.

[1314] 1. System Overview

[1315] In this system, the server collects text data related to food taste from gourmet websites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. Generative AI is then used to assign tags based on the extracted keywords and phrases, and the tagged information is stored in a database. Users can use an interface to search by area or specific, detailed criteria, and the system displays search results. An emotion engine is also incorporated to optimize search results and suggestions based on the user's emotional data.

[1316] 2. Information gathering

[1317] The server periodically runs a crawler program that collects text data about food taste from gourmet websites and social networking services. The collected data is temporarily stored and filtered to remove duplication and noise.

[1318] As a concrete example, consider a case where a server collects data from a large number of social networking services that contain comments about "ramen." From the collected comments, it extracts sentences such as "This ramen has a rich soup and is very delicious."

[1319] 3. Data cleaning and analysis

[1320] The server cleans the collected text data, removing special characters and noise, and then uses natural language processing (NLP) technology to extract taste-related keywords and phrases from the text data.

[1321] Based on the above specific example, phrases such as "rich" and "delicious" are extracted.

[1322] 4. Tagging

[1323] The server uses a generation AI based on the extracted keywords and phrases. The generation AI automatically assigns specific tags to each keyword or phrase. For example, "rich" is assigned a tag such as "rich soup" and "delicious" is assigned a tag such as "highly rated."

[1324] 5. Saving to the database

[1325] The server stores the tagged information in a database, which also includes the tags, the URL of the original text data, the store name, and detailed product information.

[1326] 6. Introducing the Emotion Engine

[1327] The server is equipped with an emotion engine that analyzes the user's emotions. The emotion engine collects and analyzes emotion data from the user's facial expressions, voice, text input, etc., and estimates the user's emotional state.

[1328] 7. Providing a search interface

[1329] Users can use a web browser or smartphone application to search by area and specific search criteria. For example, a user can search for "Shinjuku" with the criteria "rich soup" and "highly rated."

[1330] 8. Search processing and result display

[1331] The device sends the search criteria entered by the user to the server. The server queries the database based on the received criteria and extracts relevant stores and products. The emotion engine uses the user's emotional data to adjust the display order and content of the search results.

[1332] For example, if the emotion engine recognizes that the user has a "very satisfied" expression, it will prioritize displaying stores that have received high ratings in the past. Also, if the user has a "highly anticipated" expression, it will recommend new or popular stores.

[1333] 9. Displaying the results

[1334] The server sends the optimized search results to the terminal, which then displays the received search results to the user. The user can check the detailed information of the stores and products that interest them from the displayed results and make the best selection according to their emotional state.

[1335] This embodiment allows the system of the present invention to provide personalized search results based on the user's emotional state in addition to the ability to search based on detailed user preferences.

[1336] The processing flow will be explained below.

[1337] Step 1:

[1338] The server runs a crawler program based on a list of URLs specified by gourmet sites and social networking services, and automatically retrieves text data (reviews and comments) related to food taste from each URL.

[1339] Step 2:

[1340] The server temporarily stores the text data acquired by the crawler program, filtering out duplicate data and noise (such as advertisements and unnecessary text).

[1341] Step 3:

[1342] The server cleans the stored text data, removing special characters and unnecessary blank lines and formatting the data.

[1343] Step 4:

[1344] The server uses natural language processing (NLP) technology on the cleaned text data to extract keywords and phrases related to taste. For example, "rich soup" and "delicious" are extracted from the text "This ramen has a rich soup and is very delicious."

[1345] Step 5:

[1346] The server then uses generative AI to tag the extracted keywords and phrases. For example, it might tag "rich" for "rich soup" and "highly rated" for "delicious."

[1347] Step 6:

[1348] The server stores the tagged information in a database, which also includes the tags, the URL of the original text data, the store name, and detailed product information.

[1349] Step 7:

[1350] The server includes an emotion engine that analyzes the user's emotions by collecting and analyzing emotion data from the user's facial expressions, voice, and text input.

[1351] Step 8:

[1352] Users access the search interface through a web browser or smartphone app and enter the area and detailed criteria they prefer, such as "Shinjuku," "rich soup," and "highly rated."

[1353] Step 9:

[1354] The terminal transmits the input search conditions to the server.

[1355] Step 10:

[1356] The server queries the database based on the received search criteria and extracts relevant stores and products. The emotion engine optimizes the display order and content of search results based on the user's emotional data.

[1357] Step 11:

[1358] The server transmits the optimized search results to the terminal.

[1359] Step 12:

[1360] The terminal displays the received search results to the user. For example, it displays a list of ramen restaurants in "Shinjuku" that meet the conditions of "rich soup" and "highly rated."

[1361] Step 13:

[1362] Users can check the detailed information of stores and products they are interested in from the displayed search results. Recommended stores and products are displayed preferentially based on the user's emotional data.

[1363] In this way, the system of the present invention can provide optimal restaurants and products based on the user's detailed preferences and feelings.

[1364] Example 2

[1365] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1366] Conventional restaurant and product search systems have the problem that it is difficult to search based on the user's detailed preferences, and the search results are not optimized according to the user's emotional state, which limits the user experience. A system that solves these problems is needed.

[1367] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting information about restaurants and products from websites and social media, means for cleaning the collected information and extracting keywords and phrases related to specific attributes, means for tagging the extracted keywords and phrases using a machine learning model, means for saving the tagged tags in a database, means for providing a user interface that allows a user to search by specifying an area or detailed specific conditions, means for querying the database based on the specified search conditions and extracting relevant information, means for displaying the extracted search results to the user, and means for analyzing the user's emotional data and optimizing the search results. This enables advanced searches based on the user's detailed specific conditions and personalized search results based on the user's emotional state.

[1368] "Information about restaurants and products" refers to data about ingredients, dishes, drinks, menus, services, store locations, business hours, ratings, reviews, etc.

[1369] "Websites and social media" refers to media and platforms on the Internet that provide information and enable users to interact with each other, and specifically includes gourmet sites and social networking sites.

[1370] "Means of collection" refers to the use of web crawlers or APIs to automatically obtain data from designated websites and social media.

[1371] "Cleaning means" refers to the process of removing duplicate data, invalid data, and noise from collected data, preparing it for analysis.

[1372] "Means for extracting keywords and phrases" refers to a method of using natural language processing techniques to identify important words and phrases from text data.

[1373] "Means of tagging using a machine learning model" refers to the process of automatically adding relevant tags to extracted keywords and phrases using a trained AI model.

[1374] "Means of saving to a database" refers to the method of storing the processed data in a relational database, NoSQL database, etc.

[1375] "Means for providing a user interface" refers to the method of providing a web page or mobile application that allows a user to enter search criteria and view results.

[1376] "Means of querying a database and extracting relevant information" refers to a method of searching for and retrieving the required information from a database using a query language such as SQL.

[1377] The "means for displaying search results to the user" refers to a method for visually presenting the acquired information to the user in the form of a list, graph, or the like.

[1378] "Means for analyzing emotional data and optimizing search results" refers to a process that analyzes emotions from the user's facial expressions, voice, text input, etc., and displays search results in the most optimal order for the user based on the results.

[1379] The present invention provides a system for efficiently collecting information about restaurants and products and providing optimal search results based on the user's detailed preferences and emotional state. Specific embodiments of this system are described in detail below.

[1380] The server runs a crawler program that collects information about restaurants and products from websites and social media. The collected data is temporarily stored and then cleaned. Cleaning involves analyzing HTML pages using Python's BeautifulSoup library and using regular expressions to remove special characters and noise. For example, if review comments about "ramen" are collected, keywords such as "rich" and "delicious" are extracted from those comments using regular expressions.

[1381] The server then uses natural language processing (NLP) techniques to extract specific keywords and phrases from the collected data. Specifically, it uses Python's NLTK (Natural Language Toolkit) library to extract phrases such as "rich" and "delicious." The extracted data is tagged using a generative AI model, such as GPT-3. For tagging, instructions are given to the generative AI using prompt sentences. The following prompt sentences are used as examples:

[1382] "Analyze the following sentence and generate keywords and appropriate tags related to taste.

[1383] Sentence: "This ramen has a rich soup and is very tasty."

[1384] Once tagged, the data is stored in a database. This is done using a database system such as PostgreSQL, and the information stored includes the tag, the URL of the original text data, the store name, detailed product information, etc. For example, the database contains the tag "rich soup" along with information about "Store A" and "Product B."

[1385] Users can use a search interface to specify areas and detailed search criteria through a web browser or smartphone application. The criteria specified by the user are the area name "Shinjuku" and keywords such as "rich soup" and "highly rated." These criteria are sent from the device to the server via an HTTP POST request. The server executes a query to the database based on the received criteria and extracts search results. SQL is used for the query.

[1386] The extracted search results are optimized using an emotion engine. The emotion engine analyzes emotional data from the user's facial expressions, voice, text input, etc. to determine the user's emotional state in real time. For example, if the user's facial expression indicates "satisfaction," the server will place stores that have received high ratings in the past at the top of the search results. Conversely, if the user's facial expression indicates "expectation," the server will recommend new or popular stores.

[1387] Finally, the device displays the optimized search results to the user. This involves parsing the JSON data and converting it into a user-friendly format. For example, the smartphone screen displays a list of store names, product names, ratings, and summaries.

[1388] This embodiment allows users to perform advanced searches based on detailed preferences and also provides personalized search results based on emotional state, improving the user experience and enabling optimal choices.

[1389] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1390] Step 1: Gather information

[1391] The server runs a crawler program to collect information about restaurants and products from specified websites and social media. The input is the specified query or URL, and the output is the collected raw data.

[1392] Specific operation: The server uses Python's BeautifulSoup library to parse HTML pages from specific gourmet sites and social media sites and extract review comments and ratings about restaurants and products.

[1393] Step 2: Data cleaning

[1394] The server removes duplicates and noise from the collected data. The input is the raw data collected in step 1, and the output is the clean data.

[1395] Specific operation: The server uses regular expressions to remove unnecessary special characters and noise components from the text data and filters out duplicate comments.

[1396] Step 3: Extract keywords and phrases

[1397] The server then uses natural language processing (NLP) techniques to extract specific keywords and phrases from the cleaned data. The input is the clean text data, and the output is the extracted keywords and phrases.

[1398] How it works: The server uses Python's NLTK library to extract keywords and phrases such as "rich" and "delicious" from the text data.

[1399] Step 4: Tagging

[1400] The server then assigns tags using a generative AI model based on the extracted keywords and phrases. The input is the extracted keywords and phrases, and the output is the tagged data.

[1401] How it works: The server uses a generative AI model like GPT-3 to generate appropriate tags for keywords (e.g., "rich soup" for "rich") based on a prompt.

[1402] Example prompt: "Analyze the following sentence and generate taste-related keywords and appropriate tags. Sentence: 'This ramen has a rich soup and is very delicious.'"

[1403] Step 5: Saving to the Database

[1404] The server stores the tagged data in a database. The input is the tagged data and the output is the information stored in the database.

[1405] Specific operation: The server executes an INSERT query to a database system such as PostgreSQL, and saves the tag, the URL of the original text data, the store name, and detailed product information in the database.

[1406] Step 6: Collect emotion data

[1407] The server collects and analyzes emotional data from the user's facial expressions, voice, text input, etc. The input is the user's emotional data, and the output is the analyzed emotional state.

[1408] Specific operation: The server inputs data obtained from the user's webcam and microphone into an emotion analysis algorithm to determine the user's emotional state in real time.

[1409] Step 7: Provide a search interface

[1410] Users use an interface to search by specifying an area and detailed preferences. The input is the user's search query, and the output is a search request based on that query.

[1411] Specific operation: The user enters the area name and keywords into a search form via a web browser or smartphone application.

[1412] Step 8: Search process

[1413] The terminal sends the user's search criteria to the server, which then executes a search query against the database. The input is the user's search query, and the output is the search results for the relevant information.

[1414] Specific operation: Upon receiving a search query sent from the terminal, the server generates an SQL query and searches the database.

[1415] Step 9: Optimize search results

[1416] The server uses an emotion engine to optimize search results based on the user's emotional state, where the inputs are search results and the user's emotional state, and the output is the optimized search results.

[1417] Specific operation: The emotion engine analyzes the user's emotional data and, if it recognizes the emotion of "satisfaction," places stores that have received high ratings in the past at the top of the search results.

[1418] Step 10: View the results

[1419] The server sends the optimized search results to the terminal, and the terminal displays the results to the user. The input is the optimized search results, and the output is the search results presented to the user.

[1420] Specific operation: The server sends optimized search results in JSON format, and the device displays the data in a user-friendly format.

[1421] (Application example 2)

[1422] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1423] Conventional Internet search systems only provide search results based on the area or specific conditions specified by the user, and have limitations in optimizing search results in response to the user's emotional state. Furthermore, to obtain highly satisfying results, it is necessary to dynamically adjust the results based on the user's emotions, but no such system exists yet. The objective of this invention is to solve this problem.

[1424] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1425] In this invention, the server includes means for collecting text data related to food taste from gourmet sites and social networking services, means for cleaning the collected text data to extract keywords and phrases related to food taste, means for tagging the extracted keywords and phrases using generative artificial intelligence, means for saving the tagged information in a database, means for providing an interface that allows users to specify areas and detailed search criteria to perform searches, means for querying the database based on the specified search criteria and extracting relevant stores and products, means for displaying the extracted search results to the user, means for analyzing a user's facial expression image to recognize the user's emotional state, and means for optimizing the search results based on the recognized emotional state, thereby enabling the provision of personalized search results according to the user's emotional state.

[1426] A "gourmet site" is a website that provides information about restaurants and allows users to search for, browse, and rate restaurants.

[1427] A "social networking service" is a web service that allows users to interact and share information online.

[1428] "Text data" is data that mainly consists of characters and symbols, and is information expressed in the form of sentences, comments, etc.

[1429] "Cleaning" refers to the process of removing unnecessary parts from data and making it accurate and usable.

[1430] A "keyword" is an important word or phrase for extracting specific information from text data.

[1431] A "phrase" is a meaningful clause or phrase within text data, and is a collection of words and phrases related to a specific purpose.

[1432] "Generative AI" is a type of machine learning technology that is artificial intelligence that recognizes patterns from given data and generates new data and information.

[1433] A "tag" is a label or identifier assigned to data, and is used to improve the efficiency of information classification and search.

[1434] A "database" is a system for efficiently organizing, storing, searching, and managing large amounts of data.

[1435] An "interface" is a means or screen through which a user operates a system or inputs information.

[1436] A "query" is a question or command used to search for information or to instruct a database to perform an operation.

[1437] "Emotional state" represents the user's psychological state and refers to emotions estimated from facial expressions, voice, text input, etc.

[1438] "Optimization" is the process of adjusting or improving something to maximize its efficiency or effectiveness for a specific purpose.

[1439] An "expression image" is image data capturing a user's facial expression.

[1440] "Personalization" means providing experiences and services that are customized according to the characteristics and preferences of individual users.

[1441] The present invention is a system that allows users to search for restaurants and products based on area and detailed preferences, and provides optimized search results by recognizing the user's emotional state. The system includes a server, a user terminal, a search interface, and an emotion engine.

[1442] System configuration

[1443] The server collects text data related to food taste from gourmet sites and social networking services, cleans the data, and extracts keywords and phrases related to food taste. Based on the extracted keywords and phrases, generative artificial intelligence (generative AI) is used to assign specific tags, and the tagged information is stored in a database. Natural language processing (NLP) technology is used for collection, data cleaning, and keyword extraction.

[1444] The search interface is designed to allow users to search by specifying an area or specific detailed criteria. Users use this interface through a web browser or smartphone application to enter search criteria. The entered criteria are sent to the server, which queries the database to extract relevant stores and products.

[1445] The emotion engine analyzes the user's facial expressions and recognizes their emotional state. The analysis uses the image processing library OpenCV and a custom emotion analysis model (EmotionRecognizer). The analysis results are used as an index to optimize the user's search results.

[1446] Hardware and software used

[1447] Hardware:

[1448] Smartphone: The device on which the user accesses the application.

[1449] Camera: Captures the user's facial expressions using the smartphone's built-in camera.

[1450] software:

[1451] Flask: A Python web framework that acts as the backend for your application.

[1452] OpenCV: An image processing library used to analyze the user's facial expressions.

[1453] EmotionRecognizer: A custom emotion analysis model, used to recognize user emotions.

[1454] Requests: An HTTP library used to communicate with external APIs.

[1455] Specific examples of operation procedures

[1456] 1. The user launches the application, enters detailed preferences such as "I want to eat ramen in the Shinjuku area," and takes a photo of their own facial expression using their smartphone camera.

[1457] 2. The device uploads the captured facial expression image to the application and sends the area and preference conditions to the server.

[1458] 3. The server queries the database based on the received search criteria and generates a list of relevant stores.

[1459] 4. In parallel, the emotion engine analyzes the facial expression image and recognizes it as a "satisfied expression."

[1460] 5. The server optimizes the search results based on the results of sentiment analysis, prioritizing the display of highly rated ramen restaurants.

[1461] 6. The optimized search results are sent to the device, where the user can view the final results.

[1462] Prompt Sentence Examples

[1463] "The system uses images of the user's facial expressions as input to recognize the user's emotions. Based on the recognized emotions, it optimizes search results for restaurants.

[1464] Area: Shinjuku

[1465] Special requirements: Ramen

[1466] Facial image: user_image.jpg

[1467] Output: List of highly rated ramen restaurants"

[1468] In this way, the system of the present invention not only provides a search function based on the user's detailed preferences, but also provides personalized search results based on the user's emotional state.

[1469] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1470] Step 1:

[1471] The user launches the application, inputs the area and detailed preferences, and takes a photo of their facial expression with their smartphone camera. In this case, the user inputs preferences such as "I want to eat ramen in the Shinjuku area" and uploads an image of their facial expression. The input area information, preferences, and facial expression image are then acquired.

[1472] Step 2:

[1473] The device sends the search criteria entered by the user and the captured facial image to the server. Specifically, area information, preference criteria, and facial image data are transferred from the smartphone to the server. The input is area information, preference criteria, and facial image, and the output is data sent to the server.

[1474] Step 3:

[1475] The server queries the database based on the received area information and preferences, and generates a list of relevant stores. Here, it searches the tagged information stored in the database and extracts stores that match the conditions. The input is the area information and preferences, and the output is a list of relevant stores. Specifically, it searches for "ramen restaurants in the Shinjuku area."

[1476] Step 4:

[1477] In parallel, the server analyzes the facial expression image and recognizes the user's emotional state. This analysis is performed using OpenCV and EmotionRecognizer. The input is the facial expression image, and the output is the recognized emotional state. Specifically, it is analyzed as a "satisfied expression."

[1478] Step 5:

[1479] The server optimizes search results based on the emotional state obtained by the emotion engine. Specifically, if the recognized emotion is a "satisfied expression," it prioritizes displaying stores that have received high ratings in the past. The input is a list of relevant stores and the emotional state, and the output is an optimized list of stores.

[1480] Step 6:

[1481] The server sends the optimized search results to the terminal. The input is the optimized store list, and the output is the data transmission to the terminal.

[1482] Step 7:

[1483] The device displays the optimized search results to the user. The user can check the detailed information of the stores and products that interest them from the displayed results and make a selection according to their emotional state. Specifically, "highly rated ramen restaurants in Shinjuku" will be displayed.

[1484] In this way, the system of the present invention can dynamically optimize search results and provide personalized restaurant recommendations based on the user's detailed preferences and emotional state.

[1485] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1486] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1487] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1488] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1489] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1490] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1491] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1492] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1493] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1494] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1495] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1496] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1497] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1498] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1499] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1500] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1501] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1502] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1503] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1504] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1505] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1506] The following is further disclosed regarding the above embodiment.

[1507] (Claim 1)

[1508] A means of collecting text data about food taste from gourmet sites and social networking services,

[1509] A means for cleaning the collected text data and extracting keywords and phrases related to taste;

[1510] A means for tagging using generative artificial intelligence based on the extracted keywords and phrases;

[1511] means for storing the tagged information in a database;

[1512] A means to provide an interface that allows users to search by area and detailed preferences, and

[1513] A means of querying the database based on specified search criteria to extract relevant stores and products;

[1514] means for displaying the extracted search results to a user;

[1515] A system including:

[1516] (Claim 2)

[1517] 10. The system of claim 1, further comprising means for storing the URL, store name, and product details of the original text data associated with the tagged information in a database.

[1518] (Claim 3)

[1519] 10. The system of claim 1, further comprising means for a user to input search criteria using a web browser or a smartphone application and transmit the criteria to the server.

[1520] "Example 1"

[1521] (Claim 1)

[1522] A means of collecting text data about food taste from gourmet sites and social networking services,

[1523] A means for cleaning the collected text data and extracting keywords and phrases related to taste;

[1524] A means for tagging using generative artificial intelligence based on the extracted keywords and phrases;

[1525] means for storing the tagged information in a database;

[1526] A means to provide an interface that allows users to search by area and detailed preferences, and

[1527] A means of querying the database based on specified search criteria to extract relevant stores and products;

[1528] means for displaying the extracted search results to a user;

[1529] a means for periodically running a crawler program;

[1530] A means to remove special characters and noise from the cleaned data;

[1531] a means for analyzing text data using a natural language processing tool;

[1532] A means for automatically tagging using a generative AI model; and

[1533] A system including:

[1534] (Claim 2)

[1535] 10. The system of claim 1, further comprising means for storing the URL, store name, and product details of the original text data associated with the tagged information in a database.

[1536] (Claim 3)

[1537] 10. The system of claim 1, further comprising means for a user to input search criteria using a web browser or a smartphone application and transmit the criteria to the server.

[1538] "Application Example 1"

[1539] (Claim 1)

[1540] A means of collecting text data about food taste from gourmet sites and social networking services,

[1541] A means for cleaning the collected text data and extracting keywords and phrases related to taste;

[1542] A means for tagging using generative artificial intelligence based on the extracted keywords and phrases;

[1543] means for storing the tagged information in a database;

[1544] A means to provide an interface that allows users to search by area and detailed preferences, and

[1545] A means of querying the database based on specified search criteria to extract relevant stores and products;

[1546] A means for smart glasses to display product and store details based on eye tracking;

[1547] means for displaying the extracted search results to a user;

[1548] A system including:

[1549] (Claim 2)

[1550] 10. The system of claim 1, further comprising means for storing the URL, store name, and product details of the original text data associated with the tagged information in a database.

[1551] (Claim 3)

[1552] 10. The system of claim 1, further comprising means for a user to enter search criteria using a web browser or a mobile terminal application and transmit the criteria to the server.

[1553] "Example 2: Combining Emotion Engines"

[1554] (Claim 1)

[1555] A means of collecting information about restaurants and products from websites and social media,

[1556] A means of cleaning the collected information to extract keywords and phrases related to specific attributes;

[1557] A means for tagging using a machine learning model based on the extracted keywords and phrases;

[1558] A means for storing the assigned tags in a database;

[1559] A means to provide a user interface that allows users to search by specifying areas and detailed specific conditions, and

[1560] means for querying the database based on specified search criteria to extract relevant information;

[1561] means for displaying the extracted search results to a user;

[1562] A means for analyzing user sentiment data and optimizing search results;

[1563] A system including:

[1564] (Claim 2)

[1565] 10. The system of claim 1, further comprising means for storing the address, store name, and product details of the original text data associated with the tagged information in a database.

[1566] (Claim 3)

[1567] 10. The system of claim 1, further comprising means for a user to enter search criteria using a web browser or a mobile application and transmit the criteria to the server.

[1568] "Application example 2 when combining emotion engines"

[1569] (Claim 1)

[1570] A means of collecting text data about food taste from gourmet sites and social networking services,

[1571] A means for cleaning the collected text data and extracting keywords and phrases related to taste;

[1572] A means for tagging using generative artificial intelligence based on the extracted keywords and phrases;

[1573] means for storing the tagged information in a database;

[1574] A means to provide an interface that allows users to search by area and detailed preferences, and

[1575] A means of querying the database based on specified search criteria to extract relevant stores and products;

[1576] means for displaying the extracted search results to a user;

[1577] means for analyzing a facial expression image of a user to recognize the user's emotional state;

[1578] a means for optimizing search results based on a perceived emotional state;

[1579] A system including:

[1580] (Claim 2)

[1581] 10. The system of claim 1, further comprising means for storing the URL, store name, and product details of the original text data associated with the tagged information in a database.

[1582] (Claim 3)

[1583] 10. The system of claim 1, further comprising means for a user to input search criteria using a web browser or smart device application and transmit the criteria to the server. [Explanation of symbols]

[1584] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting text data about food taste from gourmet sites and social networking services, A means for cleaning the collected text data and extracting keywords and phrases related to taste; A means for tagging using generative artificial intelligence based on the extracted keywords and phrases; means for storing the tagged information in a database; A means to provide an interface that allows users to search by area and detailed preferences, and A means of querying the database based on specified search criteria to extract relevant stores and products; means for displaying the extracted search results to a user; A system including:

2. The system of claim 1 further comprising means for storing the URL, store name, and product details of the original text data associated with the tagged information in a database.

3. The system of claim 1 , further comprising means for a user to input search criteria using a web browser or a smartphone application and transmit the criteria to the server.

Images