System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving a search query input from a user terminal; means for performing natural language analysis based on the received search query; means for optimizing a search condition based on an analysis result; means for searching related data from a database using the optimized search condition; means for evaluating and ranking search results; and means for transmitting the ranked search results to the user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, where information is overflowing, it is difficult for researchers, developers, corporate marketing personnel, and others to efficiently find high-quality information. Specifically, to obtain search results that accurately reflect the information users are looking for, it is necessary to extract relevant information from vast amounts of data and accurately evaluate and compare it. However, current search systems do not adequately support this process, often resulting in inefficient and inaccurate results. For this reason, there is a need for a system that allows users to access accurate information in a short amount of time. [Means for solving the problem]

[0005] To address these challenges, the present invention provides a means for receiving a search query entered from a user terminal and analyzing it using a natural language analysis model. It also includes a means for optimizing search conditions based on the analysis results and searching for relevant data from a database using the optimized search conditions. The search results are evaluated and ranked based on reliability, freshness, and level of detail, and the results are sent to the user terminal. It also includes a means for using the user's past search history and behavioral data when optimizing the search conditions, thereby enabling the provision of information tailored to the user's specific needs.

[0006] A "user terminal" is a device such as a computer or smartphone that a user operates to input a query and display search results.

[0007] A "search query" is the text or question a user enters to search for specific information.

[0008] "Natural language analysis" is the process of analyzing the natural language text contained in a search query to understand its meaning and structure.

[0009] "Search optimization" is the process of adjusting search criteria to process a query efficiently based on a user's past behavior and search history.

[0010] A "database" is a system or collection of related information or data.

[0011] "Search Results" refers to a list of relevant information extracted from a database based on optimized search criteria.

[0012] "Ranking" is the process of ranking search results based on predefined criteria.

[0013] "Evaluation criteria" are indicators used to compare and evaluate search results, including reliability, freshness of information, and level of detail.

[0014] "Reliability" is a criterion for evaluating the accuracy and reliability of information.

[0015] "Freshness of information" is a criterion for evaluating the recency of information and the update date.

[0016] "Level of detail" is a criterion for evaluating the specificity and detail of information.

[0017] "Past search history" refers to data on search queries and operation history previously performed by a user.

[0018] "Behavioral data" is a record of the user's behavior on the system, such as the user's Internet usage and system operation history.

[0019] These definitions provide a clear understanding of each element of the present invention. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The system according to the present invention implements a series of processes that allow users to search for information efficiently and accurately using a terminal. The basic configuration of this system is made up of a user terminal, a server, and a database client.

[0042] System Program Overview

[0043] Program processing

[0044] 1. User enters a query

[0045] Users access a terminal with an interface called "Search-ru, Kura-beru" and enter specific information such as a search query, "latest natural language processing technology."

[0046] 2. Sending a request from the device to the server

[0047] The device converts this search query into JSON format and sends it to the server as a POST request.

[0048] 3. Server receives and analyzes the request

[0049] The server receives the request and passes the query to a natural language analysis model. This model analyzes the query and extracts relevant keywords and phrases. For example, a query like "latest natural language processing technology" would be broken down into three keywords: "latest," "natural language processing," and "technology."

[0050] 4. Query Optimization

[0051] The server optimizes the search query by referencing the user's past search history and behavioral data. For example, if the user has searched a lot for information about "natural language processing" in the past, it will give more weight to this keyword and adjust the search criteria.

[0052] 5. Database Search

[0053] The server uses optimized search criteria to search the database via an internal database client, where relevant articles, papers, online courses, and other information are retrieved.

[0054] 6. Compare and evaluate search results

[0055] The server evaluates the search results it receives, rating each based on reliability, freshness of information, and specificity.

[0056] 7. Ranking of search results

[0057] The server ranks the search results based on the criteria: for example, if the most recent paper is rated as highly reliable, it will be ranked highly.

[0058] 8. Sending results to the device

[0059] The server sends the ranked search results in JSON format to the device, which receives this information and displays the results in the user's interface.

[0060] 9. User confirmation of results and action

[0061] Users can check the search results displayed on their devices and view detailed information as needed. For example, they can select a specific paper or article to view its details.

[0062] Specific examples

[0063] For example, if a user enters the search query "latest AI technology," the flow is as follows:

[0064] The user enters a search query through the interface, and the terminal sends it to the server.

[0065] The server analyzes the query and extracts keywords such as "latest," "AI," and "technology."

[0066] The server uses the user's past behavioral data to optimize queries and prioritize searches for the latest information related to AI technology.

[0067] It retrieves relevant information from internal databases and external APIs, and ranks the retrieved data based on criteria such as reliability, freshness, and level of detail.

[0068] The ranked results are sent to the user's terminal and displayed.

[0069] Users can review search results and select specific articles or papers to view more details.

[0070] This system allows users to quickly access the information they need and obtain higher quality information.

[0071] The processing flow will be explained below.

[0072] Step 1:

[0073] A user accesses the Search-Ru interface and enters a search query, such as "latest natural language processing technology," in the search box of a browser or dedicated application, and clicks the submit button.

[0074] Step 2:

[0075] The device converts the user's input into JSON format, which may package the search query along with the user's past search history and preferences.

[0076] Step 3:

[0077] The device sends the converted JSON-formatted data to the server as a POST request, which includes the entered search query and related metadata.

[0078] Step 4:

[0079] The server receives the POST request and extracts the search query and related data from the request body.

[0080] Step 5:

[0081] The server uses a natural language analysis model to analyze the search query. It breaks down the query "latest natural language processing technology" into keywords such as "latest," "natural language processing," and "technology," and analyzes the meaning and relevance of each keyword.

[0082] Step 6:

[0083] The server optimizes search criteria based on the user's past search history and behavioral data. For example, if a user has frequently searched for information about "natural language processing" in the past, it will assign a weight to this keyword.

[0084] Step 7:

[0085] The server uses optimized queries to search the database through an internal database client, retrieving relevant articles, papers, videos, online courses, etc.

[0086] Step 8:

[0087] The server evaluates the search results it retrieves based on their reliability, freshness, and specificity, assigning a score to each result and comparing them based on the evaluation results.

[0088] Step 9:

[0089] The server ranks the search results based on criteria, sorting them so that the most reliable and up-to-date information is placed at the top.

[0090] Step 10:

[0091] The server encodes the ranked search results in JSON format and sends them to the device.

[0092] Step 11:

[0093] The device receives the search results and displays them in a user interface, laid out visually and clearly so that the user can easily check each result.

[0094] Step 12:

[0095] Users can review the search results and select articles or papers that interest them to read more, for example, by clicking on a highly rated paper to view its contents.

[0096] Example 1

[0097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] Conventional information search systems often lack the accuracy and relevance of results based on user-entered search queries, making it difficult for users to quickly and reliably access the information they are looking for. Providing appropriate search results is particularly challenging in today's world, where vast amounts of information are readily available. Furthermore, technologies that optimize search results by utilizing users' past search history and behavioral data are often not fully utilized. This limits the user experience, creating problems.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0100] In this invention, the server includes means for receiving a search query input from a user terminal, means for converting the received search query into a structured data format, means for transmitting the converted search query, means for performing natural language analysis based on the received search query, means for optimizing search conditions based on the analysis results, means for searching for related data from a data management system using the optimized search conditions, means for evaluating and ranking search results, means for transmitting the ranked search results to the user terminal, and means for displaying the search results received on the user terminal. This allows the user to quickly obtain optimal information based on the search query and to obtain highly accurate search results based on past search history and behavioral data.

[0101] A "user terminal" is an appliance or device used by a user to input information and display results.

[0102] A "search query" is a word or string of characters that a user enters to search for specific information.

[0103] The "means for receiving a search query" is a device or software module that receives a search query sent from a user terminal.

[0104] A "means for converting into a structured data format" is a device or software module that converts an input search query into a certain format (e.g., JSON format).

[0105] A "means for transmitting a converted search query" is a device or software module that transmits a search query that has been converted into a structured data format over a network.

[0106] "Natural language analysis" is a technology that analyzes natural language, which is human language, and extracts semantic information.

[0107] A "means for performing natural language analysis" is a device or software module that performs natural language analysis on a received search query.

[0108] A "means for optimizing search conditions" is a device or software module that uses the results of natural language analysis and the user's past search history to adjust search conditions in the most effective way.

[0109] A "data management system" is a system for storing, managing, and retrieving information in an organized manner.

[0110] A "means for retrieving relevant data" is a device or software module that retrieves relevant information from a data management system using optimized search criteria.

[0111] A "means for evaluating and ranking search results" is a device or software module that evaluates and prioritizes results based on criteria such as the authoritativeness, freshness, and specificity of the search results.

[0112] The "means for transmitting to a user terminal" is a device or software module that transmits the evaluated and ranked search results to a user terminal.

[0113] The "means for displaying the received search results on the user terminal" is a device or software module that visually displays the received search results on the user terminal.

[0114] The system according to the present invention allows users to search for information efficiently and accurately using a terminal. The basic configuration of this system is made up of a user terminal, a server, and a data management system.

[0115] First, a user accesses a "search interface" through a dedicated application or a web browser, and then enters a specific query, such as "latest natural language processing technology," into the search box on the interface.

[0116] When a user enters a query and presses the search button, the device converts the query into JSON format using JavaScript (registered trademark). For example, it is converted into {"query": "latest natural language processing technology"}. The device then sends this JSON data to the server as an HTTP POST request. The HTTPS protocol is used for transmission, and the destination URL is, for example, https: / / example.com / search.

[0117] The server receives an HTTP POST request via a web server such as Apache (registered trademark) or Nginx. After receiving the request, the server processes the request using a backend framework such as Python's Flask, Django, or Node.js. At this time, a natural language analysis model (e.g., BERT or GPT-3 (registered trademark)) is used to analyze the query and extract related keywords and phrases. For example, the keywords "latest," "natural language processing," and "technology" are extracted from the query "latest natural language processing technology."

[0118] The server then retrieves the user's past search history from the database and uses this information to optimize the search query: if the user searches a lot for "natural language processing," it will weight this keyword more highly to improve the accuracy of the search results.

[0119] The server uses an internal database client (e.g., PostgreSQL or MongoDB) to search for information in the data management system based on optimized search criteria, retrieving relevant data such as papers, articles, online courses, etc.

[0120] The retrieved search results are compared and ranked based on criteria such as reliability, freshness, and specificity. The evaluated results are then compiled into JSON format and sent to the device. For example, the format might look like this: {"results": [{"title": "The latest natural language processing technology", "url": "https: / / example.com / article"}]}. The device receives this JSON data and visually displays the results on the user interface.

[0121] Users can view the displayed search results and click on the information they are interested in to view more details. This system allows users to quickly access the information they are looking for and provides highly accurate search results based on their past search history and behavioral data.

[0122] Specific examples

[0123] For example, if a user enters the search query "latest AI technology":

[0124] The user inputs the latest AI technology through the interface, and the device sends it to the server.

[0125] The server analyzes the query and extracts keywords such as "latest," "AI," and "technology."

[0126] The server uses the user's past behavioral data to optimize queries and prioritize searches for the latest information related to AI technology.

[0127] It retrieves relevant information from internal databases and external APIs and ranks it based on criteria such as reliability, freshness, and detail.

[0128] The ranked results are sent to the user's terminal and displayed.

[0129] Users can review search results and select specific articles or papers to view more details.

[0130] Prompt Sentence Examples

[0131] For example, the prompt text to input to a generative AI model might look like this:

[0132] "I want to search for information about the latest natural language processing technology. Related keywords are 'latest,' 'natural language processing,' and 'technology,' and this user has searched a lot for information about natural language processing in the past."

[0133] The above is a specific embodiment for carrying out the present invention.

[0134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0135] Step 1:

[0136] A user accesses a search interface on a terminal and inputs a search query such as "latest natural language processing technology." The input query is stored in text format in an input field on the terminal.

[0137] Step 2:

[0138] When the user presses the search button, the device uses JavaScript to convert the search query into JSON format, for example, {"query": "latest natural language processing technology"}. This JSON data is then prepared as an HTTP POST request.

[0139] Step 3:

[0140] The terminal uses the HTTPS protocol to send this JSON-formatted query to the server. The destination URL is, for example, https: / / example.com / search. The sent JSON data arrives at the server.

[0141] Step 4:

[0142] The server receives this HTTP POST request via a web server such as Apache or Nginx and extracts the search query in JSON format from the body of the request. For example, the data extracted is {"query": "The latest natural language processing technology"}.

[0143] Step 5:

[0144] The server uses a parsing module to input the search query into a natural language parsing model (e.g., BERT or GPT-3). The model analyzes the query and extracts relevant keywords and phrases. For example, the query "latest natural language processing technology" yields the keywords "latest," "natural language processing," and "technology."

[0145] Step 6:

[0146] The server retrieves the user's past search history and behavioral data from the database. Based on this, it optimizes the weighting of queries. For example, if there have been many searches related to "natural language processing" in the past, it will increase the weighting of this keyword.

[0147] Step 7:

[0148] The server issues SQL or NoSQL queries to data management systems (e.g., PostgreSQL or MongoDB) based on the optimized search criteria, searching relevant databases for information such as related papers, articles, or online courses.

[0149] Step 8:

[0150] The server retrieves the search results and rates each item based on the criteria of authoritativeness, freshness, and specificity, using a numerical score or other criteria, and assigns each result a score for authoritativeness, freshness, and specificity.

[0151] Step 9:

[0152] The server ranks the search results based on their scores, placing highly rated information at the top of the list. The server then compiles the ranked search results in JSON format, structured as follows: {"results": [{"title": "The latest natural language processing technology", "url": "https: / / example.com / article"}]}.

[0153] Step 10:

[0154] The server sends the ranked search results as JSON data to the user's device, which receives the JSON data, parses it, and displays it on the user interface.

[0155] Step 11:

[0156] Users can view detailed information by checking the search results displayed on their device and clicking on the links to articles or papers they are interested in. For example, they can click on the title of a particular paper to read more about it.

[0157] The above are the specific processing steps of the program of this system.

[0158] (Application example 1)

[0159] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0160] In physical stores, users face the challenge of finding the products they want efficiently and quickly. Furthermore, there is a lack of systems that can provide detailed product information, reviews, price comparisons, and other information in real time to help users make appropriate purchasing decisions. These situations result in a poor user experience.

[0161] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0162] In this invention, the server includes a means for the user terminal to provide search results including location information and reviews via a display device, a means for considering reliability, freshness of information, and level of detail, and a means for using the user's past search history and behavioral data when optimizing search conditions, thereby enabling users to quickly find desired products in physical stores, obtain detailed information, and make appropriate purchasing decisions.

[0163] "User device" is a general term for devices that users use to enter queries and view search results, such as smartphones and smart glasses.

[0164] A "search query" is a string of characters or voice command that a user types to obtain information.

[0165] "Natural language analysis" is the process of converting search queries into a format that is easy for machines to understand.

[0166] "Search optimization" refers to transforming search queries into more effective forms based on user requirements and past behavioral data.

[0167] A "database" refers to a system or structure that stores retrieved information and makes it available for retrieval as needed.

[0168] "Ranking" refers to evaluating and ranking the search results obtained based on certain criteria.

[0169] A "display device" is a device that visually presents search results to users, such as a smartphone screen or a smart glasses display.

[0170] "Location information" refers to information about the location where a particular product is located.

[0171] A "review" is an evaluation or impression posted by a user who has previously purchased the product.

[0172] "Credibility" is an indicator of how trustworthy the search results are.

[0173] "Information freshness" is an indicator of how up-to-date the search results are.

[0174] "Detail" is an indicator of how detailed the search results provide.

[0175] MODE FOR CARRYING OUT THE INVENTION

[0176] As an embodiment of the present invention, a smart shopping assistant system for improving user experience in a physical store will be described as a specific example. This system is composed of a user terminal, a server, and a database.

[0177] Program Overview

[0178] 1. User Device

[0179] Display devices such as smartphones and smart glasses are used as user terminals. These terminals accept voice and text inputs and retrieve user search queries.

[0180] Example: A user speaks into their smartphone, "What's the latest chocolate product?"

[0181] 2. Submitting a query

[0182] The user terminal converts the entered search query into JSON format and sends it to the server.

[0183] 3. Server

[0184] The server receives a search query sent from a user terminal.

[0185] Incoming search queries are parsed by a natural language parsing model (e.g., BERT), which breaks the query down into keywords and converts them into an understandable format.

[0186] Based on the analysis results, the search conditions are optimized by referring to the user's past search history and behavioral data.

[0187] 4. Database

[0188] The server uses optimized search criteria to search internal databases and external APIs, which store product information, reviews, pricing, and more.

[0189] 5. Rating and Ranking of Search Results

[0190] The server evaluates and ranks the search results based on reliability, freshness of information, and level of detail.

[0191] 6. Displaying the results

[0192] The ranked search results are sent to the user's terminal in JSON format and presented to the user via a display device, including product location information and reviews.

[0193] Specific examples

[0194] For example, if a user enters a query such as "What are the latest chocolate products?", the following process will occur:

[0195] The query is sent by the terminal to the server.

[0196] The server analyzes the received query and extracts keywords such as "latest," "chocolate," and "product."

[0197] Search criteria are optimized based on the user's past search history.

[0198] Relevant product information is retrieved from an internal database and external APIs, and then evaluated and ranked based on criteria such as reliability and freshness.

[0199] The ranked results are sent to the user's device and displayed along with product location information and reviews.

[0200] Prompt Sentence Examples

[0201] Voice input: "What's the latest chocolate product?"

[0202] Text input: "Tell me about new chocolate products"

[0203] As described above, this invention enables users to quickly find the desired product in a physical store, obtain detailed information, and make appropriate purchasing decisions, thereby significantly improving the user's purchasing experience.

[0204] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0205] Step 1:

[0206] User enters a search query

[0207] How it works: A user uses a smartphone or smart glasses to enter a search query using voice or text input.

[0208] Input: "What's the latest chocolate product?" (voice or text)

[0209] Output: Query data entered on the user's terminal

[0210] Step 2:

[0211] Sending a query from the device to the server

[0212] How it works: The user device converts the entered search query into JSON format and sends it to the server.

[0213] Input: Query data entered on the user's device

[0214] Output: Query data converted to JSON format

[0215] Step 3:

[0216] Server receives and analyzes the query

[0217] How it works: The server receives the query data in JSON format and passes the query to a natural language parsing model (e.g., BERT). The model parses the query and breaks it down into keywords.

[0218] Input: Query data in JSON format

[0219] Output: Extracted keywords (e.g. "latest", "chocolate", "product")

[0220] Step 4:

[0221] Optimizing search criteria

[0222] How it works: The server references the user's past search history and behavioral data to optimize search criteria based on the search query.

[0223] Input: Keywords and user history data

[0224] Output: Optimized search criteria

[0225] Step 5:

[0226] Database search

[0227] How it works: The server uses optimized search criteria to search for product information, reviews, pricing information, etc. from internal databases and external APIs.

[0228] Input: Optimized search criteria

[0229] Output: Search result data

[0230] Step 6:

[0231] Rating and ranking of search results

[0232] How it works: The server evaluates and ranks the search results it retrieves based on their authoritativeness, freshness, and specificity.

[0233] Input: Search result data

[0234] Output: Ranked search results

[0235] Step 7:

[0236] Sending results to the device

[0237] How it works: The server sends the ranked search results in JSON format to the user's device.

[0238] Input: Ranked search results

[0239] Output: Search result data in JSON format

[0240] Step 8:

[0241] Displaying search results

[0242] How it works: The user's device parses the JSON-formatted search results data received from the server and displays them on the display device (smartphone screen or smart glasses display) along with location information and reviews.

[0243] Input: Search result data in JSON format

[0244] Output: Search results displayed on a display device (e.g. product name, location, reviews)

[0245] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0246] The system according to the present invention allows users to search for information efficiently and accurately using a terminal, and further improves the search experience by recognizing the user's emotions during the search process. The basic configuration of this system is made up of a user terminal, a server, a database client, and an emotion engine.

[0247] System Program Overview

[0248] Program processing

[0249] 1. User enters a query

[0250] A user enters a search query in the search interface, for example, entering specific information such as "latest natural language processing technology."

[0251] 2. User Emotion Recognition

[0252] The device activates an emotion engine when the user inputs something, recognizing the user's emotions in real time. The emotion engine uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice, recognizing emotions such as "excited," "calm," or "irritated."

[0253] 3. Sending a request from the device to the server

[0254] The device converts the search query and recognized emotion data into JSON format and sends it to the server as a POST request.

[0255] 4. Server Receipt and Analysis of Request

[0256] The server receives the request and analyzes the query using a natural language analysis model. The query "latest natural language processing technology" is broken down into keywords such as "latest," "natural language processing," and "technology," and the meaning and relevance of each keyword is analyzed.

[0257] 5. Query Optimization

[0258] The server references the user's past search history and behavioral data, and also optimizes search conditions based on recognized emotional data. For example, if the user is in an "excited" state, the server will prioritize searches for new technologies and innovative content.

[0259] 6. Database Search

[0260] The server searches the database through an internal database client using optimized search criteria to retrieve relevant articles, papers, videos, online courses, etc.

[0261] 7. Compare and evaluate search results

[0262] The server evaluates the search results it retrieves based on their reliability, freshness, and specificity, assigning a score to each result and comparing them based on the evaluation results.

[0263] 8. Ranking of search results

[0264] The server ranks the search results based on criteria, with the most reliable and up-to-date information ranked at the top.

[0265] 9. Sending results to the device

[0266] The server encodes the ranked search results in JSON format and sends them to the device.

[0267] 10. User confirmation of results and action

[0268] The terminal receives the search results and displays them on the user interface. The user can check the displayed search results and view the details of each result.

[0269] Specific examples

[0270] For example, if a user enters the search query "latest AI technology" and the emotion engine recognizes that the user is "excited," the flow is as follows:

[0271] The user enters a search query through the interface, and the device transmits the query and emotion data to the server.

[0272] The server analyzes the query and extracts keywords such as "latest," "AI," and "technology."

[0273] The server optimizes queries to prioritize new technologies and innovative content because users are "excited."

[0274] It retrieves relevant information from internal databases and external APIs, and ranks the retrieved data based on criteria such as reliability, freshness, and level of detail.

[0275] The ranked results are sent to the user's terminal and displayed.

[0276] Users review the search results and select the most interesting articles or papers to read more.

[0277] In this way, a more personalized search experience can be provided by taking the user's emotions into account.

[0278] The processing flow will be explained below.

[0279] Step 1:

[0280] A user accesses the "Search-ru, Kuberu" interface and enters a search query. Specifically, the user enters a query such as "latest natural language processing technology" into the search box using a browser or a dedicated application, and clicks the submit button.

[0281] Step 2:

[0282] The device receives user input and activates the emotion engine, which uses the device's camera and microphone to analyze the user's facial expressions and tone of voice in real time.

[0283] Step 3:

[0284] The device analyzes the emotional data (e.g., "excited," "calm," "irritated," etc.) and converts it into JSON format along with the search query.

[0285] Step 4:

[0286] The device sends the converted JSON-formatted data to the server as a POST request, which includes the entered search query and the recognized emotion data.

[0287] Step 5:

[0288] The server receives the POST request and extracts the search query and sentiment data from the request body.

[0289] Step 6:

[0290] The server uses a natural language analysis model to analyze the search query. It breaks down the query "latest natural language processing technology" into keywords such as "latest," "natural language processing," and "technology," and analyzes the meaning and relevance of each keyword.

[0291] Step 7:

[0292] The server optimizes search criteria based on the user's past search history, behavioral data, and emotional data. For example, if a user has frequently searched for information about "natural language processing" in the past and is currently in an "excited" state, the server will adjust search criteria to prioritize information about more innovative and new technologies.

[0293] Step 8:

[0294] The server searches the database through an internal database client using optimized search criteria to retrieve relevant articles, papers, videos, online courses, etc.

[0295] Step 9:

[0296] The server evaluates the search results it retrieves based on their reliability, freshness, and specificity, assigning a score to each result and comparing them based on the evaluation results.

[0297] Step 10:

[0298] The server ranks the search results based on criteria, with the most reliable and up-to-date information ranked at the top.

[0299] Step 11:

[0300] The server encodes the ranked search results in JSON format and sends them to the device.

[0301] Step 12:

[0302] The device receives the search results and displays them on the user interface. The display method is laid out in a visually easy-to-understand manner, taking into consideration the user's convenience.

[0303] Step 13:

[0304] Users can review the search results and select the articles or papers that interest them most for more detailed information. For example, they can click on a highly rated paper to view its contents in more detail.

[0305] Example 2

[0306] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0307] Conventional search systems are required to provide highly relevant information in response to user search queries, but it has been difficult to provide search results that take into account the user's emotions and search context. As a result, users are burdened with selecting the most relevant information from a large amount of information, resulting in a poor search experience. The present invention aims to solve this problem and provide more appropriate and personalized search results that take into account the user's emotions.

[0308] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving a search query input from a user terminal, a means for performing natural language analysis based on the received search query, and a means for optimizing search conditions based on the analysis results. This makes it possible to recognize the user's emotions in real time and optimize the search conditions based on the emotion data. This realizes a personalized search experience that reflects the user's emotions, reduces the user's search burden, and enables more appropriate information to be provided more quickly.

[0309] A "user terminal" is a communication device operated by a user, and is a device for inputting search queries, recognizing emotion data, and displaying search results.

[0310] A "search query" is a query that a user inputs in the form of a string of characters or voice to express the information they want to know or search for.

[0311] "Natural language analysis" is a technology that allows a computer to understand and analyze the meaning of text written in human language.

[0312] "Search condition optimization" is a technique that adjusts conditions to efficiently and effectively search for related information based on a received search query.

[0313] A "database" is a system for storing structured data and searching and retrieving it as needed.

[0314] "Relevant data" is information extracted from a database based on a search query, and is useful data that matches the search intent.

[0315] "Ranking of search results" refers to the process of ranking search results based on evaluation criteria and determining the priority of the results to be presented to the user.

[0316] "Real-time emotion recognition" is a technology that instantly recognizes a user's current emotions from facial expressions, tone of voice, etc.

[0317] "Emotion data" is data that indicates the emotional state of the user recognized in real time.

[0318] "User interface" is a general term for the screens and input devices that allow a user to interact with a system.

[0319] MODE FOR CARRYING OUT THE INVENTION

[0320] A specific embodiment of the system according to the present invention will be described. This system allows users to search for information efficiently and accurately using a terminal, and further improves the search experience by recognizing the user's emotions during the search process. The basic configuration of this system is made up of a user terminal, a server, a database client, and an emotion engine.

[0321] Hardware and software used

[0322] User devices: PCs, smartphones, tablets, etc.

[0323] Server: Web server (Apache HTTP Server, Nginx, etc.), application server

[0324] Database client: Database management system such as MySQL (registered trademark), PostgreSQL, etc.

[0325] Emotion engine: Camera, microphone, emotion recognition software (OpenCV, Microsoft® Azure® Cognitive Services, etc.)

[0326] Generative AI models: GPT-3, BERT, etc. for natural language analysis

[0327] System basic configuration and operation

[0328] 1. Enter a query

[0329] A user enters a search query through an interface installed on the device (such as a browser-based web application or a mobile application), for example, entering a specific information request such as "latest natural language processing technology."

[0330] 2. Emotion recognition

[0331] When a user inputs a query, the device activates an emotion engine. The emotion engine uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice, allowing it to recognize the user's emotions, such as "excited," "calm," or "irritated," in real time.

[0332] 3. Sending a request

[0333] The device converts the search query and the recognized emotion data into JSON format and sends it to the server as an HTTP POST request.

[0334] 4. Receiving and analyzing requests

[0335] The server receives the JSON-formatted request from the device. It then uses a generative AI model for natural language analysis (e.g., GPT-3 or BERT) to analyze the query, breaking down the query "latest natural language processing technology" into keywords such as "latest," "natural language processing," and "technology." It then analyzes the meaning and relevance of each keyword.

[0336] 5. Query Optimization

[0337] The server uses the user's past search history and behavioral data, as well as emotional data, to optimize search conditions. For example, if the server detects that the user is "excited," it will prioritize new technologies and innovative content.

[0338] 6. Database Search

[0339] The server searches the database through the database client using optimized search criteria, such as executing SQL queries to retrieve relevant articles, papers, videos, online courses, etc.

[0340] 7. Display of search results

[0341] The server evaluates the search results based on their reliability, freshness, and detail, assigns a score, compares them, and then ranks them based on the evaluation criteria. The ranked results are encoded in JSON format and sent to the device.

[0342] 8. User confirmation of results

[0343] The terminal receives the search results sent from the server and displays them on the user interface. The user can then review the displayed search results, select the most interesting article or paper, and view its details.

[0344] Specific examples

[0345] For example, if a user enters the search query "latest AI technology" and the emotion engine recognizes the user's emotion as "excited," the operation flow is as follows:

[0346] 1. User: Enters the query "latest AI technology."

[0347] 2. Device: The emotion engine recognizes that the user is excited based on their facial expressions and voice.

[0348] 3. Terminal: Convert the search query and sentiment data into JSON format and send it to the server as an HTTP POST request.

[0349] 4. Server: Receives the request and breaks down "latest AI technology" into the keywords "latest," "AI," and "technology."

[0350] 5. Server: Optimize queries to prioritize innovative content based on what users are "excited about."

[0351] 6. Server: Searches the database using optimized search criteria to retrieve relevant information.

[0352] 7. Server: Evaluates and ranks the acquired information.

[0353] 8. Server: Encode the ranked search results in JSON format and send them to the device.

[0354] 9. Device: Search results are displayed and the user checks the details.

[0355] Examples of usage prompt statements

[0356] Search for "latest AI technology" and describe a system that provides the most appropriate information to a user when the user is emotionally "excited."

[0357] As described above, this system takes user emotions into consideration to provide a more appropriate and personalized search experience.

[0358] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0359] Step 1:

[0360] Query Input

[0361] A user inputs a search query through the device interface. A specific example of what the user inputs is "the latest natural language processing technology." The input data is in text format.

[0362] Specific behavior:

[0363] The user enters a search query and clicks the submit button, and the device stores the entered text query in its internal memory.

[0364] Step 2:

[0365] emotion recognition

[0366] The device activates an emotion engine to recognize the user's emotions in real time. The device uses a camera and microphone to analyze the user's facial expressions and tone of voice to determine emotions.

[0367] input:

[0368] User's facial expression and voice data

[0369] Specific behavior:

[0370] The device inputs data acquired from the camera and microphone into emotion recognition software to determine the user's current emotion (e.g., "excited" or "calm"), and the analysis results are stored in internal memory.

[0371] output:

[0372] User emotion data (e.g., excited)

[0373] Step 3:

[0374] Send request

[0375] The device converts the search query and recognized emotion data into JSON format and sends it to the server as an HTTP POST request.

[0376] input:

[0377] Search query and user sentiment data

[0378] Specific behavior:

[0379] The device converts the search query and sentiment data into JSON format as key-value pairs, then generates an HTTP request and sends it to the specified URL on the server.

[0380] output:

[0381] JSON formatted HTTP POST request

[0382] Step 4:

[0383] Receiving and parsing requests

[0384] The server receives a JSON-formatted request from the device and analyzes the query using a generative AI model. For example, the query "latest natural language processing technology" is broken down into keywords such as "latest," "natural language processing," and "technology."

[0385] input:

[0386] Request data in JSON format

[0387] Specific behavior:

[0388] The server receives the request and feeds the query to the generative AI model, which analyzes the query and extracts important keywords, which are then stored in an internal database.

[0389] output:

[0390] Analyzed keyword data (e.g., "latest," "natural language processing," "technology")

[0391] Step 5:

[0392] Query optimization

[0393] The server optimizes the search criteria based on the analyzed keyword data, as well as the user's emotional data and past search history. For example, if the user is "excited," it will prioritize new technologies and innovative content.

[0394] input:

[0395] Analyzed keyword data, user sentiment data, and past search history

[0396] Specific behavior:

[0397] The server uses this data to run an optimization algorithm to determine the highest priority search terms, which results in an optimized search term.

[0398] output:

[0399] Optimized search criteria

[0400] Step 6:

[0401] Database search

[0402] The server searches the database using optimized search criteria and retrieves relevant information internally or through an external API.

[0403] input:

[0404] Optimized search criteria

[0405] Specific behavior:

[0406] The server issues SQL queries to the database to retrieve data that matches the search criteria, as well as external API calls.

[0407] output:

[0408] Retrieved search result data

[0409] Step 7:

[0410] Rating and ranking of search results

[0411] The search results obtained by the server are evaluated based on reliability, freshness, and detail, and are assigned a score to rank them.

[0412] input:

[0413] Retrieved search result data

[0414] Specific behavior:

[0415] The server evaluates each search result based on the evaluation criteria, assigns a score, and ranks the search results based on the score.

[0416] output:

[0417] Ranked search results data

[0418] Step 8:

[0419] Send results

[0420] The server encodes the ranked search results in JSON format and sends them to the device.

[0421] input:

[0422] Ranked search results data

[0423] Specific behavior:

[0424] The server encodes the search results in JSON format and sends them to the terminal as an HTTP response.

[0425] output:

[0426] JSON formatted HTTP response

[0427] Step 9:

[0428] Check the results

[0429] The terminal receives the search results sent from the server and displays them on the user interface, where the user can check the displayed search results and view their details.

[0430] input:

[0431] Search result data in JSON format

[0432] Specific behavior:

[0433] The device receives the search results, parses them, and displays them in a user interface that the user can navigate and click on results of interest to learn more.

[0434] output:

[0435] Search results displayed to users

[0436] (Application example 2)

[0437] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0438] Conventional food delivery services lack personalized meal suggestions because they do not provide a search experience that takes into account users' emotions and individual preferences. Another issue is the lack of functionality to analyze users' emotional data and optimize search criteria based on that data in order to provide optimal search results for users' search queries.

[0439] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recognizing a user's emotion, means for optimizing search conditions based on the recognized emotion, and means for searching for related data from a database using the optimized search conditions. This enables a personalized search experience based on the user's emotion, and can provide more optimal search results.

[0440] A "user terminal" is a hardware device operated by a user, such as a smartphone or tablet.

[0441] A "search query" refers to the text or voice input a user enters to search for specific information.

[0442] "Means for recognizing emotions" refers to technology that analyzes a user's words and actions, facial expressions, tone of voice, etc. to determine their emotions at that time.

[0443] "Natural language analysis" is a technology for analyzing input text or speech and understanding its meaning.

[0444] "Means for optimizing search conditions" refers to technology that performs processing to make search results more accurate and useful based on the user's input query and emotional data.

[0445] A "database" is an organized collection of data that stores information responsive to a search query.

[0446] The "means for searching related data" is a technique for extracting related information in a database using optimized search criteria.

[0447] "Means for evaluating and ranking search results" refers to technology that evaluates the search results obtained based on criteria such as reliability and freshness, and ranks them in order of importance.

[0448] "Means for sending to user terminal" refers to the technology for transferring the evaluated and ranked search results to the user terminal.

[0449] "Past search history" is a record of search queries a user has previously performed.

[0450] "Behavioral data" refers to data relating to the behavior of a user, such as what operations the user performed and which pages the user viewed.

[0451] The system according to the present invention comprises a user terminal, a server, a database, and an emotion engine. The system program performs processing in the following steps: When a user inputs a search query using a food delivery app, the emotion recognition means analyzes the user's emotion in real time, and the query is sent to the server together with that data.

[0452] The server receives the user's search query and emotion data, and optimizes the search query based on the recognized emotion. The search criteria are further refined by referring to the user's past search history and behavioral data. Based on the optimized search criteria, relevant information is searched for from the database. The obtained search results are evaluated based on criteria such as reliability, freshness, and level of detail, and a ranking is assigned. The ranked search results are sent from the server to the user's device and displayed on the user interface.

[0453] Examples of hardware and software used include smartphones and tablets as user devices, emotion engines for recognizing emotions, natural language processing models for data analysis, and servers for searching and evaluating databases.

[0454] As a concrete example, consider the case where a user searches for "Italian restaurant." If the user's device uses an emotion engine to recognize the emotion "excited," the server uses that emotion data to optimize the search criteria to prioritize new restaurants and special menus. The server then searches the database based on the optimized search criteria to retrieve information about new restaurants and special menus. The information is then ranked based on criteria such as reliability and freshness and presented to the user.

[0455] An example prompt might be:

[0456] "When a user enters a search query in a food delivery app, analyze the data and recognize the user's emotions in real time. Design a system that optimizes the query based on the user's emotions and provides search results for relevant restaurants and menus. For example, when a user is excited, suggest new restaurants or special menus, and when a user is stressed, prioritize past favorite restaurants or menus that have given them high satisfaction."

[0457] This provides a personalized search experience that takes user emotions into account, which is expected to improve satisfaction with food delivery services.

[0458] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0459] Step 1:

[0460] A user opens a food delivery app and enters a search query. Input: The user's query (e.g., "Italian restaurant") and the user's facial image and voice data. Output: The device collects the query and emotion data. Specific operation: The user enters the query into an input box on the device interface, and the device's camera and microphone are used to collect the user's facial expressions and tone of voice in real time.

[0461] Step 2:

[0462] The device activates the emotion engine and recognizes the user's emotions in real time. Input: User's facial image and voice data. Output: Recognized emotion data (e.g., "Excited"). Specific operation: The emotion engine uses face recognition algorithms and voice analysis algorithms to analyze the user's facial muscle movements and voice tone.

[0463] Step 3:

[0464] The device sends the search query and recognized emotion data to the server. Input: Query and emotion data. Output: Data sent to the server. Specific operation: The device encodes the query and emotion data into JSON format and sends it to the server using an HTTP POST request.

[0465] Step 4:

[0466] The server analyzes the received query and sentiment data and performs natural language analysis on the search query. Input: Query and sentiment data. Output: Keywords from the analyzed query (e.g., "Italian," "restaurant"). What happens: A natural language processing (NLP) model on the server tokenizes the query and extracts key keywords.

[0467] Step 5:

[0468] The server references the user's past behavioral data and search history along with the recognized emotions to optimize the search criteria. Input: Analyzed keywords, emotion data, past behavioral data, and search history. Output: Optimized search criteria. Specific operation: The server retrieves the user's past history from the database and adjusts the search algorithm along with the emotion data.

[0469] Step 6:

[0470] The server searches the database for relevant information based on the search criteria. Input: Optimized search criteria. Output: Relevant information (e.g., a list of new Italian restaurants). Specific behavior: The server executes a database query to extract relevant restaurants and menus.

[0471] Step 7:

[0472] The server evaluates and ranks the search results. Input: Retrieved search results. Output: Rated and ranked results. Specific behavior: The server assigns a score to each result using evaluation criteria such as reliability, freshness, and specificity, and then ranks them.

[0473] Step 8:

[0474] The server sends the ranked search results to the user's terminal. Input: Ranked search results. Output: Search results sent to the user's terminal. Specific operation: The server encodes the ranked results in JSON format and sends them to the user's terminal as an HTTP response.

[0475] Step 9:

[0476] The user terminal receives the search results and displays them on the user interface. Input: Ranked search results. Output: Displayed search results. Specific operation: The application on the terminal analyzes the search results and displays them to the user.

[0477] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0478] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0479] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0480] [Second embodiment]

[0481] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0482] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0483] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0484] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0485] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0486] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0487] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0488] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0489] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0490] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0491] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0492] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0493] The system according to the present invention implements a series of processes that allow users to search for information efficiently and accurately using a terminal. The basic configuration of this system is made up of a user terminal, a server, and a database client.

[0494] System Program Overview

[0495] Program processing

[0496] 1. User enters a query

[0497] Users access a terminal with an interface called "Search-ru, Kura-beru" and enter specific information such as a search query, "latest natural language processing technology."

[0498] 2. Sending a request from the device to the server

[0499] The device converts this search query into JSON format and sends it to the server as a POST request.

[0500] 3. Server receives and analyzes the request

[0501] The server receives the request and passes the query to a natural language analysis model. This model analyzes the query and extracts relevant keywords and phrases. For example, a query like "latest natural language processing technology" would be broken down into three keywords: "latest," "natural language processing," and "technology."

[0502] 4. Query Optimization

[0503] The server optimizes the search query by referencing the user's past search history and behavioral data. For example, if the user has searched a lot for information about "natural language processing" in the past, it will give more weight to this keyword and adjust the search criteria.

[0504] 5. Database Search

[0505] The server uses optimized search criteria to search the database via an internal database client, where relevant articles, papers, online courses, and other information are retrieved.

[0506] 6. Compare and evaluate search results

[0507] The server evaluates the search results it receives, rating each based on reliability, freshness of information, and specificity.

[0508] 7. Ranking of search results

[0509] The server ranks the search results based on the criteria: for example, if the most recent paper is rated as highly reliable, it will be ranked highly.

[0510] 8. Sending results to the device

[0511] The server sends the ranked search results in JSON format to the device, which receives this information and displays the results in the user's interface.

[0512] 9. User confirmation of results and action

[0513] Users can check the search results displayed on their devices and view detailed information as needed. For example, they can select a specific paper or article to view its details.

[0514] Specific examples

[0515] For example, if a user enters the search query "latest AI technology," the flow is as follows:

[0516] The user enters a search query through the interface, and the terminal sends it to the server.

[0517] The server analyzes the query and extracts keywords such as "latest," "AI," and "technology."

[0518] The server uses the user's past behavioral data to optimize queries and prioritize searches for the latest information related to AI technology.

[0519] It retrieves relevant information from internal databases and external APIs, and ranks the retrieved data based on criteria such as reliability, freshness, and level of detail.

[0520] The ranked results are sent to the user's terminal and displayed.

[0521] Users can review search results and select specific articles or papers to view more details.

[0522] This system allows users to quickly access the information they need and obtain higher quality information.

[0523] The processing flow will be explained below.

[0524] Step 1:

[0525] A user accesses the Search-Ru interface and enters a search query, such as "latest natural language processing technology," in the search box of a browser or dedicated application, and clicks the submit button.

[0526] Step 2:

[0527] The device converts the user's input into JSON format, which may package the search query along with the user's past search history and preferences.

[0528] Step 3:

[0529] The device sends the converted JSON-formatted data to the server as a POST request, which includes the entered search query and related metadata.

[0530] Step 4:

[0531] The server receives the POST request and extracts the search query and related data from the request body.

[0532] Step 5:

[0533] The server uses a natural language analysis model to analyze the search query. It breaks down the query "latest natural language processing technology" into keywords such as "latest," "natural language processing," and "technology," and analyzes the meaning and relevance of each keyword.

[0534] Step 6:

[0535] The server optimizes search criteria based on the user's past search history and behavioral data. For example, if a user has frequently searched for information about "natural language processing" in the past, it will assign a weight to this keyword.

[0536] Step 7:

[0537] The server uses optimized queries to search the database through an internal database client, retrieving relevant articles, papers, videos, online courses, etc.

[0538] Step 8:

[0539] The server evaluates the search results it retrieves based on their reliability, freshness, and specificity, assigning a score to each result and comparing them based on the evaluation results.

[0540] Step 9:

[0541] The server ranks the search results based on criteria, sorting them so that the most reliable and up-to-date information is placed at the top.

[0542] Step 10:

[0543] The server encodes the ranked search results in JSON format and sends them to the device.

[0544] Step 11:

[0545] The device receives the search results and displays them in a user interface, laid out visually and clearly so that the user can easily check each result.

[0546] Step 12:

[0547] Users can review the search results and select articles or papers that interest them to read more, for example, by clicking on a highly rated paper to view its contents.

[0548] Example 1

[0549] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0550] Conventional information search systems often lack the accuracy and relevance of results based on user-entered search queries, making it difficult for users to quickly and reliably access the information they are looking for. Providing appropriate search results is particularly challenging in today's world, where vast amounts of information are readily available. Furthermore, technologies that optimize search results by utilizing users' past search history and behavioral data are often not fully utilized. This limits the user experience, creating problems.

[0551] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0552] In this invention, the server includes means for receiving a search query input from a user terminal, means for converting the received search query into a structured data format, means for transmitting the converted search query, means for performing natural language analysis based on the received search query, means for optimizing search conditions based on the analysis results, means for searching for related data from a data management system using the optimized search conditions, means for evaluating and ranking search results, means for transmitting the ranked search results to the user terminal, and means for displaying the search results received on the user terminal. This allows the user to quickly obtain optimal information based on the search query and to obtain highly accurate search results based on past search history and behavioral data.

[0553] A "user terminal" is an appliance or device used by a user to input information and display results.

[0554] A "search query" is a word or string of characters that a user enters to search for specific information.

[0555] The "means for receiving a search query" is a device or software module that receives a search query sent from a user terminal.

[0556] A "means for converting into a structured data format" is a device or software module that converts an input search query into a certain format (e.g., JSON format).

[0557] A "means for transmitting a converted search query" is a device or software module that transmits a search query that has been converted into a structured data format over a network.

[0558] "Natural language analysis" is a technology that analyzes natural language, which is human language, and extracts semantic information.

[0559] A "means for performing natural language analysis" is a device or software module that performs natural language analysis on a received search query.

[0560] A "means for optimizing search conditions" is a device or software module that uses the results of natural language analysis and the user's past search history to adjust search conditions in the most effective way.

[0561] A "data management system" is a system for storing, managing, and retrieving information in an organized manner.

[0562] A "means for retrieving relevant data" is a device or software module that retrieves relevant information from a data management system using optimized search criteria.

[0563] A "means for evaluating and ranking search results" is a device or software module that evaluates and prioritizes results based on criteria such as the authoritativeness, freshness, and specificity of the search results.

[0564] The "means for transmitting to a user terminal" is a device or software module that transmits the evaluated and ranked search results to a user terminal.

[0565] The "means for displaying the received search results on the user terminal" is a device or software module that visually displays the received search results on the user terminal.

[0566] The system according to the present invention allows users to search for information efficiently and accurately using a terminal. The basic configuration of this system is made up of a user terminal, a server, and a data management system.

[0567] First, a user accesses a "search interface" through a dedicated application or a web browser, and then enters a specific query, such as "latest natural language processing technology," into the search box on the interface.

[0568] When a user enters a query and presses the search button, the device uses JavaScript to convert the query into JSON format. For example, it is converted into {"query": "latest natural language processing technology"}. The device then sends this JSON data to the server as an HTTP POST request. The HTTPS protocol is used for transmission, and the destination URL is, for example, https: / / example.com / search.

[0569] The server receives an HTTP POST request via a web server such as Apache or Nginx. After receiving the request, the server processes the request using a backend framework such as Python's Flask, Django, or Node.js. The server then analyzes the query using a natural language analysis model (e.g., BERT or GPT-3) to extract relevant keywords and phrases. For example, the keywords "latest," "natural language processing," and "technology" are extracted from the query "latest natural language processing technology."

[0570] The server then retrieves the user's past search history from the database and uses this information to optimize the search query: if the user searches a lot for "natural language processing," it will weight this keyword more highly to improve the accuracy of the search results.

[0571] The server uses an internal database client (e.g., PostgreSQL or MongoDB) to search for information in the data management system based on optimized search criteria, retrieving relevant data such as papers, articles, online courses, etc.

[0572] The retrieved search results are compared and ranked based on criteria such as reliability, freshness, and specificity. The evaluated results are then compiled into JSON format and sent to the device. For example, the format might look like this: {"results": [{"title": "The latest natural language processing technology", "url": "https: / / example.com / article"}]}. The device receives this JSON data and visually displays the results on the user interface.

[0573] Users can view the displayed search results and click on the information they are interested in to view more details. This system allows users to quickly access the information they are looking for and provides highly accurate search results based on their past search history and behavioral data.

[0574] Specific examples

[0575] For example, if a user enters the search query "latest AI technology":

[0576] The user inputs the latest AI technology through the interface, and the device sends it to the server.

[0577] The server analyzes the query and extracts keywords such as "latest," "AI," and "technology."

[0578] The server uses the user's past behavioral data to optimize queries and prioritize searches for the latest information related to AI technology.

[0579] It retrieves relevant information from internal databases and external APIs and ranks it based on criteria such as reliability, freshness, and detail.

[0580] The ranked results are sent to the user's terminal and displayed.

[0581] Users can review search results and select specific articles or papers to view more details.

[0582] Prompt Sentence Examples

[0583] For example, the prompt text to input to a generative AI model might look like this:

[0584] "I want to search for information about the latest natural language processing technology. Related keywords are 'latest,' 'natural language processing,' and 'technology,' and this user has searched a lot for information about natural language processing in the past."

[0585] The above is a specific embodiment for carrying out the present invention.

[0586] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0587] Step 1:

[0588] A user accesses a search interface on a terminal and inputs a search query such as "latest natural language processing technology." The input query is stored in text format in an input field on the terminal.

[0589] Step 2:

[0590] When the user presses the search button, the device uses JavaScript to convert the search query into JSON format, for example, {"query": "latest natural language processing technology"}. This JSON data is then prepared as an HTTP POST request.

[0591] Step 3:

[0592] The terminal uses the HTTPS protocol to send this JSON-formatted query to the server. The destination URL is, for example, https: / / example.com / search. The sent JSON data arrives at the server.

[0593] Step 4:

[0594] The server receives this HTTP POST request via a web server such as Apache or Nginx and extracts the search query in JSON format from the body of the request. For example, the data extracted is {"query": "The latest natural language processing technology"}.

[0595] Step 5:

[0596] The server uses a parsing module to input the search query into a natural language parsing model (e.g., BERT or GPT-3). The model analyzes the query and extracts relevant keywords and phrases. For example, the query "latest natural language processing technology" yields the keywords "latest," "natural language processing," and "technology."

[0597] Step 6:

[0598] The server retrieves the user's past search history and behavioral data from the database. Based on this, it optimizes the weighting of queries. For example, if there have been many searches related to "natural language processing" in the past, it will increase the weighting of this keyword.

[0599] Step 7:

[0600] The server issues SQL or NoSQL queries to data management systems (e.g., PostgreSQL or MongoDB) based on the optimized search criteria, searching relevant databases for information such as related papers, articles, or online courses.

[0601] Step 8:

[0602] The server retrieves the search results and rates each item based on the criteria of authoritativeness, freshness, and specificity, using a numerical score or other criteria, and assigns each result a score for authoritativeness, freshness, and specificity.

[0603] Step 9:

[0604] The server ranks the search results based on their scores, placing highly rated information at the top of the list. The server then compiles the ranked search results in JSON format, structured as follows: {"results": [{"title": "The latest natural language processing technology", "url": "https: / / example.com / article"}]}.

[0605] Step 10:

[0606] The server sends the ranked search results as JSON data to the user's device, which receives the JSON data, parses it, and displays it on the user interface.

[0607] Step 11:

[0608] Users can view detailed information by checking the search results displayed on their device and clicking on the links to articles or papers they are interested in. For example, they can click on the title of a particular paper to read more about it.

[0609] The above are the specific processing steps of the program of this system.

[0610] (Application example 1)

[0611] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0612] In physical stores, users face the challenge of finding the products they want efficiently and quickly. Furthermore, there is a lack of systems that can provide detailed product information, reviews, price comparisons, and other information in real time to help users make appropriate purchasing decisions. These situations result in a poor user experience.

[0613] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0614] In this invention, the server includes a means for the user terminal to provide search results including location information and reviews via a display device, a means for considering reliability, freshness of information, and level of detail, and a means for using the user's past search history and behavioral data when optimizing search conditions, thereby enabling users to quickly find desired products in physical stores, obtain detailed information, and make appropriate purchasing decisions.

[0615] "User device" is a general term for devices that users use to enter queries and view search results, such as smartphones and smart glasses.

[0616] A "search query" is a string of characters or voice command that a user types to obtain information.

[0617] "Natural language analysis" is the process of converting search queries into a format that is easy for machines to understand.

[0618] "Search optimization" refers to transforming search queries into more effective forms based on user requirements and past behavioral data.

[0619] A "database" refers to a system or structure that stores retrieved information and makes it available for retrieval as needed.

[0620] "Ranking" refers to evaluating and ranking the search results obtained based on certain criteria.

[0621] A "display device" is a device that visually presents search results to users, such as a smartphone screen or a smart glasses display.

[0622] "Location information" refers to information about the location where a particular product is located.

[0623] A "review" is an evaluation or impression posted by a user who has previously purchased the product.

[0624] "Credibility" is an indicator of how trustworthy the search results are.

[0625] "Information freshness" is an indicator of how up-to-date the search results are.

[0626] "Detail" is an indicator of how detailed the search results provide.

[0627] MODE FOR CARRYING OUT THE INVENTION

[0628] As an embodiment of the present invention, a smart shopping assistant system for improving user experience in a physical store will be described as a specific example. This system is composed of a user terminal, a server, and a database.

[0629] Program Overview

[0630] 1. User Device

[0631] Display devices such as smartphones and smart glasses are used as user terminals. These terminals accept voice and text inputs and retrieve user search queries.

[0632] Example: A user speaks into their smartphone, "What's the latest chocolate product?"

[0633] 2. Submitting a query

[0634] The user terminal converts the entered search query into JSON format and sends it to the server.

[0635] 3. Server

[0636] The server receives a search query sent from a user terminal.

[0637] Incoming search queries are parsed by a natural language parsing model (e.g., BERT), which breaks the query down into keywords and converts them into an understandable format.

[0638] Based on the analysis results, the search conditions are optimized by referring to the user's past search history and behavioral data.

[0639] 4. Database

[0640] The server uses optimized search criteria to search internal databases and external APIs, which store product information, reviews, pricing, and more.

[0641] 5. Rating and Ranking of Search Results

[0642] The server evaluates and ranks the search results based on reliability, freshness of information, and level of detail.

[0643] 6. Displaying the results

[0644] The ranked search results are sent to the user's terminal in JSON format and presented to the user via a display device, including product location information and reviews.

[0645] Specific examples

[0646] For example, if a user enters a query such as "What are the latest chocolate products?", the following process will occur:

[0647] The query is sent by the terminal to the server.

[0648] The server analyzes the received query and extracts keywords such as "latest," "chocolate," and "product."

[0649] Search criteria are optimized based on the user's past search history.

[0650] Relevant product information is retrieved from an internal database and external APIs, and then evaluated and ranked based on criteria such as reliability and freshness.

[0651] The ranked results are sent to the user's device and displayed along with product location information and reviews.

[0652] Prompt Sentence Examples

[0653] Voice input: "What's the latest chocolate product?"

[0654] Text input: "Tell me about new chocolate products"

[0655] As described above, this invention enables users to quickly find the desired product in a physical store, obtain detailed information, and make appropriate purchasing decisions, thereby significantly improving the user's purchasing experience.

[0656] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0657] Step 1:

[0658] User enters a search query

[0659] How it works: A user uses a smartphone or smart glasses to enter a search query using voice or text input.

[0660] Input: "What's the latest chocolate product?" (voice or text)

[0661] Output: Query data entered on the user's terminal

[0662] Step 2:

[0663] Sending a query from the device to the server

[0664] How it works: The user device converts the entered search query into JSON format and sends it to the server.

[0665] Input: Query data entered on the user's device

[0666] Output: Query data converted to JSON format

[0667] Step 3:

[0668] Server receives and analyzes the query

[0669] How it works: The server receives the query data in JSON format and passes the query to a natural language parsing model (e.g., BERT). The model parses the query and breaks it down into keywords.

[0670] Input: Query data in JSON format

[0671] Output: Extracted keywords (e.g. "latest", "chocolate", "product")

[0672] Step 4:

[0673] Optimizing search criteria

[0674] How it works: The server references the user's past search history and behavioral data to optimize search criteria based on the search query.

[0675] Input: Keywords and user history data

[0676] Output: Optimized search criteria

[0677] Step 5:

[0678] Database search

[0679] How it works: The server uses optimized search criteria to search for product information, reviews, pricing information, etc. from internal databases and external APIs.

[0680] Input: Optimized search criteria

[0681] Output: Search result data

[0682] Step 6:

[0683] Rating and ranking of search results

[0684] How it works: The server evaluates and ranks the search results it retrieves based on their authoritativeness, freshness, and specificity.

[0685] Input: Search result data

[0686] Output: Ranked search results

[0687] Step 7:

[0688] Sending results to the device

[0689] How it works: The server sends the ranked search results in JSON format to the user's device.

[0690] Input: Ranked search results

[0691] Output: Search result data in JSON format

[0692] Step 8:

[0693] Displaying search results

[0694] How it works: The user's device parses the JSON-formatted search results data received from the server and displays them on the display device (smartphone screen or smart glasses display) along with location information and reviews.

[0695] Input: Search result data in JSON format

[0696] Output: Search results displayed on a display device (e.g. product name, location, reviews)

[0697] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0698] The system according to the present invention allows users to search for information efficiently and accurately using a terminal, and further improves the search experience by recognizing the user's emotions during the search process. The basic configuration of this system is made up of a user terminal, a server, a database client, and an emotion engine.

[0699] System Program Overview

[0700] Program processing

[0701] 1. User enters a query

[0702] A user enters a search query in the search interface, for example, entering specific information such as "latest natural language processing technology."

[0703] 2. User Emotion Recognition

[0704] The device activates an emotion engine when the user inputs something, recognizing the user's emotions in real time. The emotion engine uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice, recognizing emotions such as "excited," "calm," or "irritated."

[0705] 3. Sending a request from the device to the server

[0706] The device converts the search query and recognized emotion data into JSON format and sends it to the server as a POST request.

[0707] 4. Server Receipt and Analysis of Request

[0708] The server receives the request and analyzes the query using a natural language analysis model. The query "latest natural language processing technology" is broken down into keywords such as "latest," "natural language processing," and "technology," and the meaning and relevance of each keyword is analyzed.

[0709] 5. Query Optimization

[0710] The server references the user's past search history and behavioral data, and also optimizes search conditions based on recognized emotional data. For example, if the user is in an "excited" state, the server will prioritize searches for new technologies and innovative content.

[0711] 6. Database Search

[0712] The server searches the database through an internal database client using optimized search criteria to retrieve relevant articles, papers, videos, online courses, etc.

[0713] 7. Compare and evaluate search results

[0714] The server evaluates the search results it retrieves based on their reliability, freshness, and specificity, assigning a score to each result and comparing them based on the evaluation results.

[0715] 8. Ranking of search results

[0716] The server ranks the search results based on criteria, with the most reliable and up-to-date information ranked at the top.

[0717] 9. Sending results to the device

[0718] The server encodes the ranked search results in JSON format and sends them to the device.

[0719] 10. User confirmation of results and action

[0720] The terminal receives the search results and displays them on the user interface. The user can check the displayed search results and view the details of each result.

[0721] Specific examples

[0722] For example, if a user enters the search query "latest AI technology" and the emotion engine recognizes that the user is "excited," the flow is as follows:

[0723] The user enters a search query through the interface, and the device transmits the query and emotion data to the server.

[0724] The server analyzes the query and extracts keywords such as "latest," "AI," and "technology."

[0725] The server optimizes queries to prioritize new technologies and innovative content because users are "excited."

[0726] It retrieves relevant information from internal databases and external APIs, and ranks the retrieved data based on criteria such as reliability, freshness, and level of detail.

[0727] The ranked results are sent to the user's terminal and displayed.

[0728] Users review the search results and select the most interesting articles or papers to read more.

[0729] In this way, a more personalized search experience can be provided by taking the user's emotions into account.

[0730] The processing flow will be explained below.

[0731] Step 1:

[0732] A user accesses the "Search-ru, Kuberu" interface and enters a search query. Specifically, the user enters a query such as "latest natural language processing technology" into the search box using a browser or a dedicated application, and clicks the submit button.

[0733] Step 2:

[0734] The device receives user input and activates the emotion engine, which uses the device's camera and microphone to analyze the user's facial expressions and tone of voice in real time.

[0735] Step 3:

[0736] The device analyzes the emotional data (e.g., "excited," "calm," "irritated," etc.) and converts it into JSON format along with the search query.

[0737] Step 4:

[0738] The device sends the converted JSON-formatted data to the server as a POST request, which includes the entered search query and the recognized emotion data.

[0739] Step 5:

[0740] The server receives the POST request and extracts the search query and sentiment data from the request body.

[0741] Step 6:

[0742] The server uses a natural language analysis model to analyze the search query. It breaks down the query "latest natural language processing technology" into keywords such as "latest," "natural language processing," and "technology," and analyzes the meaning and relevance of each keyword.

[0743] Step 7:

[0744] The server optimizes search criteria based on the user's past search history, behavioral data, and emotional data. For example, if a user has frequently searched for information about "natural language processing" in the past and is currently in an "excited" state, the server will adjust search criteria to prioritize information about more innovative and new technologies.

[0745] Step 8:

[0746] The server searches the database through an internal database client using optimized search criteria to retrieve relevant articles, papers, videos, online courses, etc.

[0747] Step 9:

[0748] The server evaluates the search results it retrieves based on their reliability, freshness, and specificity, assigning a score to each result and comparing them based on the evaluation results.

[0749] Step 10:

[0750] The server ranks the search results based on criteria, with the most reliable and up-to-date information ranked at the top.

[0751] Step 11:

[0752] The server encodes the ranked search results in JSON format and sends them to the device.

[0753] Step 12:

[0754] The device receives the search results and displays them on the user interface. The display method is laid out in a visually easy-to-understand manner, taking into consideration the user's convenience.

[0755] Step 13:

[0756] Users can review the search results and select the articles or papers that interest them most for more detailed information. For example, they can click on a highly rated paper to view its contents in more detail.

[0757] Example 2

[0758] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0759] Conventional search systems are required to provide highly relevant information in response to user search queries, but it has been difficult to provide search results that take into account the user's emotions and search context. As a result, users are burdened with selecting the most relevant information from a large amount of information, resulting in a poor search experience. The present invention aims to solve this problem and provide more appropriate and personalized search results that take into account the user's emotions.

[0760] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving a search query input from a user terminal, a means for performing natural language analysis based on the received search query, and a means for optimizing search conditions based on the analysis results. This makes it possible to recognize the user's emotions in real time and optimize the search conditions based on the emotion data. This realizes a personalized search experience that reflects the user's emotions, reduces the user's search burden, and enables more appropriate information to be provided more quickly.

[0761] A "user terminal" is a communication device operated by a user, and is a device for inputting search queries, recognizing emotion data, and displaying search results.

[0762] A "search query" is a query that a user inputs in the form of a string of characters or voice to express the information they want to know or search for.

[0763] "Natural language analysis" is a technology that allows a computer to understand and analyze the meaning of text written in human language.

[0764] "Search condition optimization" is a technique that adjusts conditions to efficiently and effectively search for related information based on a received search query.

[0765] A "database" is a system for storing structured data and searching and retrieving it as needed.

[0766] "Relevant data" is information extracted from a database based on a search query, and is useful data that matches the search intent.

[0767] "Ranking of search results" refers to the process of ranking search results based on evaluation criteria and determining the priority of the results to be presented to the user.

[0768] "Real-time emotion recognition" is a technology that instantly recognizes a user's current emotions from facial expressions, tone of voice, etc.

[0769] "Emotion data" is data that indicates the emotional state of the user recognized in real time.

[0770] "User interface" is a general term for the screens and input devices that allow a user to interact with a system.

[0771] MODE FOR CARRYING OUT THE INVENTION

[0772] A specific embodiment of the system according to the present invention will be described. This system allows users to search for information efficiently and accurately using a terminal, and further improves the search experience by recognizing the user's emotions during the search process. The basic configuration of this system is made up of a user terminal, a server, a database client, and an emotion engine.

[0773] Hardware and software used

[0774] User devices: PCs, smartphones, tablets, etc.

[0775] Server: Web server (Apache HTTP Server, Nginx, etc.), application server

[0776] Database client: Database management system such as MySQL, PostgreSQL, etc.

[0777] Emotion engine: Camera, microphone, emotion recognition software (OpenCV, Microsoft Azure Cognitive Services, etc.)

[0778] Generative AI models: GPT-3, BERT, etc. for natural language analysis

[0779] System basic configuration and operation

[0780] 1. Enter a query

[0781] A user enters a search query through an interface installed on the device (such as a browser-based web application or a mobile application), for example, entering a specific information request such as "latest natural language processing technology."

[0782] 2. Emotion recognition

[0783] When a user inputs a query, the device activates an emotion engine. The emotion engine uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice, allowing it to recognize the user's emotions, such as "excited," "calm," or "irritated," in real time.

[0784] 3. Sending a request

[0785] The device converts the search query and the recognized emotion data into JSON format and sends it to the server as an HTTP POST request.

[0786] 4. Receiving and analyzing requests

[0787] The server receives the JSON-formatted request from the device. It then uses a generative AI model for natural language analysis (e.g., GPT-3 or BERT) to analyze the query, breaking down the query "latest natural language processing technology" into keywords such as "latest," "natural language processing," and "technology." It then analyzes the meaning and relevance of each keyword.

[0788] 5. Query Optimization

[0789] The server uses the user's past search history and behavioral data, as well as emotional data, to optimize search conditions. For example, if the server detects that the user is "excited," it will prioritize new technologies and innovative content.

[0790] 6. Database Search

[0791] The server searches the database through the database client using optimized search criteria, such as executing SQL queries to retrieve relevant articles, papers, videos, online courses, etc.

[0792] 7. Display of search results

[0793] The server evaluates the search results based on their reliability, freshness, and detail, assigns a score, compares them, and then ranks them based on the evaluation criteria. The ranked results are encoded in JSON format and sent to the device.

[0794] 8. User confirmation of results

[0795] The terminal receives the search results sent from the server and displays them on the user interface. The user can then review the displayed search results, select the most interesting article or paper, and view its details.

[0796] Specific examples

[0797] For example, if a user enters the search query "latest AI technology" and the emotion engine recognizes the user's emotion as "excited," the operation flow is as follows:

[0798] 1. User: Enters the query "latest AI technology."

[0799] 2. Device: The emotion engine recognizes that the user is excited based on their facial expressions and voice.

[0800] 3. Terminal: Convert the search query and sentiment data into JSON format and send it to the server as an HTTP POST request.

[0801] 4. Server: Receives the request and breaks down "latest AI technology" into the keywords "latest," "AI," and "technology."

[0802] 5. Server: Optimize queries to prioritize innovative content based on what users are "excited about."

[0803] 6. Server: Searches the database using optimized search criteria to retrieve relevant information.

[0804] 7. Server: Evaluates and ranks the acquired information.

[0805] 8. Server: Encode the ranked search results in JSON format and send them to the device.

[0806] 9. Device: Search results are displayed and the user checks the details.

[0807] Examples of usage prompt statements

[0808] Search for "latest AI technology" and describe a system that provides the most appropriate information to a user when the user is emotionally "excited."

[0809] As described above, this system takes user emotions into consideration to provide a more appropriate and personalized search experience.

[0810] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0811] Step 1:

[0812] Query Input

[0813] A user inputs a search query through the device interface. A specific example of what the user inputs is "the latest natural language processing technology." The input data is in text format.

[0814] Specific behavior:

[0815] The user enters a search query and clicks the submit button, and the device stores the entered text query in its internal memory.

[0816] Step 2:

[0817] emotion recognition

[0818] The device activates an emotion engine to recognize the user's emotions in real time. The device uses a camera and microphone to analyze the user's facial expressions and tone of voice to determine emotions.

[0819] input:

[0820] User's facial expression and voice data

[0821] Specific behavior:

[0822] The device inputs data acquired from the camera and microphone into emotion recognition software to determine the user's current emotion (e.g., "excited" or "calm"), and the analysis results are stored in internal memory.

[0823] output:

[0824] User emotion data (e.g., excited)

[0825] Step 3:

[0826] Send request

[0827] The device converts the search query and recognized emotion data into JSON format and sends it to the server as an HTTP POST request.

[0828] input:

[0829] Search query and user sentiment data

[0830] Specific behavior:

[0831] The device converts the search query and sentiment data into JSON format as key-value pairs, then generates an HTTP request and sends it to the specified URL on the server.

[0832] output:

[0833] JSON formatted HTTP POST request

[0834] Step 4:

[0835] Receiving and parsing requests

[0836] The server receives a JSON-formatted request from the device and analyzes the query using a generative AI model. For example, the query "latest natural language processing technology" is broken down into keywords such as "latest," "natural language processing," and "technology."

[0837] input:

[0838] Request data in JSON format

[0839] Specific behavior:

[0840] The server receives the request and feeds the query to the generative AI model, which analyzes the query and extracts important keywords, which are then stored in an internal database.

[0841] output:

[0842] Analyzed keyword data (e.g., "latest," "natural language processing," "technology")

[0843] Step 5:

[0844] Query optimization

[0845] The server optimizes the search criteria based on the analyzed keyword data, as well as the user's emotional data and past search history. For example, if the user is "excited," it will prioritize new technologies and innovative content.

[0846] input:

[0847] Analyzed keyword data, user sentiment data, and past search history

[0848] Specific behavior:

[0849] The server uses this data to run an optimization algorithm to determine the highest priority search terms, which results in an optimized search term.

[0850] output:

[0851] Optimized search criteria

[0852] Step 6:

[0853] Database search

[0854] The server searches the database using optimized search criteria and retrieves relevant information internally or through an external API.

[0855] input:

[0856] Optimized search criteria

[0857] Specific behavior:

[0858] The server issues SQL queries to the database to retrieve data that matches the search criteria, as well as external API calls.

[0859] output:

[0860] Retrieved search result data

[0861] Step 7:

[0862] Rating and ranking of search results

[0863] The search results obtained by the server are evaluated based on reliability, freshness, and detail, and are assigned a score to rank them.

[0864] input:

[0865] Retrieved search result data

[0866] Specific behavior:

[0867] The server evaluates each search result based on the evaluation criteria, assigns a score, and ranks the search results based on the score.

[0868] output:

[0869] Ranked search results data

[0870] Step 8:

[0871] Send results

[0872] The server encodes the ranked search results in JSON format and sends them to the device.

[0873] input:

[0874] Ranked search results data

[0875] Specific behavior:

[0876] The server encodes the search results in JSON format and sends them to the terminal as an HTTP response.

[0877] output:

[0878] JSON formatted HTTP response

[0879] Step 9:

[0880] Check the results

[0881] The terminal receives the search results sent from the server and displays them on the user interface, where the user can check the displayed search results and view their details.

[0882] input:

[0883] Search result data in JSON format

[0884] Specific behavior:

[0885] The device receives the search results, parses them, and displays them in a user interface that the user can navigate and click on results of interest to learn more.

[0886] output:

[0887] Search results displayed to users

[0888] (Application example 2)

[0889] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0890] Conventional food delivery services lack personalized meal suggestions because they do not provide a search experience that takes into account users' emotions and individual preferences. Another issue is the lack of functionality to analyze users' emotional data and optimize search criteria based on that data in order to provide optimal search results for users' search queries.

[0891] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recognizing a user's emotion, means for optimizing search conditions based on the recognized emotion, and means for searching for related data from a database using the optimized search conditions. This enables a personalized search experience based on the user's emotion, and can provide more optimal search results.

[0892] A "user terminal" is a hardware device operated by a user, such as a smartphone or tablet.

[0893] A "search query" refers to the text or voice input a user enters to search for specific information.

[0894] "Means for recognizing emotions" refers to technology that analyzes a user's words and actions, facial expressions, tone of voice, etc. to determine their emotions at that time.

[0895] "Natural language analysis" is a technology for analyzing input text or speech and understanding its meaning.

[0896] "Means for optimizing search conditions" refers to technology that performs processing to make search results more accurate and useful based on the user's input query and emotional data.

[0897] A "database" is an organized collection of data that stores information responsive to a search query.

[0898] The "means for searching related data" is a technique for extracting related information in a database using optimized search criteria.

[0899] "Means for evaluating and ranking search results" refers to technology that evaluates the search results obtained based on criteria such as reliability and freshness, and ranks them in order of importance.

[0900] "Means for sending to user terminal" refers to the technology for transferring the evaluated and ranked search results to the user terminal.

[0901] "Past search history" is a record of search queries a user has previously performed.

[0902] "Behavioral data" refers to data relating to the behavior of a user, such as what operations the user performed and which pages the user viewed.

[0903] The system according to the present invention comprises a user terminal, a server, a database, and an emotion engine. The system program performs processing in the following steps: When a user inputs a search query using a food delivery app, the emotion recognition means analyzes the user's emotion in real time, and the query is sent to the server together with that data.

[0904] The server receives the user's search query and emotion data, and optimizes the search query based on the recognized emotion. The search criteria are further refined by referring to the user's past search history and behavioral data. Based on the optimized search criteria, relevant information is searched for from the database. The obtained search results are evaluated based on criteria such as reliability, freshness, and level of detail, and a ranking is assigned. The ranked search results are sent from the server to the user's device and displayed on the user interface.

[0905] Examples of hardware and software used include smartphones and tablets as user devices, emotion engines for recognizing emotions, natural language processing models for data analysis, and servers for searching and evaluating databases.

[0906] As a concrete example, consider the case where a user searches for "Italian restaurant." If the user's device uses an emotion engine to recognize the emotion "excited," the server uses that emotion data to optimize the search criteria to prioritize new restaurants and special menus. The server then searches the database based on the optimized search criteria to retrieve information about new restaurants and special menus. The information is then ranked based on criteria such as reliability and freshness and presented to the user.

[0907] An example prompt might be:

[0908] "When a user enters a search query in a food delivery app, analyze the data and recognize the user's emotions in real time. Design a system that optimizes the query based on the user's emotions and provides search results for relevant restaurants and menus. For example, when a user is excited, suggest new restaurants or special menus, and when a user is stressed, prioritize past favorite restaurants or menus that have given them high satisfaction."

[0909] This provides a personalized search experience that takes user emotions into account, which is expected to improve satisfaction with food delivery services.

[0910] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0911] Step 1:

[0912] A user opens a food delivery app and enters a search query. Input: The user's query (e.g., "Italian restaurant") and the user's facial image and voice data. Output: The device collects the query and emotion data. Specific operation: The user enters the query into an input box on the device interface, and the device's camera and microphone are used to collect the user's facial expressions and tone of voice in real time.

[0913] Step 2:

[0914] The device activates the emotion engine and recognizes the user's emotions in real time. Input: User's facial image and voice data. Output: Recognized emotion data (e.g., "Excited"). Specific operation: The emotion engine uses face recognition algorithms and voice analysis algorithms to analyze the user's facial muscle movements and voice tone.

[0915] Step 3:

[0916] The device sends the search query and recognized emotion data to the server. Input: Query and emotion data. Output: Data sent to the server. Specific operation: The device encodes the query and emotion data into JSON format and sends it to the server using an HTTP POST request.

[0917] Step 4:

[0918] The server analyzes the received query and sentiment data and performs natural language analysis on the search query. Input: Query and sentiment data. Output: Keywords from the analyzed query (e.g., "Italian," "restaurant"). What happens: A natural language processing (NLP) model on the server tokenizes the query and extracts key keywords.

[0919] Step 5:

[0920] The server references the user's past behavioral data and search history along with the recognized emotions to optimize the search criteria. Input: Analyzed keywords, emotion data, past behavioral data, and search history. Output: Optimized search criteria. Specific operation: The server retrieves the user's past history from the database and adjusts the search algorithm along with the emotion data.

[0921] Step 6:

[0922] The server searches the database for relevant information based on the search criteria. Input: Optimized search criteria. Output: Relevant information (e.g., a list of new Italian restaurants). Specific behavior: The server executes a database query to extract relevant restaurants and menus.

[0923] Step 7:

[0924] The server evaluates and ranks the search results. Input: Retrieved search results. Output: Rated and ranked results. Specific behavior: The server assigns a score to each result using evaluation criteria such as reliability, freshness, and specificity, and then ranks them.

[0925] Step 8:

[0926] The server sends the ranked search results to the user's terminal. Input: Ranked search results. Output: Search results sent to the user's terminal. Specific operation: The server encodes the ranked results in JSON format and sends them to the user's terminal as an HTTP response.

[0927] Step 9:

[0928] The user terminal receives the search results and displays them on the user interface. Input: Ranked search results. Output: Displayed search results. Specific operation: The application on the terminal analyzes the search results and displays them to the user.

[0929] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0930] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0931] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0932] [Third embodiment]

[0933] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0934] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0935] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0936] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0937] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0938] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0939] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0940] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0941] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0942] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0943] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0944] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0945] The system according to the present invention implements a series of processes that allow users to search for information efficiently and accurately using a terminal. The basic configuration of this system is made up of a user terminal, a server, and a database client.

[0946] System Program Overview

[0947] Program processing

[0948] 1. User enters a query

[0949] Users access a terminal with an interface called "Search-ru, Kura-beru" and enter specific information such as a search query, "latest natural language processing technology."

[0950] 2. Sending a request from the device to the server

[0951] The device converts this search query into JSON format and sends it to the server as a POST request.

[0952] 3. Server receives and analyzes the request

[0953] The server receives the request and passes the query to a natural language analysis model. This model analyzes the query and extracts relevant keywords and phrases. For example, a query like "latest natural language processing technology" would be broken down into three keywords: "latest," "natural language processing," and "technology."

[0954] 4. Query Optimization

[0955] The server optimizes the search query by referencing the user's past search history and behavioral data. For example, if the user has searched a lot for information about "natural language processing" in the past, it will give more weight to this keyword and adjust the search criteria.

[0956] 5. Database Search

[0957] The server uses optimized search criteria to search the database via an internal database client, where relevant articles, papers, online courses, and other information are retrieved.

[0958] 6. Compare and evaluate search results

[0959] The server evaluates the search results it receives, rating each based on reliability, freshness of information, and specificity.

[0960] 7. Ranking of search results

[0961] The server ranks the search results based on the criteria: for example, if the most recent paper is rated as highly reliable, it will be ranked highly.

[0962] 8. Sending results to the device

[0963] The server sends the ranked search results in JSON format to the device, which receives this information and displays the results in the user's interface.

[0964] 9. User confirmation of results and action

[0965] Users can check the search results displayed on their devices and view detailed information as needed. For example, they can select a specific paper or article to view its details.

[0966] Specific examples

[0967] For example, if a user enters the search query "latest AI technology," the flow is as follows:

[0968] The user enters a search query through the interface, and the terminal sends it to the server.

[0969] The server analyzes the query and extracts keywords such as "latest," "AI," and "technology."

[0970] The server uses the user's past behavioral data to optimize queries and prioritize searches for the latest information related to AI technology.

[0971] It retrieves relevant information from internal databases and external APIs, and ranks the retrieved data based on criteria such as reliability, freshness, and level of detail.

[0972] The ranked results are sent to the user's terminal and displayed.

[0973] Users can review search results and select specific articles or papers to view more details.

[0974] This system allows users to quickly access the information they need and obtain higher quality information.

[0975] The processing flow will be explained below.

[0976] Step 1:

[0977] A user accesses the Search-Ru interface and enters a search query, such as "latest natural language processing technology," in the search box of a browser or dedicated application, and clicks the submit button.

[0978] Step 2:

[0979] The device converts the user's input into JSON format, which may package the search query along with the user's past search history and preferences.

[0980] Step 3:

[0981] The device sends the converted JSON-formatted data to the server as a POST request, which includes the entered search query and related metadata.

[0982] Step 4:

[0983] The server receives the POST request and extracts the search query and related data from the request body.

[0984] Step 5:

[0985] The server uses a natural language analysis model to analyze the search query. It breaks down the query "latest natural language processing technology" into keywords such as "latest," "natural language processing," and "technology," and analyzes the meaning and relevance of each keyword.

[0986] Step 6:

[0987] The server optimizes search criteria based on the user's past search history and behavioral data. For example, if a user has frequently searched for information about "natural language processing" in the past, it will assign a weight to this keyword.

[0988] Step 7:

[0989] The server uses optimized queries to search the database through an internal database client, retrieving relevant articles, papers, videos, online courses, etc.

[0990] Step 8:

[0991] The server evaluates the search results it retrieves based on their reliability, freshness, and specificity, assigning a score to each result and comparing them based on the evaluation results.

[0992] Step 9:

[0993] The server ranks the search results based on criteria, sorting them so that the most reliable and up-to-date information is placed at the top.

[0994] Step 10:

[0995] The server encodes the ranked search results in JSON format and sends them to the device.

[0996] Step 11:

[0997] The device receives the search results and displays them in a user interface, laid out visually and clearly so that the user can easily check each result.

[0998] Step 12:

[0999] Users can review the search results and select articles or papers that interest them to read more, for example, by clicking on a highly rated paper to view its contents.

[1000] Example 1

[1001] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1002] Conventional information search systems often lack the accuracy and relevance of results based on user-entered search queries, making it difficult for users to quickly and reliably access the information they are looking for. Providing appropriate search results is particularly challenging in today's world, where vast amounts of information are readily available. Furthermore, technologies that optimize search results by utilizing users' past search history and behavioral data are often not fully utilized. This limits the user experience, creating problems.

[1003] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1004] In this invention, the server includes means for receiving a search query input from a user terminal, means for converting the received search query into a structured data format, means for transmitting the converted search query, means for performing natural language analysis based on the received search query, means for optimizing search conditions based on the analysis results, means for searching for related data from a data management system using the optimized search conditions, means for evaluating and ranking search results, means for transmitting the ranked search results to the user terminal, and means for displaying the search results received on the user terminal. This allows the user to quickly obtain optimal information based on the search query and to obtain highly accurate search results based on past search history and behavioral data.

[1005] A "user terminal" is an appliance or device used by a user to input information and display results.

[1006] A "search query" is a word or string of characters that a user enters to search for specific information.

[1007] The "means for receiving a search query" is a device or software module that receives a search query sent from a user terminal.

[1008] A "means for converting into a structured data format" is a device or software module that converts an input search query into a certain format (e.g., JSON format).

[1009] A "means for transmitting a converted search query" is a device or software module that transmits a search query that has been converted into a structured data format over a network.

[1010] "Natural language analysis" is a technology that analyzes natural language, which is human language, and extracts semantic information.

[1011] A "means for performing natural language analysis" is a device or software module that performs natural language analysis on a received search query.

[1012] A "means for optimizing search conditions" is a device or software module that uses the results of natural language analysis and the user's past search history to adjust search conditions in the most effective way.

[1013] A "data management system" is a system for storing, managing, and retrieving information in an organized manner.

[1014] A "means for retrieving relevant data" is a device or software module that retrieves relevant information from a data management system using optimized search criteria.

[1015] A "means for evaluating and ranking search results" is a device or software module that evaluates and prioritizes results based on criteria such as the authoritativeness, freshness, and specificity of the search results.

[1016] The "means for transmitting to a user terminal" is a device or software module that transmits the evaluated and ranked search results to a user terminal.

[1017] The "means for displaying the received search results on the user terminal" is a device or software module that visually displays the received search results on the user terminal.

[1018] The system according to the present invention allows users to search for information efficiently and accurately using a terminal. The basic configuration of this system is made up of a user terminal, a server, and a data management system.

[1019] First, a user accesses a "search interface" through a dedicated application or a web browser, and then enters a specific query, such as "latest natural language processing technology," into the search box on the interface.

[1020] When a user enters a query and presses the search button, the device uses JavaScript to convert the query into JSON format. For example, it is converted into {"query": "latest natural language processing technology"}. The device then sends this JSON data to the server as an HTTP POST request. The HTTPS protocol is used for transmission, and the destination URL is, for example, https: / / example.com / search.

[1021] The server receives an HTTP POST request via a web server such as Apache or Nginx. After receiving the request, the server processes the request using a backend framework such as Python's Flask, Django, or Node.js. The server then analyzes the query using a natural language analysis model (e.g., BERT or GPT-3) to extract relevant keywords and phrases. For example, the keywords "latest," "natural language processing," and "technology" are extracted from the query "latest natural language processing technology."

[1022] The server then retrieves the user's past search history from the database and uses this information to optimize the search query: if the user searches a lot for "natural language processing," it will weight this keyword more highly to improve the accuracy of the search results.

[1023] The server uses an internal database client (e.g., PostgreSQL or MongoDB) to search for information in the data management system based on optimized search criteria, retrieving relevant data such as papers, articles, online courses, etc.

[1024] The retrieved search results are compared and ranked based on criteria such as reliability, freshness, and specificity. The evaluated results are then compiled into JSON format and sent to the device. For example, the format might look like this: {"results": [{"title": "The latest natural language processing technology", "url": "https: / / example.com / article"}]}. The device receives this JSON data and visually displays the results on the user interface.

[1025] Users can view the displayed search results and click on the information they are interested in to view more details. This system allows users to quickly access the information they are looking for and provides highly accurate search results based on their past search history and behavioral data.

[1026] Specific examples

[1027] For example, if a user enters the search query "latest AI technology":

[1028] The user inputs the latest AI technology through the interface, and the device sends it to the server.

[1029] The server analyzes the query and extracts keywords such as "latest," "AI," and "technology."

[1030] The server uses the user's past behavioral data to optimize queries and prioritize searches for the latest information related to AI technology.

[1031] It retrieves relevant information from internal databases and external APIs and ranks it based on criteria such as reliability, freshness, and detail.

[1032] The ranked results are sent to the user's terminal and displayed.

[1033] Users can review search results and select specific articles or papers to view more details.

[1034] Prompt Sentence Examples

[1035] For example, the prompt text to input to a generative AI model might look like this:

[1036] "I want to search for information about the latest natural language processing technology. Related keywords are 'latest,' 'natural language processing,' and 'technology,' and this user has searched a lot for information about natural language processing in the past."

[1037] The above is a specific embodiment for carrying out the present invention.

[1038] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1039] Step 1:

[1040] A user accesses a search interface on a terminal and inputs a search query such as "latest natural language processing technology." The input query is stored in text format in an input field on the terminal.

[1041] Step 2:

[1042] When the user presses the search button, the device uses JavaScript to convert the search query into JSON format, for example, {"query": "latest natural language processing technology"}. This JSON data is then prepared as an HTTP POST request.

[1043] Step 3:

[1044] The terminal uses the HTTPS protocol to send this JSON-formatted query to the server. The destination URL is, for example, https: / / example.com / search. The sent JSON data arrives at the server.

[1045] Step 4:

[1046] The server receives this HTTP POST request via a web server such as Apache or Nginx and extracts the search query in JSON format from the body of the request. For example, the data extracted is {"query": "The latest natural language processing technology"}.

[1047] Step 5:

[1048] The server uses a parsing module to input the search query into a natural language parsing model (e.g., BERT or GPT-3). The model analyzes the query and extracts relevant keywords and phrases. For example, the query "latest natural language processing technology" yields the keywords "latest," "natural language processing," and "technology."

[1049] Step 6:

[1050] The server retrieves the user's past search history and behavioral data from the database. Based on this, it optimizes the weighting of queries. For example, if there have been many searches related to "natural language processing" in the past, it will increase the weighting of this keyword.

[1051] Step 7:

[1052] The server issues SQL or NoSQL queries to data management systems (e.g., PostgreSQL or MongoDB) based on the optimized search criteria, searching relevant databases for information such as related papers, articles, or online courses.

[1053] Step 8:

[1054] The server retrieves the search results and rates each item based on the criteria of authoritativeness, freshness, and specificity, using a numerical score or other criteria, and assigns each result a score for authoritativeness, freshness, and specificity.

[1055] Step 9:

[1056] The server ranks the search results based on their scores, placing highly rated information at the top of the list. The server then compiles the ranked search results in JSON format, structured as follows: {"results": [{"title": "The latest natural language processing technology", "url": "https: / / example.com / article"}]}.

[1057] Step 10:

[1058] The server sends the ranked search results as JSON data to the user's device, which receives the JSON data, parses it, and displays it on the user interface.

[1059] Step 11:

[1060] Users can view detailed information by checking the search results displayed on their device and clicking on the links to articles or papers they are interested in. For example, they can click on the title of a particular paper to read more about it.

[1061] The above are the specific processing steps of the program of this system.

[1062] (Application example 1)

[1063] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1064] In physical stores, users face the challenge of finding the products they want efficiently and quickly. Furthermore, there is a lack of systems that can provide detailed product information, reviews, price comparisons, and other information in real time to help users make appropriate purchasing decisions. These situations result in a poor user experience.

[1065] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1066] In this invention, the server includes a means for the user terminal to provide search results including location information and reviews via a display device, a means for considering reliability, freshness of information, and level of detail, and a means for using the user's past search history and behavioral data when optimizing search conditions, thereby enabling users to quickly find desired products in physical stores, obtain detailed information, and make appropriate purchasing decisions.

[1067] "User device" is a general term for devices that users use to enter queries and view search results, such as smartphones and smart glasses.

[1068] A "search query" is a string of characters or voice command that a user types to obtain information.

[1069] "Natural language analysis" is the process of converting search queries into a format that is easy for machines to understand.

[1070] "Search optimization" refers to transforming search queries into more effective forms based on user requirements and past behavioral data.

[1071] A "database" refers to a system or structure that stores retrieved information and makes it available for retrieval as needed.

[1072] "Ranking" refers to evaluating and ranking the search results obtained based on certain criteria.

[1073] A "display device" is a device that visually presents search results to users, such as a smartphone screen or a smart glasses display.

[1074] "Location information" refers to information about the location where a particular product is located.

[1075] A "review" is an evaluation or impression posted by a user who has previously purchased the product.

[1076] "Credibility" is an indicator of how trustworthy the search results are.

[1077] "Information freshness" is an indicator of how up-to-date the search results are.

[1078] "Detail" is an indicator of how detailed the search results provide.

[1079] MODE FOR CARRYING OUT THE INVENTION

[1080] As an embodiment of the present invention, a smart shopping assistant system for improving user experience in a physical store will be described as a specific example. This system is composed of a user terminal, a server, and a database.

[1081] Program Overview

[1082] 1. User Device

[1083] Display devices such as smartphones and smart glasses are used as user terminals. These terminals accept voice and text inputs and retrieve user search queries.

[1084] Example: A user speaks into their smartphone, "What's the latest chocolate product?"

[1085] 2. Submitting a query

[1086] The user terminal converts the entered search query into JSON format and sends it to the server.

[1087] 3. Server

[1088] The server receives a search query sent from a user terminal.

[1089] Incoming search queries are parsed by a natural language parsing model (e.g., BERT), which breaks the query down into keywords and converts them into an understandable format.

[1090] Based on the analysis results, the search conditions are optimized by referring to the user's past search history and behavioral data.

[1091] 4. Database

[1092] The server uses optimized search criteria to search internal databases and external APIs, which store product information, reviews, pricing, and more.

[1093] 5. Rating and Ranking of Search Results

[1094] The server evaluates and ranks the search results based on reliability, freshness of information, and level of detail.

[1095] 6. Displaying the results

[1096] The ranked search results are sent to the user's terminal in JSON format and presented to the user via a display device, including product location information and reviews.

[1097] Specific examples

[1098] For example, if a user enters a query such as "What are the latest chocolate products?", the following process will occur:

[1099] The query is sent by the terminal to the server.

[1100] The server analyzes the received query and extracts keywords such as "latest," "chocolate," and "product."

[1101] Search criteria are optimized based on the user's past search history.

[1102] Relevant product information is retrieved from an internal database and external APIs, and then evaluated and ranked based on criteria such as reliability and freshness.

[1103] The ranked results are sent to the user's device and displayed along with product location information and reviews.

[1104] Prompt Sentence Examples

[1105] Voice input: "What's the latest chocolate product?"

[1106] Text input: "Tell me about new chocolate products"

[1107] As described above, this invention enables users to quickly find the desired product in a physical store, obtain detailed information, and make appropriate purchasing decisions, thereby significantly improving the user's purchasing experience.

[1108] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1109] Step 1:

[1110] User enters a search query

[1111] How it works: A user uses a smartphone or smart glasses to enter a search query using voice or text input.

[1112] Input: "What's the latest chocolate product?" (voice or text)

[1113] Output: Query data entered on the user's terminal

[1114] Step 2:

[1115] Sending a query from the device to the server

[1116] How it works: The user device converts the entered search query into JSON format and sends it to the server.

[1117] Input: Query data entered on the user's device

[1118] Output: Query data converted to JSON format

[1119] Step 3:

[1120] Server receives and analyzes the query

[1121] How it works: The server receives the query data in JSON format and passes the query to a natural language parsing model (e.g., BERT). The model parses the query and breaks it down into keywords.

[1122] Input: Query data in JSON format

[1123] Output: Extracted keywords (e.g. "latest", "chocolate", "product")

[1124] Step 4:

[1125] Optimizing search criteria

[1126] How it works: The server references the user's past search history and behavioral data to optimize search criteria based on the search query.

[1127] Input: Keywords and user history data

[1128] Output: Optimized search criteria

[1129] Step 5:

[1130] Database search

[1131] How it works: The server uses optimized search criteria to search for product information, reviews, pricing information, etc. from internal databases and external APIs.

[1132] Input: Optimized search criteria

[1133] Output: Search result data

[1134] Step 6:

[1135] Rating and ranking of search results

[1136] How it works: The server evaluates and ranks the search results it retrieves based on their authoritativeness, freshness, and specificity.

[1137] Input: Search result data

[1138] Output: Ranked search results

[1139] Step 7:

[1140] Sending results to the device

[1141] How it works: The server sends the ranked search results in JSON format to the user's device.

[1142] Input: Ranked search results

[1143] Output: Search result data in JSON format

[1144] Step 8:

[1145] Displaying search results

[1146] How it works: The user's device parses the JSON-formatted search results data received from the server and displays them on the display device (smartphone screen or smart glasses display) along with location information and reviews.

[1147] Input: Search result data in JSON format

[1148] Output: Search results displayed on a display device (e.g. product name, location, reviews)

[1149] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1150] The system according to the present invention allows users to search for information efficiently and accurately using a terminal, and further improves the search experience by recognizing the user's emotions during the search process. The basic configuration of this system is made up of a user terminal, a server, a database client, and an emotion engine.

[1151] System Program Overview

[1152] Program processing

[1153] 1. User enters a query

[1154] A user enters a search query in the search interface, for example, entering specific information such as "latest natural language processing technology."

[1155] 2. User Emotion Recognition

[1156] The device activates an emotion engine when the user inputs something, recognizing the user's emotions in real time. The emotion engine uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice, recognizing emotions such as "excited," "calm," or "irritated."

[1157] 3. Sending a request from the device to the server

[1158] The device converts the search query and recognized emotion data into JSON format and sends it to the server as a POST request.

[1159] 4. Server Receipt and Analysis of Request

[1160] The server receives the request and analyzes the query using a natural language analysis model. The query "latest natural language processing technology" is broken down into keywords such as "latest," "natural language processing," and "technology," and the meaning and relevance of each keyword is analyzed.

[1161] 5. Query Optimization

[1162] The server references the user's past search history and behavioral data, and also optimizes search conditions based on recognized emotional data. For example, if the user is in an "excited" state, the server will prioritize searches for new technologies and innovative content.

[1163] 6. Database Search

[1164] The server searches the database through an internal database client using optimized search criteria to retrieve relevant articles, papers, videos, online courses, etc.

[1165] 7. Compare and evaluate search results

[1166] The server evaluates the search results it retrieves based on their reliability, freshness, and specificity, assigning a score to each result and comparing them based on the evaluation results.

[1167] 8. Ranking of search results

[1168] The server ranks the search results based on criteria, with the most reliable and up-to-date information ranked at the top.

[1169] 9. Sending results to the device

[1170] The server encodes the ranked search results in JSON format and sends them to the device.

[1171] 10. User confirmation of results and action

[1172] The terminal receives the search results and displays them on the user interface. The user can check the displayed search results and view the details of each result.

[1173] Specific examples

[1174] For example, if a user enters the search query "latest AI technology" and the emotion engine recognizes that the user is "excited," the flow is as follows:

[1175] The user enters a search query through the interface, and the device transmits the query and emotion data to the server.

[1176] The server analyzes the query and extracts keywords such as "latest," "AI," and "technology."

[1177] The server optimizes queries to prioritize new technologies and innovative content because users are "excited."

[1178] It retrieves relevant information from internal databases and external APIs, and ranks the retrieved data based on criteria such as reliability, freshness, and level of detail.

[1179] The ranked results are sent to the user's terminal and displayed.

[1180] Users review the search results and select the most interesting articles or papers to read more.

[1181] In this way, a more personalized search experience can be provided by taking the user's emotions into account.

[1182] The processing flow will be explained below.

[1183] Step 1:

[1184] A user accesses the "Search-ru, Kuberu" interface and enters a search query. Specifically, the user enters a query such as "latest natural language processing technology" into the search box using a browser or a dedicated application, and clicks the submit button.

[1185] Step 2:

[1186] The device receives user input and activates the emotion engine, which uses the device's camera and microphone to analyze the user's facial expressions and tone of voice in real time.

[1187] Step 3:

[1188] The device analyzes the emotional data (e.g., "excited," "calm," "irritated," etc.) and converts it into JSON format along with the search query.

[1189] Step 4:

[1190] The device sends the converted JSON-formatted data to the server as a POST request, which includes the entered search query and the recognized emotion data.

[1191] Step 5:

[1192] The server receives the POST request and extracts the search query and sentiment data from the request body.

[1193] Step 6:

[1194] The server uses a natural language analysis model to analyze the search query. It breaks down the query "latest natural language processing technology" into keywords such as "latest," "natural language processing," and "technology," and analyzes the meaning and relevance of each keyword.

[1195] Step 7:

[1196] The server optimizes search criteria based on the user's past search history, behavioral data, and emotional data. For example, if a user has frequently searched for information about "natural language processing" in the past and is currently in an "excited" state, the server will adjust search criteria to prioritize information about more innovative and new technologies.

[1197] Step 8:

[1198] The server searches the database through an internal database client using optimized search criteria to retrieve relevant articles, papers, videos, online courses, etc.

[1199] Step 9:

[1200] The server evaluates the search results it retrieves based on their reliability, freshness, and specificity, assigning a score to each result and comparing them based on the evaluation results.

[1201] Step 10:

[1202] The server ranks the search results based on criteria, with the most reliable and up-to-date information ranked at the top.

[1203] Step 11:

[1204] The server encodes the ranked search results in JSON format and sends them to the device.

[1205] Step 12:

[1206] The device receives the search results and displays them on the user interface. The display method is laid out in a visually easy-to-understand manner, taking into consideration the user's convenience.

[1207] Step 13:

[1208] Users can review the search results and select the articles or papers that interest them most for more detailed information. For example, they can click on a highly rated paper to view its contents in more detail.

[1209] Example 2

[1210] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1211] Conventional search systems are required to provide highly relevant information in response to user search queries, but it has been difficult to provide search results that take into account the user's emotions and search context. As a result, users are burdened with selecting the most relevant information from a large amount of information, resulting in a poor search experience. The present invention aims to solve this problem and provide more appropriate and personalized search results that take into account the user's emotions.

[1212] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving a search query input from a user terminal, a means for performing natural language analysis based on the received search query, and a means for optimizing search conditions based on the analysis results. This makes it possible to recognize the user's emotions in real time and optimize the search conditions based on the emotion data. This realizes a personalized search experience that reflects the user's emotions, reduces the user's search burden, and enables more appropriate information to be provided more quickly.

[1213] A "user terminal" is a communication device operated by a user, and is a device for inputting search queries, recognizing emotion data, and displaying search results.

[1214] A "search query" is a query that a user inputs in the form of a string of characters or voice to express the information they want to know or search for.

[1215] "Natural language analysis" is a technology that allows a computer to understand and analyze the meaning of text written in human language.

[1216] "Search condition optimization" is a technique that adjusts conditions to efficiently and effectively search for related information based on a received search query.

[1217] A "database" is a system for storing structured data and searching and retrieving it as needed.

[1218] "Relevant data" is information extracted from a database based on a search query, and is useful data that matches the search intent.

[1219] "Ranking of search results" refers to the process of ranking search results based on evaluation criteria and determining the priority of the results to be presented to the user.

[1220] "Real-time emotion recognition" is a technology that instantly recognizes a user's current emotions from facial expressions, tone of voice, etc.

[1221] "Emotion data" is data that indicates the emotional state of the user recognized in real time.

[1222] "User interface" is a general term for the screens and input devices that allow a user to interact with a system.

[1223] MODE FOR CARRYING OUT THE INVENTION

[1224] A specific embodiment of the system according to the present invention will be described. This system allows users to search for information efficiently and accurately using a terminal, and further improves the search experience by recognizing the user's emotions during the search process. The basic configuration of this system is made up of a user terminal, a server, a database client, and an emotion engine.

[1225] Hardware and software used

[1226] User devices: PCs, smartphones, tablets, etc.

[1227] Server: Web server (Apache HTTP Server, Nginx, etc.), application server

[1228] Database client: Database management system such as MySQL, PostgreSQL, etc.

[1229] Emotion engine: Camera, microphone, emotion recognition software (OpenCV, Microsoft Azure Cognitive Services, etc.)

[1230] Generative AI models: GPT-3, BERT, etc. for natural language analysis

[1231] System basic configuration and operation

[1232] 1. Enter a query

[1233] A user enters a search query through an interface installed on the device (such as a browser-based web application or a mobile application), for example, entering a specific information request such as "latest natural language processing technology."

[1234] 2. Emotion recognition

[1235] When a user inputs a query, the device activates an emotion engine. The emotion engine uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice, allowing it to recognize the user's emotions, such as "excited," "calm," or "irritated," in real time.

[1236] 3. Sending a request

[1237] The device converts the search query and the recognized emotion data into JSON format and sends it to the server as an HTTP POST request.

[1238] 4. Receiving and analyzing requests

[1239] The server receives the JSON-formatted request from the device. It then uses a generative AI model for natural language analysis (e.g., GPT-3 or BERT) to analyze the query, breaking down the query "latest natural language processing technology" into keywords such as "latest," "natural language processing," and "technology." It then analyzes the meaning and relevance of each keyword.

[1240] 5. Query Optimization

[1241] The server uses the user's past search history and behavioral data, as well as emotional data, to optimize search conditions. For example, if the server detects that the user is "excited," it will prioritize new technologies and innovative content.

[1242] 6. Database Search

[1243] The server searches the database through the database client using optimized search criteria, such as executing SQL queries to retrieve relevant articles, papers, videos, online courses, etc.

[1244] 7. Display of search results

[1245] The server evaluates the search results based on their reliability, freshness, and detail, assigns a score, compares them, and then ranks them based on the evaluation criteria. The ranked results are encoded in JSON format and sent to the device.

[1246] 8. User confirmation of results

[1247] The terminal receives the search results sent from the server and displays them on the user interface. The user can then review the displayed search results, select the most interesting article or paper, and view its details.

[1248] Specific examples

[1249] For example, if a user enters the search query "latest AI technology" and the emotion engine recognizes the user's emotion as "excited," the operation flow is as follows:

[1250] 1. User: Enters the query "latest AI technology."

[1251] 2. Device: The emotion engine recognizes that the user is excited based on their facial expressions and voice.

[1252] 3. Terminal: Convert the search query and sentiment data into JSON format and send it to the server as an HTTP POST request.

[1253] 4. Server: Receives the request and breaks down "latest AI technology" into the keywords "latest," "AI," and "technology."

[1254] 5. Server: Optimize queries to prioritize innovative content based on what users are "excited about."

[1255] 6. Server: Searches the database using optimized search criteria to retrieve relevant information.

[1256] 7. Server: Evaluates and ranks the acquired information.

[1257] 8. Server: Encode the ranked search results in JSON format and send them to the device.

[1258] 9. Device: Search results are displayed and the user checks the details.

[1259] Examples of usage prompt statements

[1260] Search for "latest AI technology" and describe a system that provides the most appropriate information to a user when the user is emotionally "excited."

[1261] As described above, this system takes user emotions into consideration to provide a more appropriate and personalized search experience.

[1262] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1263] Step 1:

[1264] Query Input

[1265] A user inputs a search query through the device interface. A specific example of what the user inputs is "the latest natural language processing technology." The input data is in text format.

[1266] Specific behavior:

[1267] The user enters a search query and clicks the submit button, and the device stores the entered text query in its internal memory.

[1268] Step 2:

[1269] emotion recognition

[1270] The device activates an emotion engine to recognize the user's emotions in real time. The device uses a camera and microphone to analyze the user's facial expressions and tone of voice to determine emotions.

[1271] input:

[1272] User's facial expression and voice data

[1273] Specific behavior:

[1274] The device inputs data acquired from the camera and microphone into emotion recognition software to determine the user's current emotion (e.g., "excited" or "calm"), and the analysis results are stored in internal memory.

[1275] output:

[1276] User emotion data (e.g., excited)

[1277] Step 3:

[1278] Send request

[1279] The device converts the search query and recognized emotion data into JSON format and sends it to the server as an HTTP POST request.

[1280] input:

[1281] Search query and user sentiment data

[1282] Specific behavior:

[1283] The device converts the search query and sentiment data into JSON format as key-value pairs, then generates an HTTP request and sends it to the specified URL on the server.

[1284] output:

[1285] JSON formatted HTTP POST request

[1286] Step 4:

[1287] Receiving and parsing requests

[1288] The server receives a JSON-formatted request from the device and analyzes the query using a generative AI model. For example, the query "latest natural language processing technology" is broken down into keywords such as "latest," "natural language processing," and "technology."

[1289] input:

[1290] Request data in JSON format

[1291] Specific behavior:

[1292] The server receives the request and feeds the query to the generative AI model, which analyzes the query and extracts important keywords, which are then stored in an internal database.

[1293] output:

[1294] Analyzed keyword data (e.g., "latest," "natural language processing," "technology")

[1295] Step 5:

[1296] Query optimization

[1297] The server optimizes the search criteria based on the analyzed keyword data, as well as the user's emotional data and past search history. For example, if the user is "excited," it will prioritize new technologies and innovative content.

[1298] input:

[1299] Analyzed keyword data, user sentiment data, and past search history

[1300] Specific behavior:

[1301] The server uses this data to run an optimization algorithm to determine the highest priority search terms, which results in an optimized search term.

[1302] output:

[1303] Optimized search criteria

[1304] Step 6:

[1305] Database search

[1306] The server searches the database using optimized search criteria and retrieves relevant information internally or through an external API.

[1307] input:

[1308] Optimized search criteria

[1309] Specific behavior:

[1310] The server issues SQL queries to the database to retrieve data that matches the search criteria, as well as external API calls.

[1311] output:

[1312] Retrieved search result data

[1313] Step 7:

[1314] Rating and ranking of search results

[1315] The search results obtained by the server are evaluated based on reliability, freshness, and detail, and are assigned a score to rank them.

[1316] input:

[1317] Retrieved search result data

[1318] Specific behavior:

[1319] The server evaluates each search result based on the evaluation criteria, assigns a score, and ranks the search results based on the score.

[1320] output:

[1321] Ranked search results data

[1322] Step 8:

[1323] Send results

[1324] The server encodes the ranked search results in JSON format and sends them to the device.

[1325] input:

[1326] Ranked search results data

[1327] Specific behavior:

[1328] The server encodes the search results in JSON format and sends them to the terminal as an HTTP response.

[1329] output:

[1330] JSON formatted HTTP response

[1331] Step 9:

[1332] Check the results

[1333] The terminal receives the search results sent from the server and displays them on the user interface, where the user can check the displayed search results and view their details.

[1334] input:

[1335] Search result data in JSON format

[1336] Specific behavior:

[1337] The device receives the search results, parses them, and displays them in a user interface that the user can navigate and click on results of interest to learn more.

[1338] output:

[1339] Search results displayed to users

[1340] (Application example 2)

[1341] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1342] Conventional food delivery services lack personalized meal suggestions because they do not provide a search experience that takes into account users' emotions and individual preferences. Another issue is the lack of functionality to analyze users' emotional data and optimize search criteria based on that data in order to provide optimal search results for users' search queries.

[1343] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recognizing a user's emotion, means for optimizing search conditions based on the recognized emotion, and means for searching for related data from a database using the optimized search conditions. This enables a personalized search experience based on the user's emotion, and can provide more optimal search results.

[1344] A "user terminal" is a hardware device operated by a user, such as a smartphone or tablet.

[1345] A "search query" refers to the text or voice input a user enters to search for specific information.

[1346] "Means for recognizing emotions" refers to technology that analyzes a user's words and actions, facial expressions, tone of voice, etc. to determine their emotions at that time.

[1347] "Natural language analysis" is a technology for analyzing input text or speech and understanding its meaning.

[1348] "Means for optimizing search conditions" refers to technology that performs processing to make search results more accurate and useful based on the user's input query and emotional data.

[1349] A "database" is an organized collection of data that stores information responsive to a search query.

[1350] The "means for searching related data" is a technique for extracting related information in a database using optimized search criteria.

[1351] "Means for evaluating and ranking search results" refers to technology that evaluates the search results obtained based on criteria such as reliability and freshness, and ranks them in order of importance.

[1352] "Means for sending to user terminal" refers to the technology for transferring the evaluated and ranked search results to the user terminal.

[1353] "Past search history" is a record of search queries a user has previously performed.

[1354] "Behavioral data" refers to data relating to the behavior of a user, such as what operations the user performed and which pages the user viewed.

[1355] The system according to the present invention comprises a user terminal, a server, a database, and an emotion engine. The system program performs processing in the following steps: When a user inputs a search query using a food delivery app, the emotion recognition means analyzes the user's emotion in real time, and the query is sent to the server together with that data.

[1356] The server receives the user's search query and emotion data, and optimizes the search query based on the recognized emotion. The search criteria are further refined by referring to the user's past search history and behavioral data. Based on the optimized search criteria, relevant information is searched for from the database. The obtained search results are evaluated based on criteria such as reliability, freshness, and level of detail, and a ranking is assigned. The ranked search results are sent from the server to the user's device and displayed on the user interface.

[1357] Examples of hardware and software used include smartphones and tablets as user devices, emotion engines for recognizing emotions, natural language processing models for data analysis, and servers for searching and evaluating databases.

[1358] As a concrete example, consider the case where a user searches for "Italian restaurant." If the user's device uses an emotion engine to recognize the emotion "excited," the server uses that emotion data to optimize the search criteria to prioritize new restaurants and special menus. The server then searches the database based on the optimized search criteria to retrieve information about new restaurants and special menus. The information is then ranked based on criteria such as reliability and freshness and presented to the user.

[1359] An example prompt might be:

[1360] "When a user enters a search query in a food delivery app, analyze the data and recognize the user's emotions in real time. Design a system that optimizes the query based on the user's emotions and provides search results for relevant restaurants and menus. For example, when a user is excited, suggest new restaurants or special menus, and when a user is stressed, prioritize past favorite restaurants or menus that have given them high satisfaction."

[1361] This provides a personalized search experience that takes user emotions into account, which is expected to improve satisfaction with food delivery services.

[1362] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1363] Step 1:

[1364] A user opens a food delivery app and enters a search query. Input: The user's query (e.g., "Italian restaurant") and the user's facial image and voice data. Output: The device collects the query and emotion data. Specific operation: The user enters the query into an input box on the device interface, and the device's camera and microphone are used to collect the user's facial expressions and tone of voice in real time.

[1365] Step 2:

[1366] The device activates the emotion engine and recognizes the user's emotions in real time. Input: User's facial image and voice data. Output: Recognized emotion data (e.g., "Excited"). Specific operation: The emotion engine uses face recognition algorithms and voice analysis algorithms to analyze the user's facial muscle movements and voice tone.

[1367] Step 3:

[1368] The device sends the search query and recognized emotion data to the server. Input: Query and emotion data. Output: Data sent to the server. Specific operation: The device encodes the query and emotion data into JSON format and sends it to the server using an HTTP POST request.

[1369] Step 4:

[1370] The server analyzes the received query and sentiment data and performs natural language analysis on the search query. Input: Query and sentiment data. Output: Keywords from the analyzed query (e.g., "Italian," "restaurant"). What happens: A natural language processing (NLP) model on the server tokenizes the query and extracts key keywords.

[1371] Step 5:

[1372] The server references the user's past behavioral data and search history along with the recognized emotions to optimize the search criteria. Input: Analyzed keywords, emotion data, past behavioral data, and search history. Output: Optimized search criteria. Specific operation: The server retrieves the user's past history from the database and adjusts the search algorithm along with the emotion data.

[1373] Step 6:

[1374] The server searches the database for relevant information based on the search criteria. Input: Optimized search criteria. Output: Relevant information (e.g., a list of new Italian restaurants). Specific behavior: The server executes a database query to extract relevant restaurants and menus.

[1375] Step 7:

[1376] The server evaluates and ranks the search results. Input: Retrieved search results. Output: Rated and ranked results. Specific behavior: The server assigns a score to each result using evaluation criteria such as reliability, freshness, and specificity, and then ranks them.

[1377] Step 8:

[1378] The server sends the ranked search results to the user's terminal. Input: Ranked search results. Output: Search results sent to the user's terminal. Specific operation: The server encodes the ranked results in JSON format and sends them to the user's terminal as an HTTP response.

[1379] Step 9:

[1380] The user terminal receives the search results and displays them on the user interface. Input: Ranked search results. Output: Displayed search results. Specific operation: The application on the terminal analyzes the search results and displays them to the user.

[1381] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1382] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1383] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1384] [Fourth embodiment]

[1385] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1386] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1387] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1388] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1389] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1390] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1391] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1392] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1393] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1394] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1395] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1396] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1397] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1398] The system according to the present invention implements a series of processes that allow users to search for information efficiently and accurately using a terminal. The basic configuration of this system is made up of a user terminal, a server, and a database client.

[1399] System Program Overview

[1400] Program processing

[1401] 1. User enters a query

[1402] Users access a terminal with an interface called "Search-ru, Kura-beru" and enter specific information such as a search query, "latest natural language processing technology."

[1403] 2. Sending a request from the device to the server

[1404] The device converts this search query into JSON format and sends it to the server as a POST request.

[1405] 3. Server receives and analyzes the request

[1406] The server receives the request and passes the query to a natural language analysis model. This model analyzes the query and extracts relevant keywords and phrases. For example, a query like "latest natural language processing technology" would be broken down into three keywords: "latest," "natural language processing," and "technology."

[1407] 4. Query Optimization

[1408] The server optimizes the search query by referencing the user's past search history and behavioral data. For example, if the user has searched a lot for information about "natural language processing" in the past, it will give more weight to this keyword and adjust the search criteria.

[1409] 5. Database Search

[1410] The server uses optimized search criteria to search the database via an internal database client, where relevant articles, papers, online courses, and other information are retrieved.

[1411] 6. Compare and evaluate search results

[1412] The server evaluates the search results it receives, rating each based on reliability, freshness of information, and specificity.

[1413] 7. Ranking of search results

[1414] The server ranks the search results based on the criteria: for example, if the most recent paper is rated as highly reliable, it will be ranked highly.

[1415] 8. Sending results to the device

[1416] The server sends the ranked search results in JSON format to the device, which receives this information and displays the results in the user's interface.

[1417] 9. User confirmation of results and action

[1418] Users can check the search results displayed on their devices and view detailed information as needed. For example, they can select a specific paper or article to view its details.

[1419] Specific examples

[1420] For example, if a user enters the search query "latest AI technology," the flow is as follows:

[1421] The user enters a search query through the interface, and the terminal sends it to the server.

[1422] The server analyzes the query and extracts keywords such as "latest," "AI," and "technology."

[1423] The server uses the user's past behavioral data to optimize queries and prioritize searches for the latest information related to AI technology.

[1424] It retrieves relevant information from internal databases and external APIs, and ranks the retrieved data based on criteria such as reliability, freshness, and level of detail.

[1425] The ranked results are sent to the user's terminal and displayed.

[1426] Users can review search results and select specific articles or papers to view more details.

[1427] This system allows users to quickly access the information they need and obtain higher quality information.

[1428] The processing flow will be explained below.

[1429] Step 1:

[1430] A user accesses the Search-Ru interface and enters a search query, such as "latest natural language processing technology," in the search box of a browser or dedicated application, and clicks the submit button.

[1431] Step 2:

[1432] The device converts the user's input into JSON format, which may package the search query along with the user's past search history and preferences.

[1433] Step 3:

[1434] The device sends the converted JSON-formatted data to the server as a POST request, which includes the entered search query and related metadata.

[1435] Step 4:

[1436] The server receives the POST request and extracts the search query and related data from the request body.

[1437] Step 5:

[1438] The server uses a natural language analysis model to analyze the search query. It breaks down the query "latest natural language processing technology" into keywords such as "latest," "natural language processing," and "technology," and analyzes the meaning and relevance of each keyword.

[1439] Step 6:

[1440] The server optimizes search criteria based on the user's past search history and behavioral data. For example, if a user has frequently searched for information about "natural language processing" in the past, it will assign a weight to this keyword.

[1441] Step 7:

[1442] The server uses optimized queries to search the database through an internal database client, retrieving relevant articles, papers, videos, online courses, etc.

[1443] Step 8:

[1444] The server evaluates the search results it retrieves based on their reliability, freshness, and specificity, assigning a score to each result and comparing them based on the evaluation results.

[1445] Step 9:

[1446] The server ranks the search results based on criteria, sorting them so that the most reliable and up-to-date information is placed at the top.

[1447] Step 10:

[1448] The server encodes the ranked search results in JSON format and sends them to the device.

[1449] Step 11:

[1450] The device receives the search results and displays them in a user interface, laid out visually and clearly so that the user can easily check each result.

[1451] Step 12:

[1452] Users can review the search results and select articles or papers that interest them to read more, for example, by clicking on a highly rated paper to view its contents.

[1453] Example 1

[1454] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1455] Conventional information search systems often lack the accuracy and relevance of results based on user-entered search queries, making it difficult for users to quickly and reliably access the information they are looking for. Providing appropriate search results is particularly challenging in today's world, where vast amounts of information are readily available. Furthermore, technologies that optimize search results by utilizing users' past search history and behavioral data are often not fully utilized. This limits the user experience, creating problems.

[1456] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1457] In this invention, the server includes means for receiving a search query input from a user terminal, means for converting the received search query into a structured data format, means for transmitting the converted search query, means for performing natural language analysis based on the received search query, means for optimizing search conditions based on the analysis results, means for searching for related data from a data management system using the optimized search conditions, means for evaluating and ranking search results, means for transmitting the ranked search results to the user terminal, and means for displaying the search results received on the user terminal. This allows the user to quickly obtain optimal information based on the search query and to obtain highly accurate search results based on past search history and behavioral data.

[1458] A "user terminal" is an appliance or device used by a user to input information and display results.

[1459] A "search query" is a word or string of characters that a user enters to search for specific information.

[1460] The "means for receiving a search query" is a device or software module that receives a search query sent from a user terminal.

[1461] A "means for converting into a structured data format" is a device or software module that converts an input search query into a certain format (e.g., JSON format).

[1462] A "means for transmitting a converted search query" is a device or software module that transmits a search query that has been converted into a structured data format over a network.

[1463] "Natural language analysis" is a technology that analyzes natural language, which is human language, and extracts semantic information.

[1464] A "means for performing natural language analysis" is a device or software module that performs natural language analysis on a received search query.

[1465] A "means for optimizing search conditions" is a device or software module that uses the results of natural language analysis and the user's past search history to adjust search conditions in the most effective way.

[1466] A "data management system" is a system for storing, managing, and retrieving information in an organized manner.

[1467] A "means for retrieving relevant data" is a device or software module that retrieves relevant information from a data management system using optimized search criteria.

[1468] A "means for evaluating and ranking search results" is a device or software module that evaluates and prioritizes results based on criteria such as the authoritativeness, freshness, and specificity of the search results.

[1469] The "means for transmitting to a user terminal" is a device or software module that transmits the evaluated and ranked search results to a user terminal.

[1470] The "means for displaying the received search results on the user terminal" is a device or software module that visually displays the received search results on the user terminal.

[1471] The system according to the present invention allows users to search for information efficiently and accurately using a terminal. The basic configuration of this system is made up of a user terminal, a server, and a data management system.

[1472] First, a user accesses a "search interface" through a dedicated application or a web browser, and then enters a specific query, such as "latest natural language processing technology," into the search box on the interface.

[1473] When a user enters a query and presses the search button, the device uses JavaScript to convert the query into JSON format. For example, it is converted into {"query": "latest natural language processing technology"}. The device then sends this JSON data to the server as an HTTP POST request. The HTTPS protocol is used for transmission, and the destination URL is, for example, https: / / example.com / search.

[1474] The server receives an HTTP POST request via a web server such as Apache or Nginx. After receiving the request, the server processes the request using a backend framework such as Python's Flask, Django, or Node.js. The server then analyzes the query using a natural language analysis model (e.g., BERT or GPT-3) to extract relevant keywords and phrases. For example, the keywords "latest," "natural language processing," and "technology" are extracted from the query "latest natural language processing technology."

[1475] The server then retrieves the user's past search history from the database and uses this information to optimize the search query: if the user searches a lot for "natural language processing," it will weight this keyword more highly to improve the accuracy of the search results.

[1476] The server uses an internal database client (e.g., PostgreSQL or MongoDB) to search for information in the data management system based on optimized search criteria, retrieving relevant data such as papers, articles, online courses, etc.

[1477] The retrieved search results are compared and ranked based on criteria such as reliability, freshness, and specificity. The evaluated results are then compiled into JSON format and sent to the device. For example, the format might look like this: {"results": [{"title": "The latest natural language processing technology", "url": "https: / / example.com / article"}]}. The device receives this JSON data and visually displays the results on the user interface.

[1478] Users can view the displayed search results and click on the information they are interested in to view more details. This system allows users to quickly access the information they are looking for and provides highly accurate search results based on their past search history and behavioral data.

[1479] Specific examples

[1480] For example, if a user enters the search query "latest AI technology":

[1481] The user inputs the latest AI technology through the interface, and the device sends it to the server.

[1482] The server analyzes the query and extracts keywords such as "latest," "AI," and "technology."

[1483] The server uses the user's past behavioral data to optimize queries and prioritize searches for the latest information related to AI technology.

[1484] It retrieves relevant information from internal databases and external APIs and ranks it based on criteria such as reliability, freshness, and detail.

[1485] The ranked results are sent to the user's terminal and displayed.

[1486] Users can review search results and select specific articles or papers to view more details.

[1487] Prompt Sentence Examples

[1488] For example, the prompt text to input to a generative AI model might look like this:

[1489] "I want to search for information about the latest natural language processing technology. Related keywords are 'latest,' 'natural language processing,' and 'technology,' and this user has searched a lot for information about natural language processing in the past."

[1490] The above is a specific embodiment for carrying out the present invention.

[1491] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1492] Step 1:

[1493] A user accesses a search interface on a terminal and inputs a search query such as "latest natural language processing technology." The input query is stored in text format in an input field on the terminal.

[1494] Step 2:

[1495] When the user presses the search button, the device uses JavaScript to convert the search query into JSON format, for example, {"query": "latest natural language processing technology"}. This JSON data is then prepared as an HTTP POST request.

[1496] Step 3:

[1497] The terminal uses the HTTPS protocol to send this JSON-formatted query to the server. The destination URL is, for example, https: / / example.com / search. The sent JSON data arrives at the server.

[1498] Step 4:

[1499] The server receives this HTTP POST request via a web server such as Apache or Nginx and extracts the search query in JSON format from the body of the request. For example, the data extracted is {"query": "The latest natural language processing technology"}.

[1500] Step 5:

[1501] The server uses a parsing module to input the search query into a natural language parsing model (e.g., BERT or GPT-3). The model analyzes the query and extracts relevant keywords and phrases. For example, the query "latest natural language processing technology" yields the keywords "latest," "natural language processing," and "technology."

[1502] Step 6:

[1503] The server retrieves the user's past search history and behavioral data from the database. Based on this, it optimizes the weighting of queries. For example, if there have been many searches related to "natural language processing" in the past, it will increase the weighting of this keyword.

[1504] Step 7:

[1505] The server issues SQL or NoSQL queries to data management systems (e.g., PostgreSQL or MongoDB) based on the optimized search criteria, searching relevant databases for information such as related papers, articles, or online courses.

[1506] Step 8:

[1507] The server retrieves the search results and rates each item based on the criteria of authoritativeness, freshness, and specificity, using a numerical score or other criteria, and assigns each result a score for authoritativeness, freshness, and specificity.

[1508] Step 9:

[1509] The server ranks the search results based on their scores, placing highly rated information at the top of the list. The server then compiles the ranked search results in JSON format, structured as follows: {"results": [{"title": "The latest natural language processing technology", "url": "https: / / example.com / article"}]}.

[1510] Step 10:

[1511] The server sends the ranked search results as JSON data to the user's device, which receives the JSON data, parses it, and displays it on the user interface.

[1512] Step 11:

[1513] Users can view detailed information by checking the search results displayed on their device and clicking on the links to articles or papers they are interested in. For example, they can click on the title of a particular paper to read more about it.

[1514] The above are the specific processing steps of the program of this system.

[1515] (Application example 1)

[1516] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1517] In physical stores, users face the challenge of finding the products they want efficiently and quickly. Furthermore, there is a lack of systems that can provide detailed product information, reviews, price comparisons, and other information in real time to help users make appropriate purchasing decisions. These situations result in a poor user experience.

[1518] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1519] In this invention, the server includes a means for the user terminal to provide search results including location information and reviews via a display device, a means for considering reliability, freshness of information, and level of detail, and a means for using the user's past search history and behavioral data when optimizing search conditions, thereby enabling users to quickly find desired products in physical stores, obtain detailed information, and make appropriate purchasing decisions.

[1520] "User device" is a general term for devices that users use to enter queries and view search results, such as smartphones and smart glasses.

[1521] A "search query" is a string of characters or voice command that a user types to obtain information.

[1522] "Natural language analysis" is the process of converting search queries into a format that is easy for machines to understand.

[1523] "Search optimization" refers to transforming search queries into more effective forms based on user requirements and past behavioral data.

[1524] A "database" refers to a system or structure that stores retrieved information and makes it available for retrieval as needed.

[1525] "Ranking" refers to evaluating and ranking the search results obtained based on certain criteria.

[1526] A "display device" is a device that visually presents search results to users, such as a smartphone screen or a smart glasses display.

[1527] "Location information" refers to information about the location where a particular product is located.

[1528] A "review" is an evaluation or impression posted by a user who has previously purchased the product.

[1529] "Credibility" is an indicator of how trustworthy the search results are.

[1530] "Information freshness" is an indicator of how up-to-date the search results are.

[1531] "Detail" is an indicator of how detailed the search results provide.

[1532] MODE FOR CARRYING OUT THE INVENTION

[1533] As an embodiment of the present invention, a smart shopping assistant system for improving user experience in a physical store will be described as a specific example. This system is composed of a user terminal, a server, and a database.

[1534] Program Overview

[1535] 1. User Device

[1536] Display devices such as smartphones and smart glasses are used as user terminals. These terminals accept voice and text inputs and retrieve user search queries.

[1537] Example: A user speaks into their smartphone, "What's the latest chocolate product?"

[1538] 2. Submitting a query

[1539] The user terminal converts the entered search query into JSON format and sends it to the server.

[1540] 3. Server

[1541] The server receives a search query sent from a user terminal.

[1542] Incoming search queries are parsed by a natural language parsing model (e.g., BERT), which breaks the query down into keywords and converts them into an understandable format.

[1543] Based on the analysis results, the search conditions are optimized by referring to the user's past search history and behavioral data.

[1544] 4. Database

[1545] The server uses optimized search criteria to search internal databases and external APIs, which store product information, reviews, pricing, and more.

[1546] 5. Rating and Ranking of Search Results

[1547] The server evaluates and ranks the search results based on reliability, freshness of information, and level of detail.

[1548] 6. Displaying the results

[1549] The ranked search results are sent to the user's terminal in JSON format and presented to the user via a display device, including product location information and reviews.

[1550] Specific examples

[1551] For example, if a user enters a query such as "What are the latest chocolate products?", the following process will occur:

[1552] The query is sent by the terminal to the server.

[1553] The server analyzes the received query and extracts keywords such as "latest," "chocolate," and "product."

[1554] Search criteria are optimized based on the user's past search history.

[1555] Relevant product information is retrieved from an internal database and external APIs, and then evaluated and ranked based on criteria such as reliability and freshness.

[1556] The ranked results are sent to the user's device and displayed along with product location information and reviews.

[1557] Prompt Sentence Examples

[1558] Voice input: "What's the latest chocolate product?"

[1559] Text input: "Tell me about new chocolate products"

[1560] As described above, this invention enables users to quickly find the desired product in a physical store, obtain detailed information, and make appropriate purchasing decisions, thereby significantly improving the user's purchasing experience.

[1561] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1562] Step 1:

[1563] User enters a search query

[1564] How it works: A user uses a smartphone or smart glasses to enter a search query using voice or text input.

[1565] Input: "What's the latest chocolate product?" (voice or text)

[1566] Output: Query data entered on the user's terminal

[1567] Step 2:

[1568] Sending a query from the device to the server

[1569] How it works: The user device converts the entered search query into JSON format and sends it to the server.

[1570] Input: Query data entered on the user's device

[1571] Output: Query data converted to JSON format

[1572] Step 3:

[1573] Server receives and analyzes the query

[1574] How it works: The server receives the query data in JSON format and passes the query to a natural language parsing model (e.g., BERT). The model parses the query and breaks it down into keywords.

[1575] Input: Query data in JSON format

[1576] Output: Extracted keywords (e.g. "latest", "chocolate", "product")

[1577] Step 4:

[1578] Optimizing search criteria

[1579] How it works: The server references the user's past search history and behavioral data to optimize search criteria based on the search query.

[1580] Input: Keywords and user history data

[1581] Output: Optimized search criteria

[1582] Step 5:

[1583] Database search

[1584] How it works: The server uses optimized search criteria to search for product information, reviews, pricing information, etc. from internal databases and external APIs.

[1585] Input: Optimized search criteria

[1586] Output: Search result data

[1587] Step 6:

[1588] Rating and ranking of search results

[1589] How it works: The server evaluates and ranks the search results it retrieves based on their authoritativeness, freshness, and specificity.

[1590] Input: Search result data

[1591] Output: Ranked search results

[1592] Step 7:

[1593] Sending results to the device

[1594] How it works: The server sends the ranked search results in JSON format to the user's device.

[1595] Input: Ranked search results

[1596] Output: Search result data in JSON format

[1597] Step 8:

[1598] Displaying search results

[1599] How it works: The user's device parses the JSON-formatted search results data received from the server and displays them on the display device (smartphone screen or smart glasses display) along with location information and reviews.

[1600] Input: Search result data in JSON format

[1601] Output: Search results displayed on a display device (e.g. product name, location, reviews)

[1602] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1603] The system according to the present invention allows users to search for information efficiently and accurately using a terminal, and further improves the search experience by recognizing the user's emotions during the search process. The basic configuration of this system is made up of a user terminal, a server, a database client, and an emotion engine.

[1604] System Program Overview

[1605] Program processing

[1606] 1. User enters a query

[1607] A user enters a search query in the search interface, for example, entering specific information such as "latest natural language processing technology."

[1608] 2. User Emotion Recognition

[1609] The device activates an emotion engine when the user inputs something, recognizing the user's emotions in real time. The emotion engine uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice, recognizing emotions such as "excited," "calm," or "irritated."

[1610] 3. Sending a request from the device to the server

[1611] The device converts the search query and recognized emotion data into JSON format and sends it to the server as a POST request.

[1612] 4. Server Receipt and Analysis of Request

[1613] The server receives the request and analyzes the query using a natural language analysis model. The query "latest natural language processing technology" is broken down into keywords such as "latest," "natural language processing," and "technology," and the meaning and relevance of each keyword is analyzed.

[1614] 5. Query Optimization

[1615] The server references the user's past search history and behavioral data, and also optimizes search conditions based on recognized emotional data. For example, if the user is in an "excited" state, the server will prioritize searches for new technologies and innovative content.

[1616] 6. Database Search

[1617] The server searches the database through an internal database client using optimized search criteria to retrieve relevant articles, papers, videos, online courses, etc.

[1618] 7. Compare and evaluate search results

[1619] The server evaluates the search results it retrieves based on their reliability, freshness, and specificity, assigning a score to each result and comparing them based on the evaluation results.

[1620] 8. Ranking of search results

[1621] The server ranks the search results based on criteria, with the most reliable and up-to-date information ranked at the top.

[1622] 9. Sending results to the device

[1623] The server encodes the ranked search results in JSON format and sends them to the device.

[1624] 10. User confirmation of results and action

[1625] The terminal receives the search results and displays them on the user interface. The user can check the displayed search results and view the details of each result.

[1626] Specific examples

[1627] For example, if a user enters the search query "latest AI technology" and the emotion engine recognizes that the user is "excited," the flow is as follows:

[1628] The user enters a search query through the interface, and the device transmits the query and emotion data to the server.

[1629] The server analyzes the query and extracts keywords such as "latest," "AI," and "technology."

[1630] The server optimizes queries to prioritize new technologies and innovative content because users are "excited."

[1631] It retrieves relevant information from internal databases and external APIs, and ranks the retrieved data based on criteria such as reliability, freshness, and level of detail.

[1632] The ranked results are sent to the user's terminal and displayed.

[1633] Users review the search results and select the most interesting articles or papers to read more.

[1634] In this way, a more personalized search experience can be provided by taking the user's emotions into account.

[1635] The processing flow will be explained below.

[1636] Step 1:

[1637] A user accesses the "Search-ru, Kuberu" interface and enters a search query. Specifically, the user enters a query such as "latest natural language processing technology" into the search box using a browser or a dedicated application, and clicks the submit button.

[1638] Step 2:

[1639] The device receives user input and activates the emotion engine, which uses the device's camera and microphone to analyze the user's facial expressions and tone of voice in real time.

[1640] Step 3:

[1641] The device analyzes the emotional data (e.g., "excited," "calm," "irritated," etc.) and converts it into JSON format along with the search query.

[1642] Step 4:

[1643] The device sends the converted JSON-formatted data to the server as a POST request, which includes the entered search query and the recognized emotion data.

[1644] Step 5:

[1645] The server receives the POST request and extracts the search query and sentiment data from the request body.

[1646] Step 6:

[1647] The server uses a natural language analysis model to analyze the search query. It breaks down the query "latest natural language processing technology" into keywords such as "latest," "natural language processing," and "technology," and analyzes the meaning and relevance of each keyword.

[1648] Step 7:

[1649] The server optimizes search criteria based on the user's past search history, behavioral data, and emotional data. For example, if a user has frequently searched for information about "natural language processing" in the past and is currently in an "excited" state, the server will adjust search criteria to prioritize information about more innovative and new technologies.

[1650] Step 8:

[1651] The server searches the database through an internal database client using optimized search criteria to retrieve relevant articles, papers, videos, online courses, etc.

[1652] Step 9:

[1653] The server evaluates the search results it retrieves based on their reliability, freshness, and specificity, assigning a score to each result and comparing them based on the evaluation results.

[1654] Step 10:

[1655] The server ranks the search results based on criteria, with the most reliable and up-to-date information ranked at the top.

[1656] Step 11:

[1657] The server encodes the ranked search results in JSON format and sends them to the device.

[1658] Step 12:

[1659] The device receives the search results and displays them on the user interface. The display method is laid out in a visually easy-to-understand manner, taking into consideration the user's convenience.

[1660] Step 13:

[1661] Users can review the search results and select the articles or papers that interest them most for more detailed information. For example, they can click on a highly rated paper to view its contents in more detail.

[1662] Example 2

[1663] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1664] Conventional search systems are required to provide highly relevant information in response to user search queries, but it has been difficult to provide search results that take into account the user's emotions and search context. As a result, users are burdened with selecting the most relevant information from a large amount of information, resulting in a poor search experience. The present invention aims to solve this problem and provide more appropriate and personalized search results that take into account the user's emotions.

[1665] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for receiving a search query input from a user terminal, a means for performing natural language analysis based on the received search query, and a means for optimizing search conditions based on the analysis results. This makes it possible to recognize the user's emotions in real time and optimize the search conditions based on the emotion data. This realizes a personalized search experience that reflects the user's emotions, reduces the user's search burden, and enables more appropriate information to be provided more quickly.

[1666] A "user terminal" is a communication device operated by a user, and is a device for inputting search queries, recognizing emotion data, and displaying search results.

[1667] A "search query" is a query that a user inputs in the form of a string of characters or voice to express the information they want to know or search for.

[1668] "Natural language analysis" is a technology that allows a computer to understand and analyze the meaning of text written in human language.

[1669] "Search condition optimization" is a technique that adjusts conditions to efficiently and effectively search for related information based on a received search query.

[1670] A "database" is a system for storing structured data and searching and retrieving it as needed.

[1671] "Relevant data" is information extracted from a database based on a search query, and is useful data that matches the search intent.

[1672] "Ranking of search results" refers to the process of ranking search results based on evaluation criteria and determining the priority of the results to be presented to the user.

[1673] "Real-time emotion recognition" is a technology that instantly recognizes a user's current emotions from facial expressions, tone of voice, etc.

[1674] "Emotion data" is data that indicates the emotional state of the user recognized in real time.

[1675] "User interface" is a general term for the screens and input devices that allow a user to interact with a system.

[1676] MODE FOR CARRYING OUT THE INVENTION

[1677] A specific embodiment of the system according to the present invention will be described. This system allows users to search for information efficiently and accurately using a terminal, and further improves the search experience by recognizing the user's emotions during the search process. The basic configuration of this system is made up of a user terminal, a server, a database client, and an emotion engine.

[1678] Hardware and software used

[1679] User devices: PCs, smartphones, tablets, etc.

[1680] Server: Web server (Apache HTTP Server, Nginx, etc.), application server

[1681] Database client: Database management system such as MySQL, PostgreSQL, etc.

[1682] Emotion engine: Camera, microphone, emotion recognition software (OpenCV, Microsoft Azure Cognitive Services, etc.)

[1683] Generative AI models: GPT-3, BERT, etc. for natural language analysis

[1684] System basic configuration and operation

[1685] 1. Enter a query

[1686] A user enters a search query through an interface installed on the device (such as a browser-based web application or a mobile application), for example, entering a specific information request such as "latest natural language processing technology."

[1687] 2. Emotion recognition

[1688] When a user inputs a query, the device activates an emotion engine. The emotion engine uses sensors such as a camera and microphone to analyze the user's facial expressions and tone of voice, allowing it to recognize the user's emotions, such as "excited," "calm," or "irritated," in real time.

[1689] 3. Sending a request

[1690] The device converts the search query and the recognized emotion data into JSON format and sends it to the server as an HTTP POST request.

[1691] 4. Receiving and analyzing requests

[1692] The server receives the JSON-formatted request from the device. It then uses a generative AI model for natural language analysis (e.g., GPT-3 or BERT) to analyze the query, breaking down the query "latest natural language processing technology" into keywords such as "latest," "natural language processing," and "technology." It then analyzes the meaning and relevance of each keyword.

[1693] 5. Query Optimization

[1694] The server uses the user's past search history and behavioral data, as well as emotional data, to optimize search conditions. For example, if the server detects that the user is "excited," it will prioritize new technologies and innovative content.

[1695] 6. Database Search

[1696] The server searches the database through the database client using optimized search criteria, such as executing SQL queries to retrieve relevant articles, papers, videos, online courses, etc.

[1697] 7. Display of search results

[1698] The server evaluates the search results based on their reliability, freshness, and detail, assigns a score, compares them, and then ranks them based on the evaluation criteria. The ranked results are encoded in JSON format and sent to the device.

[1699] 8. User confirmation of results

[1700] The terminal receives the search results sent from the server and displays them on the user interface. The user can then review the displayed search results, select the most interesting article or paper, and view its details.

[1701] Specific examples

[1702] For example, if a user enters the search query "latest AI technology" and the emotion engine recognizes the user's emotion as "excited," the operation flow is as follows:

[1703] 1. User: Enters the query "latest AI technology."

[1704] 2. Device: The emotion engine recognizes that the user is excited based on their facial expressions and voice.

[1705] 3. Terminal: Convert the search query and sentiment data into JSON format and send it to the server as an HTTP POST request.

[1706] 4. Server: Receives the request and breaks down "latest AI technology" into the keywords "latest," "AI," and "technology."

[1707] 5. Server: Optimize queries to prioritize innovative content based on what users are "excited about."

[1708] 6. Server: Searches the database using optimized search criteria to retrieve relevant information.

[1709] 7. Server: Evaluates and ranks the acquired information.

[1710] 8. Server: Encode the ranked search results in JSON format and send them to the device.

[1711] 9. Device: Search results are displayed and the user checks the details.

[1712] Examples of usage prompt statements

[1713] Search for "latest AI technology" and describe a system that provides the most appropriate information to a user when the user is emotionally "excited."

[1714] As described above, this system takes user emotions into consideration to provide a more appropriate and personalized search experience.

[1715] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1716] Step 1:

[1717] Query Input

[1718] A user inputs a search query through the device interface. A specific example of what the user inputs is "the latest natural language processing technology." The input data is in text format.

[1719] Specific behavior:

[1720] The user enters a search query and clicks the submit button, and the device stores the entered text query in its internal memory.

[1721] Step 2:

[1722] emotion recognition

[1723] The device activates an emotion engine to recognize the user's emotions in real time. The device uses a camera and microphone to analyze the user's facial expressions and tone of voice to determine emotions.

[1724] input:

[1725] User's facial expression and voice data

[1726] Specific behavior:

[1727] The device inputs data acquired from the camera and microphone into emotion recognition software to determine the user's current emotion (e.g., "excited" or "calm"), and the analysis results are stored in internal memory.

[1728] output:

[1729] User emotion data (e.g., excited)

[1730] Step 3:

[1731] Send request

[1732] The device converts the search query and recognized emotion data into JSON format and sends it to the server as an HTTP POST request.

[1733] input:

[1734] Search query and user sentiment data

[1735] Specific behavior:

[1736] The device converts the search query and sentiment data into JSON format as key-value pairs, then generates an HTTP request and sends it to the specified URL on the server.

[1737] output:

[1738] JSON formatted HTTP POST request

[1739] Step 4:

[1740] Receiving and parsing requests

[1741] The server receives a JSON-formatted request from the device and analyzes the query using a generative AI model. For example, the query "latest natural language processing technology" is broken down into keywords such as "latest," "natural language processing," and "technology."

[1742] input:

[1743] Request data in JSON format

[1744] Specific behavior:

[1745] The server receives the request and feeds the query to the generative AI model, which analyzes the query and extracts important keywords, which are then stored in an internal database.

[1746] output:

[1747] Analyzed keyword data (e.g., "latest," "natural language processing," "technology")

[1748] Step 5:

[1749] Query optimization

[1750] The server optimizes the search criteria based on the analyzed keyword data, as well as the user's emotional data and past search history. For example, if the user is "excited," it will prioritize new technologies and innovative content.

[1751] input:

[1752] Analyzed keyword data, user sentiment data, and past search history

[1753] Specific behavior:

[1754] The server uses this data to run an optimization algorithm to determine the highest priority search terms, which results in an optimized search term.

[1755] output:

[1756] Optimized search criteria

[1757] Step 6:

[1758] Database search

[1759] The server searches the database using optimized search criteria and retrieves relevant information internally or through an external API.

[1760] input:

[1761] Optimized search criteria

[1762] Specific behavior:

[1763] The server issues SQL queries to the database to retrieve data that matches the search criteria, as well as external API calls.

[1764] output:

[1765] Retrieved search result data

[1766] Step 7:

[1767] Rating and ranking of search results

[1768] The search results obtained by the server are evaluated based on reliability, freshness, and detail, and are assigned a score to rank them.

[1769] input:

[1770] Retrieved search result data

[1771] Specific behavior:

[1772] The server evaluates each search result based on the evaluation criteria, assigns a score, and ranks the search results based on the score.

[1773] output:

[1774] Ranked search results data

[1775] Step 8:

[1776] Send results

[1777] The server encodes the ranked search results in JSON format and sends them to the device.

[1778] input:

[1779] Ranked search results data

[1780] Specific behavior:

[1781] The server encodes the search results in JSON format and sends them to the terminal as an HTTP response.

[1782] output:

[1783] JSON formatted HTTP response

[1784] Step 9:

[1785] Check the results

[1786] The terminal receives the search results sent from the server and displays them on the user interface, where the user can check the displayed search results and view their details.

[1787] input:

[1788] Search result data in JSON format

[1789] Specific behavior:

[1790] The device receives the search results, parses them, and displays them in a user interface that the user can navigate and click on results of interest to learn more.

[1791] output:

[1792] Search results displayed to users

[1793] (Application example 2)

[1794] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1795] Conventional food delivery services lack personalized meal suggestions because they do not provide a search experience that takes into account users' emotions and individual preferences. Another issue is the lack of functionality to analyze users' emotional data and optimize search criteria based on that data in order to provide optimal search results for users' search queries.

[1796] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for recognizing a user's emotion, means for optimizing search conditions based on the recognized emotion, and means for searching for related data from a database using the optimized search conditions. This enables a personalized search experience based on the user's emotion, and can provide more optimal search results.

[1797] A "user terminal" is a hardware device operated by a user, such as a smartphone or tablet.

[1798] A "search query" refers to the text or voice input a user enters to search for specific information.

[1799] "Means for recognizing emotions" refers to technology that analyzes a user's words and actions, facial expressions, tone of voice, etc. to determine their emotions at that time.

[1800] "Natural language analysis" is a technology for analyzing input text or speech and understanding its meaning.

[1801] "Means for optimizing search conditions" refers to technology that performs processing to make search results more accurate and useful based on the user's input query and emotional data.

[1802] A "database" is an organized collection of data that stores information responsive to a search query.

[1803] The "means for searching related data" is a technique for extracting related information in a database using optimized search criteria.

[1804] "Means for evaluating and ranking search results" refers to technology that evaluates the search results obtained based on criteria such as reliability and freshness, and ranks them in order of importance.

[1805] "Means for sending to user terminal" refers to the technology for transferring the evaluated and ranked search results to the user terminal.

[1806] "Past search history" is a record of search queries a user has previously performed.

[1807] "Behavioral data" refers to data relating to the behavior of a user, such as what operations the user performed and which pages the user viewed.

[1808] The system according to the present invention comprises a user terminal, a server, a database, and an emotion engine. The system program performs processing in the following steps: When a user inputs a search query using a food delivery app, the emotion recognition means analyzes the user's emotion in real time, and the query is sent to the server together with that data.

[1809] The server receives the user's search query and emotion data, and optimizes the search query based on the recognized emotion. The search criteria are further refined by referring to the user's past search history and behavioral data. Based on the optimized search criteria, relevant information is searched for from the database. The obtained search results are evaluated based on criteria such as reliability, freshness, and level of detail, and a ranking is assigned. The ranked search results are sent from the server to the user's device and displayed on the user interface.

[1810] Examples of hardware and software used include smartphones and tablets as user devices, emotion engines for recognizing emotions, natural language processing models for data analysis, and servers for searching and evaluating databases.

[1811] As a concrete example, consider the case where a user searches for "Italian restaurant." If the user's device uses an emotion engine to recognize the emotion "excited," the server uses that emotion data to optimize the search criteria to prioritize new restaurants and special menus. The server then searches the database based on the optimized search criteria to retrieve information about new restaurants and special menus. The information is then ranked based on criteria such as reliability and freshness and presented to the user.

[1812] An example prompt might be:

[1813] "When a user enters a search query in a food delivery app, analyze the data and recognize the user's emotions in real time. Design a system that optimizes the query based on the user's emotions and provides search results for relevant restaurants and menus. For example, when a user is excited, suggest new restaurants or special menus, and when a user is stressed, prioritize past favorite restaurants or menus that have given them high satisfaction."

[1814] This provides a personalized search experience that takes user emotions into account, which is expected to improve satisfaction with food delivery services.

[1815] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1816] Step 1:

[1817] A user opens a food delivery app and enters a search query. Input: The user's query (e.g., "Italian restaurant") and the user's facial image and voice data. Output: The device collects the query and emotion data. Specific operation: The user enters the query into an input box on the device interface, and the device's camera and microphone are used to collect the user's facial expressions and tone of voice in real time.

[1818] Step 2:

[1819] The device activates the emotion engine and recognizes the user's emotions in real time. Input: User's facial image and voice data. Output: Recognized emotion data (e.g., "Excited"). Specific operation: The emotion engine uses face recognition algorithms and voice analysis algorithms to analyze the user's facial muscle movements and voice tone.

[1820] Step 3:

[1821] The device sends the search query and recognized emotion data to the server. Input: Query and emotion data. Output: Data sent to the server. Specific operation: The device encodes the query and emotion data into JSON format and sends it to the server using an HTTP POST request.

[1822] Step 4:

[1823] The server analyzes the received query and sentiment data and performs natural language analysis on the search query. Input: Query and sentiment data. Output: Keywords from the analyzed query (e.g., "Italian," "restaurant"). What happens: A natural language processing (NLP) model on the server tokenizes the query and extracts key keywords.

[1824] Step 5:

[1825] The server references the user's past behavioral data and search history along with the recognized emotions to optimize the search criteria. Input: Analyzed keywords, emotion data, past behavioral data, and search history. Output: Optimized search criteria. Specific operation: The server retrieves the user's past history from the database and adjusts the search algorithm along with the emotion data.

[1826] Step 6:

[1827] The server searches the database for relevant information based on the search criteria. Input: Optimized search criteria. Output: Relevant information (e.g., a list of new Italian restaurants). Specific behavior: The server executes a database query to extract relevant restaurants and menus.

[1828] Step 7:

[1829] The server evaluates and ranks the search results. Input: Retrieved search results. Output: Rated and ranked results. Specific behavior: The server assigns a score to each result using evaluation criteria such as reliability, freshness, and specificity, and then ranks them.

[1830] Step 8:

[1831] The server sends the ranked search results to the user's terminal. Input: Ranked search results. Output: Search results sent to the user's terminal. Specific operation: The server encodes the ranked results in JSON format and sends them to the user's terminal as an HTTP response.

[1832] Step 9:

[1833] The user terminal receives the search results and displays them on the user interface. Input: Ranked search results. Output: Displayed search results. Specific operation: The application on the terminal analyzes the search results and displays them to the user.

[1834] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1835] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1836] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1837] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1838] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1839] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1840] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1841] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1842] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1843] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1844] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1845] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1846] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1847] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1848] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1849] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1850] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1851] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1852] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1853] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1854] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1855] The following is further disclosed regarding the above embodiment.

[1856] (Claim 1)

[1857] means for receiving a search query input from a user terminal;

[1858] means for performing natural language analysis based on the received search query;

[1859] A means for optimizing search conditions based on the analysis results;

[1860] means for retrieving relevant data from a database using optimized search criteria;

[1861] a means of evaluating and ranking search results;

[1862] means for transmitting the ranked search results to a user terminal;

[1863] A system including:

[1864] (Claim 2)

[1865] 10. The system of claim 1, further comprising means for using a user's past search history and behavioral data when optimizing a search query.

[1866] (Claim 3)

[1867] 10. The system of claim 1, further comprising means for considering reliability, freshness of information, and specificity as criteria for evaluating search results.

[1868] "Example 1"

[1869] (Claim 1)

[1870] means for receiving a search query input from a user terminal;

[1871] means for converting the received search query into a structured data format;

[1872] means for transmitting the converted search query;

[1873] means for performing natural language analysis based on the received search query;

[1874] A means for optimizing search conditions based on the analysis results;

[1875] means for retrieving relevant data from a data management system using optimized search criteria;

[1876] a means of evaluating and ranking search results;

[1877] means for transmitting the ranked search results to a user terminal;

[1878] means for displaying the received search results on a user terminal;

[1879] A system including:

[1880] (Claim 2)

[1881] 2. The system of claim 1, wherein the system uses a user's past search history and behavioral data when optimizing search conditions.

[1882] (Claim 3)

[1883] 10. The system of claim 1, further comprising means for considering reliability, freshness of information, and specificity as criteria for evaluating search results.

[1884] "Application Example 1"

[1885] (Claim 1)

[1886] means for receiving a search query input from a user terminal;

[1887] means for performing natural language analysis based on the received search query;

[1888] A means for optimizing search conditions based on the analysis results;

[1889] means for retrieving relevant data from a database using optimized search criteria;

[1890] a means of evaluating and ranking search results;

[1891] means for transmitting the ranked search results to a user terminal;

[1892] a means for the user terminal to provide search results including location information and reviews via a display device;

[1893] A system including:

[1894] (Claim 2)

[1895] 10. The system of claim 1, further comprising means for using a user's past search history and behavioral data when optimizing a search query.

[1896] (Claim 3)

[1897] 10. The system of claim 1, further comprising means for considering reliability, freshness of information, and specificity as criteria for evaluating search results.

[1898] "Example 2: Combining Emotion Engines"

[1899] (Claim 1)

[1900] means for receiving a search query input from a user terminal;

[1901] means for performing natural language analysis based on the received search query;

[1902] A means for optimizing search conditions based on the analysis results;

[1903] means for retrieving relevant data from a database using optimized search criteria;

[1904] a means of evaluating and ranking search results;

[1905] means for transmitting the ranked search results to a user terminal;

[1906] means for recognizing user emotions in real time;

[1907] A means of optimizing search conditions based on emotion data;

[1908] A system including:

[1909] (Claim 2)

[1910] 10. The system of claim 1, further comprising means for using a user's past search history and behavioral data when optimizing a search query.

[1911] (Claim 3)

[1912] 10. The system of claim 1, further comprising means for considering reliability, freshness of information, and specificity as criteria for evaluating search results.

[1913] "Application example 2 when combining emotion engines"

[1914] (Claim 1)

[1915] means for receiving a search query input from a user terminal;

[1916] means for recognizing a user's emotion;

[1917] means for performing natural language analysis based on the received search query and the recognized sentiment;

[1918] A means for optimizing search conditions based on the analysis results and sentiment data;

[1919] means for retrieving relevant data from a database using optimized search criteria;

[1920] a means of evaluating and ranking search results;

[1921] means for transmitting the ranked search results to a user terminal;

[1922] A system including:

[1923] (Claim 2)

[1924] 10. The system of claim 1, further comprising means for using a user's past search history, behavioral data, and recognized emotion data when optimizing a search query.

[1925] (Claim 3)

[1926] 10. The system of claim 1, further comprising means for considering reliability, freshness of information, and specificity as criteria for evaluating search results. [Explanation of symbols]

[1927] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving a search query input from a user terminal; means for performing natural language analysis based on the received search query; A means for optimizing search conditions based on the analysis results; means for retrieving relevant data from a database using optimized search criteria; a means of evaluating and ranking search results; means for transmitting the ranked search results to a user terminal; A system including:

2. 10. The system of claim 1, further comprising means for using a user's past search history and behavioral data when optimizing a search query.

3. 10. The system of claim 1, further comprising means for considering reliability, freshness of information, and level of detail as criteria for evaluating search results.

Images