System

Abstract

A system is provided.SOLUTION: A system comprising: means for obtaining information from a plurality of means for communicating information; means for converting the information obtained from the means for communicating information into a uniform format; means for storing the converted information in a database; means for generating an index for the stored information; means for receiving a search query; means for searching the database based on the received search query; and means for formatting and displaying search results.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's business environment, a wide variety of information is exchanged using e-mail, chat tools, and other communication tools. However, because this information is dispersed, it is difficult for users to efficiently search for the information they need later. This poses a problem: users must spend a lot of time searching each tool individually, resulting in reduced work efficiency. The present invention aims to solve this problem by providing a system that can search and retrieve information from multiple information transmission means all at once. [Means for solving the problem]

[0005] The present invention provides a system including means for acquiring information from multiple information transmission means, means for converting the acquired information into a unified format, means for storing the converted information in a database, means for generating an index for the stored information, means for receiving a search query, means for searching the database based on the received search query, and means for formatting and displaying search results. This system allows users to search for information from multiple information sources all at once using a single search window, thereby significantly reducing the effort required for searching and improving work efficiency.

[0006] "Information transmission means" refers to tools and systems for communicating and sharing information between users, including email tools, chat tools, and other communication tools.

[0007] "Means of acquiring information" refers to the processes and functions for collecting data from information transmission means and incorporating it into the system.

[0008] A "uniform format" refers to a format for converting and expressing information of different formats into a consistent format, thereby maintaining data consistency and efficient search.

[0009] "Means of storing in a database" refers to the process or function for storing acquired information in a storage or data management system.

[0010] "Means for generating indexes" refers to the ability to create indexes based on specific keys or fields in order to efficiently search for information in a database.

[0011] "Means for receiving search queries" refers to the process or function for sending search conditions or keywords entered by a user to a server, etc., and receiving them.

[0012] "Means for searching a database" refers to the function of investigating information in a database and extracting relevant data based on a received search query.

[0013] "Search result formatting" refers to the process or function that formats search results extracted from a database in a form that is easy for a user to understand.

[0014] "Means for displaying search results" refers to the process or functionality for visually displaying the formatted search results on the user's device. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a system that searches information acquired from multiple information transmission means in one go and efficiently displays the results. This system is designed to enable users to quickly search and acquire information, improving work efficiency. The system's program processing is explained in detail below.

[0037] A natural language description of the program's operation

[0038] Server-side processing

[0039] First, the server obtains information from multiple information transmission means. Specific information transmission means include email tools, chat tools, and other communication tools. The server periodically obtains the necessary data using the APIs of these tools. For example, it obtains unread emails from email tools and the latest messages from chat tools.

[0040] Next, the acquired information is converted into a unified format. This is to unify data of different formats into a consistent format. For example, email data is converted into fields such as "subject," "body," "sender," and "date."

[0041] The converted data is stored in a database, along with metadata such as the user ID and tool type.

[0042] It generates an index for the information stored in the database, which can be quickly searched based on date, sender, content keywords, etc.

[0043] When a user enters a search query on a terminal, the query is sent to the server, which searches the database based on the received search query and retrieves a list of relevant messages.

[0044] Finally, the search results are formatted and displayed in the format of the date, hit content, sender name, information transmission means name, etc. The formatted search results are sent to the terminal and displayed on the screen.

[0045] Terminal side processing

[0046] The user enters a search keyword into the search field on the device, for example, "project progress."

[0047] When the search button is pressed, the entered search query is sent to the server.

[0048] The search results received from the server are displayed on one screen, including the date, hit content, sender name, and information transmission method name.

[0049] Specific examples

[0050] For example, if a user searches for "project progress," the following happens:

[0051] 1. User Action:

[0052] The user enters "project progress" into the search field on the terminal and clicks the search button.

[0053] 2. Request from device to server:

[0054] The terminal sends a search query containing the keyword "project progress" to the server.

[0055] 3. Server lookup process:

[0056] The server searches the database for messages that correspond to "Project Progress."

[0057] The hit messages are formatted and search results are generated.

[0058] 4. Sending from the server to the device:

[0059] Send the formatted search results to the device.

[0060] 5. Terminal display processing:

[0061] The terminal displays the received search results in a list format.

[0062] for example:

[0063] Date: 2023-10-01

[0064] Hit: "Project progress is going well"

[0065] Sender name: Yamada Taro

[0066] Name of information transmission method: Email tool

[0067] Date: 2023-09-25

[0068] Hits: "I'll report on the project progress at the next meeting."

[0069] Sender name: Hanako Sato

[0070] Name of information transmission method: Chat tool

[0071] In this way, the system of the present invention can improve work efficiency by enabling users to search and acquire necessary information from multiple information sources all at once.

[0072] The processing flow will be explained below.

[0073] Step 1: Collect data

[0074] 1. The server periodically connects to the APIs of email, chat, and other communication tools.

[0075] 2. Use API authentication information (API key, authentication token, etc.) to retrieve user data from each tool.

[0076] 3. For example, get unread emails from an email tool, or get the latest chat messages from a chat tool.

[0077] Step 2: Transform the data

[0078] 1. The server converts the acquired data into a unified format.

[0079] 2. For example, email data is converted into fields such as "Subject," "Body," "Sender," and "Date," while chat messages are converted into a format such as "Message Content," "Sender," and "Date."

[0080] Step 3: Save your data

[0081] 1. The server saves the converted data in the database.

[0082] 2. When saving, add metadata such as the user ID and tool type.

[0083] Step 4: Generate indexes

[0084] 1. The server generates an index for the stored data.

[0085] 2. Indexes are created based on fields such as date, sender, and content keywords.

[0086] 3. The index is reflected in the database, enabling fast searches.

[0087] Step 5: Receiving a search query

[0088] 1. The user enters a search keyword (e.g., "project progress") into the search field on the device.

[0089] 2. When you press the search button, the entered search query is sent from the device to the server.

[0090] Step 6: Search the database

[0091] 1. The server searches the database index based on the received search query.

[0092] 2. Get a list of matching messages as search results.

[0093] Step 7: Formatting the search results

[0094] 1. The server formats the search results.

[0095] 2. Search results are organized by date, hit content, sender name, information transmission method name, etc.

[0096] 3. Send the formatted search results to the device.

[0097] Step 8: Viewing search results

[0098] 1. The terminal receives the search results from the server.

[0099] 2. Display search results to the user on one screen.

[0100] 3. The results are displayed in the form of, for example, date, hit content, sender name, and information transmission method name.

[0101] In this way, users can search for the information they need from multiple sources in one search window, enabling them to acquire information efficiently.

[0102] Example 1

[0103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0104] Conventional information retrieval systems have had difficulty efficiently managing data from multiple information transmission methods and quickly searching and retrieving it. They also lacked sufficient functionality to convert data of different formats into a unified format and generate indexes for high-speed searches. Furthermore, they lacked the ability to input prompt statements into generative AI models to perform more advanced data retrieval processing.

[0105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0106] In this invention, the server includes means for acquiring information from a plurality of information transmission means, means for converting the information acquired from the information transmission means into a unified format, means for storing the converted information in a database, means for generating an index for the stored information, means for receiving a search query, means for searching the database based on the received search query, means for formatting and displaying search results, and means for inputting a prompt statement to the generative AI model, thereby enabling efficient search and acquisition of necessary information from a plurality of information sources in a batch.

[0107] "Information vehicles" refers to tools for communicating and sharing information between users, including electronic messaging tools, dialogue tools, and other communication tools.

[0108] "Unified format" refers to converting data of different formats into a specific, consistent format, thereby improving data integrity and search efficiency.

[0109] A "database" refers to an organized collection of information that can be used to store, manage, and retrieve data efficiently.

[0110] "Index" refers to a structure used to quickly search for information stored in a database, created based on specific fields (e.g., date and time, source identification, content keywords).

[0111] A "search query" refers to a keyword or phrase that a user enters to identify the information they want to retrieve, and which defines the conditions for searching a database.

[0112] A "generative AI model" refers to a pre-trained artificial intelligence algorithm that is capable of performing advanced data processing and search based on input prompts.

[0113] A "prompt" is a piece of text containing instructions or questions that are used as input to a generative AI model, leading to an appropriate response or action from the AI.

[0114] "Formatted display" refers to converting the search results obtained into an easy-to-read format and presenting the content in a visually easy-to-understand manner.

[0115] "Sender identification information" refers to information for identifying the sender of the message, and specifically includes the name and ID.

[0116] The present invention is a system that searches information acquired from multiple information transmission means in one go and efficiently displays the results. This system is designed to enable users to quickly search and acquire information and improve work efficiency. Specific embodiments of the system are described below.

[0117] Server-side configuration and operation

[0118] The server obtains information from multiple information transmission means (electronic messaging tools, dialogue tools, and other communication tools). Specific information transmission means include email tools (e.g., Gmail API) and chat tools (e.g., Slack API). The server periodically obtains data using the APIs of these tools. For example, the server obtains unread emails using the Gmail API and the latest messages using the Slack API.

[0119] The acquired information is converted into a unified format. This is to unify data of different formats into a consistent format. For example, email data acquired from Gmail is converted into fields of "Subject", "Body", "Sender", and "Date", and Slack message data is converted into fields of "Channel Name", "Body", "Sender", and "Date".

[0120] The converted data is stored in a database, such as MySQL or MongoDB, along with metadata such as the user ID and the name of the information transmission method.

[0121] The information stored in the database is indexed using Elasticsearch, which allows for fast searches based on date, sender, content keywords, etc. For example, you can quickly search for relevant data based on keywords such as "project progress."

[0122] When a user enters a search query from a terminal, the query is sent to the server. The server searches the database based on the received search query and retrieves a list of relevant messages. The search results are formatted and formatted to include the date, hit content, sender name, and information transmission method name. The formatted search results are sent to the terminal and displayed on the screen.

[0123] It also has a prompt input function that uses the generative AI model. For example, a prompt such as "Please search for the latest messages regarding project progress" can be input into the generative AI model to perform advanced data search processing.

[0124] Terminal configuration and operation

[0125] A user enters a search keyword into the search field on their device. For example, they enter "project progress." When the user presses the search button, the entered search query is sent to the server. The search query is sent to the server's API endpoint using an HTTPS request.

[0126] The search results received from the server are displayed on one screen on the terminal. The search result list is rendered in HTML format, and each item displays the date, hit content (excerpt of the text), sender identification information, and information transmission means identification information.

[0127] Specific examples of operation

[0128] For example, if a user searches for "project progress," the following happens:

[0129] 1. User Action:

[0130] The user enters "project progress" into the search field on the terminal and clicks the search button.

[0131] 2. Request from device to server:

[0132] The terminal sends a search query containing the keyword "project progress" to the server.

[0133] 3. Server lookup process:

[0134] The server uses Elasticsearch to search the database for messages that correspond to "project progress."

[0135] 4. Sending from the server to the device:

[0136] Send the formatted search results to the device.

[0137] 5. Terminal display processing:

[0138] The terminal displays the received search results in a list format.

[0139] In this way, the system of the present invention is capable of efficiently searching and acquiring necessary information from multiple information sources all at once.

[0140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0141] Step 1: Information Acquisition

[0142] The server obtains data from multiple information transmission means. Specific information transmission means include email tools (e.g., Gmail API) and chat tools (e.g., Slack API). When using the Gmail API to obtain unread emails, the "messages.list" endpoint is used to obtain the unread message IDs, and the "messages.get" endpoint is used to obtain the contents of each message. When using the Slack API to obtain the latest messages, the "conversations.history" endpoint is used to obtain the list of latest messages.

[0143] Input: API endpoint of the communication medium

[0144] Data processing and calculation: Acquire data in XML or JSON format by calling the API

[0145] Output: Raw data obtained (unread email list, chat messages)

[0146] Step 2: Format conversion

[0147] The server converts the information it receives into a unified format. Email data from Gmail is converted into fields such as "Subject," "Body," "Sender," and "Date," while Slack messages are converted into fields such as "Channel Name," "Body," "Sender," and "Date."

[0148] Input: Raw data (unread email list, chat messages)

[0149] Data processing and calculation: JSON parsing and field conversion operations

[0150] Output: Uniformly formatted data

[0151] Step 3: Save to database

[0152] The server stores the converted data in a database, such as MySQL or MongoDB, and can also include metadata such as the user ID and the name of the information transmission method.

[0153] Input: Uniformly formatted data

[0154] Data processing and calculation: SQL insert operation or NoSQL data insert operation

[0155] Output: Data entries stored in the database

[0156] Step 4: Index generation

[0157] The server creates an index of the stored information, using Elasticsearch to create indexes based on date, sender, content keywords, etc. This allows for fast searches.

[0158] Input: Data entry stored in a database

[0159] Data processing and calculation: Index generation operation

[0160] Output: Elasticsearch index

[0161] Step 5: Receiving a search query

[0162] The server receives a search query from the user. When the user enters a keyword into the search field of the terminal and presses the search button, the search query is sent from the terminal.

[0163] Input: User search query (e.g. "project progress")

[0164] Data processing and calculation: HTTPS request parsing

[0165] Output: Parsed search query

[0166] Step 6: Database Search

[0167] The server searches the database based on the received search query, using Elasticsearch to quickly find messages that match "Project Progress."

[0168] Input: Parsed search query

[0169] Data processing and calculation: Executing Elasticsearch queries

[0170] Output: Search results (list of matching messages)

[0171] Step 7: Formatting search results

[0172] The server formats the search results and converts them into a displayable format, including each field of the email or chat message included in the search results (date and time, excerpt of the message body, sender identification information, and information transmission method name).

[0173] Input: Search results (list of matching messages)

[0174] Data processing and calculation: Result formatting operations

[0175] Output: Formatted search results

[0176] Step 8: Submit search results

[0177] The server sends the formatted search results to the terminal, which displays the formatted results on the terminal screen.

[0178] Input: Formatted search results

[0179] Data processing and calculation: HTTP response generation

[0180] Output: Search results sent to your device

[0181] Step 9: Displaying search results

[0182] The terminal displays the search results received from the server in a list format. Using an HTML template, the search results are displayed with the date and time, hit content, sender identification information, and information transmission method name.

[0183] Input: Search results received from the server

[0184] Data processing and calculation: HTML rendering

[0185] Output: Search results displayed on the screen

[0186] Step 10: Enter a prompt into the generative AI model

[0187] The user inputs a prompt into the generative AI model, such as "Please search for the latest messages about project progress," and the AI ​​performs advanced data search processing.

[0188] Input: Prompt from the user

[0189] Data processing and calculation: Analysis and search operations using generative AI models

[0190] Output: AI-generated search results

[0191] (Application example 1)

[0192] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0193] In modern factories, data from various sources is scattered, making it difficult to efficiently collect and manage it and quickly search and retrieve specific information. Furthermore, in complex manufacturing processes, when an abnormality occurs, the person in charge must quickly identify the cause and take measures, but the dispersion of information makes this task cumbersome. For this reason, there is a need for a system that can quickly integrate data obtained from various sources within the factory and efficiently search and reference it.

[0194] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0195] In this invention, the server includes: means for acquiring information from a plurality of information transmission means; means for converting the information acquired from the information transmission means into a unified format; means for storing the converted information in a database; means for generating an index for the stored information; means for receiving a search query; means for searching the database based on the received search query; means for formatting and displaying search results; and means, which is implemented in a factory robot, for collecting, formatting, and displaying data from each information source in the factory. This makes it possible to collectively collect, and quickly search and acquire, necessary information from a plurality of data sources in the factory.

[0196] "Information transmission means" refers to means for exchanging data and information, and specifically includes email tools, chat tools, manufacturing data collection tools, and other communication tools.

[0197] "Harmonized format" refers to standards and formats for converting data captured in different formats into a consistent format.

[0198] A "database" refers to a system in which data is systematically organized and stored for retrieval and storage.

[0199] An "index" refers to a data structure created to quickly search data and is used to optimize search speed.

[0200] "Computer robot" refers to a mechanical device that operates automatically and is used to perform tasks, especially in a factory.

[0201] A "search query" refers to an instruction or request sent to a database or information system to retrieve specific information.

[0202] "Search Results" refers to a list of information or data retrieved based on a search query.

[0203] The "formatting and displaying means" refers to a method or means for converting search results into a format that is easy for the user to understand and displaying them.

[0204] "Data collection methods" refers to the methods and means used to obtain the required data from the source of interest.

[0205] This invention provides a system that is implemented in a factory robot and that collectively collects necessary information from multiple information sources within the factory, and efficiently searches and acquires the information. A specific embodiment of this system is shown below.

[0206] System Overview

[0207] The system includes the following major components:

[0208] 1. Server

[0209] 2. Factory robots

[0210] 3. Database

[0211] 4. User Interface

[0212] server

[0213] The server is responsible for collecting information from multiple sources within the factory, converting it into a unified format, and storing it in a database. It also receives search queries from users, searches the database, formats the results, and sends them to factory robots and user devices.

[0214] The server uses the following hardware and software:

[0215] Hardware: High-performance server (e.g., server with Intel Xeon processor)

[0216] Software: REST API module, MySQL database, data conversion scripts (e.g. Python scripts)

[0217] Factory robots

[0218] Factory robots work in conjunction with a server to collect, format, and display data from various sources within the factory. The robots periodically collect information using APIs and send the necessary data to the server.

[0219] Factory robots use the following hardware and software:

[0220] Hardware: Industrial robots (e.g. industrial robots)

[0221] Software: Robot control software (e.g. ROS)

[0222] Database

[0223] The database stores the data sent from the server in a unified format and generates an index to enable fast searches based on date, source, content keywords, etc.

[0224] The database uses the following software:

[0225] Software: MySQL database

[0226] User Interface

[0227] The user interface is used to input search queries and display search results. Users can search for the information they need through the interface and view the results in a list format.

[0228] Specific examples

[0229] For example, if a user searches for quality control data for products in a factory, the process would go something like this:

[0230] 1. User operation: The user enters "poor quality" in the search field of the terminal and clicks the search button.

[0231] 2. Request from device to server: The device sends a search query containing the keyword “poor quality” to the server.

[0232] 3. Server search process: The server searches the database for data that corresponds to "poor quality." It formats the hit data and generates search results.

[0233] 4. Transmission from server to terminal: The formatted search results are transmitted to the terminal.

[0234] 5. Display processing on the terminal: The terminal displays the received search results in a list format, including the date, product ID, measurement value, and reference value.

[0235] Prompt Sentence Examples

[0236] Using a generative AI model, you can generate a program to build a system that collects relevant data and efficiently searches and displays it by entering a prompt like the following:

[0237] Please create a Python program that saves the quality data obtained from the API below in a MySQL database, and searches for and displays data that meets a specific condition (e.g., measurement value greater than or equal to 10).

[0238] API URL: https: / / api.example.com / quality_data

[0239] This makes it possible to collect necessary information from multiple data sources within the factory in one go and efficiently search and retrieve it.

[0240] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0241] Step 1: Gather information

[0242] The server obtains information from various sources within the factory (email tools, chat tools, manufacturing data collection tools, etc.). These tools send data to the server through their respective APIs. The input is data from each source, and the output is the raw data collected by the server. Specifically, it periodically sends requests to the API endpoint and receives response data.

[0243] Step 2: Data conversion

[0244] The server converts the acquired information into a unified format. To convert data of different formats into a consistent format, for example, each data item is classified into fields such as "product ID," "quality standard," "measurement value," and "date." The input is the acquired raw data, and the output is data converted into a unified format. Specifically, the server uses a data conversion script (e.g., a Python script) to analyze the data and extract and convert the required information.

[0245] Step 3: Save to database

[0246] The server saves the data converted into a unified format in a database. The input is the data converted into a unified format, and the output is the data stored in the database. Specifically, it connects to a database management system (e.g., MySQL), executes an SQL query, and saves the data.

[0247] Step 4: Indexing

[0248] The server creates an index for the stored data. The index is created based on date, source, content keywords, etc., allowing for fast searches. The input is the data in the database, and the output is the indexed database. Specifically, it runs SQL indexing queries and indexes the appropriate fields.

[0249] Step 5: Search query reception

[0250] The terminal receives a search query from a user. The input is a search keyword entered by the user, and the output is a search query sent to the server. Specifically, the terminal generates a query by entering keywords through a user interface and clicking a search button.

[0251] Step 6: Data Retrieval

[0252] The server searches the database based on the received search query. The input is the received search query, and the output is a list of matching search results. Specifically, it executes the SQL search query and extracts data that matches the criteria.

[0253] Step 7: Formatting search results

[0254] The server formats the search results and converts them into a display format. The input is the raw search results, and the output is the formatted data for display. Specifically, the server organizes the search results into fields such as "date," "product ID," "measurement value," and "reference value," and formats them in a format that is easy for users to view.

[0255] Step 8: View the results

[0256] The terminal displays the formatted search results to the user. The input is the formatted search results received from the server, and the output is the result list displayed to the user. Specifically, the terminal uses widgets and components for displaying results in a user interface to display the search results on the screen.

[0257] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0258] This invention provides a user-friendly search experience by combining a system that searches information acquired from multiple information transmission means in one go and efficiently displays the results with an emotion engine that recognizes the user's emotions. This system enables users to quickly search and acquire information and provides more appropriate information according to their emotions at the time.

[0259] A natural language description of the program's operation

[0260] Server-side processing

[0261] First, the server obtains information from multiple information transmission means. Specific information transmission means include email tools, chat tools, and other communication tools. The server periodically obtains the necessary data using the APIs of these tools. For example, it obtains unread emails from email tools and the latest messages from chat tools.

[0262] Next, the acquired information is converted into a unified format. This is to unify data of different formats into a consistent format. For example, email data is converted into fields for "subject," "body," "sender," and "date."

[0263] The converted data is stored in a database, along with metadata such as the user ID and tool type.

[0264] It generates an index for the information stored in the database, which can be quickly searched based on date, sender, content keywords, etc.

[0265] When a user enters a search query from their device, the query is sent to the server. The server searches the database based on the received search query and retrieves a list of relevant messages. The server's emotion engine recognizes the user's emotions based on the user's input, operation history, voice input, and data obtained from biometric sensors. Search results are filtered or prioritized based on this recognized emotion.

[0266] Finally, the search results are formatted and displayed in the format of the date, hit content, sender name, information transmission means name, etc. The formatted search results are sent to the terminal and displayed on the screen.

[0267] Terminal side processing

[0268] The user enters a search keyword into the search field on the device, for example, "project progress."

[0269] When the search button is pressed, the entered search query is sent to the server.

[0270] The search results received from the server are displayed on one screen, including the date, hit content, sender name, and information transmission method name.

[0271] Specific examples

[0272] For example, if a user searches for "project progress," the following happens:

[0273] 1. User Action:

[0274] The user enters "project progress" into the search field on the terminal and clicks the search button.

[0275] 2. Request from device to server:

[0276] The terminal sends a search query containing the keyword "project progress" to the server.

[0277] 3. Server lookup process:

[0278] The server searches the database for messages that correspond to "Project Progress."

[0279] The emotion engine determines the user's emotion and filters or prioritizes search results based on that emotion.

[0280] The hit messages are formatted and search results are generated.

[0281] 4. Sending from the server to the device:

[0282] Send the formatted search results to the device.

[0283] 5. Terminal display processing:

[0284] The terminal displays the received search results in a list format.

[0285] for example:

[0286] Date: 2023-10-01

[0287] Hit: "Project progress is going well"

[0288] Sender name: Yamada Taro

[0289] Name of information transmission method: Email tool

[0290] Date: 2023-09-25

[0291] Hits: "I'll report on the project progress at the next meeting."

[0292] Sender name: Hanako Sato

[0293] Name of information transmission method: Chat tool

[0294] In this way, the system of the present invention allows users to search for the information they need from multiple information sources at once, and provides more appropriate information that reflects their emotions at the time, thereby improving work efficiency.

[0295] The processing flow will be explained below.

[0296] Step 1: Collect data

[0297] 1. The server periodically connects to the APIs of email, chat, and other communication tools.

[0298] 2. Use each tool's API credentials (API key, authentication token, etc.) to retrieve user data.

[0299] 3. For example, get unread emails from an email tool, or get the latest chat messages from a chat tool.

[0300] Step 2: Transform the data

[0301] 1. The server converts the acquired data into a unified format.

[0302] 2. For example, convert email data into fields of "Subject", "Body", "Sender", and "Date", and convert chat messages into the format of "Message Content", "Sender", and "Date".

[0303] Step 3: Save your data

[0304] 1. The server saves the converted data in the database.

[0305] 2. When saving, metadata such as user ID and tool type are also saved.

[0306] Step 4: Generate indexes

[0307] 1. The server generates an index for the stored data.

[0308] 2. Indexes are created based on fields such as date, sender, and content keywords to enable fast searches.

[0309] Step 5: Emotion Recognition with the Emotion Engine

[0310] 1. The server's emotion engine collects user input, operation history, voice input, and data from biometric sensors.

[0311] 2. Determine the user's emotions based on the collected data.

[0312] 3. For example, if a user is feeling stressed, generate search results that reflect that.

[0313] Step 6: Receiving a search query

[0314] 1. The user enters a search keyword (e.g., "project progress") into the search field on the device.

[0315] 2. When you press the search button, the entered search query is sent from the device to the server.

[0316] Step 7: Search the database

[0317] 1. The server searches the database index based on the received search query.

[0318] 2. As a search result, a list of relevant messages is obtained.

[0319] 3. Filter or prioritize search results based on user sentiment as recognized by the sentiment engine.

[0320] Step 8: Formatting the search results

[0321] 1. The server formats the search results.

[0322] 2. Search results are organized by date, hit content, sender name, information transmission method name, etc.

[0323] 3. Send the formatted search results to the device.

[0324] Step 9: Viewing search results

[0325] 1. The terminal displays the search results received from the server.

[0326] 2. Search results are displayed on one screen, and each item includes the date, hit content, sender name, and information transmission method name.

[0327] In this way, by using the system of the present invention, users can not only search and obtain information from multiple information sources all at once using a single search box, but also use the emotion engine to obtain the most appropriate information based on their emotions at the time.

[0328] Example 2

[0329] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0330] In today's information society, users use multiple communication methods to obtain a wide variety of information. However, this information is often provided in different formats, making it difficult for users to quickly search and obtain the information they need. In addition, information is not provided in a way that reflects the user's emotions, resulting in a poor user experience.

[0331] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0332] In this invention, the server includes means for acquiring information from a plurality of information transmission means, means for converting the information acquired from the information transmission means into a unified format, means for storing the converted information in a database, means for generating an index for the stored information, means for receiving a search query, means for searching the database based on the received search query, means for formatting and displaying search results, and means for recognizing a user's emotion and filtering or preferentially displaying search results based on the emotion, thereby enabling a user to search for necessary information from a plurality of information sources at once and providing more appropriate information according to the user's emotion at that time.

[0333] "Information transmission means" refers to email tools, chat tools, and other communication tools.

[0334] A "uniform format" is a data structure for converting information obtained from different information transmission means into a consistent format.

[0335] A "database" is a structured collection of data that stores transformed information and allows it to be efficiently searched and retrieved.

[0336] An "index" is a data structure used to quickly search for information in a database.

[0337] A "search query" is data containing keywords or phrases that a user enters to specify the information they want to obtain.

[0338] "Search results" are a collection of relevant information retrieved from a database based on a search query.

[0339] An "emotion engine" is a system for recognizing a user's emotions and filtering or prioritizing search results based on those emotions.

[0340] "Filtering" is a process of extracting only necessary information from data acquired based on specific criteria.

[0341] The "display means" is a system for visually presenting search results to the user.

[0342] "Metadata" is additional information that includes information about the content of the data (for example, user ID, type of tool, etc.).

[0343] The present invention is a system that allows users to quickly search for the information they need from multiple information sources and provides appropriate information according to their emotions at the time. This system is realized by the server and the terminals each fulfilling their respective roles.

[0344] First, the server obtains information from multiple information transmission means. Specific information transmission means include email tools (e.g., Gmail API), chat tools (e.g., Slack API), and other communication tools. The server periodically obtains the necessary data using each of these APIs. For example, it obtains unread emails from the email tool and the latest messages from the chat tool.

[0345] The server then converts the information into a unified format, allowing different data types to be unified into a consistent format. For example, email data is converted into fields for "subject," "body," "sender," and "date."

[0346] The converted data is stored in a database, including metadata such as the user ID and tool type. The server then creates an index for the information stored in the database. This index is created based on date, sender, content keywords, etc., allowing for fast searches.

[0347] When a user enters a search query into the device's search field, the query is sent to the server. The server searches the database based on the received search query and retrieves a list of relevant messages. The server's emotion engine then recognizes the user's emotions based on the user's input, operation history, voice input, and data obtained from biometric sensors. The server then filters or prioritizes search results based on the recognized emotions.

[0348] Finally, the server formats and displays the search results in the format of the date, hit content, sender name, information transmission means name, etc. The formatted search results are sent to the terminal and displayed on the user's screen.

[0349] Specific examples

[0350] For example, if a user searches for "project progress," the following happens:

[0351] The user enters "project progress" into the search field of the device and clicks the search button. The device sends a search query containing the keyword "project progress" to the server. The server searches the database for messages that match "project progress." The emotion engine determines the user's emotion and filters or prioritizes the search results based on that emotion. The matched messages are formatted and search results are generated. The formatted search results are sent to the device, and the device displays the received search results in a list format.

[0352] Prompt Sentence Examples

[0353] "Describe a system that searches for project status using multiple sources of information and filters the results based on user sentiment."

[0354] In this way, users can search for related information all at once and get a better search experience based on their current emotions.

[0355] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0356] Step 1:

[0357] The server obtains information from multiple information transmission means.

[0358] Input: API endpoint of each tool

[0359] Specific behavior: The server uses the Gmail API to retrieve unread emails and the Slack API to retrieve the latest messages.

[0360] Output: Data from each tool (e.g., email subject, body, sender, date)

[0361] Step 2:

[0362] The server converts the acquired information into a unified format.

[0363] Input: Data for each tool acquired

[0364] Specific operation: Map the data fields obtained from each API to "Subject," "Body," "Sender," and "Date."

[0365] Output: Data converted into a unified format

[0366] Step 3:

[0367] The server stores the converted information in a database.

[0368] Input: Data converted into a unified format

[0369] What it does: Inserts data into a MongoDB collection, along with metadata such as user ID and tool type.

[0370] Output: Information stored in the database

[0371] Step 4:

[0372] The server generates an index for the stored information.

[0373] Input: Information stored in a database

[0374] What it does: Uses MongoDB's indexing capabilities to generate indexes based on date, sender, and keywords.

[0375] Output: An index that allows for fast searches

[0376] Step 5:

[0377] A user enters a search query into a search field on the device.

[0378] Input: User's search keyword (e.g. "project progress")

[0379] Specific operation: The user enters keywords into the search field on the device and presses the search button.

[0380] Output: The search query is sent to the server

[0381] Step 6:

[0382] The server searches the database based on the received search query.

[0383] Input: Search query sent from the device

[0384] Specific operation: The server sends a query containing the received search keywords to MongoDB and retrieves the relevant data.

[0385] Output: A list of matching messages

[0386] Step 7:

[0387] The emotion engine of the server recognizes the emotion of the user.

[0388] Input: User operation history, voice input, biometric sensor data

[0389] Specific operation: The emotion engine on the server analyzes these data and estimates the user's current emotion.

[0390] Output: Recognized emotion

[0391] Step 8:

[0392] The server filters or prioritizes search results based on the recognized sentiment.

[0393] Input: List of relevant messages, recognized emotions

[0394] Specific behavior: For users who are stressed, prioritize relevant and important messages.

[0395] Output: Filtered or prioritized search results

[0396] Step 9:

[0397] The server formats the search results and sends them to the terminal.

[0398] Input: Filtered or prioritized search results

[0399] Specific operation: The search results are formatted in the format of "date," "hit content," "sender name," and "information transmission method name," and the results are sent to the terminal in JSON format.

[0400] Output: Formatted search results sent to your device

[0401] Step 10:

[0402] The terminal displays the received search results in a list format.

[0403] Input: Formatted search results received from the server

[0404] Specific operation: The device renders the search results in HTML and displays them in a list format on the screen.

[0405] Output: Search results presented visually to the user

[0406] (Application example 2)

[0407] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0408] Conventional information acquisition systems have had difficulty in quickly and accurately providing the information users want. Furthermore, they have not provided information that corresponds to the user's emotional state, which has prevented them from improving the user experience. In particular, virtual stores need to provide optimal products and information depending on the user's emotional state, and technology to achieve this has been in demand.

[0409] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring information from multiple information transmission means, means for converting the information acquired from the information transmission means into a unified format, means for saving the converted information in a database, means for generating an index for the saved information, means for receiving a search query, means for searching the database based on the received search query, means for acquiring and analyzing user emotion data, means for filtering or preferentially displaying search results based on the user emotion data, and means for shaping and displaying the search results. This makes it possible to provide optimal product information according to the user's emotions in a virtual store, significantly improving the user experience.

[0410] "Multiple information transmission means" refers to means for sending and receiving information in different formats, such as email tools, chat tools, and other communication tools.

[0411] A "unified format" is a data format for converting data obtained from different information sources into a unified format.

[0412] "Database" means an information management system for storing, managing, and retrieving acquired and transformed information.

[0413] An "index" is a structured index that allows fast and efficient searching of information in a database.

[0414] A "search query" is a keyword or phrase entered by a user to identify the information they are seeking.

[0415] "Emotion data" is data indicating the user's emotional state obtained from facial expressions, voice, operation history, and the like.

[0416] "Filtering" is the process of selecting data based on specific criteria.

[0417] "Priority display" is a process of displaying data in the foreground based on specific criteria.

[0418] "Search Results" are information retrieved from a database based on a search query.

[0419] "Formatting and displaying" is a process of converting search results into an easy-to-read format and displaying them on the user's terminal.

[0420] This invention relates to a system for realizing efficient product searches in a virtual store. Specifically, it describes a method for recognizing a user's emotions and providing optimal search results based on those emotions. Below, we will explain in detail each component of this system and the processing it performs.

[0421] First, the server obtains information from multiple communication tools. This includes email, chat, and other communication tools. The server periodically obtains the necessary data using the APIs of these tools and converts it into a unified format. For example, from email, the data is converted into fields such as "subject," "body," "sender," and "date."

[0422] The converted data is stored in a database, including metadata such as the user ID and tool type. An index is generated for this stored information on the server side. This index is created based on date, sender, content keywords, etc., enabling fast searches.

[0423] Users log in to the virtual store using smart glasses or a head-mounted display and enter search keywords in the search field. Once a search query is submitted, the server searches the database based on the query and retrieves a list of relevant messages. At the same time, it also acquires and analyzes the user's emotional data. This is done using an emotion recognition engine that determines emotions through facial recognition, voice tone, etc.

[0424] Based on the emotional data, the server can filter or prioritize search results. For example, if the user is in a hurry, products that are available for immediate purchase and have good reviews will be prioritized. This allows the server to provide information optimized for the user's emotional state.

[0425] Finally, the formatted search results are sent to the terminal and displayed in a list format on the user's screen, including the date, hit content, source name, and information transmission method name.

[0426] For example, if a user searches for "new smartphone" and the emotion engine determines that the user is "in a hurry," smartphones that are available for immediate purchase and have high reviews will be prioritized. On the other hand, if the user is relaxed, new products and unique suggestions will be prioritized.

[0427] Prompt Sentence Examples

[0428] Design an application that suggests the best filtered products when a user searches for "new smartphone" and detects that they are "in a hurry." The products that are displayed preferentially should be "available for immediate purchase" and have "good reviews." Please explain with specific product examples.

[0429] In this way, optimal search results based on the user's emotions can be provided in real time, enabling an efficient and satisfying shopping experience within the virtual store.

[0430] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0431] Step 1:

[0432] The server obtains information from multiple information transmission methods. In this case, it periodically collects the necessary data (unread emails, latest messages, etc.) from email tools, chat tools, and other communication tools via API. The input is the raw data from each information transmission method, and the output is the obtained raw data.

[0433] Step 2:

[0434] The server converts the information it receives into a unified format. For example, email data is converted into fields such as "subject," "body," "sender," and "date." The input is the raw data obtained in step 1, and the output is the data converted into a unified format.

[0435] Step 3:

[0436] The server stores the converted information in a database, along with metadata such as the user ID and tool type. The input is data converted into a unified format, and the output is the data stored in the database.

[0437] Step 4:

[0438] The server generates an index for the stored information. The index is created based on date, sender, content keywords, etc. The input is the information stored in the database, and the output is an index to speed up searches.

[0439] Step 5:

[0440] A user inputs a search keyword into a search field on a device. For example, the user inputs "smartphone" as a search query. The input is the user's search query, and the output is the action of sending the query to a server.

[0441] Step 6:

[0442] The server searches the database based on the received search query and retrieves a list of relevant messages from the database. The input is the search query received from the user, and the output is the search results (a list of relevant messages).

[0443] Step 7:

[0444] The server acquires and analyzes the user's emotional data in real time. Using an emotion recognition engine, it determines the user's emotional state based on facial recognition, voice tone, etc. The input is the user's facial expression and voice data, and the output is the analyzed emotional state.

[0445] Step 8:

[0446] The server filters or prioritizes search results based on the user's emotional data. For example, if the user is in a hurry, products that are available for immediate purchase and have good reviews are prioritized. The input is the analyzed emotional state and search results, and the output is the filtered or prioritized search results.

[0447] Step 9:

[0448] The server sends the formatted search results to the terminal. The results are formatted to include the date, hit content, source name, and information transmission method name. The input is the filtered search results, and the output is the formatted search results sent to the terminal.

[0449] Step 10:

[0450] The terminal displays the received search results on the screen. The user can check the search results and obtain detailed information. The input is the formatted search results sent from the server, and the output is the search results displayed on the terminal screen.

[0451] Through this process, appropriate information is quickly provided according to the user's emotions, realizing an efficient search experience within the virtual store.

[0452] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0453] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0454] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0455] [Second embodiment]

[0456] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0457] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0458] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0459] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0460] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0461] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0462] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0463] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0464] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0465] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0466] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0467] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0468] The present invention is a system that searches information acquired from multiple information transmission means in one go and efficiently displays the results. This system is designed to enable users to quickly search and acquire information, improving work efficiency. The system's program processing is explained in detail below.

[0469] A natural language description of the program's operation

[0470] Server-side processing

[0471] First, the server obtains information from multiple information transmission means. Specific information transmission means include email tools, chat tools, and other communication tools. The server periodically obtains the necessary data using the APIs of these tools. For example, it obtains unread emails from email tools and the latest messages from chat tools.

[0472] Next, the acquired information is converted into a unified format. This is to unify data of different formats into a consistent format. For example, email data is converted into fields such as "subject," "body," "sender," and "date."

[0473] The converted data is stored in a database, along with metadata such as the user ID and tool type.

[0474] It generates an index for the information stored in the database, which can be quickly searched based on date, sender, content keywords, etc.

[0475] When a user enters a search query on a terminal, the query is sent to the server, which searches the database based on the received search query and retrieves a list of relevant messages.

[0476] Finally, the search results are formatted and displayed in the format of the date, hit content, sender name, information transmission means name, etc. The formatted search results are sent to the terminal and displayed on the screen.

[0477] Terminal side processing

[0478] The user enters a search keyword into the search field on the device, for example, "project progress."

[0479] When the search button is pressed, the entered search query is sent to the server.

[0480] The search results received from the server are displayed on one screen, including the date, hit content, sender name, and information transmission method name.

[0481] Specific examples

[0482] For example, if a user searches for "project progress," the following happens:

[0483] 1. User Action:

[0484] The user enters "project progress" into the search field on the terminal and clicks the search button.

[0485] 2. Request from device to server:

[0486] The terminal sends a search query containing the keyword "project progress" to the server.

[0487] 3. Server lookup process:

[0488] The server searches the database for messages that correspond to "Project Progress."

[0489] The hit messages are formatted and search results are generated.

[0490] 4. Sending from the server to the device:

[0491] Send the formatted search results to the device.

[0492] 5. Terminal display processing:

[0493] The terminal displays the received search results in a list format.

[0494] for example:

[0495] Date: 2023-10-01

[0496] Hit: "Project progress is going well"

[0497] Sender name: Yamada Taro

[0498] Name of information transmission method: Email tool

[0499] Date: 2023-09-25

[0500] Hits: "I'll report on the project progress at the next meeting."

[0501] Sender name: Hanako Sato

[0502] Name of information transmission method: Chat tool

[0503] In this way, the system of the present invention can improve work efficiency by enabling users to search and acquire necessary information from multiple information sources all at once.

[0504] The processing flow will be explained below.

[0505] Step 1: Collect data

[0506] 1. The server periodically connects to the APIs of email, chat, and other communication tools.

[0507] 2. Use API authentication information (API key, authentication token, etc.) to retrieve user data from each tool.

[0508] 3. For example, get unread emails from an email tool, or get the latest chat messages from a chat tool.

[0509] Step 2: Transform the data

[0510] 1. The server converts the acquired data into a unified format.

[0511] 2. For example, email data is converted into fields such as "Subject," "Body," "Sender," and "Date," while chat messages are converted into a format such as "Message Content," "Sender," and "Date."

[0512] Step 3: Save your data

[0513] 1. The server saves the converted data in the database.

[0514] 2. When saving, add metadata such as the user ID and tool type.

[0515] Step 4: Generate indexes

[0516] 1. The server generates an index for the stored data.

[0517] 2. Indexes are created based on fields such as date, sender, and content keywords.

[0518] 3. The index is reflected in the database, enabling fast searches.

[0519] Step 5: Receiving a search query

[0520] 1. The user enters a search keyword (e.g., "project progress") into the search field on the device.

[0521] 2. When you press the search button, the entered search query is sent from the device to the server.

[0522] Step 6: Search the database

[0523] 1. The server searches the database index based on the received search query.

[0524] 2. Get a list of matching messages as search results.

[0525] Step 7: Formatting the search results

[0526] 1. The server formats the search results.

[0527] 2. Search results are organized by date, hit content, sender name, information transmission method name, etc.

[0528] 3. Send the formatted search results to the device.

[0529] Step 8: Viewing search results

[0530] 1. The terminal receives the search results from the server.

[0531] 2. Display search results to the user on one screen.

[0532] 3. The results are displayed in the form of, for example, date, hit content, sender name, and information transmission method name.

[0533] In this way, users can search for the information they need from multiple sources in one search window, enabling them to acquire information efficiently.

[0534] Example 1

[0535] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0536] Conventional information retrieval systems have had difficulty efficiently managing data from multiple information transmission methods and quickly searching and retrieving it. They also lacked sufficient functionality to convert data of different formats into a unified format and generate indexes for high-speed searches. Furthermore, they lacked the ability to input prompt statements into generative AI models to perform more advanced data retrieval processing.

[0537] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0538] In this invention, the server includes means for acquiring information from a plurality of information transmission means, means for converting the information acquired from the information transmission means into a unified format, means for storing the converted information in a database, means for generating an index for the stored information, means for receiving a search query, means for searching the database based on the received search query, means for formatting and displaying search results, and means for inputting a prompt statement to the generative AI model, thereby enabling efficient search and acquisition of necessary information from a plurality of information sources in a batch.

[0539] "Information vehicles" refers to tools for communicating and sharing information between users, including electronic messaging tools, dialogue tools, and other communication tools.

[0540] "Unified format" refers to converting data of different formats into a specific, consistent format, thereby improving data integrity and search efficiency.

[0541] A "database" refers to an organized collection of information that can be used to store, manage, and retrieve data efficiently.

[0542] "Index" refers to a structure used to quickly search for information stored in a database, created based on specific fields (e.g., date and time, source identification, content keywords).

[0543] A "search query" refers to a keyword or phrase that a user enters to identify the information they want to retrieve, and which defines the conditions for searching a database.

[0544] A "generative AI model" refers to a pre-trained artificial intelligence algorithm that is capable of performing advanced data processing and search based on input prompts.

[0545] A "prompt" is a piece of text containing instructions or questions that are used as input to a generative AI model, leading to an appropriate response or action from the AI.

[0546] "Formatted display" refers to converting the search results obtained into an easy-to-read format and presenting the content in a visually easy-to-understand manner.

[0547] "Sender identification information" refers to information for identifying the sender of the message, and specifically includes the name and ID.

[0548] The present invention is a system that searches information acquired from multiple information transmission means in one go and efficiently displays the results. This system is designed to enable users to quickly search and acquire information and improve work efficiency. Specific embodiments of the system are described below.

[0549] Server-side configuration and operation

[0550] The server obtains information from multiple information transmission means (electronic messaging tools, dialogue tools, and other communication tools). Specific information transmission means include email tools (e.g., Gmail API) and chat tools (e.g., Slack API). The server periodically obtains data using the APIs of these tools. For example, the server obtains unread emails using the Gmail API and the latest messages using the Slack API.

[0551] The acquired information is converted into a unified format. This is to unify data of different formats into a consistent format. For example, email data acquired from Gmail is converted into fields of "Subject", "Body", "Sender", and "Date", and Slack message data is converted into fields of "Channel Name", "Body", "Sender", and "Date".

[0552] The converted data is stored in a database, such as MySQL or MongoDB, along with metadata such as the user ID and the name of the information transmission method.

[0553] The information stored in the database is indexed using Elasticsearch, which allows for fast searches based on date, sender, content keywords, etc. For example, you can quickly search for relevant data based on keywords such as "project progress."

[0554] When a user enters a search query from a terminal, the query is sent to the server. The server searches the database based on the received search query and retrieves a list of relevant messages. The search results are formatted and formatted to include the date, hit content, sender name, and information transmission method name. The formatted search results are sent to the terminal and displayed on the screen.

[0555] It also has a prompt input function that uses the generative AI model. For example, a prompt such as "Please search for the latest messages regarding project progress" can be input into the generative AI model to perform advanced data search processing.

[0556] Terminal configuration and operation

[0557] A user enters a search keyword into the search field on their device. For example, they enter "project progress." When the user presses the search button, the entered search query is sent to the server. The search query is sent to the server's API endpoint using an HTTPS request.

[0558] The search results received from the server are displayed on one screen on the terminal. The search result list is rendered in HTML format, and each item displays the date, hit content (excerpt of the text), sender identification information, and information transmission means identification information.

[0559] Specific examples of operation

[0560] For example, if a user searches for "project progress," the following happens:

[0561] 1. User Action:

[0562] The user enters "project progress" into the search field on the terminal and clicks the search button.

[0563] 2. Request from device to server:

[0564] The terminal sends a search query containing the keyword "project progress" to the server.

[0565] 3. Server lookup process:

[0566] The server uses Elasticsearch to search the database for messages that correspond to "project progress."

[0567] 4. Sending from the server to the device:

[0568] Send the formatted search results to the device.

[0569] 5. Terminal display processing:

[0570] The terminal displays the received search results in a list format.

[0571] In this way, the system of the present invention is capable of efficiently searching and acquiring necessary information from multiple information sources all at once.

[0572] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0573] Step 1: Information Acquisition

[0574] The server obtains data from multiple information transmission means. Specific information transmission means include email tools (e.g., Gmail API) and chat tools (e.g., Slack API). When using the Gmail API to obtain unread emails, the "messages.list" endpoint is used to obtain the unread message IDs, and the "messages.get" endpoint is used to obtain the contents of each message. When using the Slack API to obtain the latest messages, the "conversations.history" endpoint is used to obtain the list of latest messages.

[0575] Input: API endpoint of the communication medium

[0576] Data processing and calculation: Acquire data in XML or JSON format by calling the API

[0577] Output: Raw data obtained (unread email list, chat messages)

[0578] Step 2: Format conversion

[0579] The server converts the information it receives into a unified format. Email data from Gmail is converted into fields such as "Subject," "Body," "Sender," and "Date," while Slack messages are converted into fields such as "Channel Name," "Body," "Sender," and "Date."

[0580] Input: Raw data (unread email list, chat messages)

[0581] Data processing and calculation: JSON parsing and field conversion operations

[0582] Output: Uniformly formatted data

[0583] Step 3: Save to database

[0584] The server stores the converted data in a database, such as MySQL or MongoDB, and can also include metadata such as the user ID and the name of the information transmission method.

[0585] Input: Uniformly formatted data

[0586] Data processing and calculation: SQL insert operation or NoSQL data insert operation

[0587] Output: Data entries stored in the database

[0588] Step 4: Index generation

[0589] The server creates an index of the stored information, using Elasticsearch to create indexes based on date, sender, content keywords, etc. This allows for fast searches.

[0590] Input: Data entry stored in a database

[0591] Data processing and calculation: Index generation operation

[0592] Output: Elasticsearch index

[0593] Step 5: Receiving a search query

[0594] The server receives a search query from the user. When the user enters a keyword into the search field of the terminal and presses the search button, the search query is sent from the terminal.

[0595] Input: User search query (e.g. "project progress")

[0596] Data processing and calculation: HTTPS request parsing

[0597] Output: Parsed search query

[0598] Step 6: Database Search

[0599] The server searches the database based on the received search query, using Elasticsearch to quickly find messages that match "Project Progress."

[0600] Input: Parsed search query

[0601] Data processing and calculation: Executing Elasticsearch queries

[0602] Output: Search results (list of matching messages)

[0603] Step 7: Formatting search results

[0604] The server formats the search results and converts them into a displayable format, including each field of the email or chat message included in the search results (date and time, excerpt of the message body, sender identification information, and information transmission method name).

[0605] Input: Search results (list of matching messages)

[0606] Data processing and calculation: Result formatting operations

[0607] Output: Formatted search results

[0608] Step 8: Submit search results

[0609] The server sends the formatted search results to the terminal, which displays the formatted results on the terminal screen.

[0610] Input: Formatted search results

[0611] Data processing and calculation: HTTP response generation

[0612] Output: Search results sent to your device

[0613] Step 9: Displaying search results

[0614] The terminal displays the search results received from the server in a list format. Using an HTML template, the search results are displayed with the date and time, hit content, sender identification information, and information transmission method name.

[0615] Input: Search results received from the server

[0616] Data processing and calculation: HTML rendering

[0617] Output: Search results displayed on the screen

[0618] Step 10: Enter a prompt into the generative AI model

[0619] The user inputs a prompt into the generative AI model, such as "Please search for the latest messages about project progress," and the AI ​​performs advanced data search processing.

[0620] Input: Prompt from the user

[0621] Data processing and calculation: Analysis and search operations using generative AI models

[0622] Output: AI-generated search results

[0623] (Application example 1)

[0624] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0625] In modern factories, data from various sources is scattered, making it difficult to efficiently collect and manage it and quickly search and retrieve specific information. Furthermore, in complex manufacturing processes, when an abnormality occurs, the person in charge must quickly identify the cause and take measures, but the dispersion of information makes this task cumbersome. For this reason, there is a need for a system that can quickly integrate data obtained from various sources within the factory and efficiently search and reference it.

[0626] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0627] In this invention, the server includes: means for acquiring information from a plurality of information transmission means; means for converting the information acquired from the information transmission means into a unified format; means for storing the converted information in a database; means for generating an index for the stored information; means for receiving a search query; means for searching the database based on the received search query; means for formatting and displaying search results; and means, which is implemented in a factory robot, for collecting, formatting, and displaying data from each information source in the factory. This makes it possible to collectively collect, and quickly search and acquire, necessary information from a plurality of data sources in the factory.

[0628] "Information transmission means" refers to means for exchanging data and information, and specifically includes email tools, chat tools, manufacturing data collection tools, and other communication tools.

[0629] "Harmonized format" refers to standards and formats for converting data captured in different formats into a consistent format.

[0630] A "database" refers to a system in which data is systematically organized and stored for retrieval and storage.

[0631] An "index" refers to a data structure created to quickly search data and is used to optimize search speed.

[0632] "Computer robot" refers to a mechanical device that operates automatically and is used to perform tasks, especially in a factory.

[0633] A "search query" refers to an instruction or request sent to a database or information system to retrieve specific information.

[0634] "Search Results" refers to a list of information or data retrieved based on a search query.

[0635] The "formatting and displaying means" refers to a method or means for converting search results into a format that is easy for the user to understand and displaying them.

[0636] "Data collection methods" refers to the methods and means used to obtain the required data from the source of interest.

[0637] This invention provides a system that is implemented in a factory robot and that collectively collects necessary information from multiple information sources within the factory, and efficiently searches and acquires the information. A specific embodiment of this system is shown below.

[0638] System Overview

[0639] The system includes the following major components:

[0640] 1. Server

[0641] 2. Factory robots

[0642] 3. Database

[0643] 4. User Interface

[0644] server

[0645] The server is responsible for collecting information from multiple sources within the factory, converting it into a unified format, and storing it in a database. It also receives search queries from users, searches the database, formats the results, and sends them to factory robots and user devices.

[0646] The server uses the following hardware and software:

[0647] Hardware: High-performance server (e.g., server with Intel Xeon processor)

[0648] Software: REST API module, MySQL database, data conversion scripts (e.g. Python scripts)

[0649] Factory robots

[0650] Factory robots work in conjunction with a server to collect, format, and display data from various sources within the factory. The robots periodically collect information using APIs and send the necessary data to the server.

[0651] Factory robots use the following hardware and software:

[0652] Hardware: Industrial robots (e.g. industrial robots)

[0653] Software: Robot control software (e.g. ROS)

[0654] Database

[0655] The database stores the data sent from the server in a unified format and generates an index to enable fast searches based on date, source, content keywords, etc.

[0656] The database uses the following software:

[0657] Software: MySQL database

[0658] User Interface

[0659] The user interface is used to input search queries and display search results. Users can search for the information they need through the interface and view the results in a list format.

[0660] Specific examples

[0661] For example, if a user searches for quality control data for products in a factory, the process would go something like this:

[0662] 1. User operation: The user enters "poor quality" in the search field of the terminal and clicks the search button.

[0663] 2. Request from device to server: The device sends a search query containing the keyword “poor quality” to the server.

[0664] 3. Server search process: The server searches the database for data that corresponds to "poor quality." It formats the hit data and generates search results.

[0665] 4. Transmission from server to terminal: The formatted search results are transmitted to the terminal.

[0666] 5. Display processing on the terminal: The terminal displays the received search results in a list format, including the date, product ID, measurement value, and reference value.

[0667] Prompt Sentence Examples

[0668] Using a generative AI model, you can generate a program to build a system that collects relevant data and efficiently searches and displays it by entering a prompt like the following:

[0669] Please create a Python program that saves the quality data obtained from the API below in a MySQL database, and searches for and displays data that meets a specific condition (e.g., measurement value greater than or equal to 10).

[0670] API URL: https: / / api.example.com / quality_data

[0671] This makes it possible to collect necessary information from multiple data sources within the factory in one go and efficiently search and retrieve it.

[0672] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0673] Step 1: Gather information

[0674] The server obtains information from various sources within the factory (email tools, chat tools, manufacturing data collection tools, etc.). These tools send data to the server through their respective APIs. The input is data from each source, and the output is the raw data collected by the server. Specifically, it periodically sends requests to the API endpoint and receives response data.

[0675] Step 2: Data conversion

[0676] The server converts the acquired information into a unified format. To convert data of different formats into a consistent format, for example, each data item is classified into fields such as "product ID," "quality standard," "measurement value," and "date." The input is the acquired raw data, and the output is data converted into a unified format. Specifically, the server uses a data conversion script (e.g., a Python script) to analyze the data and extract and convert the required information.

[0677] Step 3: Save to database

[0678] The server saves the data converted into a unified format in a database. The input is the data converted into a unified format, and the output is the data stored in the database. Specifically, it connects to a database management system (e.g., MySQL), executes an SQL query, and saves the data.

[0679] Step 4: Indexing

[0680] The server creates an index for the stored data. The index is created based on date, source, content keywords, etc., allowing for fast searches. The input is the data in the database, and the output is the indexed database. Specifically, it runs SQL indexing queries and indexes the appropriate fields.

[0681] Step 5: Search query reception

[0682] The terminal receives a search query from a user. The input is a search keyword entered by the user, and the output is a search query sent to the server. Specifically, the terminal generates a query by entering keywords through a user interface and clicking a search button.

[0683] Step 6: Data Retrieval

[0684] The server searches the database based on the received search query. The input is the received search query, and the output is a list of matching search results. Specifically, it executes the SQL search query and extracts data that matches the criteria.

[0685] Step 7: Formatting search results

[0686] The server formats the search results and converts them into a display format. The input is the raw search results, and the output is the formatted data for display. Specifically, the server organizes the search results into fields such as "date," "product ID," "measurement value," and "reference value," and formats them in a format that is easy for users to view.

[0687] Step 8: View the results

[0688] The terminal displays the formatted search results to the user. The input is the formatted search results received from the server, and the output is the result list displayed to the user. Specifically, the terminal uses widgets and components for displaying results in a user interface to display the search results on the screen.

[0689] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0690] This invention provides a user-friendly search experience by combining a system that searches information acquired from multiple information transmission means in one go and efficiently displays the results with an emotion engine that recognizes the user's emotions. This system enables users to quickly search and acquire information and provides more appropriate information according to their emotions at the time.

[0691] A natural language description of the program's operation

[0692] Server-side processing

[0693] First, the server obtains information from multiple information transmission means. Specific information transmission means include email tools, chat tools, and other communication tools. The server periodically obtains the necessary data using the APIs of these tools. For example, it obtains unread emails from email tools and the latest messages from chat tools.

[0694] Next, the acquired information is converted into a unified format. This is to unify data of different formats into a consistent format. For example, email data is converted into fields for "subject," "body," "sender," and "date."

[0695] The converted data is stored in a database, along with metadata such as the user ID and tool type.

[0696] It generates an index for the information stored in the database, which can be quickly searched based on date, sender, content keywords, etc.

[0697] When a user enters a search query from their device, the query is sent to the server. The server searches the database based on the received search query and retrieves a list of relevant messages. The server's emotion engine recognizes the user's emotions based on the user's input, operation history, voice input, and data obtained from biometric sensors. Search results are filtered or prioritized based on this recognized emotion.

[0698] Finally, the search results are formatted and displayed in the format of the date, hit content, sender name, information transmission means name, etc. The formatted search results are sent to the terminal and displayed on the screen.

[0699] Terminal side processing

[0700] The user enters a search keyword into the search field on the device, for example, "project progress."

[0701] When the search button is pressed, the entered search query is sent to the server.

[0702] The search results received from the server are displayed on one screen, including the date, hit content, sender name, and information transmission method name.

[0703] Specific examples

[0704] For example, if a user searches for "project progress," the following happens:

[0705] 1. User Action:

[0706] The user enters "project progress" into the search field on the terminal and clicks the search button.

[0707] 2. Request from device to server:

[0708] The terminal sends a search query containing the keyword "project progress" to the server.

[0709] 3. Server lookup process:

[0710] The server searches the database for messages that correspond to "Project Progress."

[0711] The emotion engine determines the user's emotion and filters or prioritizes search results based on that emotion.

[0712] The hit messages are formatted and search results are generated.

[0713] 4. Sending from the server to the device:

[0714] Send the formatted search results to the device.

[0715] 5. Terminal display processing:

[0716] The terminal displays the received search results in a list format.

[0717] for example:

[0718] Date: 2023-10-01

[0719] Hit: "Project progress is going well"

[0720] Sender name: Yamada Taro

[0721] Name of information transmission method: Email tool

[0722] Date: 2023-09-25

[0723] Hits: "I'll report on the project progress at the next meeting."

[0724] Sender name: Hanako Sato

[0725] Name of information transmission method: Chat tool

[0726] In this way, the system of the present invention allows users to search for the information they need from multiple information sources at once, and provides more appropriate information that reflects their emotions at the time, thereby improving work efficiency.

[0727] The processing flow will be explained below.

[0728] Step 1: Collect data

[0729] 1. The server periodically connects to the APIs of email, chat, and other communication tools.

[0730] 2. Use each tool's API credentials (API key, authentication token, etc.) to retrieve user data.

[0731] 3. For example, get unread emails from an email tool, or get the latest chat messages from a chat tool.

[0732] Step 2: Transform the data

[0733] 1. The server converts the acquired data into a unified format.

[0734] 2. For example, convert email data into fields of "Subject", "Body", "Sender", and "Date", and convert chat messages into the format of "Message Content", "Sender", and "Date".

[0735] Step 3: Save your data

[0736] 1. The server saves the converted data in the database.

[0737] 2. When saving, metadata such as user ID and tool type are also saved.

[0738] Step 4: Generate indexes

[0739] 1. The server generates an index for the stored data.

[0740] 2. Indexes are created based on fields such as date, sender, and content keywords to enable fast searches.

[0741] Step 5: Emotion Recognition with the Emotion Engine

[0742] 1. The server's emotion engine collects user input, operation history, voice input, and data from biometric sensors.

[0743] 2. Determine the user's emotions based on the collected data.

[0744] 3. For example, if a user is feeling stressed, generate search results that reflect that.

[0745] Step 6: Receiving a search query

[0746] 1. The user enters a search keyword (e.g., "project progress") into the search field on the device.

[0747] 2. When you press the search button, the entered search query is sent from the device to the server.

[0748] Step 7: Search the database

[0749] 1. The server searches the database index based on the received search query.

[0750] 2. As a search result, a list of relevant messages is obtained.

[0751] 3. Filter or prioritize search results based on user sentiment as recognized by the sentiment engine.

[0752] Step 8: Formatting the search results

[0753] 1. The server formats the search results.

[0754] 2. Search results are organized by date, hit content, sender name, information transmission method name, etc.

[0755] 3. Send the formatted search results to the device.

[0756] Step 9: Viewing search results

[0757] 1. The terminal displays the search results received from the server.

[0758] 2. Search results are displayed on one screen, and each item includes the date, hit content, sender name, and information transmission method name.

[0759] In this way, by using the system of the present invention, users can not only search and obtain information from multiple information sources all at once using a single search box, but also use the emotion engine to obtain the most appropriate information based on their emotions at the time.

[0760] Example 2

[0761] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0762] In today's information society, users use multiple communication methods to obtain a wide variety of information. However, this information is often provided in different formats, making it difficult for users to quickly search and obtain the information they need. In addition, information is not provided in a way that reflects the user's emotions, resulting in a poor user experience.

[0763] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0764] In this invention, the server includes means for acquiring information from a plurality of information transmission means, means for converting the information acquired from the information transmission means into a unified format, means for storing the converted information in a database, means for generating an index for the stored information, means for receiving a search query, means for searching the database based on the received search query, means for formatting and displaying search results, and means for recognizing a user's emotion and filtering or preferentially displaying search results based on the emotion, thereby enabling a user to search for necessary information from a plurality of information sources at once and providing more appropriate information according to the user's emotion at that time.

[0765] "Information transmission means" refers to email tools, chat tools, and other communication tools.

[0766] A "uniform format" is a data structure for converting information obtained from different information transmission means into a consistent format.

[0767] A "database" is a structured collection of data that stores transformed information and allows it to be efficiently searched and retrieved.

[0768] An "index" is a data structure used to quickly search for information in a database.

[0769] A "search query" is data containing keywords or phrases that a user enters to specify the information they want to obtain.

[0770] "Search results" are a collection of relevant information retrieved from a database based on a search query.

[0771] An "emotion engine" is a system for recognizing a user's emotions and filtering or prioritizing search results based on those emotions.

[0772] "Filtering" is a process of extracting only necessary information from data acquired based on specific criteria.

[0773] The "display means" is a system for visually presenting search results to the user.

[0774] "Metadata" is additional information that includes information about the content of the data (for example, user ID, type of tool, etc.).

[0775] The present invention is a system that allows users to quickly search for the information they need from multiple information sources and provides appropriate information according to their emotions at the time. This system is realized by the server and the terminals each fulfilling their respective roles.

[0776] First, the server obtains information from multiple information transmission means. Specific information transmission means include email tools (e.g., Gmail API), chat tools (e.g., Slack API), and other communication tools. The server periodically obtains the necessary data using each of these APIs. For example, it obtains unread emails from the email tool and the latest messages from the chat tool.

[0777] The server then converts the information into a unified format, allowing different data types to be unified into a consistent format. For example, email data is converted into fields for "subject," "body," "sender," and "date."

[0778] The converted data is stored in a database, including metadata such as the user ID and tool type. The server then creates an index for the information stored in the database. This index is created based on date, sender, content keywords, etc., allowing for fast searches.

[0779] When a user enters a search query into the device's search field, the query is sent to the server. The server searches the database based on the received search query and retrieves a list of relevant messages. The server's emotion engine then recognizes the user's emotions based on the user's input, operation history, voice input, and data obtained from biometric sensors. The server then filters or prioritizes search results based on the recognized emotions.

[0780] Finally, the server formats and displays the search results in the format of the date, hit content, sender name, information transmission means name, etc. The formatted search results are sent to the terminal and displayed on the user's screen.

[0781] Specific examples

[0782] For example, if a user searches for "project progress," the following happens:

[0783] The user enters "project progress" into the search field of the device and clicks the search button. The device sends a search query containing the keyword "project progress" to the server. The server searches the database for messages that match "project progress." The emotion engine determines the user's emotion and filters or prioritizes the search results based on that emotion. The matched messages are formatted and search results are generated. The formatted search results are sent to the device, and the device displays the received search results in a list format.

[0784] Prompt Sentence Examples

[0785] "Describe a system that searches for project status using multiple sources of information and filters the results based on user sentiment."

[0786] In this way, users can search for related information all at once and get a better search experience based on their current emotions.

[0787] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0788] Step 1:

[0789] The server obtains information from multiple information transmission means.

[0790] Input: API endpoint of each tool

[0791] Specific behavior: The server uses the Gmail API to retrieve unread emails and the Slack API to retrieve the latest messages.

[0792] Output: Data from each tool (e.g., email subject, body, sender, date)

[0793] Step 2:

[0794] The server converts the acquired information into a unified format.

[0795] Input: Data for each tool acquired

[0796] Specific operation: Map the data fields obtained from each API to "Subject," "Body," "Sender," and "Date."

[0797] Output: Data converted into a unified format

[0798] Step 3:

[0799] The server stores the converted information in a database.

[0800] Input: Data converted into a unified format

[0801] What it does: Inserts data into a MongoDB collection, along with metadata such as user ID and tool type.

[0802] Output: Information stored in the database

[0803] Step 4:

[0804] The server generates an index for the stored information.

[0805] Input: Information stored in a database

[0806] What it does: Uses MongoDB's indexing capabilities to generate indexes based on date, sender, and keywords.

[0807] Output: An index that allows for fast searches

[0808] Step 5:

[0809] A user enters a search query into a search field on the device.

[0810] Input: User's search keyword (e.g. "project progress")

[0811] Specific operation: The user enters keywords into the search field on the device and presses the search button.

[0812] Output: The search query is sent to the server

[0813] Step 6:

[0814] The server searches the database based on the received search query.

[0815] Input: Search query sent from the device

[0816] Specific operation: The server sends a query containing the received search keywords to MongoDB and retrieves the relevant data.

[0817] Output: A list of matching messages

[0818] Step 7:

[0819] The emotion engine of the server recognizes the emotion of the user.

[0820] Input: User operation history, voice input, biometric sensor data

[0821] Specific operation: The emotion engine on the server analyzes these data and estimates the user's current emotion.

[0822] Output: Recognized emotion

[0823] Step 8:

[0824] The server filters or prioritizes search results based on the recognized sentiment.

[0825] Input: List of relevant messages, recognized emotions

[0826] Specific behavior: For users who are stressed, prioritize relevant and important messages.

[0827] Output: Filtered or prioritized search results

[0828] Step 9:

[0829] The server formats the search results and sends them to the terminal.

[0830] Input: Filtered or prioritized search results

[0831] Specific operation: The search results are formatted in the format of "date," "hit content," "sender name," and "information transmission method name," and the results are sent to the terminal in JSON format.

[0832] Output: Formatted search results sent to your device

[0833] Step 10:

[0834] The terminal displays the received search results in a list format.

[0835] Input: Formatted search results received from the server

[0836] Specific operation: The device renders the search results in HTML and displays them in a list format on the screen.

[0837] Output: Search results presented visually to the user

[0838] (Application example 2)

[0839] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0840] Conventional information acquisition systems have had difficulty in quickly and accurately providing the information users want. Furthermore, they have not provided information that corresponds to the user's emotional state, which has prevented them from improving the user experience. In particular, virtual stores need to provide optimal products and information depending on the user's emotional state, and technology to achieve this has been in demand.

[0841] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring information from multiple information transmission means, means for converting the information acquired from the information transmission means into a unified format, means for saving the converted information in a database, means for generating an index for the saved information, means for receiving a search query, means for searching the database based on the received search query, means for acquiring and analyzing user emotion data, means for filtering or preferentially displaying search results based on the user emotion data, and means for shaping and displaying the search results. This makes it possible to provide optimal product information according to the user's emotions in a virtual store, significantly improving the user experience.

[0842] "Multiple information transmission means" refers to means for sending and receiving information in different formats, such as email tools, chat tools, and other communication tools.

[0843] A "unified format" is a data format for converting data obtained from different information sources into a unified format.

[0844] "Database" means an information management system for storing, managing, and retrieving acquired and transformed information.

[0845] An "index" is a structured index that allows fast and efficient searching of information in a database.

[0846] A "search query" is a keyword or phrase entered by a user to identify the information they are seeking.

[0847] "Emotion data" is data indicating the user's emotional state obtained from facial expressions, voice, operation history, and the like.

[0848] "Filtering" is the process of selecting data based on specific criteria.

[0849] "Priority display" is a process of displaying data in the foreground based on specific criteria.

[0850] "Search Results" are information retrieved from a database based on a search query.

[0851] "Formatting and displaying" is a process of converting search results into an easy-to-read format and displaying them on the user's terminal.

[0852] This invention relates to a system for realizing efficient product searches in a virtual store. Specifically, it describes a method for recognizing a user's emotions and providing optimal search results based on those emotions. Below, we will explain in detail each component of this system and the processing it performs.

[0853] First, the server obtains information from multiple communication tools. This includes email, chat, and other communication tools. The server periodically obtains the necessary data using the APIs of these tools and converts it into a unified format. For example, from email, the data is converted into fields such as "subject," "body," "sender," and "date."

[0854] The converted data is stored in a database, including metadata such as the user ID and tool type. An index is generated for this stored information on the server side. This index is created based on date, sender, content keywords, etc., enabling fast searches.

[0855] Users log in to the virtual store using smart glasses or a head-mounted display and enter search keywords in the search field. Once a search query is submitted, the server searches the database based on the query and retrieves a list of relevant messages. At the same time, it also acquires and analyzes the user's emotional data. This is done using an emotion recognition engine that determines emotions through facial recognition, voice tone, etc.

[0856] Based on the emotional data, the server can filter or prioritize search results. For example, if the user is in a hurry, products that are available for immediate purchase and have good reviews will be prioritized. This allows the server to provide information optimized for the user's emotional state.

[0857] Finally, the formatted search results are sent to the terminal and displayed in a list format on the user's screen, including the date, hit content, source name, and information transmission method name.

[0858] For example, if a user searches for "new smartphone" and the emotion engine determines that the user is "in a hurry," smartphones that are available for immediate purchase and have high reviews will be prioritized. On the other hand, if the user is relaxed, new products and unique suggestions will be prioritized.

[0859] Prompt Sentence Examples

[0860] Design an application that suggests the best filtered products when a user searches for "new smartphone" and detects that they are "in a hurry." The products that are displayed preferentially should be "available for immediate purchase" and have "good reviews." Please explain with specific product examples.

[0861] In this way, optimal search results based on the user's emotions can be provided in real time, enabling an efficient and satisfying shopping experience within the virtual store.

[0862] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0863] Step 1:

[0864] The server obtains information from multiple information transmission methods. In this case, it periodically collects the necessary data (unread emails, latest messages, etc.) from email tools, chat tools, and other communication tools via API. The input is the raw data from each information transmission method, and the output is the obtained raw data.

[0865] Step 2:

[0866] The server converts the information it receives into a unified format. For example, email data is converted into fields such as "subject," "body," "sender," and "date." The input is the raw data obtained in step 1, and the output is the data converted into a unified format.

[0867] Step 3:

[0868] The server stores the converted information in a database, along with metadata such as the user ID and tool type. The input is data converted into a unified format, and the output is the data stored in the database.

[0869] Step 4:

[0870] The server generates an index for the stored information. The index is created based on date, sender, content keywords, etc. The input is the information stored in the database, and the output is an index to speed up searches.

[0871] Step 5:

[0872] A user inputs a search keyword into a search field on a device. For example, the user inputs "smartphone" as a search query. The input is the user's search query, and the output is the action of sending the query to a server.

[0873] Step 6:

[0874] The server searches the database based on the received search query and retrieves a list of relevant messages from the database. The input is the search query received from the user, and the output is the search results (a list of relevant messages).

[0875] Step 7:

[0876] The server acquires and analyzes the user's emotional data in real time. Using an emotion recognition engine, it determines the user's emotional state based on facial recognition, voice tone, etc. The input is the user's facial expression and voice data, and the output is the analyzed emotional state.

[0877] Step 8:

[0878] The server filters or prioritizes search results based on the user's emotional data. For example, if the user is in a hurry, products that are available for immediate purchase and have good reviews are prioritized. The input is the analyzed emotional state and search results, and the output is the filtered or prioritized search results.

[0879] Step 9:

[0880] The server sends the formatted search results to the terminal. The results are formatted to include the date, hit content, source name, and information transmission method name. The input is the filtered search results, and the output is the formatted search results sent to the terminal.

[0881] Step 10:

[0882] The terminal displays the received search results on the screen. The user can check the search results and obtain detailed information. The input is the formatted search results sent from the server, and the output is the search results displayed on the terminal screen.

[0883] Through this process, appropriate information is quickly provided according to the user's emotions, realizing an efficient search experience within the virtual store.

[0884] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0885] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0886] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0887] [Third embodiment]

[0888] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0889] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0890] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0891] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0892] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0893] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0894] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0895] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0896] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0897] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0898] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0899] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0900] The present invention is a system that searches information acquired from multiple information transmission means in one go and efficiently displays the results. This system is designed to enable users to quickly search and acquire information, improving work efficiency. The system's program processing is explained in detail below.

[0901] A natural language description of the program's operation

[0902] Server-side processing

[0903] First, the server obtains information from multiple information transmission means. Specific information transmission means include email tools, chat tools, and other communication tools. The server periodically obtains the necessary data using the APIs of these tools. For example, it obtains unread emails from email tools and the latest messages from chat tools.

[0904] Next, the acquired information is converted into a unified format. This is to unify data of different formats into a consistent format. For example, email data is converted into fields such as "subject," "body," "sender," and "date."

[0905] The converted data is stored in a database, along with metadata such as the user ID and tool type.

[0906] It generates an index for the information stored in the database, which can be quickly searched based on date, sender, content keywords, etc.

[0907] When a user enters a search query on a terminal, the query is sent to the server, which searches the database based on the received search query and retrieves a list of relevant messages.

[0908] Finally, the search results are formatted and displayed in the format of the date, hit content, sender name, information transmission means name, etc. The formatted search results are sent to the terminal and displayed on the screen.

[0909] Terminal side processing

[0910] The user enters a search keyword into the search field on the device, for example, "project progress."

[0911] When the search button is pressed, the entered search query is sent to the server.

[0912] The search results received from the server are displayed on one screen, including the date, hit content, sender name, and information transmission method name.

[0913] Specific examples

[0914] For example, if a user searches for "project progress," the following happens:

[0915] 1. User Action:

[0916] The user enters "project progress" into the search field on the terminal and clicks the search button.

[0917] 2. Request from device to server:

[0918] The terminal sends a search query containing the keyword "project progress" to the server.

[0919] 3. Server lookup process:

[0920] The server searches the database for messages that correspond to "Project Progress."

[0921] The hit messages are formatted and search results are generated.

[0922] 4. Sending from the server to the device:

[0923] Send the formatted search results to the device.

[0924] 5. Terminal display processing:

[0925] The terminal displays the received search results in a list format.

[0926] for example:

[0927] Date: 2023-10-01

[0928] Hit: "Project progress is going well"

[0929] Sender name: Yamada Taro

[0930] Name of information transmission method: Email tool

[0931] Date: 2023-09-25

[0932] Hits: "I'll report on the project progress at the next meeting."

[0933] Sender name: Hanako Sato

[0934] Name of information transmission method: Chat tool

[0935] In this way, the system of the present invention can improve work efficiency by enabling users to search and acquire necessary information from multiple information sources all at once.

[0936] The processing flow will be explained below.

[0937] Step 1: Collect data

[0938] 1. The server periodically connects to the APIs of email, chat, and other communication tools.

[0939] 2. Use API authentication information (API key, authentication token, etc.) to retrieve user data from each tool.

[0940] 3. For example, get unread emails from an email tool, or get the latest chat messages from a chat tool.

[0941] Step 2: Transform the data

[0942] 1. The server converts the acquired data into a unified format.

[0943] 2. For example, email data is converted into fields such as "Subject," "Body," "Sender," and "Date," while chat messages are converted into a format such as "Message Content," "Sender," and "Date."

[0944] Step 3: Save your data

[0945] 1. The server saves the converted data in the database.

[0946] 2. When saving, add metadata such as the user ID and tool type.

[0947] Step 4: Generate indexes

[0948] 1. The server generates an index for the stored data.

[0949] 2. Indexes are created based on fields such as date, sender, and content keywords.

[0950] 3. The index is reflected in the database, enabling fast searches.

[0951] Step 5: Receiving a search query

[0952] 1. The user enters a search keyword (e.g., "project progress") into the search field on the device.

[0953] 2. When you press the search button, the entered search query is sent from the device to the server.

[0954] Step 6: Search the database

[0955] 1. The server searches the database index based on the received search query.

[0956] 2. Get a list of matching messages as search results.

[0957] Step 7: Formatting the search results

[0958] 1. The server formats the search results.

[0959] 2. Search results are organized by date, hit content, sender name, information transmission method name, etc.

[0960] 3. Send the formatted search results to the device.

[0961] Step 8: Viewing search results

[0962] 1. The terminal receives the search results from the server.

[0963] 2. Display search results to the user on one screen.

[0964] 3. The results are displayed in the form of, for example, date, hit content, sender name, and information transmission method name.

[0965] In this way, users can search for the information they need from multiple sources in one search window, enabling them to acquire information efficiently.

[0966] Example 1

[0967] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0968] Conventional information retrieval systems have had difficulty efficiently managing data from multiple information transmission methods and quickly searching and retrieving it. They also lacked sufficient functionality to convert data of different formats into a unified format and generate indexes for high-speed searches. Furthermore, they lacked the ability to input prompt statements into generative AI models to perform more advanced data retrieval processing.

[0969] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0970] In this invention, the server includes means for acquiring information from a plurality of information transmission means, means for converting the information acquired from the information transmission means into a unified format, means for storing the converted information in a database, means for generating an index for the stored information, means for receiving a search query, means for searching the database based on the received search query, means for formatting and displaying search results, and means for inputting a prompt statement to the generative AI model, thereby enabling efficient search and acquisition of necessary information from a plurality of information sources in a batch.

[0971] "Information vehicles" refers to tools for communicating and sharing information between users, including electronic messaging tools, dialogue tools, and other communication tools.

[0972] "Unified format" refers to converting data of different formats into a specific, consistent format, thereby improving data integrity and search efficiency.

[0973] A "database" refers to an organized collection of information that can be used to store, manage, and retrieve data efficiently.

[0974] "Index" refers to a structure used to quickly search for information stored in a database, created based on specific fields (e.g., date and time, source identification, content keywords).

[0975] A "search query" refers to a keyword or phrase that a user enters to identify the information they want to retrieve, and which defines the conditions for searching a database.

[0976] A "generative AI model" refers to a pre-trained artificial intelligence algorithm that is capable of performing advanced data processing and search based on input prompts.

[0977] A "prompt" is a piece of text containing instructions or questions that are used as input to a generative AI model, leading to an appropriate response or action from the AI.

[0978] "Formatted display" refers to converting the search results obtained into an easy-to-read format and presenting the content in a visually easy-to-understand manner.

[0979] "Sender identification information" refers to information for identifying the sender of the message, and specifically includes the name and ID.

[0980] The present invention is a system that searches information acquired from multiple information transmission means in one go and efficiently displays the results. This system is designed to enable users to quickly search and acquire information and improve work efficiency. Specific embodiments of the system are described below.

[0981] Server-side configuration and operation

[0982] The server obtains information from multiple information transmission means (electronic messaging tools, dialogue tools, and other communication tools). Specific information transmission means include email tools (e.g., Gmail API) and chat tools (e.g., Slack API). The server periodically obtains data using the APIs of these tools. For example, the server obtains unread emails using the Gmail API and the latest messages using the Slack API.

[0983] The acquired information is converted into a unified format. This is to unify data of different formats into a consistent format. For example, email data acquired from Gmail is converted into fields of "Subject", "Body", "Sender", and "Date", and Slack message data is converted into fields of "Channel Name", "Body", "Sender", and "Date".

[0984] The converted data is stored in a database, such as MySQL or MongoDB, along with metadata such as the user ID and the name of the information transmission method.

[0985] The information stored in the database is indexed using Elasticsearch, which allows for fast searches based on date, sender, content keywords, etc. For example, you can quickly search for relevant data based on keywords such as "project progress."

[0986] When a user enters a search query from a terminal, the query is sent to the server. The server searches the database based on the received search query and retrieves a list of relevant messages. The search results are formatted and formatted to include the date, hit content, sender name, and information transmission method name. The formatted search results are sent to the terminal and displayed on the screen.

[0987] It also has a prompt input function that uses the generative AI model. For example, a prompt such as "Please search for the latest messages regarding project progress" can be input into the generative AI model to perform advanced data search processing.

[0988] Terminal configuration and operation

[0989] A user enters a search keyword into the search field on their device. For example, they enter "project progress." When the user presses the search button, the entered search query is sent to the server. The search query is sent to the server's API endpoint using an HTTPS request.

[0990] The search results received from the server are displayed on one screen on the terminal. The search result list is rendered in HTML format, and each item displays the date, hit content (excerpt of the text), sender identification information, and information transmission means identification information.

[0991] Specific examples of operation

[0992] For example, if a user searches for "project progress," the following happens:

[0993] 1. User Action:

[0994] The user enters "project progress" into the search field on the terminal and clicks the search button.

[0995] 2. Request from device to server:

[0996] The terminal sends a search query containing the keyword "project progress" to the server.

[0997] 3. Server lookup process:

[0998] The server uses Elasticsearch to search the database for messages that correspond to "project progress."

[0999] 4. Sending from the server to the device:

[1000] Send the formatted search results to the device.

[1001] 5. Terminal display processing:

[1002] The terminal displays the received search results in a list format.

[1003] In this way, the system of the present invention is capable of efficiently searching and acquiring necessary information from multiple information sources all at once.

[1004] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1005] Step 1: Information Acquisition

[1006] The server obtains data from multiple information transmission means. Specific information transmission means include email tools (e.g., Gmail API) and chat tools (e.g., Slack API). When using the Gmail API to obtain unread emails, the "messages.list" endpoint is used to obtain the unread message IDs, and the "messages.get" endpoint is used to obtain the contents of each message. When using the Slack API to obtain the latest messages, the "conversations.history" endpoint is used to obtain the list of latest messages.

[1007] Input: API endpoint of the communication medium

[1008] Data processing and calculation: Acquire data in XML or JSON format by calling the API

[1009] Output: Raw data obtained (unread email list, chat messages)

[1010] Step 2: Format conversion

[1011] The server converts the information it receives into a unified format. Email data from Gmail is converted into fields such as "Subject," "Body," "Sender," and "Date," while Slack messages are converted into fields such as "Channel Name," "Body," "Sender," and "Date."

[1012] Input: Raw data (unread email list, chat messages)

[1013] Data processing and calculation: JSON parsing and field conversion operations

[1014] Output: Uniformly formatted data

[1015] Step 3: Save to database

[1016] The server stores the converted data in a database, such as MySQL or MongoDB, and can also include metadata such as the user ID and the name of the information transmission method.

[1017] Input: Uniformly formatted data

[1018] Data processing and calculation: SQL insert operation or NoSQL data insert operation

[1019] Output: Data entries stored in the database

[1020] Step 4: Index generation

[1021] The server creates an index of the stored information, using Elasticsearch to create indexes based on date, sender, content keywords, etc. This allows for fast searches.

[1022] Input: Data entry stored in a database

[1023] Data processing and calculation: Index generation operation

[1024] Output: Elasticsearch index

[1025] Step 5: Receiving a search query

[1026] The server receives a search query from the user. When the user enters a keyword into the search field of the terminal and presses the search button, the search query is sent from the terminal.

[1027] Input: User search query (e.g. "project progress")

[1028] Data processing and calculation: HTTPS request parsing

[1029] Output: Parsed search query

[1030] Step 6: Database Search

[1031] The server searches the database based on the received search query, using Elasticsearch to quickly find messages that match "Project Progress."

[1032] Input: Parsed search query

[1033] Data processing and calculation: Executing Elasticsearch queries

[1034] Output: Search results (list of matching messages)

[1035] Step 7: Formatting search results

[1036] The server formats the search results and converts them into a displayable format, including each field of the email or chat message included in the search results (date and time, excerpt of the message body, sender identification information, and information transmission method name).

[1037] Input: Search results (list of matching messages)

[1038] Data processing and calculation: Result formatting operations

[1039] Output: Formatted search results

[1040] Step 8: Submit search results

[1041] The server sends the formatted search results to the terminal, which displays the formatted results on the terminal screen.

[1042] Input: Formatted search results

[1043] Data processing and calculation: HTTP response generation

[1044] Output: Search results sent to your device

[1045] Step 9: Displaying search results

[1046] The terminal displays the search results received from the server in a list format. Using an HTML template, the search results are displayed with the date and time, hit content, sender identification information, and information transmission method name.

[1047] Input: Search results received from the server

[1048] Data processing and calculation: HTML rendering

[1049] Output: Search results displayed on the screen

[1050] Step 10: Enter a prompt into the generative AI model

[1051] The user inputs a prompt into the generative AI model, such as "Please search for the latest messages about project progress," and the AI ​​performs advanced data search processing.

[1052] Input: Prompt from the user

[1053] Data processing and calculation: Analysis and search operations using generative AI models

[1054] Output: AI-generated search results

[1055] (Application example 1)

[1056] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1057] In modern factories, data from various sources is scattered, making it difficult to efficiently collect and manage it and quickly search and retrieve specific information. Furthermore, in complex manufacturing processes, when an abnormality occurs, the person in charge must quickly identify the cause and take measures, but the dispersion of information makes this task cumbersome. For this reason, there is a need for a system that can quickly integrate data obtained from various sources within the factory and efficiently search and reference it.

[1058] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1059] In this invention, the server includes: means for acquiring information from a plurality of information transmission means; means for converting the information acquired from the information transmission means into a unified format; means for storing the converted information in a database; means for generating an index for the stored information; means for receiving a search query; means for searching the database based on the received search query; means for formatting and displaying search results; and means, which is implemented in a factory robot, for collecting, formatting, and displaying data from each information source in the factory. This makes it possible to collectively collect, and quickly search and acquire, necessary information from a plurality of data sources in the factory.

[1060] "Information transmission means" refers to means for exchanging data and information, and specifically includes email tools, chat tools, manufacturing data collection tools, and other communication tools.

[1061] "Harmonized format" refers to standards and formats for converting data captured in different formats into a consistent format.

[1062] A "database" refers to a system in which data is systematically organized and stored for retrieval and storage.

[1063] An "index" refers to a data structure created to quickly search data and is used to optimize search speed.

[1064] "Computer robot" refers to a mechanical device that operates automatically and is used to perform tasks, especially in a factory.

[1065] A "search query" refers to an instruction or request sent to a database or information system to retrieve specific information.

[1066] "Search Results" refers to a list of information or data retrieved based on a search query.

[1067] The "formatting and displaying means" refers to a method or means for converting search results into a format that is easy for the user to understand and displaying them.

[1068] "Data collection methods" refers to the methods and means used to obtain the required data from the source of interest.

[1069] This invention provides a system that is implemented in a factory robot and that collectively collects necessary information from multiple information sources within the factory, and efficiently searches and acquires the information. A specific embodiment of this system is shown below.

[1070] System Overview

[1071] The system includes the following major components:

[1072] 1. Server

[1073] 2. Factory robots

[1074] 3. Database

[1075] 4. User Interface

[1076] server

[1077] The server is responsible for collecting information from multiple sources within the factory, converting it into a unified format, and storing it in a database. It also receives search queries from users, searches the database, formats the results, and sends them to factory robots and user devices.

[1078] The server uses the following hardware and software:

[1079] Hardware: High-performance server (e.g., server with Intel Xeon processor)

[1080] Software: REST API module, MySQL database, data conversion scripts (e.g. Python scripts)

[1081] Factory robots

[1082] Factory robots work in conjunction with a server to collect, format, and display data from various sources within the factory. The robots periodically collect information using APIs and send the necessary data to the server.

[1083] Factory robots use the following hardware and software:

[1084] Hardware: Industrial robots (e.g. industrial robots)

[1085] Software: Robot control software (e.g. ROS)

[1086] Database

[1087] The database stores the data sent from the server in a unified format and generates an index to enable fast searches based on date, source, content keywords, etc.

[1088] The database uses the following software:

[1089] Software: MySQL database

[1090] User Interface

[1091] The user interface is used to input search queries and display search results. Users can search for the information they need through the interface and view the results in a list format.

[1092] Specific examples

[1093] For example, if a user searches for quality control data for products in a factory, the process would go something like this:

[1094] 1. User operation: The user enters "poor quality" in the search field of the terminal and clicks the search button.

[1095] 2. Request from device to server: The device sends a search query containing the keyword “poor quality” to the server.

[1096] 3. Server search process: The server searches the database for data that corresponds to "poor quality." It formats the hit data and generates search results.

[1097] 4. Transmission from server to terminal: The formatted search results are transmitted to the terminal.

[1098] 5. Display processing on the terminal: The terminal displays the received search results in a list format, including the date, product ID, measurement value, and reference value.

[1099] Prompt Sentence Examples

[1100] Using a generative AI model, you can generate a program to build a system that collects relevant data and efficiently searches and displays it by entering a prompt like the following:

[1101] Please create a Python program that saves the quality data obtained from the API below in a MySQL database, and searches for and displays data that meets a specific condition (e.g., measurement value greater than or equal to 10).

[1102] API URL: https: / / api.example.com / quality_data

[1103] This makes it possible to collect necessary information from multiple data sources within the factory in one go and efficiently search and retrieve it.

[1104] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1105] Step 1: Gather information

[1106] The server obtains information from various sources within the factory (email tools, chat tools, manufacturing data collection tools, etc.). These tools send data to the server through their respective APIs. The input is data from each source, and the output is the raw data collected by the server. Specifically, it periodically sends requests to the API endpoint and receives response data.

[1107] Step 2: Data conversion

[1108] The server converts the acquired information into a unified format. To convert data of different formats into a consistent format, for example, each data item is classified into fields such as "product ID," "quality standard," "measurement value," and "date." The input is the acquired raw data, and the output is data converted into a unified format. Specifically, the server uses a data conversion script (e.g., a Python script) to analyze the data and extract and convert the required information.

[1109] Step 3: Save to database

[1110] The server saves the data converted into a unified format in a database. The input is the data converted into a unified format, and the output is the data stored in the database. Specifically, it connects to a database management system (e.g., MySQL), executes an SQL query, and saves the data.

[1111] Step 4: Indexing

[1112] The server creates an index for the stored data. The index is created based on date, source, content keywords, etc., allowing for fast searches. The input is the data in the database, and the output is the indexed database. Specifically, it runs SQL indexing queries and indexes the appropriate fields.

[1113] Step 5: Search query reception

[1114] The terminal receives a search query from a user. The input is a search keyword entered by the user, and the output is a search query sent to the server. Specifically, the terminal generates a query by entering keywords through a user interface and clicking a search button.

[1115] Step 6: Data Retrieval

[1116] The server searches the database based on the received search query. The input is the received search query, and the output is a list of matching search results. Specifically, it executes the SQL search query and extracts data that matches the criteria.

[1117] Step 7: Formatting search results

[1118] The server formats the search results and converts them into a display format. The input is the raw search results, and the output is the formatted data for display. Specifically, the server organizes the search results into fields such as "date," "product ID," "measurement value," and "reference value," and formats them in a format that is easy for users to view.

[1119] Step 8: View the results

[1120] The terminal displays the formatted search results to the user. The input is the formatted search results received from the server, and the output is the result list displayed to the user. Specifically, the terminal uses widgets and components for displaying results in a user interface to display the search results on the screen.

[1121] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1122] This invention provides a user-friendly search experience by combining a system that searches information acquired from multiple information transmission means in one go and efficiently displays the results with an emotion engine that recognizes the user's emotions. This system enables users to quickly search and acquire information and provides more appropriate information according to their emotions at the time.

[1123] A natural language description of the program's operation

[1124] Server-side processing

[1125] First, the server obtains information from multiple information transmission means. Specific information transmission means include email tools, chat tools, and other communication tools. The server periodically obtains the necessary data using the APIs of these tools. For example, it obtains unread emails from email tools and the latest messages from chat tools.

[1126] Next, the acquired information is converted into a unified format. This is to unify data of different formats into a consistent format. For example, email data is converted into fields for "subject," "body," "sender," and "date."

[1127] The converted data is stored in a database, along with metadata such as the user ID and tool type.

[1128] It generates an index for the information stored in the database, which can be quickly searched based on date, sender, content keywords, etc.

[1129] When a user enters a search query from their device, the query is sent to the server. The server searches the database based on the received search query and retrieves a list of relevant messages. The server's emotion engine recognizes the user's emotions based on the user's input, operation history, voice input, and data obtained from biometric sensors. Search results are filtered or prioritized based on this recognized emotion.

[1130] Finally, the search results are formatted and displayed in the format of the date, hit content, sender name, information transmission means name, etc. The formatted search results are sent to the terminal and displayed on the screen.

[1131] Terminal side processing

[1132] The user enters a search keyword into the search field on the device, for example, "project progress."

[1133] When the search button is pressed, the entered search query is sent to the server.

[1134] The search results received from the server are displayed on one screen, including the date, hit content, sender name, and information transmission method name.

[1135] Specific examples

[1136] For example, if a user searches for "project progress," the following happens:

[1137] 1. User Action:

[1138] The user enters "project progress" into the search field on the terminal and clicks the search button.

[1139] 2. Request from device to server:

[1140] The terminal sends a search query containing the keyword "project progress" to the server.

[1141] 3. Server lookup process:

[1142] The server searches the database for messages that correspond to "Project Progress."

[1143] The emotion engine determines the user's emotion and filters or prioritizes search results based on that emotion.

[1144] The hit messages are formatted and search results are generated.

[1145] 4. Sending from the server to the device:

[1146] Send the formatted search results to the device.

[1147] 5. Terminal display processing:

[1148] The terminal displays the received search results in a list format.

[1149] for example:

[1150] Date: 2023-10-01

[1151] Hit: "Project progress is going well"

[1152] Sender name: Yamada Taro

[1153] Name of information transmission method: Email tool

[1154] Date: 2023-09-25

[1155] Hits: "I'll report on the project progress at the next meeting."

[1156] Sender name: Hanako Sato

[1157] Name of information transmission method: Chat tool

[1158] In this way, the system of the present invention allows users to search for the information they need from multiple information sources at once, and provides more appropriate information that reflects their emotions at the time, thereby improving work efficiency.

[1159] The processing flow will be explained below.

[1160] Step 1: Collect data

[1161] 1. The server periodically connects to the APIs of email, chat, and other communication tools.

[1162] 2. Use each tool's API credentials (API key, authentication token, etc.) to retrieve user data.

[1163] 3. For example, get unread emails from an email tool, or get the latest chat messages from a chat tool.

[1164] Step 2: Transform the data

[1165] 1. The server converts the acquired data into a unified format.

[1166] 2. For example, convert email data into fields of "Subject", "Body", "Sender", and "Date", and convert chat messages into the format of "Message Content", "Sender", and "Date".

[1167] Step 3: Save your data

[1168] 1. The server saves the converted data in the database.

[1169] 2. When saving, metadata such as user ID and tool type are also saved.

[1170] Step 4: Generate indexes

[1171] 1. The server generates an index for the stored data.

[1172] 2. Indexes are created based on fields such as date, sender, and content keywords to enable fast searches.

[1173] Step 5: Emotion Recognition with the Emotion Engine

[1174] 1. The server's emotion engine collects user input, operation history, voice input, and data from biometric sensors.

[1175] 2. Determine the user's emotions based on the collected data.

[1176] 3. For example, if a user is feeling stressed, generate search results that reflect that.

[1177] Step 6: Receiving a search query

[1178] 1. The user enters a search keyword (e.g., "project progress") into the search field on the device.

[1179] 2. When you press the search button, the entered search query is sent from the device to the server.

[1180] Step 7: Search the database

[1181] 1. The server searches the database index based on the received search query.

[1182] 2. As a search result, a list of relevant messages is obtained.

[1183] 3. Filter or prioritize search results based on user sentiment as recognized by the sentiment engine.

[1184] Step 8: Formatting the search results

[1185] 1. The server formats the search results.

[1186] 2. Search results are organized by date, hit content, sender name, information transmission method name, etc.

[1187] 3. Send the formatted search results to the device.

[1188] Step 9: Viewing search results

[1189] 1. The terminal displays the search results received from the server.

[1190] 2. Search results are displayed on one screen, and each item includes the date, hit content, sender name, and information transmission method name.

[1191] In this way, by using the system of the present invention, users can not only search and obtain information from multiple information sources all at once using a single search box, but also use the emotion engine to obtain the most appropriate information based on their emotions at the time.

[1192] Example 2

[1193] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1194] In today's information society, users use multiple communication methods to obtain a wide variety of information. However, this information is often provided in different formats, making it difficult for users to quickly search and obtain the information they need. In addition, information is not provided in a way that reflects the user's emotions, resulting in a poor user experience.

[1195] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1196] In this invention, the server includes means for acquiring information from a plurality of information transmission means, means for converting the information acquired from the information transmission means into a unified format, means for storing the converted information in a database, means for generating an index for the stored information, means for receiving a search query, means for searching the database based on the received search query, means for formatting and displaying search results, and means for recognizing a user's emotion and filtering or preferentially displaying search results based on the emotion, thereby enabling a user to search for necessary information from a plurality of information sources at once and providing more appropriate information according to the user's emotion at that time.

[1197] "Information transmission means" refers to email tools, chat tools, and other communication tools.

[1198] A "uniform format" is a data structure for converting information obtained from different information transmission means into a consistent format.

[1199] A "database" is a structured collection of data that stores transformed information and allows it to be efficiently searched and retrieved.

[1200] An "index" is a data structure used to quickly search for information in a database.

[1201] A "search query" is data containing keywords or phrases that a user enters to specify the information they want to obtain.

[1202] "Search results" are a collection of relevant information retrieved from a database based on a search query.

[1203] An "emotion engine" is a system for recognizing a user's emotions and filtering or prioritizing search results based on those emotions.

[1204] "Filtering" is a process of extracting only necessary information from data acquired based on specific criteria.

[1205] The "display means" is a system for visually presenting search results to the user.

[1206] "Metadata" is additional information that includes information about the content of the data (for example, user ID, type of tool, etc.).

[1207] The present invention is a system that allows users to quickly search for the information they need from multiple information sources and provides appropriate information according to their emotions at the time. This system is realized by the server and the terminals each fulfilling their respective roles.

[1208] First, the server obtains information from multiple information transmission means. Specific information transmission means include email tools (e.g., Gmail API), chat tools (e.g., Slack API), and other communication tools. The server periodically obtains the necessary data using each of these APIs. For example, it obtains unread emails from the email tool and the latest messages from the chat tool.

[1209] The server then converts the information into a unified format, allowing different data types to be unified into a consistent format. For example, email data is converted into fields for "subject," "body," "sender," and "date."

[1210] The converted data is stored in a database, including metadata such as the user ID and tool type. The server then creates an index for the information stored in the database. This index is created based on date, sender, content keywords, etc., allowing for fast searches.

[1211] When a user enters a search query into the device's search field, the query is sent to the server. The server searches the database based on the received search query and retrieves a list of relevant messages. The server's emotion engine then recognizes the user's emotions based on the user's input, operation history, voice input, and data obtained from biometric sensors. The server then filters or prioritizes search results based on the recognized emotions.

[1212] Finally, the server formats and displays the search results in the format of the date, hit content, sender name, information transmission means name, etc. The formatted search results are sent to the terminal and displayed on the user's screen.

[1213] Specific examples

[1214] For example, if a user searches for "project progress," the following happens:

[1215] The user enters "project progress" into the search field of the device and clicks the search button. The device sends a search query containing the keyword "project progress" to the server. The server searches the database for messages that match "project progress." The emotion engine determines the user's emotion and filters or prioritizes the search results based on that emotion. The matched messages are formatted and search results are generated. The formatted search results are sent to the device, and the device displays the received search results in a list format.

[1216] Prompt Sentence Examples

[1217] "Describe a system that searches for project status using multiple sources of information and filters the results based on user sentiment."

[1218] In this way, users can search for related information all at once and get a better search experience based on their current emotions.

[1219] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1220] Step 1:

[1221] The server obtains information from multiple information transmission means.

[1222] Input: API endpoint of each tool

[1223] Specific behavior: The server uses the Gmail API to retrieve unread emails and the Slack API to retrieve the latest messages.

[1224] Output: Data from each tool (e.g., email subject, body, sender, date)

[1225] Step 2:

[1226] The server converts the acquired information into a unified format.

[1227] Input: Data for each tool acquired

[1228] Specific operation: Map the data fields obtained from each API to "Subject," "Body," "Sender," and "Date."

[1229] Output: Data converted into a unified format

[1230] Step 3:

[1231] The server stores the converted information in a database.

[1232] Input: Data converted into a unified format

[1233] What it does: Inserts data into a MongoDB collection, along with metadata such as user ID and tool type.

[1234] Output: Information stored in the database

[1235] Step 4:

[1236] The server generates an index for the stored information.

[1237] Input: Information stored in a database

[1238] What it does: Uses MongoDB's indexing capabilities to generate indexes based on date, sender, and keywords.

[1239] Output: An index that allows for fast searches

[1240] Step 5:

[1241] A user enters a search query into a search field on the device.

[1242] Input: User's search keyword (e.g. "project progress")

[1243] Specific operation: The user enters keywords into the search field on the device and presses the search button.

[1244] Output: The search query is sent to the server

[1245] Step 6:

[1246] The server searches the database based on the received search query.

[1247] Input: Search query sent from the device

[1248] Specific operation: The server sends a query containing the received search keywords to MongoDB and retrieves the relevant data.

[1249] Output: A list of matching messages

[1250] Step 7:

[1251] The emotion engine of the server recognizes the emotion of the user.

[1252] Input: User operation history, voice input, biometric sensor data

[1253] Specific operation: The emotion engine on the server analyzes these data and estimates the user's current emotion.

[1254] Output: Recognized emotion

[1255] Step 8:

[1256] The server filters or prioritizes search results based on the recognized sentiment.

[1257] Input: List of relevant messages, recognized emotions

[1258] Specific behavior: For users who are stressed, prioritize relevant and important messages.

[1259] Output: Filtered or prioritized search results

[1260] Step 9:

[1261] The server formats the search results and sends them to the terminal.

[1262] Input: Filtered or prioritized search results

[1263] Specific operation: The search results are formatted in the format of "date," "hit content," "sender name," and "information transmission method name," and the results are sent to the terminal in JSON format.

[1264] Output: Formatted search results sent to your device

[1265] Step 10:

[1266] The terminal displays the received search results in a list format.

[1267] Input: Formatted search results received from the server

[1268] Specific operation: The device renders the search results in HTML and displays them in a list format on the screen.

[1269] Output: Search results presented visually to the user

[1270] (Application example 2)

[1271] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1272] Conventional information acquisition systems have had difficulty in quickly and accurately providing the information users want. Furthermore, they have not provided information that corresponds to the user's emotional state, which has prevented them from improving the user experience. In particular, virtual stores need to provide optimal products and information depending on the user's emotional state, and technology to achieve this has been in demand.

[1273] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring information from multiple information transmission means, means for converting the information acquired from the information transmission means into a unified format, means for saving the converted information in a database, means for generating an index for the saved information, means for receiving a search query, means for searching the database based on the received search query, means for acquiring and analyzing user emotion data, means for filtering or preferentially displaying search results based on the user emotion data, and means for shaping and displaying the search results. This makes it possible to provide optimal product information according to the user's emotions in a virtual store, significantly improving the user experience.

[1274] "Multiple information transmission means" refers to means for sending and receiving information in different formats, such as email tools, chat tools, and other communication tools.

[1275] A "unified format" is a data format for converting data obtained from different information sources into a unified format.

[1276] "Database" means an information management system for storing, managing, and retrieving acquired and transformed information.

[1277] An "index" is a structured index that allows fast and efficient searching of information in a database.

[1278] A "search query" is a keyword or phrase entered by a user to identify the information they are seeking.

[1279] "Emotion data" is data indicating the user's emotional state obtained from facial expressions, voice, operation history, and the like.

[1280] "Filtering" is the process of selecting data based on specific criteria.

[1281] "Priority display" is a process of displaying data in the foreground based on specific criteria.

[1282] "Search Results" are information retrieved from a database based on a search query.

[1283] "Formatting and displaying" is a process of converting search results into an easy-to-read format and displaying them on the user's terminal.

[1284] This invention relates to a system for realizing efficient product searches in a virtual store. Specifically, it describes a method for recognizing a user's emotions and providing optimal search results based on those emotions. Below, we will explain in detail each component of this system and the processing it performs.

[1285] First, the server obtains information from multiple communication tools. This includes email, chat, and other communication tools. The server periodically obtains the necessary data using the APIs of these tools and converts it into a unified format. For example, from email, the data is converted into fields such as "subject," "body," "sender," and "date."

[1286] The converted data is stored in a database, including metadata such as the user ID and tool type. An index is generated for this stored information on the server side. This index is created based on date, sender, content keywords, etc., enabling fast searches.

[1287] Users log in to the virtual store using smart glasses or a head-mounted display and enter search keywords in the search field. Once a search query is submitted, the server searches the database based on the query and retrieves a list of relevant messages. At the same time, it also acquires and analyzes the user's emotional data. This is done using an emotion recognition engine that determines emotions through facial recognition, voice tone, etc.

[1288] Based on the emotional data, the server can filter or prioritize search results. For example, if the user is in a hurry, products that are available for immediate purchase and have good reviews will be prioritized. This allows the server to provide information optimized for the user's emotional state.

[1289] Finally, the formatted search results are sent to the terminal and displayed in a list format on the user's screen, including the date, hit content, source name, and information transmission method name.

[1290] For example, if a user searches for "new smartphone" and the emotion engine determines that the user is "in a hurry," smartphones that are available for immediate purchase and have high reviews will be prioritized. On the other hand, if the user is relaxed, new products and unique suggestions will be prioritized.

[1291] Prompt Sentence Examples

[1292] Design an application that suggests the best filtered products when a user searches for "new smartphone" and detects that they are "in a hurry." The products that are displayed preferentially should be "available for immediate purchase" and have "good reviews." Please explain with specific product examples.

[1293] In this way, optimal search results based on the user's emotions can be provided in real time, enabling an efficient and satisfying shopping experience within the virtual store.

[1294] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1295] Step 1:

[1296] The server obtains information from multiple information transmission methods. In this case, it periodically collects the necessary data (unread emails, latest messages, etc.) from email tools, chat tools, and other communication tools via API. The input is the raw data from each information transmission method, and the output is the obtained raw data.

[1297] Step 2:

[1298] The server converts the information it receives into a unified format. For example, email data is converted into fields such as "subject," "body," "sender," and "date." The input is the raw data obtained in step 1, and the output is the data converted into a unified format.

[1299] Step 3:

[1300] The server stores the converted information in a database, along with metadata such as the user ID and tool type. The input is data converted into a unified format, and the output is the data stored in the database.

[1301] Step 4:

[1302] The server generates an index for the stored information. The index is created based on date, sender, content keywords, etc. The input is the information stored in the database, and the output is an index to speed up searches.

[1303] Step 5:

[1304] A user inputs a search keyword into a search field on a device. For example, the user inputs "smartphone" as a search query. The input is the user's search query, and the output is the action of sending the query to a server.

[1305] Step 6:

[1306] The server searches the database based on the received search query and retrieves a list of relevant messages from the database. The input is the search query received from the user, and the output is the search results (a list of relevant messages).

[1307] Step 7:

[1308] The server acquires and analyzes the user's emotional data in real time. Using an emotion recognition engine, it determines the user's emotional state based on facial recognition, voice tone, etc. The input is the user's facial expression and voice data, and the output is the analyzed emotional state.

[1309] Step 8:

[1310] The server filters or prioritizes search results based on the user's emotional data. For example, if the user is in a hurry, products that are available for immediate purchase and have good reviews are prioritized. The input is the analyzed emotional state and search results, and the output is the filtered or prioritized search results.

[1311] Step 9:

[1312] The server sends the formatted search results to the terminal. The results are formatted to include the date, hit content, source name, and information transmission method name. The input is the filtered search results, and the output is the formatted search results sent to the terminal.

[1313] Step 10:

[1314] The terminal displays the received search results on the screen. The user can check the search results and obtain detailed information. The input is the formatted search results sent from the server, and the output is the search results displayed on the terminal screen.

[1315] Through this process, appropriate information is quickly provided according to the user's emotions, realizing an efficient search experience within the virtual store.

[1316] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1317] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1318] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1319] [Fourth embodiment]

[1320] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1321] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1322] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1323] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1324] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1325] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1326] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1327] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1328] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1329] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1330] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1331] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1332] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1333] The present invention is a system that searches information acquired from multiple information transmission means in one go and efficiently displays the results. This system is designed to enable users to quickly search and acquire information, improving work efficiency. The system's program processing is explained in detail below.

[1334] A natural language description of the program's operation

[1335] Server-side processing

[1336] First, the server obtains information from multiple information transmission means. Specific information transmission means include email tools, chat tools, and other communication tools. The server periodically obtains the necessary data using the APIs of these tools. For example, it obtains unread emails from email tools and the latest messages from chat tools.

[1337] Next, the acquired information is converted into a unified format. This is to unify data of different formats into a consistent format. For example, email data is converted into fields such as "subject," "body," "sender," and "date."

[1338] The converted data is stored in a database, along with metadata such as the user ID and tool type.

[1339] It generates an index for the information stored in the database, which can be quickly searched based on date, sender, content keywords, etc.

[1340] When a user enters a search query on a terminal, the query is sent to the server, which searches the database based on the received search query and retrieves a list of relevant messages.

[1341] Finally, the search results are formatted and displayed in the format of the date, hit content, sender name, information transmission means name, etc. The formatted search results are sent to the terminal and displayed on the screen.

[1342] Terminal side processing

[1343] The user enters a search keyword into the search field on the device, for example, "project progress."

[1344] When the search button is pressed, the entered search query is sent to the server.

[1345] The search results received from the server are displayed on one screen, including the date, hit content, sender name, and information transmission method name.

[1346] Specific examples

[1347] For example, if a user searches for "project progress," the following happens:

[1348] 1. User Action:

[1349] The user enters "project progress" into the search field on the terminal and clicks the search button.

[1350] 2. Request from device to server:

[1351] The terminal sends a search query containing the keyword "project progress" to the server.

[1352] 3. Server lookup process:

[1353] The server searches the database for messages that correspond to "Project Progress."

[1354] The hit messages are formatted and search results are generated.

[1355] 4. Sending from the server to the device:

[1356] Send the formatted search results to the device.

[1357] 5. Terminal display processing:

[1358] The terminal displays the received search results in a list format.

[1359] for example:

[1360] Date: 2023-10-01

[1361] Hit: "Project progress is going well"

[1362] Sender name: Yamada Taro

[1363] Name of information transmission method: Email tool

[1364] Date: 2023-09-25

[1365] Hits: "I'll report on the project progress at the next meeting."

[1366] Sender name: Hanako Sato

[1367] Name of information transmission method: Chat tool

[1368] In this way, the system of the present invention can improve work efficiency by enabling users to search and acquire necessary information from multiple information sources all at once.

[1369] The processing flow will be explained below.

[1370] Step 1: Collect data

[1371] 1. The server periodically connects to the APIs of email, chat, and other communication tools.

[1372] 2. Use API authentication information (API key, authentication token, etc.) to retrieve user data from each tool.

[1373] 3. For example, get unread emails from an email tool, or get the latest chat messages from a chat tool.

[1374] Step 2: Transform the data

[1375] 1. The server converts the acquired data into a unified format.

[1376] 2. For example, email data is converted into fields such as "Subject," "Body," "Sender," and "Date," while chat messages are converted into a format such as "Message Content," "Sender," and "Date."

[1377] Step 3: Save your data

[1378] 1. The server saves the converted data in the database.

[1379] 2. When saving, add metadata such as the user ID and tool type.

[1380] Step 4: Generate indexes

[1381] 1. The server generates an index for the stored data.

[1382] 2. Indexes are created based on fields such as date, sender, and content keywords.

[1383] 3. The index is reflected in the database, enabling fast searches.

[1384] Step 5: Receiving a search query

[1385] 1. The user enters a search keyword (e.g., "project progress") into the search field on the device.

[1386] 2. When you press the search button, the entered search query is sent from the device to the server.

[1387] Step 6: Search the database

[1388] 1. The server searches the database index based on the received search query.

[1389] 2. Get a list of matching messages as search results.

[1390] Step 7: Formatting the search results

[1391] 1. The server formats the search results.

[1392] 2. Search results are organized by date, hit content, sender name, information transmission method name, etc.

[1393] 3. Send the formatted search results to the device.

[1394] Step 8: Viewing search results

[1395] 1. The terminal receives the search results from the server.

[1396] 2. Display search results to the user on one screen.

[1397] 3. The results are displayed in the form of, for example, date, hit content, sender name, and information transmission method name.

[1398] In this way, users can search for the information they need from multiple sources in one search window, enabling them to acquire information efficiently.

[1399] Example 1

[1400] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1401] Conventional information retrieval systems have had difficulty efficiently managing data from multiple information transmission methods and quickly searching and retrieving it. They also lacked sufficient functionality to convert data of different formats into a unified format and generate indexes for high-speed searches. Furthermore, they lacked the ability to input prompt statements into generative AI models to perform more advanced data retrieval processing.

[1402] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1403] In this invention, the server includes means for acquiring information from a plurality of information transmission means, means for converting the information acquired from the information transmission means into a unified format, means for storing the converted information in a database, means for generating an index for the stored information, means for receiving a search query, means for searching the database based on the received search query, means for formatting and displaying search results, and means for inputting a prompt statement to the generative AI model, thereby enabling efficient search and acquisition of necessary information from a plurality of information sources in a batch.

[1404] "Information vehicles" refers to tools for communicating and sharing information between users, including electronic messaging tools, dialogue tools, and other communication tools.

[1405] "Unified format" refers to converting data of different formats into a specific, consistent format, thereby improving data integrity and search efficiency.

[1406] A "database" refers to an organized collection of information that can be used to store, manage, and retrieve data efficiently.

[1407] "Index" refers to a structure used to quickly search for information stored in a database, created based on specific fields (e.g., date and time, source identification, content keywords).

[1408] A "search query" refers to a keyword or phrase that a user enters to identify the information they want to retrieve, and which defines the conditions for searching a database.

[1409] A "generative AI model" refers to a pre-trained artificial intelligence algorithm that is capable of performing advanced data processing and search based on input prompts.

[1410] A "prompt" is a piece of text containing instructions or questions that are used as input to a generative AI model, leading to an appropriate response or action from the AI.

[1411] "Formatted display" refers to converting the search results obtained into an easy-to-read format and presenting the content in a visually easy-to-understand manner.

[1412] "Sender identification information" refers to information for identifying the sender of the message, and specifically includes the name and ID.

[1413] The present invention is a system that searches information acquired from multiple information transmission means in one go and efficiently displays the results. This system is designed to enable users to quickly search and acquire information and improve work efficiency. Specific embodiments of the system are described below.

[1414] Server-side configuration and operation

[1415] The server obtains information from multiple information transmission means (electronic messaging tools, dialogue tools, and other communication tools). Specific information transmission means include email tools (e.g., Gmail API) and chat tools (e.g., Slack API). The server periodically obtains data using the APIs of these tools. For example, the server obtains unread emails using the Gmail API and the latest messages using the Slack API.

[1416] The acquired information is converted into a unified format. This is to unify data of different formats into a consistent format. For example, email data acquired from Gmail is converted into fields of "Subject", "Body", "Sender", and "Date", and Slack message data is converted into fields of "Channel Name", "Body", "Sender", and "Date".

[1417] The converted data is stored in a database, such as MySQL or MongoDB, along with metadata such as the user ID and the name of the information transmission method.

[1418] The information stored in the database is indexed using Elasticsearch, which allows for fast searches based on date, sender, content keywords, etc. For example, you can quickly search for relevant data based on keywords such as "project progress."

[1419] When a user enters a search query from a terminal, the query is sent to the server. The server searches the database based on the received search query and retrieves a list of relevant messages. The search results are formatted and formatted to include the date, hit content, sender name, and information transmission method name. The formatted search results are sent to the terminal and displayed on the screen.

[1420] It also has a prompt input function that uses the generative AI model. For example, a prompt such as "Please search for the latest messages regarding project progress" can be input into the generative AI model to perform advanced data search processing.

[1421] Terminal configuration and operation

[1422] A user enters a search keyword into the search field on their device. For example, they enter "project progress." When the user presses the search button, the entered search query is sent to the server. The search query is sent to the server's API endpoint using an HTTPS request.

[1423] The search results received from the server are displayed on one screen on the terminal. The search result list is rendered in HTML format, and each item displays the date, hit content (excerpt of the text), sender identification information, and information transmission means identification information.

[1424] Specific examples of operation

[1425] For example, if a user searches for "project progress," the following happens:

[1426] 1. User Action:

[1427] The user enters "project progress" into the search field on the terminal and clicks the search button.

[1428] 2. Request from device to server:

[1429] The terminal sends a search query containing the keyword "project progress" to the server.

[1430] 3. Server lookup process:

[1431] The server uses Elasticsearch to search the database for messages that correspond to "project progress."

[1432] 4. Sending from the server to the device:

[1433] Send the formatted search results to the device.

[1434] 5. Terminal display processing:

[1435] The terminal displays the received search results in a list format.

[1436] In this way, the system of the present invention is capable of efficiently searching and acquiring necessary information from multiple information sources all at once.

[1437] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1438] Step 1: Information Acquisition

[1439] The server obtains data from multiple information transmission means. Specific information transmission means include email tools (e.g., Gmail API) and chat tools (e.g., Slack API). When using the Gmail API to obtain unread emails, the "messages.list" endpoint is used to obtain the unread message IDs, and the "messages.get" endpoint is used to obtain the contents of each message. When using the Slack API to obtain the latest messages, the "conversations.history" endpoint is used to obtain the list of latest messages.

[1440] Input: API endpoint of the communication medium

[1441] Data processing and calculation: Acquire data in XML or JSON format by calling the API

[1442] Output: Raw data obtained (unread email list, chat messages)

[1443] Step 2: Format conversion

[1444] The server converts the information it receives into a unified format. Email data from Gmail is converted into fields such as "Subject," "Body," "Sender," and "Date," while Slack messages are converted into fields such as "Channel Name," "Body," "Sender," and "Date."

[1445] Input: Raw data (unread email list, chat messages)

[1446] Data processing and calculation: JSON parsing and field conversion operations

[1447] Output: Uniformly formatted data

[1448] Step 3: Save to database

[1449] The server stores the converted data in a database, such as MySQL or MongoDB, and can also include metadata such as the user ID and the name of the information transmission method.

[1450] Input: Uniformly formatted data

[1451] Data processing and calculation: SQL insert operation or NoSQL data insert operation

[1452] Output: Data entries stored in the database

[1453] Step 4: Index generation

[1454] The server creates an index of the stored information, using Elasticsearch to create indexes based on date, sender, content keywords, etc. This allows for fast searches.

[1455] Input: Data entry stored in a database

[1456] Data processing and calculation: Index generation operation

[1457] Output: Elasticsearch index

[1458] Step 5: Receiving a search query

[1459] The server receives a search query from the user. When the user enters a keyword into the search field of the terminal and presses the search button, the search query is sent from the terminal.

[1460] Input: User search query (e.g. "project progress")

[1461] Data processing and calculation: HTTPS request parsing

[1462] Output: Parsed search query

[1463] Step 6: Database Search

[1464] The server searches the database based on the received search query, using Elasticsearch to quickly find messages that match "Project Progress."

[1465] Input: Parsed search query

[1466] Data processing and calculation: Executing Elasticsearch queries

[1467] Output: Search results (list of matching messages)

[1468] Step 7: Formatting search results

[1469] The server formats the search results and converts them into a displayable format, including each field of the email or chat message included in the search results (date and time, excerpt of the message body, sender identification information, and information transmission method name).

[1470] Input: Search results (list of matching messages)

[1471] Data processing and calculation: Result formatting operations

[1472] Output: Formatted search results

[1473] Step 8: Submit search results

[1474] The server sends the formatted search results to the terminal, which displays the formatted results on the terminal screen.

[1475] Input: Formatted search results

[1476] Data processing and calculation: HTTP response generation

[1477] Output: Search results sent to your device

[1478] Step 9: Displaying search results

[1479] The terminal displays the search results received from the server in a list format. Using an HTML template, the search results are displayed with the date and time, hit content, sender identification information, and information transmission method name.

[1480] Input: Search results received from the server

[1481] Data processing and calculation: HTML rendering

[1482] Output: Search results displayed on the screen

[1483] Step 10: Enter a prompt into the generative AI model

[1484] The user inputs a prompt into the generative AI model, such as "Please search for the latest messages about project progress," and the AI ​​performs advanced data search processing.

[1485] Input: Prompt from the user

[1486] Data processing and calculation: Analysis and search operations using generative AI models

[1487] Output: AI-generated search results

[1488] (Application example 1)

[1489] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1490] In modern factories, data from various sources is scattered, making it difficult to efficiently collect and manage it and quickly search and retrieve specific information. Furthermore, in complex manufacturing processes, when an abnormality occurs, the person in charge must quickly identify the cause and take measures, but the dispersion of information makes this task cumbersome. For this reason, there is a need for a system that can quickly integrate data obtained from various sources within the factory and efficiently search and reference it.

[1491] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1492] In this invention, the server includes: means for acquiring information from a plurality of information transmission means; means for converting the information acquired from the information transmission means into a unified format; means for storing the converted information in a database; means for generating an index for the stored information; means for receiving a search query; means for searching the database based on the received search query; means for formatting and displaying search results; and means, which is implemented in a factory robot, for collecting, formatting, and displaying data from each information source in the factory. This makes it possible to collectively collect, and quickly search and acquire, necessary information from a plurality of data sources in the factory.

[1493] "Information transmission means" refers to means for exchanging data and information, and specifically includes email tools, chat tools, manufacturing data collection tools, and other communication tools.

[1494] "Harmonized format" refers to standards and formats for converting data captured in different formats into a consistent format.

[1495] A "database" refers to a system in which data is systematically organized and stored for retrieval and storage.

[1496] An "index" refers to a data structure created to quickly search data and is used to optimize search speed.

[1497] "Computer robot" refers to a mechanical device that operates automatically and is used to perform tasks, especially in a factory.

[1498] A "search query" refers to an instruction or request sent to a database or information system to retrieve specific information.

[1499] "Search Results" refers to a list of information or data retrieved based on a search query.

[1500] The "formatting and displaying means" refers to a method or means for converting search results into a format that is easy for the user to understand and displaying them.

[1501] "Data collection methods" refers to the methods and means used to obtain the required data from the source of interest.

[1502] This invention provides a system that is implemented in a factory robot and that collectively collects necessary information from multiple information sources within the factory, and efficiently searches and acquires the information. A specific embodiment of this system is shown below.

[1503] System Overview

[1504] The system includes the following major components:

[1505] 1. Server

[1506] 2. Factory robots

[1507] 3. Database

[1508] 4. User Interface

[1509] server

[1510] The server is responsible for collecting information from multiple sources within the factory, converting it into a unified format, and storing it in a database. It also receives search queries from users, searches the database, formats the results, and sends them to factory robots and user devices.

[1511] The server uses the following hardware and software:

[1512] Hardware: High-performance server (e.g., server with Intel Xeon processor)

[1513] Software: REST API module, MySQL database, data conversion scripts (e.g. Python scripts)

[1514] Factory robots

[1515] Factory robots work in conjunction with a server to collect, format, and display data from various sources within the factory. The robots periodically collect information using APIs and send the necessary data to the server.

[1516] Factory robots use the following hardware and software:

[1517] Hardware: Industrial robots (e.g. industrial robots)

[1518] Software: Robot control software (e.g. ROS)

[1519] Database

[1520] The database stores the data sent from the server in a unified format and generates an index to enable fast searches based on date, source, content keywords, etc.

[1521] The database uses the following software:

[1522] Software: MySQL database

[1523] User Interface

[1524] The user interface is used to input search queries and display search results. Users can search for the information they need through the interface and view the results in a list format.

[1525] Specific examples

[1526] For example, if a user searches for quality control data for products in a factory, the process would go something like this:

[1527] 1. User operation: The user enters "poor quality" in the search field of the terminal and clicks the search button.

[1528] 2. Request from device to server: The device sends a search query containing the keyword “poor quality” to the server.

[1529] 3. Server search process: The server searches the database for data that corresponds to "poor quality." It formats the hit data and generates search results.

[1530] 4. Transmission from server to terminal: The formatted search results are transmitted to the terminal.

[1531] 5. Display processing on the terminal: The terminal displays the received search results in a list format, including the date, product ID, measurement value, and reference value.

[1532] Prompt Sentence Examples

[1533] Using a generative AI model, you can generate a program to build a system that collects relevant data and efficiently searches and displays it by entering a prompt like the following:

[1534] Please create a Python program that saves the quality data obtained from the API below in a MySQL database, and searches for and displays data that meets a specific condition (e.g., measurement value greater than or equal to 10).

[1535] API URL: https: / / api.example.com / quality_data

[1536] This makes it possible to collect necessary information from multiple data sources within the factory in one go and efficiently search and retrieve it.

[1537] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1538] Step 1: Gather information

[1539] The server obtains information from various sources within the factory (email tools, chat tools, manufacturing data collection tools, etc.). These tools send data to the server through their respective APIs. The input is data from each source, and the output is the raw data collected by the server. Specifically, it periodically sends requests to the API endpoint and receives response data.

[1540] Step 2: Data conversion

[1541] The server converts the acquired information into a unified format. To convert data of different formats into a consistent format, for example, each data item is classified into fields such as "product ID," "quality standard," "measurement value," and "date." The input is the acquired raw data, and the output is data converted into a unified format. Specifically, the server uses a data conversion script (e.g., a Python script) to analyze the data and extract and convert the required information.

[1542] Step 3: Save to database

[1543] The server saves the data converted into a unified format in a database. The input is the data converted into a unified format, and the output is the data stored in the database. Specifically, it connects to a database management system (e.g., MySQL), executes an SQL query, and saves the data.

[1544] Step 4: Indexing

[1545] The server creates an index for the stored data. The index is created based on date, source, content keywords, etc., allowing for fast searches. The input is the data in the database, and the output is the indexed database. Specifically, it runs SQL indexing queries and indexes the appropriate fields.

[1546] Step 5: Search query reception

[1547] The terminal receives a search query from a user. The input is a search keyword entered by the user, and the output is a search query sent to the server. Specifically, the terminal generates a query by entering keywords through a user interface and clicking a search button.

[1548] Step 6: Data Retrieval

[1549] The server searches the database based on the received search query. The input is the received search query, and the output is a list of matching search results. Specifically, it executes the SQL search query and extracts data that matches the criteria.

[1550] Step 7: Formatting search results

[1551] The server formats the search results and converts them into a display format. The input is the raw search results, and the output is the formatted data for display. Specifically, the server organizes the search results into fields such as "date," "product ID," "measurement value," and "reference value," and formats them in a format that is easy for users to view.

[1552] Step 8: View the results

[1553] The terminal displays the formatted search results to the user. The input is the formatted search results received from the server, and the output is the result list displayed to the user. Specifically, the terminal uses widgets and components for displaying results in a user interface to display the search results on the screen.

[1554] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1555] This invention provides a user-friendly search experience by combining a system that searches information acquired from multiple information transmission means in one go and efficiently displays the results with an emotion engine that recognizes the user's emotions. This system enables users to quickly search and acquire information and provides more appropriate information according to their emotions at the time.

[1556] A natural language description of the program's operation

[1557] Server-side processing

[1558] First, the server obtains information from multiple information transmission means. Specific information transmission means include email tools, chat tools, and other communication tools. The server periodically obtains the necessary data using the APIs of these tools. For example, it obtains unread emails from email tools and the latest messages from chat tools.

[1559] Next, the acquired information is converted into a unified format. This is to unify data of different formats into a consistent format. For example, email data is converted into fields for "subject," "body," "sender," and "date."

[1560] The converted data is stored in a database, along with metadata such as the user ID and tool type.

[1561] It generates an index for the information stored in the database, which can be quickly searched based on date, sender, content keywords, etc.

[1562] When a user enters a search query from their device, the query is sent to the server. The server searches the database based on the received search query and retrieves a list of relevant messages. The server's emotion engine recognizes the user's emotions based on the user's input, operation history, voice input, and data obtained from biometric sensors. Search results are filtered or prioritized based on this recognized emotion.

[1563] Finally, the search results are formatted and displayed in the format of the date, hit content, sender name, information transmission means name, etc. The formatted search results are sent to the terminal and displayed on the screen.

[1564] Terminal side processing

[1565] The user enters a search keyword into the search field on the device, for example, "project progress."

[1566] When the search button is pressed, the entered search query is sent to the server.

[1567] The search results received from the server are displayed on one screen, including the date, hit content, sender name, and information transmission method name.

[1568] Specific examples

[1569] For example, if a user searches for "project progress," the following happens:

[1570] 1. User Action:

[1571] The user enters "project progress" into the search field on the terminal and clicks the search button.

[1572] 2. Request from device to server:

[1573] The terminal sends a search query containing the keyword "project progress" to the server.

[1574] 3. Server lookup process:

[1575] The server searches the database for messages that correspond to "Project Progress."

[1576] The emotion engine determines the user's emotion and filters or prioritizes search results based on that emotion.

[1577] The hit messages are formatted and search results are generated.

[1578] 4. Sending from the server to the device:

[1579] Send the formatted search results to the device.

[1580] 5. Terminal display processing:

[1581] The terminal displays the received search results in a list format.

[1582] for example:

[1583] Date: 2023-10-01

[1584] Hit: "Project progress is going well"

[1585] Sender name: Yamada Taro

[1586] Name of information transmission method: Email tool

[1587] Date: 2023-09-25

[1588] Hits: "I'll report on the project progress at the next meeting."

[1589] Sender name: Hanako Sato

[1590] Name of information transmission method: Chat tool

[1591] In this way, the system of the present invention allows users to search for the information they need from multiple information sources at once, and provides more appropriate information that reflects their emotions at the time, thereby improving work efficiency.

[1592] The processing flow will be explained below.

[1593] Step 1: Collect data

[1594] 1. The server periodically connects to the APIs of email, chat, and other communication tools.

[1595] 2. Use each tool's API credentials (API key, authentication token, etc.) to retrieve user data.

[1596] 3. For example, get unread emails from an email tool, or get the latest chat messages from a chat tool.

[1597] Step 2: Transform the data

[1598] 1. The server converts the acquired data into a unified format.

[1599] 2. For example, convert email data into fields of "Subject", "Body", "Sender", and "Date", and convert chat messages into the format of "Message Content", "Sender", and "Date".

[1600] Step 3: Save your data

[1601] 1. The server saves the converted data in the database.

[1602] 2. When saving, metadata such as user ID and tool type are also saved.

[1603] Step 4: Generate indexes

[1604] 1. The server generates an index for the stored data.

[1605] 2. Indexes are created based on fields such as date, sender, and content keywords to enable fast searches.

[1606] Step 5: Emotion Recognition with the Emotion Engine

[1607] 1. The server's emotion engine collects user input, operation history, voice input, and data from biometric sensors.

[1608] 2. Determine the user's emotions based on the collected data.

[1609] 3. For example, if a user is feeling stressed, generate search results that reflect that.

[1610] Step 6: Receiving a search query

[1611] 1. The user enters a search keyword (e.g., "project progress") into the search field on the device.

[1612] 2. When you press the search button, the entered search query is sent from the device to the server.

[1613] Step 7: Search the database

[1614] 1. The server searches the database index based on the received search query.

[1615] 2. As a search result, a list of relevant messages is obtained.

[1616] 3. Filter or prioritize search results based on user sentiment as recognized by the sentiment engine.

[1617] Step 8: Formatting the search results

[1618] 1. The server formats the search results.

[1619] 2. Search results are organized by date, hit content, sender name, information transmission method name, etc.

[1620] 3. Send the formatted search results to the device.

[1621] Step 9: Viewing search results

[1622] 1. The terminal displays the search results received from the server.

[1623] 2. Search results are displayed on one screen, and each item includes the date, hit content, sender name, and information transmission method name.

[1624] In this way, by using the system of the present invention, users can not only search and obtain information from multiple information sources all at once using a single search box, but also use the emotion engine to obtain the most appropriate information based on their emotions at the time.

[1625] Example 2

[1626] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1627] In today's information society, users use multiple communication methods to obtain a wide variety of information. However, this information is often provided in different formats, making it difficult for users to quickly search and obtain the information they need. In addition, information is not provided in a way that reflects the user's emotions, resulting in a poor user experience.

[1628] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1629] In this invention, the server includes means for acquiring information from a plurality of information transmission means, means for converting the information acquired from the information transmission means into a unified format, means for storing the converted information in a database, means for generating an index for the stored information, means for receiving a search query, means for searching the database based on the received search query, means for formatting and displaying search results, and means for recognizing a user's emotion and filtering or preferentially displaying search results based on the emotion, thereby enabling a user to search for necessary information from a plurality of information sources at once and providing more appropriate information according to the user's emotion at that time.

[1630] "Information transmission means" refers to email tools, chat tools, and other communication tools.

[1631] A "uniform format" is a data structure for converting information obtained from different information transmission means into a consistent format.

[1632] A "database" is a structured collection of data that stores transformed information and allows it to be efficiently searched and retrieved.

[1633] An "index" is a data structure used to quickly search for information in a database.

[1634] A "search query" is data containing keywords or phrases that a user enters to specify the information they want to obtain.

[1635] "Search results" are a collection of relevant information retrieved from a database based on a search query.

[1636] An "emotion engine" is a system for recognizing a user's emotions and filtering or prioritizing search results based on those emotions.

[1637] "Filtering" is a process of extracting only necessary information from data acquired based on specific criteria.

[1638] The "display means" is a system for visually presenting search results to the user.

[1639] "Metadata" is additional information that includes information about the content of the data (for example, user ID, type of tool, etc.).

[1640] The present invention is a system that allows users to quickly search for the information they need from multiple information sources and provides appropriate information according to their emotions at the time. This system is realized by the server and the terminals each fulfilling their respective roles.

[1641] First, the server obtains information from multiple information transmission means. Specific information transmission means include email tools (e.g., Gmail API), chat tools (e.g., Slack API), and other communication tools. The server periodically obtains the necessary data using each of these APIs. For example, it obtains unread emails from the email tool and the latest messages from the chat tool.

[1642] The server then converts the information into a unified format, allowing different data types to be unified into a consistent format. For example, email data is converted into fields for "subject," "body," "sender," and "date."

[1643] The converted data is stored in a database, including metadata such as the user ID and tool type. The server then creates an index for the information stored in the database. This index is created based on date, sender, content keywords, etc., allowing for fast searches.

[1644] When a user enters a search query into the device's search field, the query is sent to the server. The server searches the database based on the received search query and retrieves a list of relevant messages. The server's emotion engine then recognizes the user's emotions based on the user's input, operation history, voice input, and data obtained from biometric sensors. The server then filters or prioritizes search results based on the recognized emotions.

[1645] Finally, the server formats and displays the search results in the format of the date, hit content, sender name, information transmission means name, etc. The formatted search results are sent to the terminal and displayed on the user's screen.

[1646] Specific examples

[1647] For example, if a user searches for "project progress," the following happens:

[1648] The user enters "project progress" into the search field of the device and clicks the search button. The device sends a search query containing the keyword "project progress" to the server. The server searches the database for messages that match "project progress." The emotion engine determines the user's emotion and filters or prioritizes the search results based on that emotion. The matched messages are formatted and search results are generated. The formatted search results are sent to the device, and the device displays the received search results in a list format.

[1649] Prompt Sentence Examples

[1650] "Describe a system that searches for project status using multiple sources of information and filters the results based on user sentiment."

[1651] In this way, users can search for related information all at once and get a better search experience based on their current emotions.

[1652] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1653] Step 1:

[1654] The server obtains information from multiple information transmission means.

[1655] Input: API endpoint of each tool

[1656] Specific behavior: The server uses the Gmail API to retrieve unread emails and the Slack API to retrieve the latest messages.

[1657] Output: Data from each tool (e.g., email subject, body, sender, date)

[1658] Step 2:

[1659] The server converts the acquired information into a unified format.

[1660] Input: Data for each tool acquired

[1661] Specific operation: Map the data fields obtained from each API to "Subject," "Body," "Sender," and "Date."

[1662] Output: Data converted into a unified format

[1663] Step 3:

[1664] The server stores the converted information in a database.

[1665] Input: Data converted into a unified format

[1666] What it does: Inserts data into a MongoDB collection, along with metadata such as user ID and tool type.

[1667] Output: Information stored in the database

[1668] Step 4:

[1669] The server generates an index for the stored information.

[1670] Input: Information stored in a database

[1671] What it does: Uses MongoDB's indexing capabilities to generate indexes based on date, sender, and keywords.

[1672] Output: An index that allows for fast searches

[1673] Step 5:

[1674] A user enters a search query into a search field on the device.

[1675] Input: User's search keyword (e.g. "project progress")

[1676] Specific operation: The user enters keywords into the search field on the device and presses the search button.

[1677] Output: The search query is sent to the server

[1678] Step 6:

[1679] The server searches the database based on the received search query.

[1680] Input: Search query sent from the device

[1681] Specific operation: The server sends a query containing the received search keywords to MongoDB and retrieves the relevant data.

[1682] Output: A list of matching messages

[1683] Step 7:

[1684] The emotion engine of the server recognizes the emotion of the user.

[1685] Input: User operation history, voice input, biometric sensor data

[1686] Specific operation: The emotion engine on the server analyzes these data and estimates the user's current emotion.

[1687] Output: Recognized emotion

[1688] Step 8:

[1689] The server filters or prioritizes search results based on the recognized sentiment.

[1690] Input: List of relevant messages, recognized emotions

[1691] Specific behavior: For users who are stressed, prioritize relevant and important messages.

[1692] Output: Filtered or prioritized search results

[1693] Step 9:

[1694] The server formats the search results and sends them to the terminal.

[1695] Input: Filtered or prioritized search results

[1696] Specific operation: The search results are formatted in the format of "date," "hit content," "sender name," and "information transmission method name," and the results are sent to the terminal in JSON format.

[1697] Output: Formatted search results sent to your device

[1698] Step 10:

[1699] The terminal displays the received search results in a list format.

[1700] Input: Formatted search results received from the server

[1701] Specific operation: The device renders the search results in HTML and displays them in a list format on the screen.

[1702] Output: Search results presented visually to the user

[1703] (Application example 2)

[1704] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1705] Conventional information acquisition systems have had difficulty in quickly and accurately providing the information users want. Furthermore, they have not provided information that corresponds to the user's emotional state, which has prevented them from improving the user experience. In particular, virtual stores need to provide optimal products and information depending on the user's emotional state, and technology to achieve this has been in demand.

[1706] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring information from multiple information transmission means, means for converting the information acquired from the information transmission means into a unified format, means for saving the converted information in a database, means for generating an index for the saved information, means for receiving a search query, means for searching the database based on the received search query, means for acquiring and analyzing user emotion data, means for filtering or preferentially displaying search results based on the user emotion data, and means for shaping and displaying the search results. This makes it possible to provide optimal product information according to the user's emotions in a virtual store, significantly improving the user experience.

[1707] "Multiple information transmission means" refers to means for sending and receiving information in different formats, such as email tools, chat tools, and other communication tools.

[1708] A "unified format" is a data format for converting data obtained from different information sources into a unified format.

[1709] "Database" means an information management system for storing, managing, and retrieving acquired and transformed information.

[1710] An "index" is a structured index that allows fast and efficient searching of information in a database.

[1711] A "search query" is a keyword or phrase entered by a user to identify the information they are seeking.

[1712] "Emotion data" is data indicating the user's emotional state obtained from facial expressions, voice, operation history, and the like.

[1713] "Filtering" is the process of selecting data based on specific criteria.

[1714] "Priority display" is a process of displaying data in the foreground based on specific criteria.

[1715] "Search Results" are information retrieved from a database based on a search query.

[1716] "Formatting and displaying" is a process of converting search results into an easy-to-read format and displaying them on the user's terminal.

[1717] This invention relates to a system for realizing efficient product searches in a virtual store. Specifically, it describes a method for recognizing a user's emotions and providing optimal search results based on those emotions. Below, we will explain in detail each component of this system and the processing it performs.

[1718] First, the server obtains information from multiple communication tools. This includes email, chat, and other communication tools. The server periodically obtains the necessary data using the APIs of these tools and converts it into a unified format. For example, from email, the data is converted into fields such as "subject," "body," "sender," and "date."

[1719] The converted data is stored in a database, including metadata such as the user ID and tool type. An index is generated for this stored information on the server side. This index is created based on date, sender, content keywords, etc., enabling fast searches.

[1720] Users log in to the virtual store using smart glasses or a head-mounted display and enter search keywords in the search field. Once a search query is submitted, the server searches the database based on the query and retrieves a list of relevant messages. At the same time, it also acquires and analyzes the user's emotional data. This is done using an emotion recognition engine that determines emotions through facial recognition, voice tone, etc.

[1721] Based on the emotional data, the server can filter or prioritize search results. For example, if the user is in a hurry, products that are available for immediate purchase and have good reviews will be prioritized. This allows the server to provide information optimized for the user's emotional state.

[1722] Finally, the formatted search results are sent to the terminal and displayed in a list format on the user's screen, including the date, hit content, source name, and information transmission method name.

[1723] For example, if a user searches for "new smartphone" and the emotion engine determines that the user is "in a hurry," smartphones that are available for immediate purchase and have high reviews will be prioritized. On the other hand, if the user is relaxed, new products and unique suggestions will be prioritized.

[1724] Prompt Sentence Examples

[1725] Design an application that suggests the best filtered products when a user searches for "new smartphone" and detects that they are "in a hurry." The products that are displayed preferentially should be "available for immediate purchase" and have "good reviews." Please explain with specific product examples.

[1726] In this way, optimal search results based on the user's emotions can be provided in real time, enabling an efficient and satisfying shopping experience within the virtual store.

[1727] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1728] Step 1:

[1729] The server obtains information from multiple information transmission methods. In this case, it periodically collects the necessary data (unread emails, latest messages, etc.) from email tools, chat tools, and other communication tools via API. The input is the raw data from each information transmission method, and the output is the obtained raw data.

[1730] Step 2:

[1731] The server converts the information it receives into a unified format. For example, email data is converted into fields such as "subject," "body," "sender," and "date." The input is the raw data obtained in step 1, and the output is the data converted into a unified format.

[1732] Step 3:

[1733] The server stores the converted information in a database, along with metadata such as the user ID and tool type. The input is data converted into a unified format, and the output is the data stored in the database.

[1734] Step 4:

[1735] The server generates an index for the stored information. The index is created based on date, sender, content keywords, etc. The input is the information stored in the database, and the output is an index to speed up searches.

[1736] Step 5:

[1737] A user inputs a search keyword into a search field on a device. For example, the user inputs "smartphone" as a search query. The input is the user's search query, and the output is the action of sending the query to a server.

[1738] Step 6:

[1739] The server searches the database based on the received search query and retrieves a list of relevant messages from the database. The input is the search query received from the user, and the output is the search results (a list of relevant messages).

[1740] Step 7:

[1741] The server acquires and analyzes the user's emotional data in real time. Using an emotion recognition engine, it determines the user's emotional state based on facial recognition, voice tone, etc. The input is the user's facial expression and voice data, and the output is the analyzed emotional state.

[1742] Step 8:

[1743] The server filters or prioritizes search results based on the user's emotional data. For example, if the user is in a hurry, products that are available for immediate purchase and have good reviews are prioritized. The input is the analyzed emotional state and search results, and the output is the filtered or prioritized search results.

[1744] Step 9:

[1745] The server sends the formatted search results to the terminal. The results are formatted to include the date, hit content, source name, and information transmission method name. The input is the filtered search results, and the output is the formatted search results sent to the terminal.

[1746] Step 10:

[1747] The terminal displays the received search results on the screen. The user can check the search results and obtain detailed information. The input is the formatted search results sent from the server, and the output is the search results displayed on the terminal screen.

[1748] Through this process, appropriate information is quickly provided according to the user's emotions, realizing an efficient search experience within the virtual store.

[1749] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1750] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1751] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1752] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1753] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1754] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1755] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1756] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1757] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1758] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1759] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1760] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1761] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1762] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1763] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1764] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1765] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1766] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1767] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1768] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1769] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1770] The following is further disclosed regarding the above embodiment.

[1771] (Claim 1)

[1772] means for acquiring information from a plurality of information transmission means;

[1773] means for converting information acquired from the information transmission means into a unified format;

[1774] a means for storing the converted information in a database;

[1775] means for generating an index for the stored information;

[1776] means for receiving a search query;

[1777] means for searching the database based on the received search query;

[1778] means for formatting and displaying the search results;

[1779] A system including:

[1780] (Claim 2)

[1781] 2. The system according to claim 1, wherein the plurality of information transmission means include an email tool, a chat tool, and other communication tools.

[1782] (Claim 3)

[1783] 2. The system according to claim 1, wherein the search result formatting and displaying means displays the search results in a format including the date, hit content, sender name, and information transmission means name.

[1784] "Example 1"

[1785] (Claim 1)

[1786] means for acquiring information from a plurality of information transmission means;

[1787] means for converting information acquired from the information transmission means into a unified format;

[1788] a means for storing the converted information in a database;

[1789] means for generating an index for the stored information;

[1790] means for receiving a search query;

[1791] means for searching the database based on the received search query;

[1792] means for formatting and displaying the search results;

[1793] a means for inputting a prompt sentence to the generative AI model;

[1794] A system including:

[1795] (Claim 2)

[1796] 10. The system of claim 1, wherein the plurality of communication vehicles includes electronic messaging tools, dialogue tools, and other communication tools.

[1797] (Claim 3)

[1798] 2. The system according to claim 1, wherein the search result formatting and displaying means displays the search results in a format including date and time, hit content, sender identification information, and information transmission means identification information.

[1799] "Application Example 1"

[1800] (Claim 1)

[1801] means for acquiring information from a plurality of information transmission means;

[1802] means for converting information acquired from the information transmission means into a unified format;

[1803] a means for storing the converted information in a database;

[1804] means for generating an index for the stored information;

[1805] means for receiving a search query;

[1806] means for searching the database based on the received search query;

[1807] means for formatting and displaying the search results;

[1808] A means implemented in the factory robot to collect, format and display data from various sources within the factory;

[1809] A system including:

[1810] (Claim 2)

[1811] 2. The system according to claim 1, wherein the plurality of information transmission means include an email tool, a chat tool, a manufacturing data collection tool, and other communication tools.

[1812] (Claim 3)

[1813] The system according to claim 1, characterized in that the means for formatting and displaying the search results displays the search results in a format including the date, hit content, sender name, information transmission means name, and product ID.

[1814] "Example 2: Combining Emotion Engines"

[1815] (Claim 1)

[1816] means for acquiring information from a plurality of information transmission means;

[1817] means for converting information acquired from the information transmission means into a unified format;

[1818] a means for storing the converted information in a database;

[1819] means for generating an index for the stored information;

[1820] means for receiving a search query;

[1821] means for searching the database based on the received search query;

[1822] means for formatting and displaying the search results;

[1823] means for recognizing a user's emotion and filtering or prioritizing search results based on the emotion;

[1824] A system including:

[1825] (Claim 2)

[1826] 2. The system according to claim 1, wherein the plurality of information transmission means include an email tool, a chat tool, and other communication tools.

[1827] (Claim 3)

[1828] 2. The system according to claim 1, wherein the search result formatting and displaying means displays the search results in a format including the date, hit content, sender name, and information transmission means name.

[1829] "Application example 2 when combining emotion engines"

[1830] (Claim 1)

[1831] means for acquiring information from a plurality of information transmission means;

[1832] means for converting information acquired from the information transmission means into a unified format;

[1833] a means for storing the converted information in a database;

[1834] means for generating an index for the stored information;

[1835] means for receiving a search query;

[1836] means for searching the database based on the received search query;

[1837] A means for acquiring and analyzing user emotion data;

[1838] means for filtering or prioritizing search results based on user emotion data;

[1839] means for formatting and displaying the search results;

[1840] A system including:

[1841] (Claim 2)

[1842] 2. The system according to claim 1, wherein the plurality of information transmission means include an email tool, a chat tool, and other communication tools.

[1843] (Claim 3)

[1844] 2. The system according to claim 1, wherein the search result formatting and displaying means displays the search results in a format including the date, hit content, sender name, and information transmission means name. [Explanation of symbols]

[1845] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for acquiring information from a plurality of information transmission means; means for converting information acquired from the information transmission means into a unified format; a means for storing the converted information in a database; means for generating an index for the stored information; means for receiving a search query; means for searching the database based on the received search query; means for formatting and displaying the search results; A system including:

2. 2. The system according to claim 1, wherein the plurality of information transmission means include an email tool, a chat tool, and other communication tools.

3. 2. The system according to claim 1, wherein the search result formatting and displaying means displays the search results in a format including the date, hit content, sender name, and information transmission means name.

Images