System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving information entered by a user; means for storing the received information in a database; means for generating metadata for the stored information; means for tagging and indexing the metadata; means for parsing a user search request using natural language processing techniques; means for retrieving information from the database based on the parsed search request; and means for presenting search results to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, information overload makes it difficult to quickly find the information you need. Because humans have limited memory and time, they often struggle to remember where they recorded what. In particular, working adults and students who want to use information efficiently need a way to quickly retrieve saved information. Furthermore, the lack of a system that notifies users of set reminders at the appropriate time increases the risk of missing important information or tasks. To solve these issues, an efficient information management system utilizing natural language processing technology and machine learning is needed. [Means for solving the problem]

[0005] The present invention provides a system that receives and stores information entered by a user, generates metadata for that information, tags it, and indexes it. It also provides a system that includes a means for analyzing a user's search request using natural language processing technology and searching for related information from a database. It also adds a means for setting a reminder, saving the reminder information in a database, scheduling notifications based on set dates and conditions, and notifying the user. The system also includes a means for analyzing a user's past search history and saved information using a machine learning algorithm to infer and present related information. These means enable users to quickly and efficiently retrieve the information they need and receive reminders at the appropriate time.

[0006] "User" refers to a person who uses this system.

[0007] "Information" refers to all data entered or saved by users, including text, audio, images, video, and conversations with AI.

[0008] "Input" refers to the act or operation by which a user provides information to a system.

[0009] "Metadata" is attribute information about the saved information, and includes the date and time of saving, tags, user ID, type of content, etc.

[0010] "Tags" refer to keywords or labels that are assigned to classify and organize information.

[0011] An "index" refers to a structure that allows efficient searching of information in a database.

[0012] "Natural language processing" refers to the technology that allows computers to understand and analyze human language.

[0013] A "search request" refers to the keywords or phrases a user enters to find specific information.

[0014] A "database" refers to a structured collection of data for the purpose of systematically managing and storing information.

[0015] A "machine learning algorithm" is a set of rules or steps used to learn patterns from data and make predictions or classifications.

[0016] "Reminder" refers to a setting that sends notifications to users at specific times or under specific conditions.

[0017] "Notifications" refer to messages or alerts that inform users of information based on set conditions or times.

[0018] "Analysis" refers to the process of examining data or information to find meaning and patterns.

[0019] "Related information" refers to data that the system determines to be relevant to the information the user is currently searching for or using.

[0020] "Inference" refers to the process of discovering new information or possibilities based on existing data and patterns.

[0021] By defining it in this way, we can clarify the meaning of important words related to the memory assist lens "Memoria." [Brief explanation of the drawings]

[0022] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0023] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0024] First, the terms used in the following description will be explained.

[0025] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0026] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0027] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0028] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0029] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0030] [First embodiment]

[0031] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0032] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0033] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0034] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0035] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0036] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0037] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0038] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0039] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0040] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0041] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0042] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0043] The present invention relates to a system that efficiently manages and rediscovers information stored by users (text, audio, images, videos, conversations with AI, etc.) and provides the information to users in an optimal way. Specific program processing and its implementation are described below.

[0044] Retention of Information

[0045] Subject: User

[0046] The user opens an application on the device and inputs the required information as text, or records voice, images, or videos. For example, if the user wants to input an idea for a new project as text, the user enters the idea in a dedicated input field.

[0047] Subject: Terminal

[0048] The device temporarily stores the input information and sends it to the server, generating metadata such as the date and time of storage, tags, user ID, and content type.

[0049] Subject: Server

[0050] The server stores the received information and metadata in a database, tags and indexes the stored information, and updates the metadata.

[0051] Information Search and Retrieval

[0052] Subject: User

[0053] Users can enter keywords related to the information they need in the application's search screen, for example, by entering "project ideas," to search for previously saved project ideas.

[0054] Subject: Terminal

[0055] The terminal acquires the input keywords and sends a search request to the server.

[0056] Subject: Server

[0057] The server receives the search request, analyzes the keywords using a natural language processing module, searches for related information from the database based on the analysis results, generates a list of results, and sends it to the terminal.

[0058] Subject: Terminal

[0059] The terminal displays the search result list received from the server to the user, who can then refer to the list to find the information they need.

[0060] Reminders and notifications

[0061] Subject: User

[0062] Users can set reminders for the information they save, for example, "Remind me to review new project ideas tomorrow at 10 AM."

[0063] Subject: Terminal

[0064] The device sends the reminder setting information to the server.

[0065] Subject: Server

[0066] The server receives the reminder setting information, stores it in a database, schedules notifications based on the reminder date and time and conditions, and notifies the user when the set date and time or conditions are met.

[0067] Subject: Terminal

[0068] When the set date, time, or conditions are met, the device will send a reminder notification to the user, who can then check the notification and take action as needed.

[0069] Related information suggestions

[0070] Subject: Server

[0071] The server periodically analyzes the user's past search history and saved information, and uses natural language processing technology and machine learning algorithms to predict related information and generate a list of suggestions.

[0072] Subject: Terminal

[0073] The terminal presents the user with a list of suggestions, which the user can refer to for quick access to relevant information.

[0074] As described above, the system of the present invention integrates multiple functions to enable users to efficiently manage and rediscover information. As a specific example, users can save ideas for new projects, retrieve information when needed by keyword search, and set reminders to receive notifications at appropriate times. This system allows users to manage important information without missing it and utilize it efficiently.

[0075] The processing flow will be explained below.

[0076] Retention of Information

[0077] Subject: User

[0078] Step 1:

[0079] The user opens the application using a terminal.

[0080] Step 2:

[0081] The user inputs information, either through text, voice, image, video, or conversation with the AI.

[0082] Subject: Terminal

[0083] Step 3:

[0084] The terminal stores the input information in temporary storage.

[0085] Step 4:

[0086] The device generates metadata including the date and time the information was saved, tags, user ID, and content type.

[0087] Step 5:

[0088] The device sends the input information and metadata to the server.

[0089] Subject: Server

[0090] Step 6:

[0091] The server stores the received information and metadata in a database.

[0092] Step 7:

[0093] The server tags and indexes the stored information.

[0094] Information Search and Retrieval

[0095] Subject: User

[0096] Step 1:

[0097] The user opens the application's search screen on their device and enters keywords.

[0098] Subject: Terminal

[0099] Step 2:

[0100] The terminal acquires the input keyword and sends a search request to the server.

[0101] Subject: Server

[0102] Step 3:

[0103] The server receives the search request and passes the keywords to the natural language processing module.

[0104] Step 4:

[0105] A natural language processing module analyzes keywords and extracts related tags and metadata.

[0106] Step 5:

[0107] The server generates a database query to retrieve the relevant information from the database.

[0108] Step 6:

[0109] The server lists the search results and sends them to the terminal.

[0110] Subject: Terminal

[0111] Step 7:

[0112] The terminal displays the search result list received from the server to the user.

[0113] Reminders and notifications

[0114] Subject: User

[0115] Step 1:

[0116] The user uses the device to select information for which they want to set a reminder.

[0117] Step 2:

[0118] The user sets the date, time, and conditions for the reminder.

[0119] Subject: Terminal

[0120] Step 3:

[0121] The device stores the reminder setting information in temporary storage and sends it to the server.

[0122] Subject: Server

[0123] Step 4:

[0124] The server receives the reminder setting information and stores it in a database.

[0125] Step 5:

[0126] The server generates a notification schedule based on the reminder date and time and conditions.

[0127] Step 6:

[0128] The server stores the notification schedule in temporary storage and registers it with the system clock.

[0129] Subject: Terminal

[0130] Step 7:

[0131] When the reminder date, time, or conditions are met, the device will notify the user.

[0132] Related information suggestions

[0133] Subject: Server

[0134] Step 1:

[0135] The server periodically analyzes the user's past search history and saved information.

[0136] Step 2:

[0137] A natural language processing module is used to calculate the relationships between stored information.

[0138] Step 3:

[0139] Machine learning algorithms are used to generate inferential models of relevant information.

[0140] Step 4:

[0141] The server generates a list of related information based on the inference model.

[0142] Subject: Terminal

[0143] Step 5:

[0144] The terminal obtains the related information list from the server and presents it to the user.

[0145] Example 1

[0146] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0147] Users generate and store a large amount of information on a daily basis, but it is difficult to efficiently manage that information and quickly find it again when needed. Users also desire a reminder function to ensure they don't forget important information, but manually setting this function is cumbersome. Furthermore, there is a lack of systems that automatically suggest related information based on a user's past behavior. An integrated information management system that can solve these issues is needed.

[0148] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0149] In this invention, the server includes means for receiving information entered by a user, means for saving the received information in a storage device, means for generating additional information for the saved information, means for tagging and indexing the additional information, means for analyzing a user's search request using natural language processing technology, means for searching for information from the storage device based on the analyzed search request, means for presenting search results to the user, means for receiving reminders set by the user, means for saving reminder information in a storage device, means for scheduling notifications based on the reminder date and time and conditions, means for notifying the user when the set date and time and conditions are met, means for analyzing the user's past search history and saved information, means for using a machine learning algorithm to infer related information, means for generating a list of inferred related information, and means for presenting the generated related information list to the user. This allows the user to efficiently manage saved information and quickly rediscover it when needed, and the reminders allow the user to manage important information without missing it and also provide suggestions of related information based on past behavior.

[0150] "Means for receiving" refers to the function that allows a terminal or server to receive information or requests sent by a user.

[0151] "Storage device" means a physical or virtual device for storing data, including hard drives, SSDs, cloud storage, etc.

[0152] The "means for saving" is a function for writing received information or data to a storage device.

[0153] "Additional information" refers to additional information related to the content of the saved information, such as metadata such as tags, save date and time, and user ID.

[0154] "Means for generating" is a function for automatically generating specific additional information based on certain information or data.

[0155] "Tagging" is a method of adding specific keywords or categories to information to make it easier to organize and search.

[0156] "Indexing" is the process of building a data structure so that information can be efficiently searched.

[0157] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, and includes text analysis and language generation.

[0158] A "search request" is a request containing keywords or phrases entered by a user in search of specific information.

[0159] "Means of analysis" refers to the ability to understand the received search request and perform processing to find appropriate information.

[0160] The "searching means" is a function for searching for related information from a storage device based on the analyzed keywords.

[0161] "Means of presentation" refers to the function that allows a server or terminal to display search results and notifications to the user.

[0162] A "reminder" is a notification that a user sets for a specific date, time, or condition.

[0163] "Scheduling tools" are functions that allow you to plan notifications based on reminder dates and conditions.

[0164] "Means of notification" is a function for sending alerts or messages to users when a set date, time, or condition is met.

[0165] "Search history" refers to a record of searches a user has conducted in the past.

[0166] A "machine learning algorithm" is a computational method for analyzing data, discovering patterns, and using those patterns to predict future data.

[0167] A "related information list" is a list of information that is presumed to be useful to the user based on past search history and saved information.

[0168] The present invention relates to a system for efficiently managing user-generated information (text, audio, images, video, etc.) and rediscovering it when needed. Specific embodiments of the system are described below.

[0169] Retention of Information

[0170] 1. User input

[0171] Users open the application on their device and enter the required information: for text input, they write the information in a dedicated input field, for voice input, they press the microphone button to start recording, or for images and videos, they use the camera function.

[0172] Specific working example:

[0173] Enter your new project idea as "New website design" or, if you use voice recognition, say something like "Record a new project idea."

[0174] 2. Temporary storage on the device

[0175] The device saves the input information in temporary storage, generating metadata such as the save date and time, tags, user ID, and content type. The smartphone's internal memory is used as temporary storage.

[0176] 3. Sending from the device to the server

[0177] The device sends the stored information and metadata to the server, typically using an HTTP POST request to send the data.

[0178] 4. Server-based information storage and metadata updates

[0179] The server stores the received information and metadata in a database (e.g., MySQL), and a tagging engine runs to add appropriate tags to the text information.

[0180] Information Search and Retrieval

[0181] 1. User enters search keywords

[0182] Users enter keywords related to the information they need on the application's search screen.

[0183] Specific working example:

[0184] A user types in "project ideas" and presses the search button.

[0185] 2. Sending a search request via the device

[0186] The device receives the input keywords and sends a search request to the server. It generates an HTTP GET request and sends it to the server.

[0187] 3. Keyword analysis and search result generation by the server

[0188] The server uses a natural language processing library (e.g., NLTK or SpaCy) to analyze the keywords and search for relevant information in a storage device (database). The result list is generated in JSON format and sent to the device.

[0189] 4. Displaying search results on your device

[0190] The terminal displays the search result list received from the server to the user, displaying the search results in a list view so that the user can access the information they need.

[0191] Reminders and notifications

[0192] 1. User-defined reminder settings

[0193] Users can set reminders for saved information.

[0194] Specific working example:

[0195] The user enters the instruction "Set a reminder to check for new project ideas tomorrow at 10 AM."

[0196] 2. Send reminders via device

[0197] The device sends this reminder setting information to the server.

[0198] 3. Server-based reminder storage and scheduling

[0199] The server stores the reminder information in a database and sets up a notification scheduler (e.g., a Cron job or Quartz Scheduler).

[0200] 4. Device-based reminder notifications

[0201] When the set date, time, or conditions are met, a notification is triggered from the server and the device sends a reminder notification to the user.

[0202] Related information suggestions

[0203] 1. Server Generation of Proposal List

[0204] The server analyzes the user's past search history and saved information, and uses natural language processing techniques and machine learning algorithms (e.g., TensorFlow and PyTorch) to infer relevant information. A list of suggestions is generated periodically.

[0205] 2. Displaying a list of suggestions on the device

[0206] The terminal presents the user with a list of suggestions received from the server, and based on the information provided, the user can quickly access relevant information.

[0207] Examples and prompts

[0208] Examples:

[0209] 1. Storage of Information:

[0210] User: Texts a new project idea: "New website design."

[0211] Device: Sends information along with metadata to the server.

[0212] Server: Stores the information in a database and tags it.

[0213] 2. Information Search:

[0214] User: Search for "project ideas."

[0215] Device: Sends a search request to the server.

[0216] Server: Searches for relevant information and generates a list of results.

[0217] Device: Display search results to the user.

[0218] 3. Reminder:

[0219] User: Set a reminder to review new project ideas tomorrow at 10 AM.

[0220] Device: Sends reminder setting information to the server.

[0221] Server: Schedule reminders.

[0222] On your device: Reminds you at the set time.

[0223] Example prompt:

[0224] "Set a reminder to review new project ideas tomorrow at 10 AM."

[0225] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0226] Step 1:

[0227] The user opens an application on their device and inputs new information, for example, by typing a new project idea as text, "About a new website design," or by recording an audio, image, or video. It takes the user's data (text, audio, image, video) as input and generates data as output that is stored in temporary storage.

[0228] Step 2:

[0229] The device saves the information entered by the user in temporary storage. At this time, additional information such as the save date and time, tag, user ID, and content type is generated for the data saved in temporary storage. This allows information to be organized efficiently. Data from the user is received as input, and data with additional information is generated as output.

[0230] Step 3:

[0231] The device sends the stored information and additional information to the server. Specifically, it sends data using an HTTP POST request. It takes the data stored in temporary storage as input and generates an HTTP request that is sent to the server as output.

[0232] Step 4:

[0233] The server stores the received information and additional information in a database, performs appropriate tagging and indexing on the stored information, and updates the additional information. The server takes the data received by the server as input and generates information to be stored in the database as output.

[0234] Step 5:

[0235] The user enters keywords related to the information they need into the application's search screen. For example, they can enter "project ideas" to search for previously saved project ideas. The application takes search keywords as input and generates a search request as output.

[0236] Step 6:

[0237] The device acquires the search keywords and sends a search request to the server. Specifically, it generates an HTTP GET request and sends it to the server. It receives the search keywords from the user as input and generates and sends a search request as output.

[0238] Step 7:

[0239] The server receives the search request and analyzes the entered keywords using natural language processing techniques. Based on the analysis results, it searches for relevant information from the database. It takes the search request received by the server as input and generates a list of search results as output.

[0240] Step 8:

[0241] The terminal displays the search result list received from the server to the user. Specifically, the terminal displays the search results in a list view. The terminal receives the search results received from the server as input and generates the list of search results that is displayed to the user as output.

[0242] Step 9:

[0243] The user sets a reminder for the saved information, for example, "Remind me to review new project ideas tomorrow at 10 AM." The system takes the reminder setting information as input and generates a reminder setting request as output.

[0244] Step 10:

[0245] The device sends reminder setting information to the server. Specifically, it generates an HTTP POST request and sends it to the server. It receives reminder setting information from the user as input and generates and sends a reminder setting request as output.

[0246] Step 11:

[0247] The server receives the reminder setting information and stores it in a database. It schedules notifications based on the reminder date and time and conditions. It takes the reminder setting information received by the server as input and generates the scheduled notifications as output.

[0248] Step 12:

[0249] When the set date, time, or conditions are met, the device will notify the user with a reminder, for example, by displaying a reminder in the notification bar. It takes a reminder notification received from the server as input and generates a reminder notification to be displayed to the user as output.

[0250] Step 13:

[0251] The server analyzes the user's past search history and saved information, and uses natural language processing technology and machine learning algorithms to infer related information and generate a list of suggestions. It receives past search history and saved information as input and generates a list of suggestions as output.

[0252] Step 14:

[0253] The terminal presents the suggestion list received from the server to the user, allowing the user to quickly access relevant information. It takes the suggestion list received from the server as input and generates the suggestion list that is presented to the user as output.

[0254] (Application example 1)

[0255] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0256] In logistics centers, workers handle a wide variety of product information management tasks, and manual information entry and search is time-consuming, resulting in problems such as reduced accuracy and increased risk of errors. Furthermore, a lack of efficient reminder functions and suggestions for related information often leads to reduced work efficiency. There is a need for a system that can solve these problems, enable workers to quickly and accurately access the information they need, and improve work efficiency.

[0257] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0258] In this invention, the server includes means for receiving information entered by a user, means for storing the received information in a database, means for generating metadata for the stored information, means for tagging and indexing the metadata, means for analyzing a user's search request using natural language processing technology, means for searching for information from the database based on the analyzed search request, means for presenting search results to the user, means for acquiring information using a wearable device worn by a worker, and means for transmitting the acquired information to the server in real time. This makes it possible to use the wearable device to quickly and accurately manage information and improve work efficiency.

[0259] "User" refers to the entity that uses an information system.

[0260] A "wearable device" refers to an electronic device that can be worn by the user.

[0261] "Means for receiving information" refers to the mechanism by which user-entered information is incorporated into the data system.

[0262] "Database" refers to a system for systematically collecting, storing, and managing data.

[0263] "Metadata" refers to information about stored information, i.e., additional descriptions or attributes about that data.

[0264] "Tagging" refers to the process of classifying and organizing information or data by attaching labels or identification codes to it.

[0265] "Indexing" refers to the procedures and structures that enable quick search for necessary information from a large amount of data.

[0266] "Natural language processing technology" refers to technology for processing and analyzing human language (natural language) using a computer.

[0267] "Search Request" means a request or inquiry entered by a User to locate specific information.

[0268] "Real-time" refers to a situation in which information is acquired and processed instantly, with almost no time delay.

[0269] This invention is a system that uses wearable devices to manage and search information in order to improve work efficiency at logistics centers. Specifically, information is acquired through the wearable devices, sent to a server, and the necessary information is searched and displayed in real time.

[0270] Overall structure

[0271] The system consists of the following main components:

[0272] 1. Wearable devices: Devices worn by workers to input and display information, including smart glasses and smart watches.

[0273] 2. Server: The central system for storing, analyzing, searching, and notifying information.

[0274] 3. Database: A system connected to a server for storing saved information, metadata, search history, and reminder information.

[0275] Receiving and storing information

[0276] The wearable device captures information through voice input and barcode scanning by the worker. For example, a worker can scan the barcode of a new product and enter its details by voice. It can also record the product's condition using video recording. This information is sent in real time to a server, which stores it in a database. At the same time, metadata is generated, including the date and time of storage, tag, worker ID, and content type.

[0277] Information Search and Retrieval

[0278] Workers use the smart glasses to search for the information they need using voice commands. For example, they can say, "Tell me the inventory status of shelf number A3." The voice command is sent to the server, which analyzes it using natural language processing technology and searches for relevant information in the database. The results are displayed on the smart glasses, allowing workers to quickly obtain the information.

[0279] Reminders and notifications

[0280] Reminders are set by workers through voice input. For example, they can say, "Remind me tomorrow at 9:00 AM for the next loading / unloading operation." The reminder information is sent to the server and stored in a database. When the set date, time, or conditions are met, the server automatically sends a notification to the wearable device and displays the reminder. This helps workers avoid missing important tasks.

[0281] Related information suggestions

[0282] The server uses machine learning algorithms to analyze past search history and stored information. Based on this, it infers relevant information and makes suggestions to improve work efficiency. For example, if a frequently searched item is low in stock, it will make a replenishment suggestion. These suggestions are displayed to the worker through the smart glasses, allowing the worker to take action quickly.

[0283] Examples and prompts

[0284] For example, consider a scenario in which a worker saves details of a new item.

[0285] Example prompt sentence:

[0286] "Scan the barcode, then speak the product details."

[0287] Execution example:

[0288] Barcode scan: "1234567890123"

[0289] Speak: "This item is part of batch process B and is located on shelf number A3."

[0290] This information is immediately stored on the server and used for future searches and related information suggestions.

[0291] Specific examples of hardware and software used

[0292] Hardware: smart glasses (e.g., Google Glass), servers (high-performance servers in data centers), database servers (e.g., PostgreSQL)

[0293] Software: Natural language processing libraries (spaCy, NLTK), machine learning algorithms (TensorFlow, scikit-learn)

[0294] In this way, it is possible to efficiently manage and search information in the logistics center, provide reminder functions, and suggest related information.

[0295] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0296] Step 1:

[0297] Users input information using a wearable device. Specifically, a worker scans the barcode of a product with smart glasses and inputs detailed information about the product by voice. This input information includes the barcode value and voice text.

[0298] Input: Barcode scan data, voice input text

[0299] Output: Collected product information

[0300] Step 2:

[0301] The device sends the collected product information to a server in real time. Specifically, the smart glasses upload the barcode scan results and voice-input text to the server via the Internet, along with metadata such as the save date and time, tag, and worker ID.

[0302] Input: Collected product information (barcode data, voice text)

[0303] Output: Data sent to the server

[0304] Step 3:

[0305] The server stores the received information in a database, specifically by recording the information and metadata in a database system (e.g., PostgreSQL), including tagging and indexing.

[0306] Input: Data sent to the server

[0307] Output: Information stored in the database

[0308] Step 4:

[0309] Users can search for the information they need using voice commands. Specifically, a worker speaks into the smart glasses, saying, "Tell me the inventory status of shelf number A3." This voice is converted into text by the device and sent to the server.

[0310] Input: Voice command

[0311] Output: Text data of voice commands

[0312] Step 5:

[0313] The server receives the search request and analyzes it using natural language processing technology. Specifically, it uses a natural language processing library (e.g., spaCy) to analyze the voice command and extract the keywords necessary for the search.

[0314] Input: Text data of voice command

[0315] Output: Parsed keywords

[0316] Step 6:

[0317] The server searches for information from the database based on the analyzed keywords, generates a database query to retrieve the relevant information, and formats the query results into a list and sends it to the smart glasses.

[0318] Input: Parsed keyword

[0319] Output: Search result list

[0320] Step 7:

[0321] The terminal receives the search results from the server and displays them to the user. Specifically, the search results are visually displayed on the smart glasses display, allowing the worker to confirm the necessary information.

[0322] Input: Search result list

[0323] Output: Visually displayed search results

[0324] Step 8:

[0325] The user sets the reminder by voice. Specifically, the worker commands the smart glasses to "remind me for the next loading / unloading operation tomorrow at 9:00 AM." This voice data is converted into text by the device and sent to the server.

[0326] Input: Voice reminder

[0327] Output: Reminder text data

[0328] Step 9:

[0329] The server stores the reminder information in a database and sets the schedule. Specifically, the server appropriately records the reminder information in the database and sets the notification to be sent at the date and time specified in the schedule function.

[0330] Input: Reminder text data

[0331] Output: Reminder information and schedule settings stored in the database

[0332] Step 10:

[0333] The device will notify the user of the reminder at the set date and time by displaying a notification on the smart glasses and prompting the user to take the necessary action.

[0334] Input: Reminder information based on schedule settings

[0335] Output: Reminder notification

[0336] Step 11:

[0337] The server analyzes past search history and saved information and uses machine learning algorithms to predict related information. Specifically, it analyzes data using machine learning libraries (e.g., TensorFlow) and generates a list of related information.

[0338] Input: User's past search history and saved information

[0339] Output: Related information list

[0340] Step 12:

[0341] The terminal presents the generated related information list to the user, specifically by displaying the related information on smart glasses, helping the worker respond quickly.

[0342] Input: Related Information List

[0343] Output: Visually displayed relevant information

[0344] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0345] The present invention relates to a system that efficiently manages and rediscovers user-entered information, and further optimizes information search, reminder notifications, and related information suggestions by recognizing and utilizing user emotions. Specific embodiments of the present invention will be described below.

[0346] Retention of Information

[0347] Subject: User

[0348] The user opens the application on their device and inputs the required information using text, music, images, videos, or through conversation with the AI. For example, if they want to input an idea for a new project, they can enter their idea in a dedicated input field.

[0349] Subject: Terminal

[0350] The device saves the input information in temporary storage and sends it to the server, generating metadata about the save date and time, tags, user ID, and content type.

[0351] Subject: Server

[0352] The server stores the received information and metadata in a database, tags the stored information, creates an index, and updates the metadata.

[0353] Information Search and Retrieval

[0354] Subject: User

[0355] Users can use the application to open a search screen and enter keywords, such as "project ideas," to search for previously saved project ideas.

[0356] Subject: Terminal

[0357] The terminal acquires the input keywords and sends a search request to the server.

[0358] Subject: Server

[0359] The server receives the search request, analyzes the keywords using a natural language processing module, searches for related information from a database based on the analysis results, and sends the results list to the terminal.

[0360] Subject: Terminal

[0361] The terminal displays the search result list sent from the server to the user, who can then refer to the list and check the information they need.

[0362] Reminders and notifications

[0363] Subject: User

[0364] The user uses the application to select the information for which they want to set a reminder, and enter the date, time, and conditions for the reminder, such as "Remind me to review new project ideas tomorrow at 10 AM."

[0365] Subject: Terminal

[0366] The device sends the reminder setting information to the server.

[0367] Subject: Server

[0368] The server receives the reminder setting information, stores it in a database, schedules notifications based on the reminder date and time and conditions, and notifies the user when the set date and time or conditions are met.

[0369] Subject: Terminal

[0370] When the date, time, or conditions set for the reminder are met, the device will notify the user, allowing the user to check the notification and take action as necessary.

[0371] Related information suggestions

[0372] Subject: Server

[0373] The server periodically analyzes the user's past search history and saved information, and uses natural language processing technology and machine learning algorithms to predict related information and generate a list of suggestions.

[0374] Subject: Terminal

[0375] The terminal presents the user with a list of suggestions, which the user can refer to for quick access to relevant information.

[0376] Incorporating an emotion engine

[0377] Subject: Terminal

[0378] The device analyzes the user's input information, behavior, and voice tone, and uses an emotion engine to recognize the user's emotions. This emotional data is then sent to the server in real time.

[0379] Subject: Server

[0380] The server then uses the received emotional data to optimize information searches and reminder notifications based on the user's emotions. For example, if the user is feeling stressed, the server will change the relevant information suggestions and adjust the reminder notification time. It also records changes in emotions over time and uses this information for future pattern analysis.

[0381] Examples:

[0382] If a reminder is set for when the user is tired, the system will use its emotion engine to detect the user's fatigue and delay the reminder. Search results will also be optimized based on the user's emotional state, prioritizing more relaxing content and information the user prefers.

[0383] In this way, the system according to the present invention not only allows users to efficiently manage and rediscover information, but also utilizes an emotion engine to provide optimal information adapted to the user's emotions.

[0384] The processing flow will be explained below.

[0385] Retention of Information

[0386] Subject: User

[0387] Step 1:

[0388] The user opens the application using a terminal.

[0389] Step 2:

[0390] The user inputs information, either through text, voice, image, video, or conversation with the AI.

[0391] Subject: Terminal

[0392] Step 3:

[0393] The terminal stores the input information in temporary storage.

[0394] Step 4:

[0395] The device generates metadata including the date and time the information was saved, tags, user ID, and content type.

[0396] Step 5:

[0397] The device sends the input information and metadata to the server.

[0398] Subject: Server

[0399] Step 6:

[0400] The server stores the received information and metadata in a database.

[0401] Step 7:

[0402] The server tags and indexes the stored information.

[0403] Information Search and Retrieval

[0404] Subject: User

[0405] Step 1:

[0406] The user opens the application's search screen on their device and enters keywords.

[0407] Subject: Terminal

[0408] Step 2:

[0409] The terminal acquires the input keyword and sends a search request to the server.

[0410] Subject: Server

[0411] Step 3:

[0412] The server receives the search request and passes the keywords to the natural language processing module.

[0413] Step 4:

[0414] A natural language processing module analyzes keywords and extracts related tags and metadata.

[0415] Step 5:

[0416] The server generates a database query to retrieve the relevant information from the database.

[0417] Step 6:

[0418] The server lists the search results and sends them to the terminal.

[0419] Subject: Terminal

[0420] Step 7:

[0421] The terminal displays the search result list received from the server to the user.

[0422] Reminders and notifications

[0423] Subject: User

[0424] Step 1:

[0425] The user uses the device to select information for which they want to set a reminder.

[0426] Step 2:

[0427] The user sets the date, time, and conditions for the reminder.

[0428] Subject: Terminal

[0429] Step 3:

[0430] The device stores the reminder setting information in temporary storage and sends it to the server.

[0431] Subject: Server

[0432] Step 4:

[0433] The server receives the reminder setting information and stores it in a database.

[0434] Step 5:

[0435] The server generates a notification schedule based on the reminder date and time and conditions.

[0436] Step 6:

[0437] The server stores the notification schedule in temporary storage and registers it with the system clock.

[0438] Subject: Terminal

[0439] Step 7:

[0440] When the reminder date, time, or conditions are met, the device will notify the user.

[0441] Related information suggestions

[0442] Subject: Server

[0443] Step 1:

[0444] The server periodically analyzes the user's past search history and saved information.

[0445] Step 2:

[0446] A natural language processing module is used to calculate the relationships between stored information.

[0447] Step 3:

[0448] Machine learning algorithms are used to generate inferential models of relevant information.

[0449] Step 4:

[0450] The server generates a list of related information based on the inference model.

[0451] Subject: Terminal

[0452] Step 5:

[0453] The terminal obtains the related information list from the server and presents it to the user.

[0454] Incorporating an emotion engine

[0455] Subject: Terminal

[0456] Step 1:

[0457] The device analyzes the user's input information, behavior, voice tone, etc., and recognizes the user's emotions using an emotion engine.

[0458] Step 2:

[0459] Emotion data is sent to the server in real time.

[0460] Subject: Server

[0461] Step 3:

[0462] Based on the emotional data received by the server, information searches and reminder notifications are optimized according to the user's emotions.

[0463] Step 4:

[0464] Emotional changes are recorded over time and used for future pattern analysis.

[0465] Specific examples

[0466] Retention of Information

[0467] Subject: User

[0468] Step 1:

[0469] A user enters a new project idea as text into an application input field.

[0470] Subject: Terminal

[0471] Step 2:

[0472] The terminal stores the input text in temporary storage and sends it to the server.

[0473] Step 3:

[0474] The device generates metadata including the save date and time and the tag "Project Idea."

[0475] Subject: Server

[0476] Step 4:

[0477] The server stores the received text and metadata in a database.

[0478] Step 5:

[0479] The server tags and indexes the text.

[0480] Information Search and Retrieval

[0481] Subject: User

[0482] Step 1:

[0483] A user searches for the keyword "project ideas."

[0484] Subject: Terminal

[0485] Step 2:

[0486] The terminal sends a search request to the server.

[0487] Subject: Server

[0488] Step 3:

[0489] The server analyzes the search request and extracts relevant information from a database.

[0490] Subject: Terminal

[0491] Step 4:

[0492] The device displays the search results to the user, who then confirms the information.

[0493] Incorporating an emotion engine

[0494] Subject: User

[0495] Step 1:

[0496] Users speak topics related to their project ideas.

[0497] Subject: Terminal

[0498] Step 2:

[0499] The device analyzes the user's voice and recognizes emotions using an emotion engine.

[0500] Step 3:

[0501] The device transmits the emotion data to the server.

[0502] Subject: Server

[0503] Step 4:

[0504] The server optimizes search results based on emotional data and displays the most appropriate information for the user.

[0505] Step 5:

[0506] The server records the user's emotional data and uses it for future data analysis.

[0507] This system allows users to efficiently manage information and receive optimal information tailored to their emotions.

[0508] Example 2

[0509] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0510] Modern information management systems provide basic functions such as efficiently storing, searching, and retrieving information input by users. However, they are unable to provide optimal information and notifications tailored to the user's emotions and circumstances. As a result, when a user is emotionally exhausted or stressed, reminder notifications and information suggestions can be annoying, and the user is unable to quickly access the information they need. It is also difficult to provide individually customized information based on the user's emotions and past behavioral history. To solve these problems, a system that can dynamically adjust information provision and notifications based on the user's emotions is needed.

[0511] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0512] In this invention, the server includes means for receiving information entered by a user, means for storing the received information in a database, means for generating metadata for the stored information, means for tagging and indexing the metadata, means for analyzing a user's search request using natural language processing technology, means for searching for information from the database based on the analyzed search request, means for presenting search results to the user, means for collecting and analyzing user emotion data, and means for adjusting reminder notifications and search results based on the collected emotion data. This makes it possible to analyze a user's emotional state and past behavior history and optimize information searches and reminder notifications based on the analyzed data.

[0513] "User" refers to any individual or entity that uses the system to enter information, search, set reminders, etc.

[0514] "Means for receiving" refers to the device or software functions that allow the server to obtain information and reminder settings entered by the user.

[0515] "Database" refers to a system or location for organizing and storing received information, reminder settings, user emotional data, etc.

[0516] "Metadata" refers to additional information about stored information (e.g., date and time of storage, tags, user ID, content type, etc.).

[0517] "Tagging" refers to the act of assigning specific labels or keywords to information to make it easier to search for later.

[0518] "Indexing" refers to the process of creating an index for efficient retrieval of information.

[0519] "Natural language processing" refers to the technology that enables computers to understand human language and appropriately analyze and generate it.

[0520] A "search request" refers to the keywords or question a user enters to find specific information.

[0521] "Emotional data" refers to information about a user's emotional state extracted from their input, behavior, vocal tone, etc.

[0522] "Analyzing" refers to structuring and understanding the received information and emotional data and processing it for a specific purpose.

[0523] "Adjustment" refers to features that dynamically change reminder notifications and search results based on collected emotional data and past behavioral history.

[0524] A "related information list" refers to a list of information that is useful to the user, generated using a machine learning algorithm based on the user's past search history and saved information.

[0525] "Inferring" refers to the process of using machine learning algorithms to predict and provide information relevant to the user.

[0526] The present invention relates to a system that efficiently manages and finds user-entered information, recognizes and utilizes user emotions, and optimizes information search, reminder notifications, and related information suggestions. Specific embodiments of the present invention are described in detail below.

[0527] Retention of Information

[0528] user

[0529] Users open the application on their device and input the required information using text, music, images, videos, or through conversation with the AI. For example, if they want to input an idea for a new project, they can enter their idea in a dedicated input field.

[0530] Terminal

[0531] The device saves the input information in temporary storage and sends it to the server. It also generates metadata about the save date and time, tags, user ID, and content type. For example, a saved project idea might be tagged with "Project," "New Technology," and "October 2023."

[0532] server

[0533] The server stores the received information and metadata in a database, tags the stored information, and creates an index to make it easier to search for the information.

[0534] Information Search and Retrieval

[0535] user

[0536] Users can use the application to open a search screen and enter keywords, such as "project ideas," to search for previously saved project ideas.

[0537] Terminal

[0538] The terminal acquires the input keywords and sends a search request to the server.

[0539] server

[0540] The server receives the search request, analyzes the keywords using natural language processing technology, searches for related information from a database based on the analysis results, and sends the search result list to the terminal.

[0541] Terminal

[0542] The terminal displays the search result list sent from the server to the user, who can then refer to the list and check the information they need.

[0543] Reminders and notifications

[0544] user

[0545] The user uses the application to select the information for which they want to set a reminder, and enter the date, time, and conditions for the reminder, such as "Remind me to review new project ideas tomorrow at 10 AM."

[0546] Terminal

[0547] The device sends the reminder setting information to the server.

[0548] server

[0549] The server receives the reminder setting information and stores it in a database. If the reminder conditions are met, the server schedules the notification and prepares to send the reminder at the set date and time.

[0550] Terminal

[0551] When the date, time, or conditions set for the reminder are met, the device will notify the user, allowing the user to check the notification and take action as necessary.

[0552] Related information suggestions

[0553] server

[0554] The server periodically analyzes the user's past search history and saved information, and uses natural language processing and machine learning algorithms to infer relevant information and generate a list of suggestions.

[0555] Terminal

[0556] The terminal displays the list of suggestions sent from the server to the user, who can then refer to the list to quickly access relevant information.

[0557] Incorporating an emotion engine

[0558] Terminal

[0559] The device analyzes the user's input information, behavior, voice tone, etc., and recognizes the user's emotions using an emotion engine. This emotional data is then sent to the server in real time.

[0560] server

[0561] The server then uses the received emotional data to optimize information searches and reminder notifications based on the user's emotions. For example, if it senses that the user is tired, it will delay reminder notifications. It also records changes in emotions over time and uses this information for future pattern analysis.

[0562] Specific examples

[0563] If a reminder is set when the user is tired, the system can use the emotion engine to detect the user's tiredness and delay the reminder by 30 minutes.

[0564] If a user searches for "project ideas," and they are feeling stressed, the suggestions could include relaxing music and videos.

[0565] Examples of prompt statements

[0566] Generate a natural language description of a program that adjusts the timing of reminders based on the user's emotional state.

[0567] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0568] Program processing flow

[0569] Retention of Information

[0570] Step 1:

[0571] user

[0572] The user opens an application on the device.

[0573] Input: Launch application.

[0574] Output: The application home screen is displayed.

[0575] Step 2:

[0576] user

[0577] Enter the required information in the form of text, music, images, videos, or even a conversation with the AI. For example, enter an idea for a new project in text.

[0578] Input: Project idea in text format.

[0579] Output: The input field is filled with the idea.

[0580] Step 3:

[0581] Terminal

[0582] The device saves the input information in temporary storage and generates metadata about the save date and time, tags, user ID, and content type.

[0583] Input: Information entered by the user.

[0584] Output: Save to temporary storage, generate metadata.

[0585] Step 4:

[0586] Terminal

[0587] The device encrypts the stored information and metadata and sends it to the server.

[0588] Input: Information and metadata stored in temporary storage.

[0589] Output: Data sent to the server.

[0590] Step 5:

[0591] server

[0592] The server stores the received information and metadata in a database, where it tags and indexes the stored information.

[0593] Input: Information and metadata sent by the device.

[0594] Output: Save to database and create index.

[0595] Information Search and Retrieval

[0596] Step 6:

[0597] user

[0598] A user opens a search screen in an application and enters a search keyword, for example, "project ideas."

[0599] Input: Search keyword.

[0600] Output: A search request is generated.

[0601] Step 7:

[0602] Terminal

[0603] The terminal acquires the input keywords and sends a search request to the server.

[0604] Input: Search keyword.

[0605] Output: A search request to the server.

[0606] Step 8:

[0607] server

[0608] The server receives the search request and analyzes the keywords using natural language processing techniques.

[0609] Input: A search request.

[0610] Output: Parsed keywords.

[0611] Step 9:

[0612] server

[0613] Based on the analysis results, relevant information is searched from the database and a list of the results is generated.

[0614] Input: The parsed keyword.

[0615] Output: Search result list.

[0616] Step 10:

[0617] server

[0618] The server transmits the search result list to the terminal.

[0619] Input: Search results list.

[0620] Output: Data sent to the terminal.

[0621] Step 11:

[0622] Terminal

[0623] The terminal receives the search result list sent from the server and displays it to the user.

[0624] Input: Search results list.

[0625] Output: Display search results to the user.

[0626] Reminders and notifications

[0627] Step 12:

[0628] user

[0629] The user opens the reminder setting screen and enters the date, time, and conditions for the reminder. For example, they can set a reminder to review new project ideas at 10:00 AM tomorrow.

[0630] Input: Reminder date, time and conditions.

[0631] Output: Reminder settings data.

[0632] Step 13:

[0633] Terminal

[0634] The device sends the reminder setting information to the server.

[0635] Input: Reminder setting data.

[0636] Output: Data sent to the server.

[0637] Step 14:

[0638] server

[0639] The server receives the reminder setting information and stores it in a database.

[0640] Input: Reminder setting data.

[0641] Output: Save to database.

[0642] Step 15:

[0643] server

[0644] When the conditions for a reminder are met, the server schedules the notification and prepares the reminder notification.

[0645] Input: Reminder date, time and conditions.

[0646] Output: Notification schedule created.

[0647] Step 16:

[0648] Terminal

[0649] When the set date, time or conditions are met, the device will send a reminder notification to the user.

[0650] Input: Notification Schedule.

[0651] Output: Notification to the user.

[0652] Related information suggestions

[0653] Step 17:

[0654] server

[0655] The server periodically analyzes the user's past search history and saved information.

[0656] Input: Your past search history and saved information.

[0657] Output: Analysis results.

[0658] Step 18:

[0659] server

[0660] It uses natural language processing technology and machine learning algorithms to infer relevant information and generate a list of suggestions.

[0661] Input: Analysis results.

[0662] Output: A list of suggestions.

[0663] Step 19:

[0664] server

[0665] The server transmits the generated proposal list to the terminal.

[0666] Input: Suggestion list.

[0667] Output: Data sent to the terminal.

[0668] Step 20:

[0669] Terminal

[0670] The terminal displays the proposal list received from the server to the user.

[0671] Input: Suggestion list.

[0672] Output: What is displayed to the user.

[0673] Incorporating an emotion engine

[0674] Step 21:

[0675] Terminal

[0676] The device collects user input information, behavior, voice tone, etc. and analyzes them using an emotion engine.

[0677] Input: User input, behavior, and tone of voice.

[0678] Output: Emotion data.

[0679] Step 22:

[0680] Terminal

[0681] The acquired emotion data is sent to the server in real time.

[0682] Input: Emotion data.

[0683] Output: Data sent to the server.

[0684] Step 23:

[0685] server

[0686] The server adjusts reminder notifications and search results based on the received emotion data.

[0687] Input: Emotion data.

[0688] Output: Tailored reminder notifications and search results.

[0689] Specific examples

[0690] Example 1:

[0691] Users search for "project ideas," and if the device detects that the user is feeling fatigued, it will suggest relaxing music and videos.

[0692] Example 2:

[0693] If a reminder notification is set when the user is tired, the server can use the emotion engine to delay the notification by 30 minutes.

[0694] (Application example 2)

[0695] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0696] Conventional information management systems are limited to efficiently managing and rediscovering user-entered information and metadata, but do not optimize search results, adjust reminder notifications, or suggest related information based on the user's emotional state. This lack of systems allows users to comfortably use information search and reminder functions. Even in physical stores, there are no product suggestions or notification functions that reflect the user's emotional state, hindering improvements to the customer experience.

[0697] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0698] In this invention, the server includes means for receiving information entered by a user, means for storing the received information in a database, means for generating metadata for the stored information, means for tagging and indexing the metadata, means for analyzing a user's search request using natural language processing technology, means for searching for information from the database based on the analyzed search request, means for presenting search results to the user, means for analyzing emotional data of the indicated information or image, and means for optimizing information based on the emotional data. This makes it possible to optimize information search and reminder notifications and suggest related information based on the user's emotional state.

[0699] "User" refers to any individual or corporation that uses this system.

[0700] "Input Information" refers to data in the form of text, music, images, video, or conversations provided by a user through a device or application.

[0701] "Means for receiving" refers to a set of hardware and software for taking input information into a computer system.

[0702] "Database" means a digital system for the structured storage and management of received information and associated metadata.

[0703] "Metadata" is data that indicates the nature and attributes of stored information, and includes the date and time of storage, tags, user ID, type of content, etc.

[0704] "Tagging and indexing methods" refers to techniques that attach identifiable labels to stored information and create data structures to facilitate efficient searching.

[0705] "Natural language processing technology" refers to a series of technologies that enable computers to understand, generate, and analyze natural human language.

[0706] A "search request" refers to the keywords or phrases a user enters to find specific information.

[0707] "Means for analyzing the emotional data of specified information or images" refers to technology that analyzes the data input by the user and recognizes the emotions contained within that data.

[0708] "Emotional data" refers to data that indicates a user's emotional state, such as information extracted from voice tone, facial expressions, and input content.

[0709] "Optimization measures" refer to methods for adjusting information search results and the timing of reminder notifications based on analyzed emotional data.

[0710] "Reminder" refers to a feature that notifies users in advance of specific events or tasks that they have set.

[0711] A "machine learning algorithm" refers to a set of computational techniques that derive rules from past data and make predictions and classifications.

[0712] "Related information" refers to additional information inferred based on a user's interests and needs, based on their past search history and saved information.

[0713] "Inferred related information list" refers to a set of related information generated by a machine learning algorithm.

[0714] "Notification" means a message or alert sent to inform the User of a reminder or relevant information.

[0715] "Suggestion" refers to useful information or recommended actions that the system provides to the user based on the user's input information and emotional data.

[0716] The present invention provides a system for efficiently managing information input by a user, recognizing the user's emotions, and optimizing search results, reminder notifications, and suggestions of related information. Specific embodiments are described below.

[0717] System configuration

[0718] Hardware

[0719] 1. Smart glasses: A device equipped with a voice recognition microphone, camera, and display that acquires and displays user input information.

[0720] 2. Server: A central system equipped with a database, natural language processing module, and emotion recognition engine, which analyzes, stores, and searches information.

[0721] 3. Surveillance camera: A device that recognizes customers' facial expressions and collects emotional data.

[0722] software

[0723] 1. Speech recognition API (such as Google Speech-to-Text): Software that converts user voice input into text data.

[0724] 2. Natural Language Processing API (such as Google NLP API): Software for analyzing text data and extracting important information.

[0725] 3. Emotion recognition API (such as Microsoft Azure Emotion API): Software for recognizing a user's emotional state from images and audio.

[0726] Operation overview

[0727] Retention of Information

[0728] The user uses the smart glasses to input information by voice. For example, they might say, "I want this red dress." This voice data is converted into text data by the smart glasses' voice recognition API and temporarily saved. This data is then sent to the server. The server stores the received information in a database, generates and tags metadata for the saved information, such as the date and time of saving, tags, and user ID, and creates an index.

[0729] Information Search and Retrieval

[0730] When a user searches for specific information through the smart glasses, they input the search query by voice. For example, a search request such as "black shoes" can be input. The smart glasses convert this voice into text and send the search request to the server. The server's natural language processing module analyzes the search request, searches for relevant information from the database, and sends the results to the smart glasses. The user can then view the search results on the display.

[0731] Reminders and notifications

[0732] Users can set reminders for later purchases. For example, they can set a reminder to "check out this red dress tomorrow at 10 AM." The smart glasses send this request to the server, which stores the setting information in a database. Reminders are scheduled based on the set date, time, and conditions, and notifications are sent to the smart glasses when the conditions are met. Furthermore, if the emotion recognition engine detects the user's fatigue, the timing of reminder notifications is adjusted.

[0733] Related information suggestions

[0734] The server periodically analyzes the user's past search history and saved information. It uses machine learning algorithms to predict relevant information and generate a list of suggestions. This list is presented to the user through smart glasses. For example, it may present highly relevant information such as "black shoes that go well with a red dress you previously purchased." The user can then choose their next action based on this list of suggestions.

[0735] Emotional data analysis and optimization

[0736] Smart glasses and surveillance cameras collect the user's facial expressions and tone of voice, and analyze the emotional data through an emotion recognition API. This data is sent to a server, which then optimizes information searches, reminder notifications, and related information suggestions based on the user's emotional state. For example, if the user is feeling stressed, product information that helps them relax will be displayed first.

[0737] Specific examples

[0738] 1. Voice input: The user inputs information by voice, such as "I like this blue jacket." The data is converted to text via a voice recognition API and sent to the server where it is saved.

[0739] 2. Emotional Data Analysis: As users walk around the store, the smart glasses' cameras and surveillance cameras collect facial expressions and voice tones. The emotion recognition API analyzes whether the user is in an excited state. This information is sent to the server, and product recommendations are optimized to match the user's excitement level.

[0740] Prompt Sentence Examples

[0741] "Save the voice data of the user saying, 'I want this red dress,' and set a reminder."

[0742] "Based on the search query 'black shoes', search for relevant information from the database and display it on the smart glasses."

[0743] "If the user's emotional state indicates stress, please suggest products that will help them relax."

[0744] Following these steps, we will build a system that provides information efficiently and based on emotions.

[0745] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0746] Step 1:

[0747] User voice input

[0748] The user puts on the smart glasses and uses voice input to describe the products they want or are interested in. For example, they might say, "I want this red dress." This input information is sent to the smart glasses' voice recognition API, which converts the voice data into text data.

[0749] Input: User voice input

[0750] Output: Speech-to-text data

[0751] Step 2:

[0752] Converting audio data to text

[0753] The device converts the user's voice into text data using a speech recognition API, such as Google Speech-to-Text, and the converted data is stored in temporary storage within the smart glasses.

[0754] Input: Audio data sent to the speech recognition API

[0755] Output: User input converted to text data

[0756] Step 3:

[0757] Sending text data to the server

[0758] The text data is sent from the temporary storage to the server, which stores it in a database and generates metadata such as the save date and time, tags, and user ID.

[0759] Input: Text data saved in temporary storage

[0760] Output: Text data and metadata stored on the server

[0761] Step 4:

[0762] Metadata generation and indexing

[0763] The server generates metadata and tags the received text data, and then creates an index to efficiently manage user information.

[0764] Input: Text data received by the server

[0765] Output: Generated metadata and indexing

[0766] Step 5:

[0767] Information search request

[0768] To search for specific information using the smart glasses, users input keywords by voice, such as a search request for "black shoes." The device converts this speech into text using a speech recognition API and sends it to the server.

[0769] Input: User's voice search request

[0770] Output: Text search request sent to the server

[0771] Step 6:

[0772] Parsing a search request

[0773] The server's natural language processing API analyzes the search request and searches the database for relevant information. As a result of the analysis, related product information is obtained.

[0774] Input: A textual search request

[0775] Output: Search results for related product information

[0776] Step 7:

[0777] Presenting search results

[0778] The server sends the search results to the smart glasses and displays them to the user, who can then view the relevant information on the smart glasses display.

[0779] Input: Server-generated search results

[0780] Output: Search results displayed on smart glasses

[0781] Step 8:

[0782] Set reminders

[0783] A user sets a reminder to check a specific product later, for example, "Check out this red dress tomorrow at 10 AM." The device sends the reminder setting information to the server.

[0784] Input: Reminder information set by the user

[0785] Output: Reminder information stored on the server

[0786] Step 9:

[0787] Scheduling reminders

[0788] The server schedules the reminder based on the reminder settings, and when the set time and conditions are met, a reminder notification is sent to the user.

[0789] Input: Reminder information stored on the server

[0790] Output: Scheduled reminder notifications

[0791] Step 10:

[0792] Emotional Data Analysis

[0793] Smart glasses and surveillance cameras collect the user's facial expressions and voice tone, and use emotion recognition APIs to analyze the emotional data, for example, to detect the user's excitement or fatigue, and send this information to a server.

[0794] Input: facial expression and tone of voice data

[0795] Output: Parsed emotion data

[0796] Step 11:

[0797] Emotion-Based Optimization

[0798] The server then uses the analyzed emotion data to adjust search results and the timing of reminder notifications. For example, if fatigue is detected, the server can delay reminder notifications.

[0799] Input: Parsed emotion data

[0800] Output: Optimized information search results and reminder notifications

[0801] Step 12:

[0802] Related information suggestions

[0803] The server uses machine learning algorithms to analyze the user's past search history and saved information to infer relevant information, and the generated suggestion list is sent to the smart glasses and presented to the user.

[0804] Input: User's past search history and saved information

[0805] Output: A generated list of related information suggestions

[0806] Through each of these steps, it becomes possible to provide optimal information and reminder functions based on the user's emotions.

[0807] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0808] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0809] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0810] [Second embodiment]

[0811] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0812] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0813] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0814] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0815] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0816] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0817] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0818] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0819] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0820] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0821] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0822] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0823] The present invention relates to a system that efficiently manages and rediscovers information stored by users (text, audio, images, videos, conversations with AI, etc.) and provides the information to users in the most optimal way. Specific program processing and its implementation are described below.

[0824] Retention of Information

[0825] Subject: User

[0826] The user opens an application on the device and inputs the required information as text, or records voice, images, or videos. For example, if the user wants to input an idea for a new project as text, the user enters the idea in a dedicated input field.

[0827] Subject: Terminal

[0828] The device temporarily stores the input information and sends it to the server, generating metadata such as the date and time of storage, tags, user ID, and content type.

[0829] Subject: Server

[0830] The server stores the received information and metadata in a database, tags and indexes the stored information, and updates the metadata.

[0831] Information Search and Retrieval

[0832] Subject: User

[0833] Users can enter keywords related to the information they need in the application's search screen, for example, by entering "project ideas," to search for previously saved project ideas.

[0834] Subject: Terminal

[0835] The terminal acquires the input keywords and sends a search request to the server.

[0836] Subject: Server

[0837] The server receives the search request, analyzes the keywords using a natural language processing module, searches for related information from the database based on the analysis results, generates a list of results, and sends it to the terminal.

[0838] Subject: Terminal

[0839] The terminal displays the search result list received from the server to the user, who can then refer to the list to find the information they need.

[0840] Reminders and notifications

[0841] Subject: User

[0842] Users can set reminders for the information they save, for example, "Remind me to review new project ideas tomorrow at 10 AM."

[0843] Subject: Terminal

[0844] The device sends the reminder setting information to the server.

[0845] Subject: Server

[0846] The server receives the reminder setting information, stores it in a database, schedules notifications based on the reminder date and time and conditions, and notifies the user when the set date and time or conditions are met.

[0847] Subject: Terminal

[0848] When the set date, time, or conditions are met, the device will send a reminder notification to the user, who can then check the notification and take action as needed.

[0849] Related information suggestions

[0850] Subject: Server

[0851] The server periodically analyzes the user's past search history and saved information, and uses natural language processing technology and machine learning algorithms to predict related information and generate a list of suggestions.

[0852] Subject: Terminal

[0853] The terminal presents the user with a list of suggestions, which the user can refer to for quick access to relevant information.

[0854] As described above, the system of the present invention integrates multiple functions to enable users to efficiently manage and rediscover information. As a specific example, users can save ideas for new projects, retrieve information when needed by keyword search, and set reminders to receive notifications at appropriate times. This system allows users to manage important information without missing it and utilize it efficiently.

[0855] The processing flow will be explained below.

[0856] Retention of Information

[0857] Subject: User

[0858] Step 1:

[0859] The user opens the application using a terminal.

[0860] Step 2:

[0861] The user inputs information, either through text, voice, image, video, or conversation with the AI.

[0862] Subject: Terminal

[0863] Step 3:

[0864] The terminal stores the input information in temporary storage.

[0865] Step 4:

[0866] The device generates metadata including the date and time the information was saved, tags, user ID, and content type.

[0867] Step 5:

[0868] The device sends the input information and metadata to the server.

[0869] Subject: Server

[0870] Step 6:

[0871] The server stores the received information and metadata in a database.

[0872] Step 7:

[0873] The server tags and indexes the stored information.

[0874] Information Search and Retrieval

[0875] Subject: User

[0876] Step 1:

[0877] The user opens the application's search screen on their device and enters keywords.

[0878] Subject: Terminal

[0879] Step 2:

[0880] The terminal acquires the input keyword and sends a search request to the server.

[0881] Subject: Server

[0882] Step 3:

[0883] The server receives the search request and passes the keywords to the natural language processing module.

[0884] Step 4:

[0885] A natural language processing module analyzes keywords and extracts related tags and metadata.

[0886] Step 5:

[0887] The server generates a database query to retrieve the relevant information from the database.

[0888] Step 6:

[0889] The server lists the search results and sends them to the terminal.

[0890] Subject: Terminal

[0891] Step 7:

[0892] The terminal displays the search result list received from the server to the user.

[0893] Reminders and notifications

[0894] Subject: User

[0895] Step 1:

[0896] The user uses the device to select information for which they want to set a reminder.

[0897] Step 2:

[0898] The user sets the date, time, and conditions for the reminder.

[0899] Subject: Terminal

[0900] Step 3:

[0901] The device stores the reminder setting information in temporary storage and sends it to the server.

[0902] Subject: Server

[0903] Step 4:

[0904] The server receives the reminder setting information and stores it in a database.

[0905] Step 5:

[0906] The server generates a notification schedule based on the reminder date and time and conditions.

[0907] Step 6:

[0908] The server stores the notification schedule in temporary storage and registers it with the system clock.

[0909] Subject: Terminal

[0910] Step 7:

[0911] When the reminder date, time, or conditions are met, the device will notify the user.

[0912] Related information suggestions

[0913] Subject: Server

[0914] Step 1:

[0915] The server periodically analyzes the user's past search history and saved information.

[0916] Step 2:

[0917] A natural language processing module is used to calculate the relationships between stored information.

[0918] Step 3:

[0919] Machine learning algorithms are used to generate inferential models of relevant information.

[0920] Step 4:

[0921] The server generates a list of related information based on the inference model.

[0922] Subject: Terminal

[0923] Step 5:

[0924] The terminal obtains the related information list from the server and presents it to the user.

[0925] Example 1

[0926] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0927] Users generate and store a large amount of information on a daily basis, but it is difficult to efficiently manage that information and quickly find it again when needed. Users also desire a reminder function to ensure they don't forget important information, but manually setting this function is cumbersome. Furthermore, there is a lack of systems that automatically suggest related information based on a user's past behavior. An integrated information management system that can solve these issues is needed.

[0928] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0929] In this invention, the server includes means for receiving information entered by a user, means for saving the received information in a storage device, means for generating additional information for the saved information, means for tagging and indexing the additional information, means for analyzing a user's search request using natural language processing technology, means for searching for information from the storage device based on the analyzed search request, means for presenting search results to the user, means for receiving reminders set by the user, means for saving reminder information in a storage device, means for scheduling notifications based on the reminder date and time and conditions, means for notifying the user when the set date and time and conditions are met, means for analyzing the user's past search history and saved information, means for using a machine learning algorithm to infer related information, means for generating a list of inferred related information, and means for presenting the generated related information list to the user. This allows the user to efficiently manage saved information and quickly rediscover it when needed, and the reminders allow the user to manage important information without missing it and also provide suggestions of related information based on past behavior.

[0930] "Means for receiving" refers to the function that allows a terminal or server to receive information or requests sent by a user.

[0931] "Storage device" means a physical or virtual device for storing data, including hard drives, SSDs, cloud storage, etc.

[0932] The "means for saving" is a function for writing received information or data to a storage device.

[0933] "Additional information" refers to additional information related to the content of the saved information, such as metadata such as tags, save date and time, and user ID.

[0934] "Means for generating" is a function for automatically generating specific additional information based on certain information or data.

[0935] "Tagging" is a method of adding specific keywords or categories to information to make it easier to organize and search.

[0936] "Indexing" is the process of building a data structure so that information can be efficiently searched.

[0937] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, and includes text analysis and language generation.

[0938] A "search request" is a request containing keywords or phrases entered by a user in search of specific information.

[0939] "Means of analysis" refers to the ability to understand the received search request and perform processing to find appropriate information.

[0940] The "searching means" is a function for searching for related information from a storage device based on the analyzed keywords.

[0941] "Means of presentation" refers to the function that allows a server or terminal to display search results and notifications to the user.

[0942] A "reminder" is a notification that a user sets for a specific date, time, or condition.

[0943] "Scheduling tools" are functions that allow you to plan notifications based on reminder dates and conditions.

[0944] "Means of notification" is a function for sending alerts or messages to users when a set date, time, or condition is met.

[0945] "Search history" refers to a record of searches a user has conducted in the past.

[0946] A "machine learning algorithm" is a computational method for analyzing data, discovering patterns, and using those patterns to predict future data.

[0947] A "related information list" is a list of information that is presumed to be useful to the user based on past search history and saved information.

[0948] The present invention relates to a system for efficiently managing user-generated information (text, audio, images, video, etc.) and rediscovering it when needed. Specific embodiments of the system are described below.

[0949] Retention of Information

[0950] 1. User input

[0951] Users open the application on their device and enter the required information: for text input, they write the information in a dedicated input field, for voice input, they press the microphone button to start recording, or for images and videos, they use the camera function.

[0952] Specific working example:

[0953] Enter your new project idea as "New website design" or, if you use voice recognition, say something like "Record a new project idea."

[0954] 2. Temporary storage on the device

[0955] The device saves the input information in temporary storage, generating metadata such as the save date and time, tags, user ID, and content type. The smartphone's internal memory is used as temporary storage.

[0956] 3. Sending from the device to the server

[0957] The device sends the stored information and metadata to the server, typically using an HTTP POST request to send the data.

[0958] 4. Server-based information storage and metadata updates

[0959] The server stores the received information and metadata in a database (e.g., MySQL), and a tagging engine runs to add appropriate tags to the text information.

[0960] Information Search and Retrieval

[0961] 1. User enters search keywords

[0962] Users enter keywords related to the information they need on the application's search screen.

[0963] Specific working example:

[0964] A user types in "project ideas" and presses the search button.

[0965] 2. Sending a search request via the device

[0966] The device receives the input keywords and sends a search request to the server. It generates an HTTP GET request and sends it to the server.

[0967] 3. Keyword analysis and search result generation by the server

[0968] The server uses a natural language processing library (e.g., NLTK or SpaCy) to analyze the keywords and search for relevant information in a storage device (database). The result list is generated in JSON format and sent to the device.

[0969] 4. Displaying search results on your device

[0970] The terminal displays the search result list received from the server to the user, displaying the search results in a list view so that the user can access the information they need.

[0971] Reminders and notifications

[0972] 1. User-defined reminder settings

[0973] Users can set reminders for saved information.

[0974] Specific working example:

[0975] The user enters the instruction "Set a reminder to check for new project ideas tomorrow at 10 AM."

[0976] 2. Send reminders via device

[0977] The device sends this reminder setting information to the server.

[0978] 3. Server-based reminder storage and scheduling

[0979] The server stores the reminder information in a database and sets up a notification scheduler (e.g., a Cron job or Quartz Scheduler).

[0980] 4. Device-based reminder notifications

[0981] When the set date, time, or conditions are met, a notification is triggered from the server and the device sends a reminder notification to the user.

[0982] Related information suggestions

[0983] 1. Server Generation of Proposal List

[0984] The server analyzes the user's past search history and saved information, and uses natural language processing techniques and machine learning algorithms (e.g., TensorFlow and PyTorch) to infer relevant information. A list of suggestions is generated periodically.

[0985] 2. Displaying a list of suggestions on the device

[0986] The terminal presents the user with a list of suggestions received from the server, and based on the information provided, the user can quickly access relevant information.

[0987] Examples and prompts

[0988] Examples:

[0989] 1. Storage of Information:

[0990] User: Texts a new project idea: "New website design."

[0991] Device: Sends information along with metadata to the server.

[0992] Server: Stores the information in a database and tags it.

[0993] 2. Information Search:

[0994] User: Search for "project ideas."

[0995] Device: Sends a search request to the server.

[0996] Server: Searches for relevant information and generates a list of results.

[0997] Device: Display search results to the user.

[0998] 3. Reminder:

[0999] User: Set a reminder to review new project ideas tomorrow at 10 AM.

[1000] Device: Sends reminder setting information to the server.

[1001] Server: Schedule reminders.

[1002] On your device: Reminds you at the set time.

[1003] Example prompt:

[1004] "Set a reminder to review new project ideas tomorrow at 10 AM."

[1005] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1006] Step 1:

[1007] The user opens an application on their device and inputs new information, for example, by typing a new project idea as text, "About a new website design," or by recording an audio, image, or video. It takes the user's data (text, audio, image, video) as input and generates data as output that is stored in temporary storage.

[1008] Step 2:

[1009] The device saves the information entered by the user in temporary storage. At this time, additional information such as the save date and time, tag, user ID, and content type is generated for the data saved in temporary storage. This allows information to be organized efficiently. Data from the user is received as input, and data with additional information is generated as output.

[1010] Step 3:

[1011] The device sends the stored information and additional information to the server. Specifically, it sends data using an HTTP POST request. It takes the data stored in temporary storage as input and generates an HTTP request that is sent to the server as output.

[1012] Step 4:

[1013] The server stores the received information and additional information in a database, performs appropriate tagging and indexing on the stored information, and updates the additional information. The server takes the data received by the server as input and generates information to be stored in the database as output.

[1014] Step 5:

[1015] The user enters keywords related to the information they need into the application's search screen. For example, they can enter "project ideas" to search for previously saved project ideas. The application takes search keywords as input and generates a search request as output.

[1016] Step 6:

[1017] The device acquires the search keywords and sends a search request to the server. Specifically, it generates an HTTP GET request and sends it to the server. It receives the search keywords from the user as input and generates and sends a search request as output.

[1018] Step 7:

[1019] The server receives the search request and analyzes the entered keywords using natural language processing techniques. Based on the analysis results, it searches for relevant information from the database. It takes the search request received by the server as input and generates a list of search results as output.

[1020] Step 8:

[1021] The terminal displays the search result list received from the server to the user. Specifically, the terminal displays the search results in a list view. The terminal receives the search results received from the server as input and generates the list of search results that is displayed to the user as output.

[1022] Step 9:

[1023] The user sets a reminder for the saved information, for example, "Remind me to review new project ideas tomorrow at 10 AM." The system takes the reminder setting information as input and generates a reminder setting request as output.

[1024] Step 10:

[1025] The device sends reminder setting information to the server. Specifically, it generates an HTTP POST request and sends it to the server. It receives reminder setting information from the user as input and generates and sends a reminder setting request as output.

[1026] Step 11:

[1027] The server receives the reminder setting information and stores it in a database. It schedules notifications based on the reminder date and time and conditions. It takes the reminder setting information received by the server as input and generates the scheduled notifications as output.

[1028] Step 12:

[1029] When the set date, time, or conditions are met, the device will notify the user with a reminder, for example, by displaying a reminder in the notification bar. It takes a reminder notification received from the server as input and generates a reminder notification to be displayed to the user as output.

[1030] Step 13:

[1031] The server analyzes the user's past search history and saved information, and uses natural language processing technology and machine learning algorithms to infer related information and generate a list of suggestions. It receives past search history and saved information as input and generates a list of suggestions as output.

[1032] Step 14:

[1033] The terminal presents the suggestion list received from the server to the user, allowing the user to quickly access relevant information. It takes the suggestion list received from the server as input and generates the suggestion list that is presented to the user as output.

[1034] (Application example 1)

[1035] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1036] In logistics centers, workers handle a wide variety of product information management tasks, and manual information entry and search is time-consuming, resulting in problems such as reduced accuracy and increased risk of errors. Furthermore, a lack of efficient reminder functions and suggestions for related information often leads to reduced work efficiency. There is a need for a system that can solve these problems, enable workers to quickly and accurately access the information they need, and improve work efficiency.

[1037] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1038] In this invention, the server includes means for receiving information entered by a user, means for storing the received information in a database, means for generating metadata for the stored information, means for tagging and indexing the metadata, means for analyzing a user's search request using natural language processing technology, means for searching for information from the database based on the analyzed search request, means for presenting search results to the user, means for acquiring information using a wearable device worn by a worker, and means for transmitting the acquired information to the server in real time. This makes it possible to use the wearable device to quickly and accurately manage information and improve work efficiency.

[1039] "User" refers to the entity that uses an information system.

[1040] A "wearable device" refers to an electronic device that can be worn by the user.

[1041] "Means for receiving information" refers to the mechanism by which user-entered information is incorporated into the data system.

[1042] "Database" refers to a system for systematically collecting, storing, and managing data.

[1043] "Metadata" refers to information about stored information, i.e., additional descriptions or attributes about that data.

[1044] "Tagging" refers to the process of classifying and organizing information or data by attaching labels or identification codes to it.

[1045] "Indexing" refers to the procedures and structures that enable quick search for necessary information from a large amount of data.

[1046] "Natural language processing technology" refers to technology for processing and analyzing human language (natural language) using a computer.

[1047] "Search Request" means a request or inquiry entered by a User to locate specific information.

[1048] "Real-time" refers to a situation in which information is acquired and processed instantly, with almost no time delay.

[1049] This invention is a system that uses wearable devices to manage and search information in order to improve work efficiency at logistics centers. Specifically, information is acquired through the wearable devices, sent to a server, and the necessary information is searched and displayed in real time.

[1050] Overall structure

[1051] The system consists of the following main components:

[1052] 1. Wearable devices: Devices worn by workers to input and display information, including smart glasses and smart watches.

[1053] 2. Server: The central system for storing, analyzing, searching, and notifying information.

[1054] 3. Database: A system connected to a server for storing saved information, metadata, search history, and reminder information.

[1055] Receiving and storing information

[1056] The wearable device captures information through voice input and barcode scanning by the worker. For example, a worker can scan the barcode of a new product and enter its details by voice. It can also record the product's condition using video recording. This information is sent in real time to a server, which stores it in a database. At the same time, metadata is generated, including the date and time of storage, tag, worker ID, and content type.

[1057] Information Search and Retrieval

[1058] Workers use the smart glasses to search for the information they need using voice commands. For example, they can say, "Tell me the inventory status of shelf number A3." The voice command is sent to the server, which analyzes it using natural language processing technology and searches for relevant information in the database. The results are displayed on the smart glasses, allowing workers to quickly obtain the information.

[1059] Reminders and notifications

[1060] Reminders are set by workers through voice input. For example, they can say, "Remind me tomorrow at 9:00 AM for the next loading / unloading operation." The reminder information is sent to the server and stored in a database. When the set date, time, or conditions are met, the server automatically sends a notification to the wearable device and displays the reminder. This helps workers avoid missing important tasks.

[1061] Related information suggestions

[1062] The server uses machine learning algorithms to analyze past search history and stored information. Based on this, it infers relevant information and makes suggestions to improve work efficiency. For example, if a frequently searched item is low in stock, it will make a replenishment suggestion. These suggestions are displayed to the worker through the smart glasses, allowing the worker to take action quickly.

[1063] Examples and prompts

[1064] For example, consider a scenario in which a worker saves details of a new item.

[1065] Example prompt sentence:

[1066] "Scan the barcode, then speak the product details."

[1067] Execution example:

[1068] Barcode scan: "1234567890123"

[1069] Speak: "This item is part of batch process B and is located on shelf number A3."

[1070] This information is immediately stored on the server and used for future searches and related information suggestions.

[1071] Specific examples of hardware and software used

[1072] Hardware: smart glasses (e.g., Google Glass), servers (high-performance servers in data centers), database servers (e.g., PostgreSQL)

[1073] Software: Natural language processing libraries (spaCy, NLTK), machine learning algorithms (TensorFlow, scikit-learn)

[1074] In this way, it is possible to efficiently manage and search information in the logistics center, provide reminder functions, and suggest related information.

[1075] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1076] Step 1:

[1077] Users input information using a wearable device. Specifically, a worker scans the barcode of a product with smart glasses and inputs detailed information about the product by voice. This input information includes the barcode value and voice text.

[1078] Input: Barcode scan data, voice input text

[1079] Output: Collected product information

[1080] Step 2:

[1081] The device sends the collected product information to a server in real time. Specifically, the smart glasses upload the barcode scan results and voice-input text to the server via the Internet, along with metadata such as the save date and time, tag, and worker ID.

[1082] Input: Collected product information (barcode data, voice text)

[1083] Output: Data sent to the server

[1084] Step 3:

[1085] The server stores the received information in a database, specifically by recording the information and metadata in a database system (e.g., PostgreSQL), including tagging and indexing.

[1086] Input: Data sent to the server

[1087] Output: Information stored in the database

[1088] Step 4:

[1089] Users can search for the information they need using voice commands. Specifically, a worker speaks into the smart glasses, saying, "Tell me the inventory status of shelf number A3." This voice is converted into text by the device and sent to the server.

[1090] Input: Voice command

[1091] Output: Text data of voice commands

[1092] Step 5:

[1093] The server receives the search request and analyzes it using natural language processing technology. Specifically, it uses a natural language processing library (e.g., spaCy) to analyze the voice command and extract the keywords necessary for the search.

[1094] Input: Text data of voice command

[1095] Output: Parsed keywords

[1096] Step 6:

[1097] The server searches for information from the database based on the analyzed keywords, generates a database query to retrieve the relevant information, and formats the query results into a list and sends it to the smart glasses.

[1098] Input: Parsed keyword

[1099] Output: Search result list

[1100] Step 7:

[1101] The terminal receives the search results from the server and displays them to the user. Specifically, the search results are visually displayed on the smart glasses display, allowing the worker to confirm the necessary information.

[1102] Input: Search result list

[1103] Output: Visually displayed search results

[1104] Step 8:

[1105] The user sets the reminder by voice. Specifically, the worker commands the smart glasses to "remind me for the next loading / unloading operation tomorrow at 9:00 AM." This voice data is converted into text by the device and sent to the server.

[1106] Input: Voice reminder

[1107] Output: Reminder text data

[1108] Step 9:

[1109] The server stores the reminder information in a database and sets the schedule. Specifically, the server records the reminder information appropriately in the database and sets the notification to be sent at the date and time specified in the schedule function.

[1110] Input: Reminder text data

[1111] Output: Reminder information and schedule settings stored in the database

[1112] Step 10:

[1113] The device will notify the user of the reminder at the set date and time by displaying a notification on the smart glasses and prompting the user to take the necessary action.

[1114] Input: Reminder information based on schedule settings

[1115] Output: Reminder notification

[1116] Step 11:

[1117] The server analyzes past search history and saved information and uses machine learning algorithms to predict related information. Specifically, it analyzes data using machine learning libraries (e.g., TensorFlow) and generates a list of related information.

[1118] Input: User's past search history and saved information

[1119] Output: Related information list

[1120] Step 12:

[1121] The terminal presents the generated related information list to the user, specifically by displaying the related information on smart glasses, helping the worker respond quickly.

[1122] Input: Related Information List

[1123] Output: Visually displayed relevant information

[1124] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1125] The present invention relates to a system that efficiently manages and rediscovers user-entered information, and further optimizes information search, reminder notifications, and related information suggestions by recognizing and utilizing user emotions. Specific embodiments of the present invention will be described below.

[1126] Retention of Information

[1127] Subject: User

[1128] The user opens the application on their device and inputs the required information using text, music, images, videos, or through conversation with the AI. For example, if they want to input an idea for a new project, they can enter their idea in a dedicated input field.

[1129] Subject: Terminal

[1130] The device saves the input information in temporary storage and sends it to the server, generating metadata about the save date and time, tags, user ID, and content type.

[1131] Subject: Server

[1132] The server stores the received information and metadata in a database, tags the stored information, creates an index, and updates the metadata.

[1133] Information Search and Retrieval

[1134] Subject: User

[1135] Users can use the application to open a search screen and enter keywords, such as "project ideas," to search for previously saved project ideas.

[1136] Subject: Terminal

[1137] The terminal acquires the input keywords and sends a search request to the server.

[1138] Subject: Server

[1139] The server receives the search request, analyzes the keywords using a natural language processing module, searches for related information from a database based on the analysis results, and sends the results list to the terminal.

[1140] Subject: Terminal

[1141] The terminal displays the search result list sent from the server to the user, who can then refer to the list and check the information they need.

[1142] Reminders and notifications

[1143] Subject: User

[1144] The user uses the application to select the information for which they want to set a reminder, and enter the date, time, and conditions for the reminder, such as "Remind me to review new project ideas tomorrow at 10 AM."

[1145] Subject: Terminal

[1146] The device sends the reminder setting information to the server.

[1147] Subject: Server

[1148] The server receives the reminder setting information, stores it in a database, schedules notifications based on the reminder date and time and conditions, and notifies the user when the set date and time or conditions are met.

[1149] Subject: Terminal

[1150] When the date, time, or conditions set for the reminder are met, the device will notify the user, allowing the user to check the notification and take action as necessary.

[1151] Related information suggestions

[1152] Subject: Server

[1153] The server periodically analyzes the user's past search history and saved information, and uses natural language processing technology and machine learning algorithms to predict related information and generate a list of suggestions.

[1154] Subject: Terminal

[1155] The terminal presents the user with a list of suggestions, which the user can refer to for quick access to relevant information.

[1156] Incorporating an emotion engine

[1157] Subject: Terminal

[1158] The device analyzes the user's input information, behavior, and voice tone, and uses an emotion engine to recognize the user's emotions. This emotional data is then sent to the server in real time.

[1159] Subject: Server

[1160] The server then uses the received emotional data to optimize information searches and reminder notifications based on the user's emotions. For example, if the user is feeling stressed, the server will change the relevant information suggestions and adjust the reminder notification time. It also records changes in emotions over time and uses this information for future pattern analysis.

[1161] Examples:

[1162] If a reminder is set for when the user is tired, the system will use its emotion engine to detect the user's fatigue and delay the reminder. Search results will also be optimized based on the user's emotional state, prioritizing more relaxing content and information the user prefers.

[1163] In this way, the system according to the present invention not only allows users to efficiently manage and rediscover information, but also utilizes an emotion engine to provide optimal information adapted to the user's emotions.

[1164] The processing flow will be explained below.

[1165] Retention of Information

[1166] Subject: User

[1167] Step 1:

[1168] The user opens the application using a terminal.

[1169] Step 2:

[1170] The user inputs information, either through text, voice, image, video, or conversation with the AI.

[1171] Subject: Terminal

[1172] Step 3:

[1173] The terminal stores the input information in temporary storage.

[1174] Step 4:

[1175] The device generates metadata including the date and time the information was saved, tags, user ID, and content type.

[1176] Step 5:

[1177] The device sends the input information and metadata to the server.

[1178] Subject: Server

[1179] Step 6:

[1180] The server stores the received information and metadata in a database.

[1181] Step 7:

[1182] The server tags and indexes the stored information.

[1183] Information Search and Retrieval

[1184] Subject: User

[1185] Step 1:

[1186] The user opens the application's search screen on their device and enters keywords.

[1187] Subject: Terminal

[1188] Step 2:

[1189] The terminal acquires the input keyword and sends a search request to the server.

[1190] Subject: Server

[1191] Step 3:

[1192] The server receives the search request and passes the keywords to the natural language processing module.

[1193] Step 4:

[1194] A natural language processing module analyzes keywords and extracts related tags and metadata.

[1195] Step 5:

[1196] The server generates a database query to retrieve the relevant information from the database.

[1197] Step 6:

[1198] The server lists the search results and sends them to the terminal.

[1199] Subject: Terminal

[1200] Step 7:

[1201] The terminal displays the search result list received from the server to the user.

[1202] Reminders and notifications

[1203] Subject: User

[1204] Step 1:

[1205] The user uses the device to select information for which they want to set a reminder.

[1206] Step 2:

[1207] The user sets the date, time, and conditions for the reminder.

[1208] Subject: Terminal

[1209] Step 3:

[1210] The device stores the reminder setting information in temporary storage and sends it to the server.

[1211] Subject: Server

[1212] Step 4:

[1213] The server receives the reminder setting information and stores it in a database.

[1214] Step 5:

[1215] The server generates a notification schedule based on the reminder date and time and conditions.

[1216] Step 6:

[1217] The server stores the notification schedule in temporary storage and registers it with the system clock.

[1218] Subject: Terminal

[1219] Step 7:

[1220] When the reminder date, time, or conditions are met, the device will notify the user.

[1221] Related information suggestions

[1222] Subject: Server

[1223] Step 1:

[1224] The server periodically analyzes the user's past search history and saved information.

[1225] Step 2:

[1226] A natural language processing module is used to calculate the relationships between stored information.

[1227] Step 3:

[1228] Machine learning algorithms are used to generate inferential models of relevant information.

[1229] Step 4:

[1230] The server generates a list of related information based on the inference model.

[1231] Subject: Terminal

[1232] Step 5:

[1233] The terminal obtains the related information list from the server and presents it to the user.

[1234] Incorporating an emotion engine

[1235] Subject: Terminal

[1236] Step 1:

[1237] The device analyzes the user's input information, behavior, voice tone, etc., and recognizes the user's emotions using an emotion engine.

[1238] Step 2:

[1239] Emotion data is sent to the server in real time.

[1240] Subject: Server

[1241] Step 3:

[1242] Based on the emotional data received by the server, information searches and reminder notifications are optimized according to the user's emotions.

[1243] Step 4:

[1244] Emotional changes are recorded over time and used for future pattern analysis.

[1245] Specific examples

[1246] Retention of Information

[1247] Subject: User

[1248] Step 1:

[1249] A user enters a new project idea as text into an application input field.

[1250] Subject: Terminal

[1251] Step 2:

[1252] The terminal stores the input text in temporary storage and sends it to the server.

[1253] Step 3:

[1254] The device generates metadata including the save date and time and the tag "Project Idea."

[1255] Subject: Server

[1256] Step 4:

[1257] The server stores the received text and metadata in a database.

[1258] Step 5:

[1259] The server tags and indexes the text.

[1260] Information Search and Retrieval

[1261] Subject: User

[1262] Step 1:

[1263] A user searches for the keyword "project ideas."

[1264] Subject: Terminal

[1265] Step 2:

[1266] The terminal sends a search request to the server.

[1267] Subject: Server

[1268] Step 3:

[1269] The server analyzes the search request and extracts relevant information from a database.

[1270] Subject: Terminal

[1271] Step 4:

[1272] The device displays the search results to the user, who then confirms the information.

[1273] Incorporating an emotion engine

[1274] Subject: User

[1275] Step 1:

[1276] Users speak topics related to their project ideas.

[1277] Subject: Terminal

[1278] Step 2:

[1279] The device analyzes the user's voice and recognizes emotions using an emotion engine.

[1280] Step 3:

[1281] The device transmits the emotion data to the server.

[1282] Subject: Server

[1283] Step 4:

[1284] The server optimizes search results based on emotional data and displays the most appropriate information for the user.

[1285] Step 5:

[1286] The server records the user's emotional data and uses it for future data analysis.

[1287] This system allows users to efficiently manage information and receive optimal information tailored to their emotions.

[1288] Example 2

[1289] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1290] Modern information management systems provide basic functions such as efficiently storing, searching, and retrieving information input by users. However, they are unable to provide optimal information and notifications tailored to the user's emotions and circumstances. As a result, when a user is emotionally exhausted or stressed, reminder notifications and information suggestions can be annoying, and the user is unable to quickly access the information they need. It is also difficult to provide individually customized information based on the user's emotions and past behavioral history. To solve these problems, a system that can dynamically adjust information provision and notifications based on the user's emotions is needed.

[1291] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1292] In this invention, the server includes means for receiving information entered by a user, means for storing the received information in a database, means for generating metadata for the stored information, means for tagging and indexing the metadata, means for analyzing a user's search request using natural language processing technology, means for searching for information from the database based on the analyzed search request, means for presenting search results to the user, means for collecting and analyzing user emotion data, and means for adjusting reminder notifications and search results based on the collected emotion data. This makes it possible to analyze a user's emotional state and past behavior history and optimize information searches and reminder notifications based on the analyzed data.

[1293] "User" refers to any individual or entity that uses the system to enter information, search, set reminders, etc.

[1294] "Means for receiving" refers to the device or software functions that allow the server to obtain information and reminder settings entered by the user.

[1295] "Database" refers to a system or location for organizing and storing received information, reminder settings, user emotional data, etc.

[1296] "Metadata" refers to additional information about stored information (e.g., date and time of storage, tags, user ID, content type, etc.).

[1297] "Tagging" refers to the act of assigning specific labels or keywords to information to make it easier to search for later.

[1298] "Indexing" refers to the process of creating an index for efficient retrieval of information.

[1299] "Natural language processing" refers to the technology that enables computers to understand human language and appropriately analyze and generate it.

[1300] A "search request" refers to the keywords or question a user enters to find specific information.

[1301] "Emotional data" refers to information about a user's emotional state extracted from their input, behavior, vocal tone, etc.

[1302] "Analyzing" refers to structuring and understanding the received information and emotional data and processing it for a specific purpose.

[1303] "Adjustment" refers to features that dynamically change reminder notifications and search results based on collected emotional data and past behavioral history.

[1304] A "related information list" refers to a list of information that is useful to the user, generated using a machine learning algorithm based on the user's past search history and saved information.

[1305] "Inferring" refers to the process of using machine learning algorithms to predict and provide information relevant to the user.

[1306] The present invention relates to a system that efficiently manages and finds user-entered information, recognizes and utilizes user emotions, and optimizes information search, reminder notifications, and related information suggestions. Specific embodiments of the present invention are described in detail below.

[1307] Retention of Information

[1308] user

[1309] Users open the application on their device and input the required information using text, music, images, videos, or through conversation with the AI. For example, if they want to input an idea for a new project, they can enter their idea in a dedicated input field.

[1310] Terminal

[1311] The device saves the input information in temporary storage and sends it to the server. It also generates metadata about the save date and time, tags, user ID, and content type. For example, a saved project idea might be tagged with "Project," "New Technology," and "October 2023."

[1312] server

[1313] The server stores the received information and metadata in a database, tags the stored information, and creates an index to make it easier to search for the information.

[1314] Information Search and Retrieval

[1315] user

[1316] Users can use the application to open a search screen and enter keywords, such as "project ideas," to search for previously saved project ideas.

[1317] Terminal

[1318] The terminal acquires the input keywords and sends a search request to the server.

[1319] server

[1320] The server receives the search request, analyzes the keywords using natural language processing technology, searches for related information from a database based on the analysis results, and sends the search result list to the terminal.

[1321] Terminal

[1322] The terminal displays the search result list sent from the server to the user, who can then refer to the list and check the information they need.

[1323] Reminders and notifications

[1324] user

[1325] The user uses the application to select the information for which they want to set a reminder, and enter the date, time, and conditions for the reminder, such as "Remind me to review new project ideas tomorrow at 10 AM."

[1326] Terminal

[1327] The device sends the reminder setting information to the server.

[1328] server

[1329] The server receives the reminder setting information and stores it in a database. If the reminder conditions are met, the server schedules the notification and prepares to send the reminder at the set date and time.

[1330] Terminal

[1331] When the date, time, or conditions set for the reminder are met, the device will notify the user, allowing the user to check the notification and take action as necessary.

[1332] Related information suggestions

[1333] server

[1334] The server periodically analyzes the user's past search history and saved information, and uses natural language processing and machine learning algorithms to infer relevant information and generate a list of suggestions.

[1335] Terminal

[1336] The terminal displays the list of suggestions sent from the server to the user, who can then refer to the list to quickly access relevant information.

[1337] Incorporating an emotion engine

[1338] Terminal

[1339] The device analyzes the user's input information, behavior, voice tone, etc., and recognizes the user's emotions using an emotion engine. This emotional data is then sent to the server in real time.

[1340] server

[1341] The server then uses the received emotional data to optimize information searches and reminder notifications based on the user's emotions. For example, if it senses that the user is tired, it will delay reminder notifications. It also records changes in emotions over time and uses this information for future pattern analysis.

[1342] Specific examples

[1343] If a reminder is set when the user is tired, the system can use the emotion engine to detect the user's tiredness and delay the reminder by 30 minutes.

[1344] If a user searches for "project ideas," and they are feeling stressed, the suggestions could include relaxing music and videos.

[1345] Examples of prompt statements

[1346] Generate a natural language description of a program that adjusts the timing of reminders based on the user's emotional state.

[1347] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1348] Program processing flow

[1349] Retention of Information

[1350] Step 1:

[1351] user

[1352] The user opens an application on the device.

[1353] Input: Launch application.

[1354] Output: The application home screen is displayed.

[1355] Step 2:

[1356] user

[1357] Enter the required information in the form of text, music, images, videos, or even a conversation with the AI. For example, enter an idea for a new project in text.

[1358] Input: Project idea in text format.

[1359] Output: The input field is filled with the idea.

[1360] Step 3:

[1361] Terminal

[1362] The device saves the input information in temporary storage and generates metadata about the save date and time, tags, user ID, and content type.

[1363] Input: Information entered by the user.

[1364] Output: Save to temporary storage, generate metadata.

[1365] Step 4:

[1366] Terminal

[1367] The device encrypts the stored information and metadata and sends it to the server.

[1368] Input: Information and metadata stored in temporary storage.

[1369] Output: Data sent to the server.

[1370] Step 5:

[1371] server

[1372] The server stores the received information and metadata in a database, where it tags and indexes the stored information.

[1373] Input: Information and metadata sent by the device.

[1374] Output: Save to database and create index.

[1375] Information Search and Retrieval

[1376] Step 6:

[1377] user

[1378] A user opens a search screen in an application and enters a search keyword, for example, "project ideas."

[1379] Input: Search keyword.

[1380] Output: A search request is generated.

[1381] Step 7:

[1382] Terminal

[1383] The terminal acquires the input keywords and sends a search request to the server.

[1384] Input: Search keyword.

[1385] Output: A search request to the server.

[1386] Step 8:

[1387] server

[1388] The server receives the search request and analyzes the keywords using natural language processing techniques.

[1389] Input: A search request.

[1390] Output: Parsed keywords.

[1391] Step 9:

[1392] server

[1393] Based on the analysis results, relevant information is searched from the database and a list of the results is generated.

[1394] Input: The parsed keyword.

[1395] Output: Search result list.

[1396] Step 10:

[1397] server

[1398] The server transmits the search result list to the terminal.

[1399] Input: Search results list.

[1400] Output: Data sent to the terminal.

[1401] Step 11:

[1402] Terminal

[1403] The terminal receives the search result list sent from the server and displays it to the user.

[1404] Input: Search results list.

[1405] Output: Display search results to the user.

[1406] Reminders and notifications

[1407] Step 12:

[1408] user

[1409] The user opens the reminder setting screen and enters the date, time, and conditions for the reminder. For example, they can set a reminder to review new project ideas at 10:00 AM tomorrow.

[1410] Input: Reminder date, time and conditions.

[1411] Output: Reminder settings data.

[1412] Step 13:

[1413] Terminal

[1414] The device sends the reminder setting information to the server.

[1415] Input: Reminder setting data.

[1416] Output: Data sent to the server.

[1417] Step 14:

[1418] server

[1419] The server receives the reminder setting information and stores it in a database.

[1420] Input: Reminder setting data.

[1421] Output: Save to database.

[1422] Step 15:

[1423] server

[1424] When the conditions for a reminder are met, the server schedules the notification and prepares the reminder notification.

[1425] Input: Reminder date, time and conditions.

[1426] Output: Notification schedule created.

[1427] Step 16:

[1428] Terminal

[1429] When the set date, time or conditions are met, the device will send a reminder notification to the user.

[1430] Input: Notification Schedule.

[1431] Output: Notification to the user.

[1432] Related information suggestions

[1433] Step 17:

[1434] server

[1435] The server periodically analyzes the user's past search history and saved information.

[1436] Input: Your past search history and saved information.

[1437] Output: Analysis results.

[1438] Step 18:

[1439] server

[1440] It uses natural language processing technology and machine learning algorithms to infer relevant information and generate a list of suggestions.

[1441] Input: Analysis results.

[1442] Output: A list of suggestions.

[1443] Step 19:

[1444] server

[1445] The server transmits the generated proposal list to the terminal.

[1446] Input: Suggestion list.

[1447] Output: Data sent to the terminal.

[1448] Step 20:

[1449] Terminal

[1450] The terminal displays the proposal list received from the server to the user.

[1451] Input: Suggestion list.

[1452] Output: What is displayed to the user.

[1453] Incorporating an emotion engine

[1454] Step 21:

[1455] Terminal

[1456] The device collects user input information, behavior, voice tone, etc. and analyzes them using an emotion engine.

[1457] Input: User input, behavior, and tone of voice.

[1458] Output: Emotion data.

[1459] Step 22:

[1460] Terminal

[1461] The acquired emotion data is sent to the server in real time.

[1462] Input: Emotion data.

[1463] Output: Data sent to the server.

[1464] Step 23:

[1465] server

[1466] The server adjusts reminder notifications and search results based on the received emotion data.

[1467] Input: Emotion data.

[1468] Output: Tailored reminder notifications and search results.

[1469] Specific examples

[1470] Example 1:

[1471] Users search for "project ideas," and if the device detects that the user is feeling fatigued, it will suggest relaxing music and videos.

[1472] Example 2:

[1473] If a reminder notification is set when the user is tired, the server can use the emotion engine to delay the notification by 30 minutes.

[1474] (Application example 2)

[1475] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1476] Conventional information management systems are limited to efficiently managing and rediscovering user-entered information and metadata, but do not optimize search results, adjust reminder notifications, or suggest related information based on the user's emotional state. This lack of systems allows users to comfortably use information search and reminder functions. Even in physical stores, there are no product suggestions or notification functions that reflect the user's emotional state, hindering improvements to the customer experience.

[1477] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1478] In this invention, the server includes means for receiving information entered by a user, means for storing the received information in a database, means for generating metadata for the stored information, means for tagging and indexing the metadata, means for analyzing a user's search request using natural language processing technology, means for searching for information from the database based on the analyzed search request, means for presenting search results to the user, means for analyzing emotional data of the indicated information or image, and means for optimizing information based on the emotional data. This makes it possible to optimize information search and reminder notifications and suggest related information based on the user's emotional state.

[1479] "User" refers to any individual or corporation that uses this system.

[1480] "Input Information" refers to data in the form of text, music, images, video, or conversations provided by a user through a device or application.

[1481] "Means for receiving" refers to a set of hardware and software for taking input information into a computer system.

[1482] "Database" means a digital system for the structured storage and management of received information and associated metadata.

[1483] "Metadata" is data that indicates the nature and attributes of stored information, and includes the date and time of storage, tags, user ID, type of content, etc.

[1484] "Tagging and indexing methods" refers to techniques that attach identifiable labels to stored information and create data structures to facilitate efficient searching.

[1485] "Natural language processing technology" refers to a series of technologies that enable computers to understand, generate, and analyze natural human language.

[1486] A "search request" refers to the keywords or phrases a user enters to find specific information.

[1487] "Means for analyzing the emotional data of specified information or images" refers to technology that analyzes the data input by the user and recognizes the emotions contained within that data.

[1488] "Emotional data" refers to data that indicates a user's emotional state, such as information extracted from voice tone, facial expressions, and input content.

[1489] "Optimization measures" refer to methods for adjusting information search results and the timing of reminder notifications based on analyzed emotional data.

[1490] "Reminder" refers to a feature that notifies users in advance of specific events or tasks that they have set.

[1491] A "machine learning algorithm" refers to a set of computational techniques that derive rules from past data and make predictions and classifications.

[1492] "Related information" refers to additional information inferred based on a user's interests and needs, based on their past search history and saved information.

[1493] "Inferred related information list" refers to a set of related information generated by a machine learning algorithm.

[1494] "Notification" means a message or alert sent to inform the User of a reminder or relevant information.

[1495] "Suggestion" refers to useful information or recommended actions that the system provides to the user based on the user's input information and emotional data.

[1496] The present invention provides a system for efficiently managing information input by a user, recognizing the user's emotions, and optimizing search results, reminder notifications, and suggestions of related information. Specific embodiments are described below.

[1497] System configuration

[1498] Hardware

[1499] 1. Smart glasses: A device equipped with a voice recognition microphone, camera, and display that acquires and displays user input information.

[1500] 2. Server: A central system equipped with a database, natural language processing module, and emotion recognition engine, which analyzes, stores, and searches information.

[1501] 3. Surveillance camera: A device that recognizes customers' facial expressions and collects emotional data.

[1502] software

[1503] 1. Speech recognition API (such as Google Speech-to-Text): Software that converts user voice input into text data.

[1504] 2. Natural Language Processing API (such as Google NLP API): Software for analyzing text data and extracting important information.

[1505] 3. Emotion recognition API (such as Microsoft Azure Emotion API): Software for recognizing a user's emotional state from images and audio.

[1506] Operation overview

[1507] Retention of Information

[1508] The user uses the smart glasses to input information by voice. For example, they might say, "I want this red dress." This voice data is converted into text data by the smart glasses' voice recognition API and temporarily saved. This data is then sent to the server. The server stores the received information in a database, generates and tags metadata for the saved information, such as the date and time of saving, tags, and user ID, and creates an index.

[1509] Information Search and Retrieval

[1510] When a user searches for specific information through the smart glasses, they input the search query by voice. For example, a search request such as "black shoes" can be input. The smart glasses convert this voice into text and send the search request to the server. The server's natural language processing module analyzes the search request, searches for relevant information from the database, and sends the results to the smart glasses. The user can then view the search results on the display.

[1511] Reminders and notifications

[1512] Users can set reminders for later purchases. For example, they can set a reminder to "check out this red dress tomorrow at 10 AM." The smart glasses send this request to the server, which stores the setting information in a database. Reminders are scheduled based on the set date, time, and conditions, and notifications are sent to the smart glasses when the conditions are met. Furthermore, if the emotion recognition engine detects the user's fatigue, the timing of reminder notifications is adjusted.

[1513] Related information suggestions

[1514] The server periodically analyzes the user's past search history and saved information. It uses machine learning algorithms to predict relevant information and generate a list of suggestions. This list is presented to the user through smart glasses. For example, it may present highly relevant information such as "black shoes that go well with a red dress you previously purchased." The user can then choose their next action based on this list of suggestions.

[1515] Emotional data analysis and optimization

[1516] Smart glasses and surveillance cameras collect the user's facial expressions and tone of voice, and analyze the emotional data through an emotion recognition API. This data is sent to a server, which then optimizes information searches, reminder notifications, and related information suggestions based on the user's emotional state. For example, if the user is feeling stressed, product information that helps them relax will be displayed first.

[1517] Specific examples

[1518] 1. Voice input: The user inputs information by voice, such as "I like this blue jacket." The data is converted to text via a voice recognition API and sent to the server where it is saved.

[1519] 2. Emotional Data Analysis: As users walk around the store, the smart glasses' cameras and surveillance cameras collect facial expressions and voice tones. The emotion recognition API analyzes whether the user is in an excited state. This information is sent to the server, and product recommendations are optimized to match the user's excitement level.

[1520] Prompt Sentence Examples

[1521] "Save the voice data of the user saying, 'I want this red dress,' and set a reminder."

[1522] "Based on the search query 'black shoes', search for relevant information from the database and display it on the smart glasses."

[1523] "If the user's emotional state indicates stress, please suggest products that will help them relax."

[1524] Following these steps, we will build a system that provides information efficiently and based on emotions.

[1525] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1526] Step 1:

[1527] User voice input

[1528] The user puts on the smart glasses and uses voice input to describe the products they want or are interested in. For example, they might say, "I want this red dress." This input information is sent to the smart glasses' voice recognition API, which converts the voice data into text data.

[1529] Input: User voice input

[1530] Output: Speech-to-text data

[1531] Step 2:

[1532] Converting audio data to text

[1533] The device converts the user's voice into text data using a speech recognition API, such as Google Speech-to-Text, and the converted data is stored in temporary storage within the smart glasses.

[1534] Input: Audio data sent to the speech recognition API

[1535] Output: User input converted to text data

[1536] Step 3:

[1537] Sending text data to the server

[1538] The text data is sent from the temporary storage to the server, which stores it in a database and generates metadata such as the save date and time, tags, and user ID.

[1539] Input: Text data saved in temporary storage

[1540] Output: Text data and metadata stored on the server

[1541] Step 4:

[1542] Metadata generation and indexing

[1543] The server generates metadata and tags the received text data, and then creates an index to efficiently manage user information.

[1544] Input: Text data received by the server

[1545] Output: Generated metadata and indexing

[1546] Step 5:

[1547] Information search request

[1548] To search for specific information using the smart glasses, users input keywords by voice, such as a search request for "black shoes." The device converts this speech into text using a speech recognition API and sends it to the server.

[1549] Input: User's voice search request

[1550] Output: Text search request sent to the server

[1551] Step 6:

[1552] Parsing a search request

[1553] The server's natural language processing API analyzes the search request and searches the database for relevant information. As a result of the analysis, related product information is obtained.

[1554] Input: A textual search request

[1555] Output: Search results for related product information

[1556] Step 7:

[1557] Presenting search results

[1558] The server sends the search results to the smart glasses and displays them to the user, who can then view the relevant information on the smart glasses display.

[1559] Input: Server-generated search results

[1560] Output: Search results displayed on smart glasses

[1561] Step 8:

[1562] Set reminders

[1563] A user sets a reminder to check a specific product later, for example, "Check out this red dress tomorrow at 10 AM." The device sends the reminder setting information to the server.

[1564] Input: Reminder information set by the user

[1565] Output: Reminder information stored on the server

[1566] Step 9:

[1567] Scheduling reminders

[1568] The server schedules the reminder based on the reminder settings, and when the set time and conditions are met, a reminder notification is sent to the user.

[1569] Input: Reminder information stored on the server

[1570] Output: Scheduled reminder notifications

[1571] Step 10:

[1572] Emotional Data Analysis

[1573] Smart glasses and surveillance cameras collect the user's facial expressions and voice tone, and use emotion recognition APIs to analyze the emotional data, for example, to detect the user's excitement or fatigue, and send this information to a server.

[1574] Input: facial expression and tone of voice data

[1575] Output: Parsed emotion data

[1576] Step 11:

[1577] Emotion-Based Optimization

[1578] The server then uses the analyzed emotion data to adjust search results and the timing of reminder notifications. For example, if fatigue is detected, the server can delay reminder notifications.

[1579] Input: Parsed emotion data

[1580] Output: Optimized information search results and reminder notifications

[1581] Step 12:

[1582] Related information suggestions

[1583] The server uses machine learning algorithms to analyze the user's past search history and saved information to infer relevant information, and the generated suggestion list is sent to the smart glasses and presented to the user.

[1584] Input: User's past search history and saved information

[1585] Output: A generated list of related information suggestions

[1586] Through each of these steps, it becomes possible to provide optimal information and reminder functions based on the user's emotions.

[1587] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1588] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1589] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1590] [Third embodiment]

[1591] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1592] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1593] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1594] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1595] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1596] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1597] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1598] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1599] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1600] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1601] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1602] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1603] The present invention relates to a system that efficiently manages and rediscovers information stored by users (text, audio, images, videos, conversations with AI, etc.) and provides the information to users in an optimal way. Specific program processing and its implementation are described below.

[1604] Retention of Information

[1605] Subject: User

[1606] The user opens an application on the device and inputs the required information as text, or records voice, images, or videos. For example, if the user wants to input an idea for a new project as text, the user enters the idea in a dedicated input field.

[1607] Subject: Terminal

[1608] The device temporarily stores the input information and sends it to the server, generating metadata such as the date and time of storage, tags, user ID, and content type.

[1609] Subject: Server

[1610] The server stores the received information and metadata in a database, tags and indexes the stored information, and updates the metadata.

[1611] Information Search and Retrieval

[1612] Subject: User

[1613] Users can enter keywords related to the information they need in the application's search screen, for example, by entering "project ideas," to search for previously saved project ideas.

[1614] Subject: Terminal

[1615] The terminal acquires the input keywords and sends a search request to the server.

[1616] Subject: Server

[1617] The server receives the search request, analyzes the keywords using a natural language processing module, searches for related information from the database based on the analysis results, generates a list of results, and sends it to the terminal.

[1618] Subject: Terminal

[1619] The terminal displays the search result list received from the server to the user, who can then refer to the list to find the information they need.

[1620] Reminders and notifications

[1621] Subject: User

[1622] Users can set reminders for the information they save, for example, "Remind me to review new project ideas tomorrow at 10 AM."

[1623] Subject: Terminal

[1624] The device sends the reminder setting information to the server.

[1625] Subject: Server

[1626] The server receives the reminder setting information, stores it in a database, schedules notifications based on the reminder date and time and conditions, and notifies the user when the set date and time or conditions are met.

[1627] Subject: Terminal

[1628] When the set date, time, or conditions are met, the device will send a reminder notification to the user, who can then check the notification and take action as needed.

[1629] Related information suggestions

[1630] Subject: Server

[1631] The server periodically analyzes the user's past search history and saved information, and uses natural language processing technology and machine learning algorithms to predict related information and generate a list of suggestions.

[1632] Subject: Terminal

[1633] The terminal presents the user with a list of suggestions, which the user can refer to for quick access to relevant information.

[1634] As described above, the system of the present invention integrates multiple functions to enable users to efficiently manage and rediscover information. As a specific example, users can save ideas for new projects, retrieve information when needed by keyword search, and set reminders to receive notifications at appropriate times. This system allows users to manage important information without missing it and utilize it efficiently.

[1635] The processing flow will be explained below.

[1636] Retention of Information

[1637] Subject: User

[1638] Step 1:

[1639] The user opens the application using a terminal.

[1640] Step 2:

[1641] The user inputs information, either through text, voice, image, video, or conversation with the AI.

[1642] Subject: Terminal

[1643] Step 3:

[1644] The terminal stores the input information in temporary storage.

[1645] Step 4:

[1646] The device generates metadata including the date and time the information was saved, tags, user ID, and content type.

[1647] Step 5:

[1648] The device sends the input information and metadata to the server.

[1649] Subject: Server

[1650] Step 6:

[1651] The server stores the received information and metadata in a database.

[1652] Step 7:

[1653] The server tags and indexes the stored information.

[1654] Information Search and Retrieval

[1655] Subject: User

[1656] Step 1:

[1657] The user opens the application's search screen on their device and enters keywords.

[1658] Subject: Terminal

[1659] Step 2:

[1660] The terminal acquires the input keyword and sends a search request to the server.

[1661] Subject: Server

[1662] Step 3:

[1663] The server receives the search request and passes the keywords to the natural language processing module.

[1664] Step 4:

[1665] A natural language processing module analyzes keywords and extracts related tags and metadata.

[1666] Step 5:

[1667] The server generates a database query to retrieve the relevant information from the database.

[1668] Step 6:

[1669] The server lists the search results and sends them to the terminal.

[1670] Subject: Terminal

[1671] Step 7:

[1672] The terminal displays the search result list received from the server to the user.

[1673] Reminders and notifications

[1674] Subject: User

[1675] Step 1:

[1676] The user uses the device to select information for which they want to set a reminder.

[1677] Step 2:

[1678] The user sets the date, time, and conditions for the reminder.

[1679] Subject: Terminal

[1680] Step 3:

[1681] The device stores the reminder setting information in temporary storage and sends it to the server.

[1682] Subject: Server

[1683] Step 4:

[1684] The server receives the reminder setting information and stores it in a database.

[1685] Step 5:

[1686] The server generates a notification schedule based on the reminder date and time and conditions.

[1687] Step 6:

[1688] The server stores the notification schedule in temporary storage and registers it with the system clock.

[1689] Subject: Terminal

[1690] Step 7:

[1691] When the reminder date, time, or conditions are met, the device will notify the user.

[1692] Related information suggestions

[1693] Subject: Server

[1694] Step 1:

[1695] The server periodically analyzes the user's past search history and saved information.

[1696] Step 2:

[1697] A natural language processing module is used to calculate the relationships between stored information.

[1698] Step 3:

[1699] Machine learning algorithms are used to generate inferential models of relevant information.

[1700] Step 4:

[1701] The server generates a list of related information based on the inference model.

[1702] Subject: Terminal

[1703] Step 5:

[1704] The terminal obtains the related information list from the server and presents it to the user.

[1705] Example 1

[1706] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1707] Users generate and store a large amount of information on a daily basis, but it is difficult to efficiently manage that information and quickly find it again when needed. Users also desire a reminder function to ensure they don't forget important information, but manually setting this function is cumbersome. Furthermore, there is a lack of systems that automatically suggest related information based on a user's past behavior. An integrated information management system that can solve these issues is needed.

[1708] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1709] In this invention, the server includes means for receiving information entered by a user, means for saving the received information in a storage device, means for generating additional information for the saved information, means for tagging and indexing the additional information, means for analyzing a user's search request using natural language processing technology, means for searching for information from the storage device based on the analyzed search request, means for presenting search results to the user, means for receiving reminders set by the user, means for saving reminder information in a storage device, means for scheduling notifications based on the reminder date and time and conditions, means for notifying the user when the set date and time and conditions are met, means for analyzing the user's past search history and saved information, means for using a machine learning algorithm to infer related information, means for generating a list of inferred related information, and means for presenting the generated related information list to the user. This allows the user to efficiently manage saved information and quickly rediscover it when needed, and the reminders allow the user to manage important information without missing it and also provide suggestions of related information based on past behavior.

[1710] "Means for receiving" refers to the function that allows a terminal or server to receive information or requests sent by a user.

[1711] "Storage device" means a physical or virtual device for storing data, including hard drives, SSDs, cloud storage, etc.

[1712] The "means for saving" is a function for writing received information or data to a storage device.

[1713] "Additional information" refers to additional information related to the content of the saved information, such as metadata such as tags, save date and time, and user ID.

[1714] "Means for generating" is a function for automatically generating specific additional information based on certain information or data.

[1715] "Tagging" is a method of adding specific keywords or categories to information to make it easier to organize and search.

[1716] "Indexing" is the process of building a data structure so that information can be efficiently searched.

[1717] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, and includes text analysis and language generation.

[1718] A "search request" is a request containing keywords or phrases entered by a user in search of specific information.

[1719] "Means of analysis" refers to the ability to understand the received search request and perform processing to find appropriate information.

[1720] The "searching means" is a function for searching for related information from a storage device based on the analyzed keywords.

[1721] "Means of presentation" refers to the function that allows a server or terminal to display search results and notifications to the user.

[1722] A "reminder" is a notification that a user sets for a specific date, time, or condition.

[1723] "Scheduling tools" are functions that allow you to plan notifications based on reminder dates and conditions.

[1724] "Means of notification" is a function for sending alerts or messages to users when a set date, time, or condition is met.

[1725] "Search history" refers to a record of searches a user has conducted in the past.

[1726] A "machine learning algorithm" is a computational method for analyzing data, discovering patterns, and using those patterns to predict future data.

[1727] A "related information list" is a list of information that is presumed to be useful to the user based on past search history and saved information.

[1728] The present invention relates to a system for efficiently managing user-generated information (text, audio, images, video, etc.) and rediscovering it when needed. Specific embodiments of the system are described below.

[1729] Retention of Information

[1730] 1. User input

[1731] Users open the application on their device and enter the required information: for text input, they write the information in a dedicated input field, for voice input, they press the microphone button to start recording, or for images and videos, they use the camera function.

[1732] Specific working example:

[1733] Enter your new project idea as "New website design" or, if you use voice recognition, say something like "Record a new project idea."

[1734] 2. Temporary storage on the device

[1735] The device saves the input information in temporary storage, generating metadata such as the save date and time, tags, user ID, and content type. The smartphone's internal memory is used as temporary storage.

[1736] 3. Sending from the device to the server

[1737] The device sends the stored information and metadata to the server, typically using an HTTP POST request to send the data.

[1738] 4. Server-based information storage and metadata updates

[1739] The server stores the received information and metadata in a database (e.g., MySQL), and a tagging engine runs to add appropriate tags to the text information.

[1740] Information Search and Retrieval

[1741] 1. User enters search keywords

[1742] Users enter keywords related to the information they need on the application's search screen.

[1743] Specific working example:

[1744] A user types in "project ideas" and presses the search button.

[1745] 2. Sending a search request via the device

[1746] The device receives the input keywords and sends a search request to the server. It generates an HTTP GET request and sends it to the server.

[1747] 3. Keyword analysis and search result generation by the server

[1748] The server uses a natural language processing library (e.g., NLTK or SpaCy) to analyze the keywords and search for relevant information in a storage device (database). The result list is generated in JSON format and sent to the device.

[1749] 4. Displaying search results on your device

[1750] The terminal displays the search result list received from the server to the user, displaying the search results in a list view so that the user can access the information they need.

[1751] Reminders and notifications

[1752] 1. User-defined reminder settings

[1753] Users can set reminders for saved information.

[1754] Specific working example:

[1755] The user enters the instruction "Set a reminder to check for new project ideas tomorrow at 10 AM."

[1756] 2. Send reminders via device

[1757] The device sends this reminder setting information to the server.

[1758] 3. Server-based reminder storage and scheduling

[1759] The server stores the reminder information in a database and sets up a notification scheduler (e.g., a Cron job or Quartz Scheduler).

[1760] 4. Device-based reminder notifications

[1761] When the set date, time, or conditions are met, a notification is triggered from the server and the device sends a reminder notification to the user.

[1762] Related information suggestions

[1763] 1. Server Generation of Proposal List

[1764] The server analyzes the user's past search history and saved information, and uses natural language processing techniques and machine learning algorithms (e.g., TensorFlow and PyTorch) to infer relevant information. A list of suggestions is generated periodically.

[1765] 2. Displaying a list of suggestions on the device

[1766] The terminal presents the user with a list of suggestions received from the server, and based on the information provided, the user can quickly access relevant information.

[1767] Examples and prompts

[1768] Examples:

[1769] 1. Storage of Information:

[1770] User: Texts a new project idea: "New website design."

[1771] Device: Sends information along with metadata to the server.

[1772] Server: Stores the information in a database and tags it.

[1773] 2. Information Search:

[1774] User: Search for "project ideas."

[1775] Device: Sends a search request to the server.

[1776] Server: Searches for relevant information and generates a list of results.

[1777] Device: Display search results to the user.

[1778] 3. Reminder:

[1779] User: Set a reminder to review new project ideas tomorrow at 10 AM.

[1780] Device: Sends reminder setting information to the server.

[1781] Server: Schedule reminders.

[1782] On your device: Reminds you at the set time.

[1783] Example prompt:

[1784] "Set a reminder to review new project ideas tomorrow at 10 AM."

[1785] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1786] Step 1:

[1787] The user opens an application on their device and inputs new information, for example, by typing a new project idea as text, "About a new website design," or by recording an audio, image, or video. It takes the user's data (text, audio, image, video) as input and generates data as output that is stored in temporary storage.

[1788] Step 2:

[1789] The device saves the information entered by the user in temporary storage. At this time, additional information such as the save date and time, tag, user ID, and content type is generated for the data saved in temporary storage. This allows information to be organized efficiently. Data from the user is received as input, and data with additional information is generated as output.

[1790] Step 3:

[1791] The device sends the stored information and additional information to the server. Specifically, it sends data using an HTTP POST request. It takes the data stored in temporary storage as input and generates an HTTP request that is sent to the server as output.

[1792] Step 4:

[1793] The server stores the received information and additional information in a database, performs appropriate tagging and indexing on the stored information, and updates the additional information. The server takes the data received by the server as input and generates information to be stored in the database as output.

[1794] Step 5:

[1795] The user enters keywords related to the information they need into the application's search screen. For example, they can enter "project ideas" to search for previously saved project ideas. The application takes search keywords as input and generates a search request as output.

[1796] Step 6:

[1797] The device acquires the search keywords and sends a search request to the server. Specifically, it generates an HTTP GET request and sends it to the server. It receives the search keywords from the user as input and generates and sends a search request as output.

[1798] Step 7:

[1799] The server receives the search request and analyzes the entered keywords using natural language processing techniques. Based on the analysis results, it searches for relevant information from the database. It takes the search request received by the server as input and generates a list of search results as output.

[1800] Step 8:

[1801] The terminal displays the search result list received from the server to the user. Specifically, the terminal displays the search results in a list view. The terminal receives the search results received from the server as input and generates the list of search results that is displayed to the user as output.

[1802] Step 9:

[1803] The user sets a reminder for the saved information, for example, "Remind me to review new project ideas tomorrow at 10 AM." The system takes the reminder setting information as input and generates a reminder setting request as output.

[1804] Step 10:

[1805] The device sends reminder setting information to the server. Specifically, it generates an HTTP POST request and sends it to the server. It receives reminder setting information from the user as input and generates and sends a reminder setting request as output.

[1806] Step 11:

[1807] The server receives the reminder setting information and stores it in a database. It schedules notifications based on the reminder date and time and conditions. It takes the reminder setting information received by the server as input and generates the scheduled notifications as output.

[1808] Step 12:

[1809] When the set date, time, or conditions are met, the device will notify the user with a reminder, for example, by displaying a reminder in the notification bar. It takes a reminder notification received from the server as input and generates a reminder notification to be displayed to the user as output.

[1810] Step 13:

[1811] The server analyzes the user's past search history and saved information, and uses natural language processing technology and machine learning algorithms to infer related information and generate a list of suggestions. It receives past search history and saved information as input and generates a list of suggestions as output.

[1812] Step 14:

[1813] The terminal presents the suggestion list received from the server to the user, allowing the user to quickly access relevant information. It takes the suggestion list received from the server as input and generates the suggestion list that is presented to the user as output.

[1814] (Application example 1)

[1815] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1816] In logistics centers, workers handle a wide variety of product information management tasks, and manual information entry and search is time-consuming, resulting in problems such as reduced accuracy and increased risk of errors. Furthermore, a lack of efficient reminder functions and suggestions for related information often leads to reduced work efficiency. There is a need for a system that can solve these problems, enable workers to quickly and accurately access the information they need, and improve work efficiency.

[1817] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1818] In this invention, the server includes means for receiving information entered by a user, means for storing the received information in a database, means for generating metadata for the stored information, means for tagging and indexing the metadata, means for analyzing a user's search request using natural language processing technology, means for searching for information from the database based on the analyzed search request, means for presenting search results to the user, means for acquiring information using a wearable device worn by a worker, and means for transmitting the acquired information to the server in real time. This makes it possible to use the wearable device to quickly and accurately manage information and improve work efficiency.

[1819] "User" refers to the entity that uses an information system.

[1820] A "wearable device" refers to an electronic device that can be worn by the user.

[1821] "Means for receiving information" refers to the mechanism by which user-entered information is incorporated into the data system.

[1822] "Database" refers to a system for systematically collecting, storing, and managing data.

[1823] "Metadata" refers to information about stored information, i.e., additional descriptions or attributes about that data.

[1824] "Tagging" refers to the process of classifying and organizing information or data by attaching labels or identification codes to it.

[1825] "Indexing" refers to the procedures and structures that enable quick search for necessary information from a large amount of data.

[1826] "Natural language processing technology" refers to technology for processing and analyzing human language (natural language) using a computer.

[1827] "Search Request" means a request or inquiry entered by a User to locate specific information.

[1828] "Real-time" refers to a situation in which information is acquired and processed instantly, with almost no time delay.

[1829] This invention is a system that uses wearable devices to manage and search information in order to improve work efficiency at logistics centers. Specifically, information is acquired through the wearable devices, sent to a server, and the necessary information is searched and displayed in real time.

[1830] Overall structure

[1831] The system consists of the following main components:

[1832] 1. Wearable devices: Devices worn by workers to input and display information, including smart glasses and smart watches.

[1833] 2. Server: The central system for storing, analyzing, searching, and notifying information.

[1834] 3. Database: A system connected to a server for storing saved information, metadata, search history, and reminder information.

[1835] Receiving and storing information

[1836] The wearable device captures information through voice input and barcode scanning by the worker. For example, a worker can scan the barcode of a new product and enter its details by voice. It can also record the product's condition using video recording. This information is sent in real time to a server, which stores it in a database. At the same time, metadata is generated, including the date and time of storage, tag, worker ID, and content type.

[1837] Information Search and Retrieval

[1838] Workers use the smart glasses to search for the information they need using voice commands. For example, they can say, "Tell me the inventory status of shelf number A3." The voice command is sent to the server, which analyzes it using natural language processing technology and searches for relevant information in the database. The results are displayed on the smart glasses, allowing workers to quickly obtain the information.

[1839] Reminders and notifications

[1840] Reminders are set by workers through voice input. For example, they can say, "Remind me tomorrow at 9:00 AM for the next loading / unloading operation." The reminder information is sent to the server and stored in a database. When the set date, time, or conditions are met, the server automatically sends a notification to the wearable device and displays the reminder. This helps workers avoid missing important tasks.

[1841] Related information suggestions

[1842] The server uses machine learning algorithms to analyze past search history and stored information. Based on this, it infers relevant information and makes suggestions to improve work efficiency. For example, if a frequently searched item is low in stock, it will make a replenishment suggestion. These suggestions are displayed to the worker through the smart glasses, allowing the worker to take action quickly.

[1843] Examples and prompts

[1844] For example, consider a scenario in which a worker saves details of a new item.

[1845] Example prompt sentence:

[1846] "Scan the barcode, then speak the product details."

[1847] Execution example:

[1848] Barcode scan: "1234567890123"

[1849] Speak: "This item is part of batch process B and is located on shelf number A3."

[1850] This information is immediately stored on the server and used for future searches and related information suggestions.

[1851] Specific examples of hardware and software used

[1852] Hardware: smart glasses (e.g., Google Glass), servers (high-performance servers in data centers), database servers (e.g., PostgreSQL)

[1853] Software: Natural language processing libraries (spaCy, NLTK), machine learning algorithms (TensorFlow, scikit-learn)

[1854] In this way, it is possible to efficiently manage and search information in the logistics center, provide reminder functions, and suggest related information.

[1855] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1856] Step 1:

[1857] Users input information using a wearable device. Specifically, a worker scans the barcode of a product with smart glasses and inputs detailed information about the product by voice. This input information includes the barcode value and voice text.

[1858] Input: Barcode scan data, voice input text

[1859] Output: Collected product information

[1860] Step 2:

[1861] The device sends the collected product information to a server in real time. Specifically, the smart glasses upload the barcode scan results and voice-input text to the server via the Internet, along with metadata such as the save date and time, tag, and worker ID.

[1862] Input: Collected product information (barcode data, voice text)

[1863] Output: Data sent to the server

[1864] Step 3:

[1865] The server stores the received information in a database, specifically by recording the information and metadata in a database system (e.g., PostgreSQL), including tagging and indexing.

[1866] Input: Data sent to the server

[1867] Output: Information stored in the database

[1868] Step 4:

[1869] Users can search for the information they need using voice commands. Specifically, a worker speaks into the smart glasses, saying, "Tell me the inventory status of shelf number A3." This voice is converted into text by the device and sent to the server.

[1870] Input: Voice command

[1871] Output: Text data of voice commands

[1872] Step 5:

[1873] The server receives the search request and analyzes it using natural language processing technology. Specifically, it uses a natural language processing library (e.g., spaCy) to analyze the voice command and extract the keywords necessary for the search.

[1874] Input: Text data of voice command

[1875] Output: Parsed keywords

[1876] Step 6:

[1877] The server searches for information from the database based on the analyzed keywords, generates a database query to retrieve the relevant information, and formats the query results into a list and sends it to the smart glasses.

[1878] Input: Parsed keyword

[1879] Output: Search result list

[1880] Step 7:

[1881] The terminal receives the search results from the server and displays them to the user. Specifically, the search results are visually displayed on the smart glasses display, allowing the worker to confirm the necessary information.

[1882] Input: Search result list

[1883] Output: Visually displayed search results

[1884] Step 8:

[1885] The user sets the reminder by voice. Specifically, the worker commands the smart glasses to "remind me for the next loading / unloading operation tomorrow at 9:00 AM." This voice data is converted into text by the device and sent to the server.

[1886] Input: Voice reminder

[1887] Output: Reminder text data

[1888] Step 9:

[1889] The server stores the reminder information in a database and sets the schedule. Specifically, the server appropriately records the reminder information in the database and sets the notification to be sent at the date and time specified in the schedule function.

[1890] Input: Reminder text data

[1891] Output: Reminder information and schedule settings stored in the database

[1892] Step 10:

[1893] The device will notify the user of the reminder at the set date and time by displaying a notification on the smart glasses and prompting the user to take the necessary action.

[1894] Input: Reminder information based on schedule settings

[1895] Output: Reminder notification

[1896] Step 11:

[1897] The server analyzes past search history and saved information and uses machine learning algorithms to predict related information. Specifically, it analyzes data using machine learning libraries (e.g., TensorFlow) and generates a list of related information.

[1898] Input: User's past search history and saved information

[1899] Output: Related information list

[1900] Step 12:

[1901] The terminal presents the generated related information list to the user, specifically by displaying the related information on smart glasses, helping the worker respond quickly.

[1902] Input: Related Information List

[1903] Output: Visually displayed relevant information

[1904] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1905] The present invention relates to a system that efficiently manages and rediscovers user-entered information, and further optimizes information search, reminder notifications, and related information suggestions by recognizing and utilizing user emotions. Specific embodiments of the present invention will be described below.

[1906] Retention of Information

[1907] Subject: User

[1908] The user opens the application on their device and inputs the required information using text, music, images, videos, or through conversation with the AI. For example, if they want to input an idea for a new project, they can enter their idea in a dedicated input field.

[1909] Subject: Terminal

[1910] The device saves the input information in temporary storage and sends it to the server, generating metadata about the save date and time, tags, user ID, and content type.

[1911] Subject: Server

[1912] The server stores the received information and metadata in a database, tags the stored information, creates an index, and updates the metadata.

[1913] Information Search and Retrieval

[1914] Subject: User

[1915] Users can use the application to open a search screen and enter keywords, such as "project ideas," to search for previously saved project ideas.

[1916] Subject: Terminal

[1917] The terminal acquires the input keywords and sends a search request to the server.

[1918] Subject: Server

[1919] The server receives the search request, analyzes the keywords using a natural language processing module, searches for related information from a database based on the analysis results, and sends the results list to the terminal.

[1920] Subject: Terminal

[1921] The terminal displays the search result list sent from the server to the user, who can then refer to the list and check the information they need.

[1922] Reminders and notifications

[1923] Subject: User

[1924] The user uses the application to select the information for which they want to set a reminder, and enter the date, time, and conditions for the reminder, such as "Remind me to review new project ideas tomorrow at 10 AM."

[1925] Subject: Terminal

[1926] The device sends the reminder setting information to the server.

[1927] Subject: Server

[1928] The server receives the reminder setting information, stores it in a database, schedules notifications based on the reminder date and time and conditions, and notifies the user when the set date and time or conditions are met.

[1929] Subject: Terminal

[1930] When the date, time, or conditions set for the reminder are met, the device will notify the user, allowing the user to check the notification and take action as necessary.

[1931] Related information suggestions

[1932] Subject: Server

[1933] The server periodically analyzes the user's past search history and saved information, and uses natural language processing technology and machine learning algorithms to predict related information and generate a list of suggestions.

[1934] Subject: Terminal

[1935] The terminal presents the user with a list of suggestions, which the user can refer to for quick access to relevant information.

[1936] Incorporating an emotion engine

[1937] Subject: Terminal

[1938] The device analyzes the user's input information, behavior, and voice tone, and uses an emotion engine to recognize the user's emotions. This emotional data is then sent to the server in real time.

[1939] Subject: Server

[1940] The server then uses the received emotional data to optimize information searches and reminder notifications based on the user's emotions. For example, if the user is feeling stressed, the server will change the relevant information suggestions and adjust the reminder notification time. It also records changes in emotions over time and uses this information for future pattern analysis.

[1941] Examples:

[1942] If a reminder is set for when the user is tired, the system will use its emotion engine to detect the user's fatigue and delay the reminder. Search results will also be optimized based on the user's emotional state, prioritizing more relaxing content and information the user prefers.

[1943] In this way, the system according to the present invention not only allows users to efficiently manage and rediscover information, but also utilizes an emotion engine to provide optimal information adapted to the user's emotions.

[1944] The processing flow will be explained below.

[1945] Retention of Information

[1946] Subject: User

[1947] Step 1:

[1948] The user opens the application using a terminal.

[1949] Step 2:

[1950] The user inputs information, either through text, voice, image, video, or conversation with the AI.

[1951] Subject: Terminal

[1952] Step 3:

[1953] The terminal stores the input information in temporary storage.

[1954] Step 4:

[1955] The device generates metadata including the date and time the information was saved, tags, user ID, and content type.

[1956] Step 5:

[1957] The device sends the input information and metadata to the server.

[1958] Subject: Server

[1959] Step 6:

[1960] The server stores the received information and metadata in a database.

[1961] Step 7:

[1962] The server tags and indexes the stored information.

[1963] Information Search and Retrieval

[1964] Subject: User

[1965] Step 1:

[1966] The user opens the application's search screen on their device and enters keywords.

[1967] Subject: Terminal

[1968] Step 2:

[1969] The terminal acquires the input keyword and sends a search request to the server.

[1970] Subject: Server

[1971] Step 3:

[1972] The server receives the search request and passes the keywords to the natural language processing module.

[1973] Step 4:

[1974] A natural language processing module analyzes keywords and extracts related tags and metadata.

[1975] Step 5:

[1976] The server generates a database query to retrieve the relevant information from the database.

[1977] Step 6:

[1978] The server lists the search results and sends them to the terminal.

[1979] Subject: Terminal

[1980] Step 7:

[1981] The terminal displays the search result list received from the server to the user.

[1982] Reminders and notifications

[1983] Subject: User

[1984] Step 1:

[1985] The user uses the device to select information for which they want to set a reminder.

[1986] Step 2:

[1987] The user sets the date, time, and conditions for the reminder.

[1988] Subject: Terminal

[1989] Step 3:

[1990] The device stores the reminder setting information in temporary storage and sends it to the server.

[1991] Subject: Server

[1992] Step 4:

[1993] The server receives the reminder setting information and stores it in a database.

[1994] Step 5:

[1995] The server generates a notification schedule based on the reminder date and time and conditions.

[1996] Step 6:

[1997] The server stores the notification schedule in temporary storage and registers it with the system clock.

[1998] Subject: Terminal

[1999] Step 7:

[2000] When the reminder date, time, or conditions are met, the device will notify the user.

[2001] Related information suggestions

[2002] Subject: Server

[2003] Step 1:

[2004] The server periodically analyzes the user's past search history and saved information.

[2005] Step 2:

[2006] A natural language processing module is used to calculate the relationships between stored information.

[2007] Step 3:

[2008] Machine learning algorithms are used to generate inferential models of relevant information.

[2009] Step 4:

[2010] The server generates a list of related information based on the inference model.

[2011] Subject: Terminal

[2012] Step 5:

[2013] The terminal obtains the related information list from the server and presents it to the user.

[2014] Incorporating an emotion engine

[2015] Subject: Terminal

[2016] Step 1:

[2017] The device analyzes the user's input information, behavior, voice tone, etc., and recognizes the user's emotions using an emotion engine.

[2018] Step 2:

[2019] Emotion data is sent to the server in real time.

[2020] Subject: Server

[2021] Step 3:

[2022] Based on the emotional data received by the server, information searches and reminder notifications are optimized according to the user's emotions.

[2023] Step 4:

[2024] Emotional changes are recorded over time and used for future pattern analysis.

[2025] Specific examples

[2026] Retention of Information

[2027] Subject: User

[2028] Step 1:

[2029] A user enters a new project idea as text into an application input field.

[2030] Subject: Terminal

[2031] Step 2:

[2032] The terminal stores the input text in temporary storage and sends it to the server.

[2033] Step 3:

[2034] The device generates metadata including the save date and time and the tag "Project Idea."

[2035] Subject: Server

[2036] Step 4:

[2037] The server stores the received text and metadata in a database.

[2038] Step 5:

[2039] The server tags and indexes the text.

[2040] Information Search and Retrieval

[2041] Subject: User

[2042] Step 1:

[2043] A user searches for the keyword "project ideas."

[2044] Subject: Terminal

[2045] Step 2:

[2046] The terminal sends a search request to the server.

[2047] Subject: Server

[2048] Step 3:

[2049] The server analyzes the search request and extracts relevant information from a database.

[2050] Subject: Terminal

[2051] Step 4:

[2052] The device displays the search results to the user, who then confirms the information.

[2053] Incorporating an emotion engine

[2054] Subject: User

[2055] Step 1:

[2056] Users speak topics related to their project ideas.

[2057] Subject: Terminal

[2058] Step 2:

[2059] The device analyzes the user's voice and recognizes emotions using an emotion engine.

[2060] Step 3:

[2061] The device transmits the emotion data to the server.

[2062] Subject: Server

[2063] Step 4:

[2064] The server optimizes search results based on emotional data and displays the most appropriate information for the user.

[2065] Step 5:

[2066] The server records the user's emotional data and uses it for future data analysis.

[2067] This system allows users to efficiently manage information and receive optimal information tailored to their emotions.

[2068] Example 2

[2069] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2070] Modern information management systems provide basic functions such as efficiently storing, searching, and retrieving information input by users. However, they are unable to provide optimal information and notifications tailored to the user's emotions and circumstances. As a result, when a user is emotionally exhausted or stressed, reminder notifications and information suggestions can be annoying, and the user is unable to quickly access the information they need. It is also difficult to provide individually customized information based on the user's emotions and past behavioral history. To solve these problems, a system that can dynamically adjust information provision and notifications based on the user's emotions is needed.

[2071] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2072] In this invention, the server includes means for receiving information entered by a user, means for storing the received information in a database, means for generating metadata for the stored information, means for tagging and indexing the metadata, means for analyzing a user's search request using natural language processing technology, means for searching for information from the database based on the analyzed search request, means for presenting search results to the user, means for collecting and analyzing user emotion data, and means for adjusting reminder notifications and search results based on the collected emotion data. This makes it possible to analyze a user's emotional state and past behavior history and optimize information searches and reminder notifications based on the analyzed data.

[2073] "User" refers to any individual or entity that uses the system to enter information, search, set reminders, etc.

[2074] "Means for receiving" refers to the device or software functions that allow the server to obtain information and reminder settings entered by the user.

[2075] "Database" refers to a system or location for organizing and storing received information, reminder settings, user emotional data, etc.

[2076] "Metadata" refers to additional information about stored information (e.g., date and time of storage, tags, user ID, content type, etc.).

[2077] "Tagging" refers to the act of assigning specific labels or keywords to information to make it easier to search for later.

[2078] "Indexing" refers to the process of creating an index for efficient retrieval of information.

[2079] "Natural language processing" refers to the technology that enables computers to understand human language and appropriately analyze and generate it.

[2080] A "search request" refers to the keywords or question a user enters to find specific information.

[2081] "Emotional data" refers to information about a user's emotional state extracted from their input, behavior, vocal tone, etc.

[2082] "Analyzing" refers to structuring and understanding the received information and emotional data and processing it for a specific purpose.

[2083] "Adjustment" refers to features that dynamically change reminder notifications and search results based on collected emotional data and past behavioral history.

[2084] A "related information list" refers to a list of information that is useful to the user, generated using a machine learning algorithm based on the user's past search history and saved information.

[2085] "Inferring" refers to the process of using machine learning algorithms to predict and provide information relevant to the user.

[2086] The present invention relates to a system that efficiently manages and finds user-entered information, recognizes and utilizes user emotions, and optimizes information search, reminder notifications, and related information suggestions. Specific embodiments of the present invention are described in detail below.

[2087] Retention of Information

[2088] user

[2089] Users open the application on their device and input the required information using text, music, images, videos, or through conversation with the AI. For example, if they want to input an idea for a new project, they can enter their idea in a dedicated input field.

[2090] Terminal

[2091] The device saves the input information in temporary storage and sends it to the server. It also generates metadata about the save date and time, tags, user ID, and content type. For example, a saved project idea might be tagged with "Project," "New Technology," and "October 2023."

[2092] server

[2093] The server stores the received information and metadata in a database, tags the stored information, and creates an index to make it easier to search for the information.

[2094] Information Search and Retrieval

[2095] user

[2096] Users can use the application to open a search screen and enter keywords, such as "project ideas," to search for previously saved project ideas.

[2097] Terminal

[2098] The terminal acquires the input keywords and sends a search request to the server.

[2099] server

[2100] The server receives the search request, analyzes the keywords using natural language processing technology, searches for related information from a database based on the analysis results, and sends the search result list to the terminal.

[2101] Terminal

[2102] The terminal displays the search result list sent from the server to the user, who can then refer to the list and check the information they need.

[2103] Reminders and notifications

[2104] user

[2105] The user uses the application to select the information for which they want to set a reminder, and enter the date, time, and conditions for the reminder, such as "Remind me to review new project ideas tomorrow at 10 AM."

[2106] Terminal

[2107] The device sends the reminder setting information to the server.

[2108] server

[2109] The server receives the reminder setting information and stores it in a database. If the reminder conditions are met, the server schedules the notification and prepares to send the reminder at the set date and time.

[2110] Terminal

[2111] When the date, time, or conditions set for the reminder are met, the device will notify the user, allowing the user to check the notification and take action as necessary.

[2112] Related information suggestions

[2113] server

[2114] The server periodically analyzes the user's past search history and saved information, and uses natural language processing and machine learning algorithms to infer relevant information and generate a list of suggestions.

[2115] Terminal

[2116] The terminal displays the list of suggestions sent from the server to the user, who can then refer to the list to quickly access relevant information.

[2117] Incorporating an emotion engine

[2118] Terminal

[2119] The device analyzes the user's input information, behavior, voice tone, etc., and recognizes the user's emotions using an emotion engine. This emotional data is then sent to the server in real time.

[2120] server

[2121] The server then uses the received emotional data to optimize information searches and reminder notifications based on the user's emotions. For example, if it senses that the user is tired, it will delay reminder notifications. It also records changes in emotions over time and uses this information for future pattern analysis.

[2122] Specific examples

[2123] If a reminder is set when the user is tired, the system can use the emotion engine to detect the user's tiredness and delay the reminder by 30 minutes.

[2124] If a user searches for "project ideas," and they are feeling stressed, the suggestions could include relaxing music and videos.

[2125] Examples of prompt statements

[2126] Generate a natural language description of a program that adjusts the timing of reminders based on the user's emotional state.

[2127] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2128] Program processing flow

[2129] Retention of Information

[2130] Step 1:

[2131] user

[2132] The user opens an application on the device.

[2133] Input: Launch application.

[2134] Output: The application home screen is displayed.

[2135] Step 2:

[2136] user

[2137] Enter the required information in the form of text, music, images, videos, or even a conversation with the AI. For example, enter an idea for a new project in text.

[2138] Input: Project idea in text format.

[2139] Output: The input field is filled with the idea.

[2140] Step 3:

[2141] Terminal

[2142] The device saves the input information in temporary storage and generates metadata about the save date and time, tags, user ID, and content type.

[2143] Input: Information entered by the user.

[2144] Output: Save to temporary storage, generate metadata.

[2145] Step 4:

[2146] Terminal

[2147] The device encrypts the stored information and metadata and sends it to the server.

[2148] Input: Information and metadata stored in temporary storage.

[2149] Output: Data sent to the server.

[2150] Step 5:

[2151] server

[2152] The server stores the received information and metadata in a database, where it tags and indexes the stored information.

[2153] Input: Information and metadata sent by the device.

[2154] Output: Save to database and create index.

[2155] Information Search and Retrieval

[2156] Step 6:

[2157] user

[2158] A user opens a search screen in an application and enters a search keyword, for example, "project ideas."

[2159] Input: Search keyword.

[2160] Output: A search request is generated.

[2161] Step 7:

[2162] Terminal

[2163] The terminal acquires the input keywords and sends a search request to the server.

[2164] Input: Search keyword.

[2165] Output: A search request to the server.

[2166] Step 8:

[2167] server

[2168] The server receives the search request and analyzes the keywords using natural language processing techniques.

[2169] Input: A search request.

[2170] Output: Parsed keywords.

[2171] Step 9:

[2172] server

[2173] Based on the analysis results, relevant information is searched from the database and a list of the results is generated.

[2174] Input: The parsed keyword.

[2175] Output: Search result list.

[2176] Step 10:

[2177] server

[2178] The server transmits the search result list to the terminal.

[2179] Input: Search results list.

[2180] Output: Data sent to the terminal.

[2181] Step 11:

[2182] Terminal

[2183] The terminal receives the search result list sent from the server and displays it to the user.

[2184] Input: Search results list.

[2185] Output: Display search results to the user.

[2186] Reminders and notifications

[2187] Step 12:

[2188] user

[2189] The user opens the reminder setting screen and enters the date, time, and conditions for the reminder. For example, they can set a reminder to review new project ideas at 10:00 AM tomorrow.

[2190] Input: Reminder date, time and conditions.

[2191] Output: Reminder settings data.

[2192] Step 13:

[2193] Terminal

[2194] The device sends the reminder setting information to the server.

[2195] Input: Reminder setting data.

[2196] Output: Data sent to the server.

[2197] Step 14:

[2198] server

[2199] The server receives the reminder setting information and stores it in a database.

[2200] Input: Reminder setting data.

[2201] Output: Save to database.

[2202] Step 15:

[2203] server

[2204] When the conditions for a reminder are met, the server schedules the notification and prepares the reminder notification.

[2205] Input: Reminder date, time and conditions.

[2206] Output: Notification schedule created.

[2207] Step 16:

[2208] Terminal

[2209] When the set date, time or conditions are met, the device will send a reminder notification to the user.

[2210] Input: Notification Schedule.

[2211] Output: Notification to the user.

[2212] Related information suggestions

[2213] Step 17:

[2214] server

[2215] The server periodically analyzes the user's past search history and saved information.

[2216] Input: Your past search history and saved information.

[2217] Output: Analysis results.

[2218] Step 18:

[2219] server

[2220] It uses natural language processing technology and machine learning algorithms to infer relevant information and generate a list of suggestions.

[2221] Input: Analysis results.

[2222] Output: A list of suggestions.

[2223] Step 19:

[2224] server

[2225] The server transmits the generated proposal list to the terminal.

[2226] Input: Suggestion list.

[2227] Output: Data sent to the terminal.

[2228] Step 20:

[2229] Terminal

[2230] The terminal displays the proposal list received from the server to the user.

[2231] Input: Suggestion list.

[2232] Output: What is displayed to the user.

[2233] Incorporating an emotion engine

[2234] Step 21:

[2235] Terminal

[2236] The device collects user input information, behavior, voice tone, etc. and analyzes them using an emotion engine.

[2237] Input: User input, behavior, and tone of voice.

[2238] Output: Emotion data.

[2239] Step 22:

[2240] Terminal

[2241] The acquired emotion data is sent to the server in real time.

[2242] Input: Emotion data.

[2243] Output: Data sent to the server.

[2244] Step 23:

[2245] server

[2246] The server adjusts reminder notifications and search results based on the received emotion data.

[2247] Input: Emotion data.

[2248] Output: Tailored reminder notifications and search results.

[2249] Specific examples

[2250] Example 1:

[2251] Users search for "project ideas," and if the device detects that the user is feeling fatigued, it will suggest relaxing music and videos.

[2252] Example 2:

[2253] If a reminder notification is set when the user is tired, the server can use the emotion engine to delay the notification by 30 minutes.

[2254] (Application example 2)

[2255] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2256] Conventional information management systems are limited to efficiently managing and rediscovering user-entered information and metadata, but do not optimize search results, adjust reminder notifications, or suggest related information based on the user's emotional state. This lack of systems allows users to comfortably use information search and reminder functions. Even in physical stores, there are no product suggestions or notification functions that reflect the user's emotional state, hindering improvements to the customer experience.

[2257] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2258] In this invention, the server includes means for receiving information entered by a user, means for storing the received information in a database, means for generating metadata for the stored information, means for tagging and indexing the metadata, means for analyzing a user's search request using natural language processing technology, means for searching for information from the database based on the analyzed search request, means for presenting search results to the user, means for analyzing emotional data of the indicated information or image, and means for optimizing information based on the emotional data. This makes it possible to optimize information search and reminder notifications and suggest related information based on the user's emotional state.

[2259] "User" refers to any individual or corporation that uses this system.

[2260] "Input Information" refers to data in the form of text, music, images, video, or conversations provided by a user through a device or application.

[2261] "Means for receiving" refers to a set of hardware and software for taking input information into a computer system.

[2262] "Database" means a digital system for the structured storage and management of received information and associated metadata.

[2263] "Metadata" is data that indicates the nature and attributes of stored information, and includes the date and time of storage, tags, user ID, type of content, etc.

[2264] "Tagging and indexing methods" refers to techniques that attach identifiable labels to stored information and create data structures to facilitate efficient searching.

[2265] "Natural language processing technology" refers to a series of technologies that enable computers to understand, generate, and analyze natural human language.

[2266] A "search request" refers to the keywords or phrases a user enters to find specific information.

[2267] "Means for analyzing the emotional data of specified information or images" refers to technology that analyzes the data input by the user and recognizes the emotions contained within that data.

[2268] "Emotional data" refers to data that indicates a user's emotional state, such as information extracted from voice tone, facial expressions, and input content.

[2269] "Optimization measures" refer to methods for adjusting information search results and the timing of reminder notifications based on analyzed emotional data.

[2270] "Reminder" refers to a feature that notifies users in advance of specific events or tasks that they have set.

[2271] A "machine learning algorithm" refers to a set of computational techniques that derive rules from past data and make predictions and classifications.

[2272] "Related information" refers to additional information inferred based on a user's interests and needs, based on their past search history and saved information.

[2273] "Inferred related information list" refers to a set of related information generated by a machine learning algorithm.

[2274] "Notification" means a message or alert sent to inform the User of a reminder or relevant information.

[2275] "Suggestion" refers to useful information or recommended actions that the system provides to the user based on the user's input information and emotional data.

[2276] The present invention provides a system for efficiently managing information input by a user, recognizing the user's emotions, and optimizing search results, reminder notifications, and suggestions of related information. Specific embodiments are described below.

[2277] System configuration

[2278] Hardware

[2279] 1. Smart glasses: A device equipped with a voice recognition microphone, camera, and display that acquires and displays user input information.

[2280] 2. Server: A central system equipped with a database, natural language processing module, and emotion recognition engine, which analyzes, stores, and searches information.

[2281] 3. Surveillance camera: A device that recognizes customers' facial expressions and collects emotional data.

[2282] software

[2283] 1. Speech recognition API (such as Google Speech-to-Text): Software that converts user voice input into text data.

[2284] 2. Natural Language Processing API (such as Google NLP API): Software for analyzing text data and extracting important information.

[2285] 3. Emotion recognition API (such as Microsoft Azure Emotion API): Software for recognizing a user's emotional state from images and audio.

[2286] Operation overview

[2287] Retention of Information

[2288] The user uses the smart glasses to input information by voice. For example, they might say, "I want this red dress." This voice data is converted into text data by the smart glasses' voice recognition API and temporarily saved. This data is then sent to the server. The server stores the received information in a database, generates and tags metadata for the saved information, such as the date and time of saving, tags, and user ID, and creates an index.

[2289] Information Search and Retrieval

[2290] When a user searches for specific information through the smart glasses, they input the search query by voice. For example, a search request such as "black shoes" can be input. The smart glasses convert this voice into text and send the search request to the server. The server's natural language processing module analyzes the search request, searches for relevant information from the database, and sends the results to the smart glasses. The user can then view the search results on the display.

[2291] Reminders and notifications

[2292] Users can set reminders for later purchases. For example, they can set a reminder to "check out this red dress tomorrow at 10 AM." The smart glasses send this request to the server, which stores the setting information in a database. Reminders are scheduled based on the set date, time, and conditions, and notifications are sent to the smart glasses when the conditions are met. Furthermore, if the emotion recognition engine detects the user's fatigue, the timing of reminder notifications is adjusted.

[2293] Related information suggestions

[2294] The server periodically analyzes the user's past search history and saved information. It uses machine learning algorithms to predict relevant information and generate a list of suggestions. This list is presented to the user through smart glasses. For example, it may present highly relevant information such as "black shoes that go well with a red dress you previously purchased." The user can then choose their next action based on this list of suggestions.

[2295] Emotional data analysis and optimization

[2296] Smart glasses and surveillance cameras collect the user's facial expressions and tone of voice, and analyze the emotional data through an emotion recognition API. This data is sent to a server, which then optimizes information searches, reminder notifications, and related information suggestions based on the user's emotional state. For example, if the user is feeling stressed, product information that helps them relax will be displayed first.

[2297] Specific examples

[2298] 1. Voice input: The user inputs information by voice, such as "I like this blue jacket." The data is converted to text via a voice recognition API and sent to the server where it is saved.

[2299] 2. Emotional Data Analysis: As users walk around the store, the smart glasses' cameras and surveillance cameras collect facial expressions and voice tones. The emotion recognition API analyzes whether the user is in an excited state. This information is sent to the server, and product recommendations are optimized to match the user's excitement level.

[2300] Prompt Sentence Examples

[2301] "Save the voice data of the user saying, 'I want this red dress,' and set a reminder."

[2302] "Based on the search query 'black shoes', search for relevant information from the database and display it on the smart glasses."

[2303] "If the user's emotional state indicates stress, please suggest products that will help them relax."

[2304] Following these steps, we will build a system that provides information efficiently and based on emotions.

[2305] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2306] Step 1:

[2307] User voice input

[2308] The user puts on the smart glasses and uses voice input to describe the products they want or are interested in. For example, they might say, "I want this red dress." This input information is sent to the smart glasses' voice recognition API, which converts the voice data into text data.

[2309] Input: User voice input

[2310] Output: Speech-to-text data

[2311] Step 2:

[2312] Converting audio data to text

[2313] The device converts the user's voice into text data using a speech recognition API, such as Google Speech-to-Text, and the converted data is stored in temporary storage within the smart glasses.

[2314] Input: Audio data sent to the speech recognition API

[2315] Output: User input converted to text data

[2316] Step 3:

[2317] Sending text data to the server

[2318] The text data is sent from the temporary storage to the server, which stores it in a database and generates metadata such as the save date and time, tags, and user ID.

[2319] Input: Text data saved in temporary storage

[2320] Output: Text data and metadata stored on the server

[2321] Step 4:

[2322] Metadata generation and indexing

[2323] The server generates metadata and tags the received text data, and then creates an index to efficiently manage user information.

[2324] Input: Text data received by the server

[2325] Output: Generated metadata and indexing

[2326] Step 5:

[2327] Information search request

[2328] To search for specific information using the smart glasses, users input keywords by voice, such as a search request for "black shoes." The device converts this speech into text using a speech recognition API and sends it to the server.

[2329] Input: User's voice search request

[2330] Output: Text search request sent to the server

[2331] Step 6:

[2332] Parsing a search request

[2333] The server's natural language processing API analyzes the search request and searches the database for relevant information. As a result of the analysis, related product information is obtained.

[2334] Input: A textual search request

[2335] Output: Search results for related product information

[2336] Step 7:

[2337] Presenting search results

[2338] The server sends the search results to the smart glasses and displays them to the user, who can then view the relevant information on the smart glasses display.

[2339] Input: Server-generated search results

[2340] Output: Search results displayed on smart glasses

[2341] Step 8:

[2342] Set reminders

[2343] A user sets a reminder to check a specific product later, for example, "Check out this red dress tomorrow at 10 AM." The device sends the reminder setting information to the server.

[2344] Input: Reminder information set by the user

[2345] Output: Reminder information stored on the server

[2346] Step 9:

[2347] Scheduling reminders

[2348] The server schedules the reminder based on the reminder settings, and when the set time and conditions are met, a reminder notification is sent to the user.

[2349] Input: Reminder information stored on the server

[2350] Output: Scheduled reminder notifications

[2351] Step 10:

[2352] Emotional Data Analysis

[2353] Smart glasses and surveillance cameras collect the user's facial expressions and voice tone, and use emotion recognition APIs to analyze the emotional data, for example, to detect the user's excitement or fatigue, and send this information to a server.

[2354] Input: facial expression and tone of voice data

[2355] Output: Parsed emotion data

[2356] Step 11:

[2357] Emotion-Based Optimization

[2358] The server then uses the analyzed emotion data to adjust search results and the timing of reminder notifications. For example, if fatigue is detected, the server can delay reminder notifications.

[2359] Input: Parsed emotion data

[2360] Output: Optimized information search results and reminder notifications

[2361] Step 12:

[2362] Related information suggestions

[2363] The server uses machine learning algorithms to analyze the user's past search history and saved information to infer relevant information, and the generated suggestion list is sent to the smart glasses and presented to the user.

[2364] Input: User's past search history and saved information

[2365] Output: A generated list of related information suggestions

[2366] Through each of these steps, it becomes possible to provide optimal information and reminder functions based on the user's emotions.

[2367] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2368] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2369] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2370] [Fourth embodiment]

[2371] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2372] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2373] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2374] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2375] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2376] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2377] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2378] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2379] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2380] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2381] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2382] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2383] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2384] The present invention relates to a system that efficiently manages and rediscovers information stored by users (text, audio, images, videos, conversations with AI, etc.) and provides the information to users in an optimal way. Specific program processing and its implementation are described below.

[2385] Retention of Information

[2386] Subject: User

[2387] The user opens an application on the device and inputs the required information as text, or records voice, images, or videos. For example, if the user wants to input an idea for a new project as text, the user enters the idea in a dedicated input field.

[2388] Subject: Terminal

[2389] The device temporarily stores the input information and sends it to the server, generating metadata such as the date and time of storage, tags, user ID, and content type.

[2390] Subject: Server

[2391] The server stores the received information and metadata in a database, tags and indexes the stored information, and updates the metadata.

[2392] Information Search and Retrieval

[2393] Subject: User

[2394] Users can enter keywords related to the information they need in the application's search screen, for example, by entering "project ideas," to search for previously saved project ideas.

[2395] Subject: Terminal

[2396] The terminal acquires the input keywords and sends a search request to the server.

[2397] Subject: Server

[2398] The server receives the search request, analyzes the keywords using a natural language processing module, searches for related information from the database based on the analysis results, generates a list of results, and sends it to the terminal.

[2399] Subject: Terminal

[2400] The terminal displays the search result list received from the server to the user, who can then refer to the list to find the information they need.

[2401] Reminders and notifications

[2402] Subject: User

[2403] Users can set reminders for the information they save, for example, "Remind me to review new project ideas tomorrow at 10 AM."

[2404] Subject: Terminal

[2405] The device sends the reminder setting information to the server.

[2406] Subject: Server

[2407] The server receives the reminder setting information, stores it in a database, schedules notifications based on the reminder date and time and conditions, and notifies the user when the set date and time or conditions are met.

[2408] Subject: Terminal

[2409] When the set date, time, or conditions are met, the device will send a reminder notification to the user, who can then check the notification and take action as needed.

[2410] Related information suggestions

[2411] Subject: Server

[2412] The server periodically analyzes the user's past search history and saved information, and uses natural language processing technology and machine learning algorithms to predict related information and generate a list of suggestions.

[2413] Subject: Terminal

[2414] The terminal presents the user with a list of suggestions, which the user can refer to for quick access to relevant information.

[2415] As described above, the system of the present invention integrates multiple functions to enable users to efficiently manage and rediscover information. As a specific example, users can save ideas for new projects, retrieve information when needed by keyword search, and set reminders to receive notifications at appropriate times. This system allows users to manage important information without missing it and utilize it efficiently.

[2416] The processing flow will be explained below.

[2417] Retention of Information

[2418] Subject: User

[2419] Step 1:

[2420] The user opens the application using a terminal.

[2421] Step 2:

[2422] The user inputs information, either through text, voice, image, video, or conversation with the AI.

[2423] Subject: Terminal

[2424] Step 3:

[2425] The terminal stores the input information in temporary storage.

[2426] Step 4:

[2427] The device generates metadata including the date and time the information was saved, tags, user ID, and content type.

[2428] Step 5:

[2429] The device sends the input information and metadata to the server.

[2430] Subject: Server

[2431] Step 6:

[2432] The server stores the received information and metadata in a database.

[2433] Step 7:

[2434] The server tags and indexes the stored information.

[2435] Information Search and Retrieval

[2436] Subject: User

[2437] Step 1:

[2438] The user opens the application's search screen on their device and enters keywords.

[2439] Subject: Terminal

[2440] Step 2:

[2441] The terminal acquires the input keyword and sends a search request to the server.

[2442] Subject: Server

[2443] Step 3:

[2444] The server receives the search request and passes the keywords to the natural language processing module.

[2445] Step 4:

[2446] A natural language processing module analyzes keywords and extracts related tags and metadata.

[2447] Step 5:

[2448] The server generates a database query to retrieve the relevant information from the database.

[2449] Step 6:

[2450] The server lists the search results and sends them to the terminal.

[2451] Subject: Terminal

[2452] Step 7:

[2453] The terminal displays the search result list received from the server to the user.

[2454] Reminders and notifications

[2455] Subject: User

[2456] Step 1:

[2457] The user uses the device to select information for which they want to set a reminder.

[2458] Step 2:

[2459] The user sets the date, time, and conditions for the reminder.

[2460] Subject: Terminal

[2461] Step 3:

[2462] The device stores the reminder setting information in temporary storage and sends it to the server.

[2463] Subject: Server

[2464] Step 4:

[2465] The server receives the reminder setting information and stores it in a database.

[2466] Step 5:

[2467] The server generates a notification schedule based on the reminder date and time and conditions.

[2468] Step 6:

[2469] The server stores the notification schedule in temporary storage and registers it with the system clock.

[2470] Subject: Terminal

[2471] Step 7:

[2472] When the reminder date, time, or conditions are met, the device will notify the user.

[2473] Related information suggestions

[2474] Subject: Server

[2475] Step 1:

[2476] The server periodically analyzes the user's past search history and saved information.

[2477] Step 2:

[2478] A natural language processing module is used to calculate the relationships between stored information.

[2479] Step 3:

[2480] Machine learning algorithms are used to generate inferential models of relevant information.

[2481] Step 4:

[2482] The server generates a list of related information based on the inference model.

[2483] Subject: Terminal

[2484] Step 5:

[2485] The terminal obtains the related information list from the server and presents it to the user.

[2486] Example 1

[2487] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2488] Users generate and store a large amount of information on a daily basis, but it is difficult to efficiently manage that information and quickly find it again when needed. Users also desire a reminder function to ensure they don't forget important information, but manually setting this function is cumbersome. Furthermore, there is a lack of systems that automatically suggest related information based on a user's past behavior. An integrated information management system that can solve these issues is needed.

[2489] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2490] In this invention, the server includes means for receiving information entered by a user, means for saving the received information in a storage device, means for generating additional information for the saved information, means for tagging and indexing the additional information, means for analyzing a user's search request using natural language processing technology, means for searching for information from the storage device based on the analyzed search request, means for presenting search results to the user, means for receiving reminders set by the user, means for saving reminder information in a storage device, means for scheduling notifications based on the reminder date and time and conditions, means for notifying the user when the set date and time and conditions are met, means for analyzing the user's past search history and saved information, means for using a machine learning algorithm to infer related information, means for generating a list of inferred related information, and means for presenting the generated related information list to the user. This allows the user to efficiently manage saved information and quickly rediscover it when needed, and the reminders allow the user to manage important information without missing it and also provide suggestions of related information based on past behavior.

[2491] "Means for receiving" refers to the function that allows a terminal or server to receive information or requests sent by a user.

[2492] "Storage device" means a physical or virtual device for storing data, including hard drives, SSDs, cloud storage, etc.

[2493] The "means for saving" is a function for writing received information or data to a storage device.

[2494] "Additional information" refers to additional information related to the content of the saved information, such as metadata such as tags, save date and time, and user ID.

[2495] "Means for generating" is a function for automatically generating specific additional information based on certain information or data.

[2496] "Tagging" is a method of adding specific keywords or categories to information to make it easier to organize and search.

[2497] "Indexing" is the process of building a data structure so that information can be efficiently searched.

[2498] "Natural language processing technology" is a technology that enables computers to understand and analyze human language, and includes text analysis and language generation.

[2499] A "search request" is a request containing keywords or phrases entered by a user in search of specific information.

[2500] "Means of analysis" refers to the ability to understand the received search request and perform processing to find appropriate information.

[2501] The "searching means" is a function for searching for related information from a storage device based on the analyzed keywords.

[2502] "Means of presentation" refers to the function that allows a server or terminal to display search results and notifications to the user.

[2503] A "reminder" is a notification that a user sets for a specific date, time, or condition.

[2504] "Scheduling tools" are functions that allow you to plan notifications based on reminder dates and conditions.

[2505] "Means of notification" is a function for sending alerts or messages to users when a set date, time, or condition is met.

[2506] "Search history" refers to a record of searches a user has conducted in the past.

[2507] A "machine learning algorithm" is a computational method for analyzing data, discovering patterns, and using those patterns to predict future data.

[2508] A "related information list" is a list of information that is presumed to be useful to the user based on past search history and saved information.

[2509] The present invention relates to a system for efficiently managing user-generated information (text, audio, images, video, etc.) and rediscovering it when needed. Specific embodiments of the system are described below.

[2510] Retention of Information

[2511] 1. User input

[2512] Users open the application on their device and enter the required information: for text input, they write the information in a dedicated input field, for voice input, they press the microphone button to start recording, or for images and videos, they use the camera function.

[2513] Specific working example:

[2514] Enter your new project idea as "New website design" or, if you use voice recognition, say something like "Record a new project idea."

[2515] 2. Temporary storage on the device

[2516] The device saves the input information in temporary storage, generating metadata such as the save date and time, tags, user ID, and content type. The smartphone's internal memory is used as temporary storage.

[2517] 3. Sending from the device to the server

[2518] The device sends the stored information and metadata to the server, typically using an HTTP POST request to send the data.

[2519] 4. Server-based information storage and metadata updates

[2520] The server stores the received information and metadata in a database (e.g., MySQL), and a tagging engine runs to add appropriate tags to the text information.

[2521] Information Search and Retrieval

[2522] 1. User enters search keywords

[2523] Users enter keywords related to the information they need on the application's search screen.

[2524] Specific working example:

[2525] A user types in "project ideas" and presses the search button.

[2526] 2. Sending a search request via the device

[2527] The device receives the input keywords and sends a search request to the server. It generates an HTTP GET request and sends it to the server.

[2528] 3. Keyword analysis and search result generation by the server

[2529] The server uses a natural language processing library (e.g., NLTK or SpaCy) to analyze the keywords and search for relevant information in a storage device (database). The result list is generated in JSON format and sent to the device.

[2530] 4. Displaying search results on your device

[2531] The terminal displays the search result list received from the server to the user, displaying the search results in a list view so that the user can access the information they need.

[2532] Reminders and notifications

[2533] 1. User-defined reminder settings

[2534] Users can set reminders for saved information.

[2535] Specific working example:

[2536] The user enters the instruction "Set a reminder to check for new project ideas tomorrow at 10 AM."

[2537] 2. Send reminders via device

[2538] The device sends this reminder setting information to the server.

[2539] 3. Server-based reminder storage and scheduling

[2540] The server stores the reminder information in a database and sets up a notification scheduler (e.g., a Cron job or Quartz Scheduler).

[2541] 4. Device-based reminder notifications

[2542] When the set date, time, or conditions are met, a notification is triggered from the server and the device sends a reminder notification to the user.

[2543] Related information suggestions

[2544] 1. Server Generation of Proposal List

[2545] The server analyzes the user's past search history and saved information, and uses natural language processing techniques and machine learning algorithms (e.g., TensorFlow and PyTorch) to infer relevant information. A list of suggestions is generated periodically.

[2546] 2. Displaying a list of suggestions on the device

[2547] The terminal presents the user with a list of suggestions received from the server, and based on the information provided, the user can quickly access relevant information.

[2548] Examples and prompts

[2549] Examples:

[2550] 1. Storage of Information:

[2551] User: Texts a new project idea: "New website design."

[2552] Device: Sends information along with metadata to the server.

[2553] Server: Stores the information in a database and tags it.

[2554] 2. Information Search:

[2555] User: Search for "project ideas."

[2556] Device: Sends a search request to the server.

[2557] Server: Searches for relevant information and generates a list of results.

[2558] Device: Display search results to the user.

[2559] 3. Reminder:

[2560] User: Set a reminder to review new project ideas tomorrow at 10 AM.

[2561] Device: Sends reminder setting information to the server.

[2562] Server: Schedule reminders.

[2563] On your device: Reminds you at the set time.

[2564] Example prompt:

[2565] "Set a reminder to review new project ideas tomorrow at 10 AM."

[2566] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2567] Step 1:

[2568] The user opens an application on their device and inputs new information, for example, by typing a new project idea as text, "About a new website design," or by recording an audio, image, or video. It takes the user's data (text, audio, image, video) as input and generates data as output that is stored in temporary storage.

[2569] Step 2:

[2570] The device saves the information entered by the user in temporary storage. At this time, additional information such as the save date and time, tag, user ID, and content type is generated for the data saved in temporary storage. This allows information to be organized efficiently. Data from the user is received as input, and data with additional information is generated as output.

[2571] Step 3:

[2572] The device sends the stored information and additional information to the server. Specifically, it sends data using an HTTP POST request. It takes the data stored in temporary storage as input and generates an HTTP request that is sent to the server as output.

[2573] Step 4:

[2574] The server stores the received information and additional information in a database, performs appropriate tagging and indexing on the stored information, and updates the additional information. The server takes the data received by the server as input and generates information to be stored in the database as output.

[2575] Step 5:

[2576] The user enters keywords related to the information they need into the application's search screen. For example, they can enter "project ideas" to search for previously saved project ideas. The application takes search keywords as input and generates a search request as output.

[2577] Step 6:

[2578] The device acquires the search keywords and sends a search request to the server. Specifically, it generates an HTTP GET request and sends it to the server. It receives the search keywords from the user as input and generates and sends a search request as output.

[2579] Step 7:

[2580] The server receives the search request and analyzes the entered keywords using natural language processing techniques. Based on the analysis results, it searches for relevant information from the database. It takes the search request received by the server as input and generates a list of search results as output.

[2581] Step 8:

[2582] The terminal displays the search result list received from the server to the user. Specifically, the terminal displays the search results in a list view. The terminal receives the search results received from the server as input and generates the list of search results that is displayed to the user as output.

[2583] Step 9:

[2584] The user sets a reminder for the saved information, for example, "Remind me to review new project ideas tomorrow at 10 AM." The system takes the reminder setting information as input and generates a reminder setting request as output.

[2585] Step 10:

[2586] The device sends reminder setting information to the server. Specifically, it generates an HTTP POST request and sends it to the server. It receives reminder setting information from the user as input and generates and sends a reminder setting request as output.

[2587] Step 11:

[2588] The server receives the reminder setting information and stores it in a database. It schedules notifications based on the reminder date and time and conditions. It takes the reminder setting information received by the server as input and generates the scheduled notifications as output.

[2589] Step 12:

[2590] When the set date, time, or conditions are met, the device will notify the user with a reminder, for example, by displaying a reminder in the notification bar. It takes a reminder notification received from the server as input and generates a reminder notification to be displayed to the user as output.

[2591] Step 13:

[2592] The server analyzes the user's past search history and saved information, and uses natural language processing technology and machine learning algorithms to infer related information and generate a list of suggestions. It receives past search history and saved information as input and generates a list of suggestions as output.

[2593] Step 14:

[2594] The terminal presents the suggestion list received from the server to the user, allowing the user to quickly access relevant information. It takes the suggestion list received from the server as input and generates the suggestion list that is presented to the user as output.

[2595] (Application example 1)

[2596] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2597] In logistics centers, workers handle a wide variety of product information management tasks, and manual information entry and search is time-consuming, resulting in problems such as reduced accuracy and increased risk of errors. Furthermore, a lack of efficient reminder functions and suggestions for related information often leads to reduced work efficiency. There is a need for a system that can solve these problems, enable workers to quickly and accurately access the information they need, and improve work efficiency.

[2598] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2599] In this invention, the server includes means for receiving information entered by a user, means for storing the received information in a database, means for generating metadata for the stored information, means for tagging and indexing the metadata, means for analyzing a user's search request using natural language processing technology, means for searching for information from the database based on the analyzed search request, means for presenting search results to the user, means for acquiring information using a wearable device worn by a worker, and means for transmitting the acquired information to the server in real time. This makes it possible to use the wearable device to quickly and accurately manage information and improve work efficiency.

[2600] "User" refers to the entity that uses an information system.

[2601] A "wearable device" refers to an electronic device that can be worn by the user.

[2602] "Means for receiving information" refers to the mechanism by which user-entered information is incorporated into the data system.

[2603] "Database" refers to a system for systematically collecting, storing, and managing data.

[2604] "Metadata" refers to information about stored information, i.e., additional descriptions or attributes about that data.

[2605] "Tagging" refers to the process of classifying and organizing information or data by attaching labels or identification codes to it.

[2606] "Indexing" refers to the procedures and structures that enable quick search for necessary information from a large amount of data.

[2607] "Natural language processing technology" refers to technology for processing and analyzing human language (natural language) using a computer.

[2608] "Search Request" means a request or inquiry entered by a User to locate specific information.

[2609] "Real-time" refers to a situation in which information is acquired and processed instantly, with almost no time delay.

[2610] This invention is a system that uses wearable devices to manage and search information in order to improve work efficiency at logistics centers. Specifically, information is acquired through the wearable devices, sent to a server, and the necessary information is searched and displayed in real time.

[2611] Overall structure

[2612] The system consists of the following main components:

[2613] 1. Wearable devices: Devices worn by workers to input and display information, including smart glasses and smart watches.

[2614] 2. Server: The central system for storing, analyzing, searching, and notifying information.

[2615] 3. Database: A system connected to a server for storing saved information, metadata, search history, and reminder information.

[2616] Receiving and storing information

[2617] The wearable device captures information through voice input and barcode scanning by the worker. For example, a worker can scan the barcode of a new product and enter its details by voice. It can also record the product's condition using video recording. This information is sent in real time to a server, which stores it in a database. At the same time, metadata is generated, including the date and time of storage, tag, worker ID, and content type.

[2618] Information Search and Retrieval

[2619] Workers use the smart glasses to search for the information they need using voice commands. For example, they can say, "Tell me the inventory status of shelf number A3." The voice command is sent to the server, which analyzes it using natural language processing technology and searches for relevant information in the database. The results are displayed on the smart glasses, allowing workers to quickly obtain the information.

[2620] Reminders and notifications

[2621] Reminders are set by workers through voice input. For example, they can say, "Remind me tomorrow at 9:00 AM for the next loading / unloading operation." The reminder information is sent to the server and stored in a database. When the set date, time, or conditions are met, the server automatically sends a notification to the wearable device and displays the reminder. This helps workers avoid missing important tasks.

[2622] Related information suggestions

[2623] The server uses machine learning algorithms to analyze past search history and stored information. Based on this, it infers relevant information and makes suggestions to improve work efficiency. For example, if a frequently searched item is low in stock, it will make a replenishment suggestion. These suggestions are displayed to the worker through the smart glasses, allowing the worker to take action quickly.

[2624] Examples and prompts

[2625] For example, consider a scenario in which a worker saves details of a new item.

[2626] Example prompt sentence:

[2627] "Scan the barcode, then speak the product details."

[2628] Execution example:

[2629] Barcode scan: "1234567890123"

[2630] Speak: "This item is part of batch process B and is located on shelf number A3."

[2631] This information is immediately stored on the server and used for future searches and related information suggestions.

[2632] Specific examples of hardware and software used

[2633] Hardware: smart glasses (e.g., Google Glass), servers (high-performance servers in data centers), database servers (e.g., PostgreSQL)

[2634] Software: Natural language processing libraries (spaCy, NLTK), machine learning algorithms (TensorFlow, scikit-learn)

[2635] In this way, it is possible to efficiently manage and search information in the logistics center, provide reminder functions, and suggest related information.

[2636] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2637] Step 1:

[2638] Users input information using a wearable device. Specifically, a worker scans the barcode of a product with smart glasses and inputs detailed information about the product by voice. This input information includes the barcode value and voice text.

[2639] Input: Barcode scan data, voice input text

[2640] Output: Collected product information

[2641] Step 2:

[2642] The device sends the collected product information to a server in real time. Specifically, the smart glasses upload the barcode scan results and voice-input text to the server via the Internet, along with metadata such as the save date and time, tag, and worker ID.

[2643] Input: Collected product information (barcode data, voice text)

[2644] Output: Data sent to the server

[2645] Step 3:

[2646] The server stores the received information in a database, specifically by recording the information and metadata in a database system (e.g., PostgreSQL), including tagging and indexing.

[2647] Input: Data sent to the server

[2648] Output: Information stored in the database

[2649] Step 4:

[2650] Users can search for the information they need using voice commands. Specifically, a worker speaks into the smart glasses, saying, "Tell me the inventory status of shelf number A3." This voice is converted into text by the device and sent to the server.

[2651] Input: Voice command

[2652] Output: Text data of voice commands

[2653] Step 5:

[2654] The server receives the search request and analyzes it using natural language processing technology. Specifically, it uses a natural language processing library (e.g., spaCy) to analyze the voice command and extract the keywords necessary for the search.

[2655] Input: Text data of voice command

[2656] Output: Parsed keywords

[2657] Step 6:

[2658] The server searches for information from the database based on the analyzed keywords, generates a database query to retrieve the relevant information, and formats the query results into a list and sends it to the smart glasses.

[2659] Input: Parsed keyword

[2660] Output: Search result list

[2661] Step 7:

[2662] The terminal receives the search results from the server and displays them to the user. Specifically, the search results are visually displayed on the smart glasses display, allowing the worker to confirm the necessary information.

[2663] Input: Search result list

[2664] Output: Visually displayed search results

[2665] Step 8:

[2666] The user sets the reminder by voice. Specifically, the worker commands the smart glasses to "remind me for the next loading / unloading operation tomorrow at 9:00 AM." This voice data is converted into text by the device and sent to the server.

[2667] Input: Voice reminder

[2668] Output: Reminder text data

[2669] Step 9:

[2670] The server stores the reminder information in a database and sets the schedule. Specifically, the server records the reminder information appropriately in the database and sets the notification to be sent at the date and time specified in the schedule function.

[2671] Input: Reminder text data

[2672] Output: Reminder information and schedule settings stored in the database

[2673] Step 10:

[2674] The device will notify the user of the reminder at the set date and time by displaying a notification on the smart glasses and prompting the user to take the necessary action.

[2675] Input: Reminder information based on schedule settings

[2676] Output: Reminder notification

[2677] Step 11:

[2678] The server analyzes past search history and saved information and uses machine learning algorithms to predict related information. Specifically, it analyzes data using machine learning libraries (e.g., TensorFlow) and generates a list of related information.

[2679] Input: User's past search history and saved information

[2680] Output: Related information list

[2681] Step 12:

[2682] The terminal presents the generated related information list to the user, specifically by displaying the related information on smart glasses, helping the worker respond quickly.

[2683] Input: Related Information List

[2684] Output: Visually displayed relevant information

[2685] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2686] The present invention relates to a system that efficiently manages and rediscovers user-entered information, and further optimizes information search, reminder notifications, and related information suggestions by recognizing and utilizing user emotions. Specific embodiments of the present invention will be described below.

[2687] Retention of Information

[2688] Subject: User

[2689] The user opens the application on their device and inputs the required information using text, music, images, videos, or through conversation with the AI. For example, if they want to input an idea for a new project, they can enter their idea in a dedicated input field.

[2690] Subject: Terminal

[2691] The device saves the input information in temporary storage and sends it to the server, generating metadata about the save date and time, tags, user ID, and content type.

[2692] Subject: Server

[2693] The server stores the received information and metadata in a database, tags the stored information, creates an index, and updates the metadata.

[2694] Information Search and Retrieval

[2695] Subject: User

[2696] Users can use the application to open a search screen and enter keywords, such as "project ideas," to search for previously saved project ideas.

[2697] Subject: Terminal

[2698] The terminal acquires the input keywords and sends a search request to the server.

[2699] Subject: Server

[2700] The server receives the search request, analyzes the keywords using a natural language processing module, searches for related information from a database based on the analysis results, and sends the results list to the terminal.

[2701] Subject: Terminal

[2702] The terminal displays the search result list sent from the server to the user, who can then refer to the list and check the information they need.

[2703] Reminders and notifications

[2704] Subject: User

[2705] The user uses the application to select the information for which they want to set a reminder, and enter the date, time, and conditions for the reminder, such as "Remind me to review new project ideas tomorrow at 10 AM."

[2706] Subject: Terminal

[2707] The device sends the reminder setting information to the server.

[2708] Subject: Server

[2709] The server receives the reminder setting information, stores it in a database, schedules notifications based on the reminder date and time and conditions, and notifies the user when the set date and time or conditions are met.

[2710] Subject: Terminal

[2711] When the date, time, or conditions set for the reminder are met, the device will notify the user, allowing the user to check the notification and take action as necessary.

[2712] Related information suggestions

[2713] Subject: Server

[2714] The server periodically analyzes the user's past search history and saved information, and uses natural language processing technology and machine learning algorithms to predict related information and generate a list of suggestions.

[2715] Subject: Terminal

[2716] The terminal presents the user with a list of suggestions, which the user can refer to for quick access to relevant information.

[2717] Incorporating an emotion engine

[2718] Subject: Terminal

[2719] The device analyzes the user's input information, behavior, and voice tone, and uses an emotion engine to recognize the user's emotions. This emotional data is then sent to the server in real time.

[2720] Subject: Server

[2721] The server then uses the received emotional data to optimize information searches and reminder notifications based on the user's emotions. For example, if the user is feeling stressed, the server will change the relevant information suggestions and adjust the reminder notification time. It also records changes in emotions over time and uses this information for future pattern analysis.

[2722] Examples:

[2723] If a reminder is set for when the user is tired, the system will use its emotion engine to detect the user's fatigue and delay the reminder. Search results will also be optimized based on the user's emotional state, prioritizing more relaxing content and information the user prefers.

[2724] In this way, the system according to the present invention not only allows users to efficiently manage and rediscover information, but also utilizes an emotion engine to provide optimal information adapted to the user's emotions.

[2725] The processing flow will be explained below.

[2726] Retention of Information

[2727] Subject: User

[2728] Step 1:

[2729] The user opens the application using a terminal.

[2730] Step 2:

[2731] The user inputs information, either through text, voice, image, video, or conversation with the AI.

[2732] Subject: Terminal

[2733] Step 3:

[2734] The terminal stores the input information in temporary storage.

[2735] Step 4:

[2736] The device generates metadata including the date and time the information was saved, tags, user ID, and content type.

[2737] Step 5:

[2738] The device sends the input information and metadata to the server.

[2739] Subject: Server

[2740] Step 6:

[2741] The server stores the received information and metadata in a database.

[2742] Step 7:

[2743] The server tags and indexes the stored information.

[2744] Information Search and Retrieval

[2745] Subject: User

[2746] Step 1:

[2747] The user opens the application's search screen on their device and enters keywords.

[2748] Subject: Terminal

[2749] Step 2:

[2750] The terminal acquires the input keyword and sends a search request to the server.

[2751] Subject: Server

[2752] Step 3:

[2753] The server receives the search request and passes the keywords to the natural language processing module.

[2754] Step 4:

[2755] A natural language processing module analyzes keywords and extracts related tags and metadata.

[2756] Step 5:

[2757] The server generates a database query to retrieve the relevant information from the database.

[2758] Step 6:

[2759] The server lists the search results and sends them to the terminal.

[2760] Subject: Terminal

[2761] Step 7:

[2762] The terminal displays the search result list received from the server to the user.

[2763] Reminders and notifications

[2764] Subject: User

[2765] Step 1:

[2766] The user uses the device to select information for which they want to set a reminder.

[2767] Step 2:

[2768] The user sets the date, time, and conditions for the reminder.

[2769] Subject: Terminal

[2770] Step 3:

[2771] The device stores the reminder setting information in temporary storage and sends it to the server.

[2772] Subject: Server

[2773] Step 4:

[2774] The server receives the reminder setting information and stores it in a database.

[2775] Step 5:

[2776] The server generates a notification schedule based on the reminder date and time and conditions.

[2777] Step 6:

[2778] The server stores the notification schedule in temporary storage and registers it with the system clock.

[2779] Subject: Terminal

[2780] Step 7:

[2781] When the reminder date, time, or conditions are met, the device will notify the user.

[2782] Related information suggestions

[2783] Subject: Server

[2784] Step 1:

[2785] The server periodically analyzes the user's past search history and saved information.

[2786] Step 2:

[2787] A natural language processing module is used to calculate the relationships between stored information.

[2788] Step 3:

[2789] Machine learning algorithms are used to generate inferential models of relevant information.

[2790] Step 4:

[2791] The server generates a list of related information based on the inference model.

[2792] Subject: Terminal

[2793] Step 5:

[2794] The terminal obtains the related information list from the server and presents it to the user.

[2795] Incorporating an emotion engine

[2796] Subject: Terminal

[2797] Step 1:

[2798] The device analyzes the user's input information, behavior, voice tone, etc., and recognizes the user's emotions using an emotion engine.

[2799] Step 2:

[2800] Emotion data is sent to the server in real time.

[2801] Subject: Server

[2802] Step 3:

[2803] Based on the emotional data received by the server, information searches and reminder notifications are optimized according to the user's emotions.

[2804] Step 4:

[2805] Emotional changes are recorded over time and used for future pattern analysis.

[2806] Specific examples

[2807] Retention of Information

[2808] Subject: User

[2809] Step 1:

[2810] A user enters a new project idea as text into an application input field.

[2811] Subject: Terminal

[2812] Step 2:

[2813] The terminal stores the input text in temporary storage and sends it to the server.

[2814] Step 3:

[2815] The device generates metadata including the save date and time and the tag "Project Idea."

[2816] Subject: Server

[2817] Step 4:

[2818] The server stores the received text and metadata in a database.

[2819] Step 5:

[2820] The server tags and indexes the text.

[2821] Information Search and Retrieval

[2822] Subject: User

[2823] Step 1:

[2824] A user searches for the keyword "project ideas."

[2825] Subject: Terminal

[2826] Step 2:

[2827] The terminal sends a search request to the server.

[2828] Subject: Server

[2829] Step 3:

[2830] The server analyzes the search request and extracts relevant information from a database.

[2831] Subject: Terminal

[2832] Step 4:

[2833] The device displays the search results to the user, who then confirms the information.

[2834] Incorporating an emotion engine

[2835] Subject: User

[2836] Step 1:

[2837] Users speak topics related to their project ideas.

[2838] Subject: Terminal

[2839] Step 2:

[2840] The device analyzes the user's voice and recognizes emotions using an emotion...

Claims

1. a means for receiving user-entered information; means for storing the received information in a database; means for generating metadata for the stored information; a means of tagging and indexing metadata; A means for analyzing user search requests using natural language processing technology; means for retrieving information from the database based on the parsed search request; a means for presenting search results to the user; and A system including:

2. a means for receiving reminders set by the user; a means for storing reminder information in a database; A way to schedule notifications based on reminder dates and conditions, and A means to notify users when set dates, times, or conditions are met; The system of claim 1 further comprising:

3. A means of analyzing users' past search history and saved information, using machine learning algorithms to infer relevant information; means for generating an inferred related information list; means for presenting the generated related information list to a user; The system of claim 1 further comprising:

Images