System

Abstract

A system is provided.SOLUTION: A system comprising: means for providing information through an interactive user interface; means for analyzing a user request and extracting related keywords; means for searching for related information from information sources such as in-house documents, minutes, personal memos, and external library documents; means for providing the user with an interactive message generated based on the extracted information; and means for additionally learning documents and memos registered by the user and reflecting them in subsequent request responses.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The information we refer to in our daily work is diverse and enormous, including internal documents, meeting minutes, personal notes, and documents from external libraries. Therefore, following numerous links to find the information we need is extremely inefficient and time-consuming. Furthermore, existing note-taking tools often have security and cost constraints, making them difficult to use within a company. Therefore, to improve work efficiency, a system is needed that can efficiently manage this information and quickly retrieve it when needed. [Means for solving the problem]

[0005] The present invention is a system including a means for providing information through an interactive user interface, a means for analyzing a user's request and extracting relevant keywords, a means for searching for relevant information from information sources such as in-house documents, minutes, personal notes, and documents in an external library, a means for providing the user with an interactive message generated based on the extracted information, and a means for additionally learning documents and notes registered by the user and reflecting them in subsequent request responses. Furthermore, by including a means for generating easy-to-read memo and document templates based on the content of the user's request, a means for receiving and analyzing voice instructions from the user, and a means for learning the user's past usage history and improving the accuracy of information provision, it is possible to significantly improve business efficiency.

[0006] An "interactive user interface" is an interface that allows a user and a system to exchange information in an interactive format.

[0007] "Information provision" refers to the system collecting, analyzing, generating, and presenting information required by users to users.

[0008] "Request analysis" is the process of analyzing the content of requests and questions sent by users to the system using natural language processing technology.

[0009] "Keyword extraction" refers to identifying and extracting important words and phrases from the text obtained from request analysis.

[0010] A "source" is a location or file that contains the information you need, such as internal company documents, meeting minutes, personal notes, or documents from an external library.

[0011] "Related information search" is the process of searching for relevant information from information sources based on the extracted keywords.

[0012] "Interactive message generation" is the process of generating a message based on retrieved information in a form that is easy for the user to understand.

[0013] "Additional learning" is the process by which the system learns from newly registered documents and notes by the user and reflects them in future request responses.

[0014] "Template generation" refers to automatically creating templates for documents and notes based on the user's request.

[0015] "Analysis of voice instructions" is the process of analyzing voice input from a user, understanding its content, and taking appropriate action.

[0016] "Usage history learning" is the process of learning users' past system usage data and using that knowledge to improve the accuracy of information provided. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention relates to an AI system that integrates all notes and documents, providing information through an interactive user interface and streamlining user work. This system has the ability to analyze user requests and provide relevant information quickly and accurately. It can also learn additional information from documents and notes registered by the user, improving the accuracy of the information provided.

[0039] Providing information through an interactive UI

[0040] Example: A scene where you request the minutes of an internal meeting

[0041] 1. User:

[0042] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[0043] 2. Terminal:

[0044] The terminal receives this request and sends it to the server.

[0045] 3. Server:

[0046] The server analyzes the received request through a natural language processing (NLP) module, extracting keywords such as "last month," "management meeting," and "minutes."

[0047] 4. Server:

[0048] Based on the extracted keywords, the system searches for relevant meeting minutes documents from internal databases and external libraries. When relevant documents are found, the information is passed to the generation AI.

[0049] 5. Generation AI:

[0050] Based on the search results, the AI ​​generates a conversational message to be provided to the user, such as "The minutes of last month's management meeting can be viewed at the link below."

[0051] 6. Server:

[0052] Send the generated message to the terminal.

[0053] 7. Terminal:

[0054] Display received messages to the user.

[0055] 8. Users:

[0056] The user can view the minutes of the meeting in question by clicking the provided link.

[0057] Assistance in creating notes and documents

[0058] Example: Support for taking notes on presentation materials

[0059] 1. User:

[0060] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[0061] 2. Terminal:

[0062] The terminal sends this request to the server.

[0063] 3. Server:

[0064] The server receives the request and analyzes it using the NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[0065] 4. Server:

[0066] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[0067] 5. Generation AI:

[0068] Based on the search results, the AI ​​creates a memo template for presentation materials to be provided to the user. It creates specific memos such as, "At the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[0069] 6. Server:

[0070] The generated memo is sent to the terminal.

[0071] 7. Terminal:

[0072] Display received notes to the user.

[0073] 8. Users:

[0074] Users can review the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[0075] Additional learning of registered information

[0076] Example: Adding a new internal guideline

[0077] 1. User:

[0078] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[0079] 2. Terminal:

[0080] The terminal transmits the user's instructions to the server.

[0081] 3. Server:

[0082] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[0083] 4. User:

[0084] The user uploads a guideline document.

[0085] 5. Terminal:

[0086] The uploaded document is sent to the server.

[0087] 6. Server:

[0088] The server receives the uploaded document and passes it to the generative AI's learning module.

[0089] 7. Generation AI:

[0090] The generative AI analyzes new documents and adds their contents as learning data, so that the new guideline information will be reflected in future information provision.

[0091] 8. Server:

[0092] Sends a message to the terminal indicating that the update is complete.

[0093] 9. Terminal:

[0094] The terminal will display a message to the user indicating that the update is complete.

[0095] 10. Users:

[0096] The user can check the displayed messages and check the operation of the system as needed.

[0097] This enables the system to provide users with information that is optimized for them, greatly contributing to improved business efficiency.

[0098] The processing flow will be explained below.

[0099] Providing information through an interactive UI

[0100] Example: A scene where you request the minutes of an internal meeting

[0101] Processing Steps

[0102] Step 1:

[0103] User:

[0104] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[0105] Step 2:

[0106] Device:

[0107] The terminal receives the user's request and sends it to the server.

[0108] Step 3:

[0109] server:

[0110] The server receives the request and calls a natural language processing (NLP) module to parse the request, extracting keywords such as "last month," "management meeting," and "minutes."

[0111] Step 4:

[0112] server:

[0113] Based on the extracted keywords, search for relevant meeting minutes documents from internal databases or external libraries.

[0114] Step 5:

[0115] server:

[0116] The search results are passed to the generation AI.

[0117] Step 6:

[0118] Generation AI:

[0119] Based on the search results, the generative AI generates a conversational message to be delivered to the user, such as, "The minutes of last month's management meeting can be viewed at the link below."

[0120] Step 7:

[0121] server:

[0122] Send the generated message to the terminal.

[0123] Step 8:

[0124] Device:

[0125] Display received messages to the user.

[0126] Step 9:

[0127] User:

[0128] The user can view the minutes of the meeting by clicking the provided link.

[0129] Assistance in creating notes and documents

[0130] Example: Support for taking notes on presentation materials

[0131] Processing Steps

[0132] Step 1:

[0133] User:

[0134] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[0135] Step 2:

[0136] Device:

[0137] The terminal sends this request to the server.

[0138] Step 3:

[0139] server:

[0140] The server receives the request and analyzes it using the NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[0141] Step 4:

[0142] server:

[0143] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[0144] Step 5:

[0145] Generation AI:

[0146] Based on the search results, the AI ​​creates a memo template for presentation materials to be provided to the user. For example, it might create a memo such as, "At the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[0147] Step 6:

[0148] server:

[0149] The generated memo is sent to the terminal.

[0150] Step 7:

[0151] Device:

[0152] Display received notes to the user.

[0153] Step 8:

[0154] User:

[0155] The user can check the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[0156] Additional learning of registered information

[0157] Example: Adding a new internal guideline

[0158] Processing Steps

[0159] Step 1:

[0160] User:

[0161] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[0162] Step 2:

[0163] Device:

[0164] The terminal transmits the user's instructions to the server.

[0165] Step 3:

[0166] server:

[0167] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[0168] Step 4:

[0169] User:

[0170] The user uploads a guideline document.

[0171] Step 5:

[0172] Device:

[0173] Sends the uploaded document to the server.

[0174] Step 6:

[0175] server:

[0176] The server receives the uploaded document and passes it to the generative AI's learning module.

[0177] Step 7:

[0178] Generation AI:

[0179] The generative AI analyzes new documents and adds their contents as learning data, so that new guideline information is reflected in future information provision.

[0180] Step 8:

[0181] server:

[0182] A message indicating that learning is complete is sent to the device.

[0183] Step 9:

[0184] Device:

[0185] The terminal displays a message to the user indicating that learning is complete.

[0186] Step 10:

[0187] User:

[0188] The user checks the displayed message and checks the operation of the system as necessary.

[0189] Example 1

[0190] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0191] Existing information provision systems often have delays in responding to user requests or provide inaccurate information. Furthermore, even when a user adds new information, it takes time for that information to be reflected in the system. Furthermore, few systems can also respond to voice commands, which limits user convenience.

[0192] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0193] In this invention, the server includes a means for analyzing the request through a natural language processing module and extracting relevant keywords, a means for searching for relevant information from an internal database or external information source based on the extracted keywords, and a means for passing the search results to a generative AI model to generate a dialogue-style message. This enables a quick and accurate response to a user's request. Furthermore, even if new information is added, the generative AI model can instantly learn from it and reflect it in subsequent request responses. Furthermore, support for voice instructions improves user convenience.

[0194] "User" means an individual or organization that uses the system.

[0195] A "terminal" is a device used by a user to send requests or receive information.

[0196] A "request" is a request for information made by a user to the system.

[0197] A "natural language processing module" is a software component that analyzes the text of a request and extracts meaning and keywords.

[0198] "Keywords" are important words or phrases extracted from a user's request.

[0199] An "internal database" is a database within the system where information such as user documents and notes is stored.

[0200] "External sources" refers to information obtained from libraries or databases outside the system.

[0201] A "generative AI model" is an artificial intelligence model that automatically generates messages to be provided to users based on search results and learning data.

[0202] "Interactive messaging" refers to messaging that is in a natural format for interacting with a user.

[0203] A "new document or note" is new information that a user adds to the system.

[0204] "Learning" is the process by which a generative AI model takes in new information and uses it to improve its responses in future scenarios.

[0205] "Voice instructions" are instructions sent by the user to the system through voice.

[0206] This invention relates to an AI system that integrates all memos and documents, providing information through an interactive user interface and streamlining user work. This system has the ability to analyze user requests and provide relevant information quickly and accurately. It can also learn from documents and memos registered by users to improve the accuracy of the information provided.

[0207] The following hardware and software are used to implement the present invention.

[0208] 1. Hardware

[0209] Device: The device through which the user enters the request (e.g., PC, smartphone, tablet).

[0210] Server: A server that analyzes requests, searches for information, generates messages, etc.

[0211] 2. Software

[0212] Natural Language Processing Module: Software for analyzing requests. Specifically, it uses the Google Cloud Natural Language API.

[0213] Internal database: A database that stores information such as user notes and documents. Specifically, MongoDB is used.

[0214] Generative AI model: An artificial intelligence model that generates messages to be provided to users. Specifically, we use OpenAI's GPT-4.

[0215] Examples of concrete examples and prompts

[0216] 1. Requesting minutes of an internal meeting

[0217] User: The user sends a request from their terminal saying, "I want to see the minutes of last month's management meeting."

[0218] Terminal: The terminal receives the user's request and sends it to the server.

[0219] Server: The server parses the request using the Google Cloud Natural Language API and extracts keywords such as "last month," "management meeting," and "minutes."

[0220] The server searches for relevant meeting minutes from MongoDB based on keywords and passes them to the generative AI model.

[0221] Based on the search results, a generative AI model (OpenAI GPT-4) generates a message such as, "The minutes of last month's management meeting can be viewed at the link below."

[0222] Server: Sends the generated message to the device.

[0223] Terminal: Displays received messages to the user.

[0224] User: View the minutes via the provided link.

[0225] Example prompt sentence:

[0226] "I'd like to see the minutes of last month's management meeting."

[0227] 2. Support for taking notes on presentation materials

[0228] User: The user makes a request from their device saying, "I want to create notes for the presentation materials for the next customer briefing."

[0229] Device: The device sends a request to the server.

[0230] Server: The server analyzes the request using the Google Cloud Natural Language API and extracts keywords such as "customer briefing," "presentation materials," and "memo."

[0231] The server searches MongoDB for relevant past documents and notes based on the extracted keywords and passes them to the generative AI model.

[0232] A generative AI model (OpenAI GPT-4) generates notes such as, "At your next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[0233] Server: Sends the generated memo to the device.

[0234] Terminal: Displays received notes to the user.

[0235] User: Review, edit and supplement notes, and create final presentation materials.

[0236] Example prompt sentence:

[0237] "I want to make notes for the presentation materials for the next client briefing."

[0238] 3. Addition of new internal guidelines

[0239] User: The user submits a request to add a new internal guideline document to the AI ​​and have it learn it.

[0240] Device: The device sends a request to the server.

[0241] Server: The server receives the request and sends a message to the user prompting them to upload a guideline document.

[0242] User: Upload a guideline document.

[0243] Terminal: Sends the document to the server.

[0244] Server: The server receives the document and passes it to the learning module of the generative AI model, which then learns from it and applies it to future responses.

[0245] Server: Sends a message to the device indicating that the update is complete.

[0246] Terminal: Display a message to the user that the update is complete.

[0247] User: Check the displayed messages and check the system operation.

[0248] Example prompt sentence:

[0249] "We want to add new internal guideline documents to the AI ​​and have it learn them."

[0250] As described above, the present invention is a system that combines a natural language processing module, an internal database, and a generative AI model to provide efficient information provision, enabling quick and accurate responses to user requests and improving business efficiency.

[0251] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0252] Step 1:

[0253] The user sends a request from the terminal.

[0254] Example: A user types or voice requests, "I'd like to see the minutes of last month's management meeting."

[0255] Input: User's text or voice request.

[0256] Output: Received request on terminal.

[0257] Step 2:

[0258] The terminal receives the user's request and sends it to the server.

[0259] Example: A device sends a request to the server saying, "I want to see the minutes of last month's management meeting."

[0260] Input: The user request.

[0261] Output: The request sent to the server.

[0262] Step 3:

[0263] The server analyzes the received request through a natural language processing module.

[0264] Example: The server passes the request to the Google Cloud Natural Language API for parsing.

[0265] Input: The user request received from the device.

[0266] Output: Keywords such as "last month", "management meeting", and "minutes".

[0267] Step 4:

[0268] The server searches for relevant information from internal databases and external information sources based on the extracted keywords.

[0269] Example: The server searches MongoDB for documents matching "last month", "management meeting", and "minutes".

[0270] Input: Extracted keywords.

[0271] Output: A document of the relevant minutes.

[0272] Step 5:

[0273] The server passes the search results to a generative AI model, which generates conversational messages.

[0274] Example: The server uses the search results to generate a message using a generative AI model (OpenAI GPT-4).

[0275] Input: Relevant minutes document.

[0276] Output: The message "The minutes of last month's management meeting can be found at the link below."

[0277] Step 6:

[0278] The server sends the generated message to the terminal.

[0279] Example: The server sends a generated message to the user terminal.

[0280] Input: Message from the generative AI model.

[0281] Output: Messages sent to the terminal.

[0282] Step 7:

[0283] The terminal displays the received message to the user.

[0284] Example: The device displays the message "The minutes of last month's management meeting can be viewed at the link below."

[0285] Input: The message sent by the server.

[0286] Output: The message displayed to the user.

[0287] Step 8:

[0288] The user checks the minutes of the meeting from the provided link.

[0289] Example: A user clicks on a displayed link to view the meeting minutes.

[0290] Input: The link displayed on the terminal.

[0291] Output: The minutes to be viewed.

[0292] (Application example 1)

[0293] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0294] Conventional information provision systems using interactive user interfaces have had difficulty in providing prompt and appropriate information in response to user requests. In particular, in virtual and physical stores, the lack of a function to provide product information desired by users in real time limited the user experience. Therefore, a system that allows users to quickly and accurately obtain the information they desire is needed.

[0295] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0296] In this invention, the server includes means for providing information through an interactive user interface, means for analyzing a user request and extracting related keywords, means for searching for related information from internal and external information sources, means for providing the user with an interactive message generated based on the search results, means for additionally learning documents and notes registered by the user and reflecting them in subsequent request responses, and means for providing product information in a virtual physical store. This allows users to quickly and accurately obtain product information in real time even in a virtual physical store.

[0297] An "interactive user interface" refers to a mechanism that allows two-way communication between the user and the system.

[0298] "Means for extracting relevant keywords" refers to a method for identifying and extracting important words and phrases from the user's request.

[0299] "Internal sources" refers to information stored within a company or organization, such as databases, documents, and notes.

[0300] "External sources" refer to information that exists outside the organization, such as external libraries or databases on the Internet.

[0301] "Conversational messages" refer to text that provides information to users in the form of natural conversation.

[0302] "Virtual brick-and-mortar store" refers to a virtual shopping environment operated on the Internet.

[0303] "Means for additional learning from registered documents and notes" refers to a method by which the system incorporates new information provided by the user and uses this information to improve the accuracy of subsequent answers.

[0304] "Means for providing product information" refers to a method for displaying data such as product details, reviews, and use examples to a user in response to a user's request.

[0305] "Generative AI" refers to artificial intelligence technology that generates new data and sentences based on learned data.

[0306] To implement this invention, the following system program is required: This system is composed of the following elements: a server, a terminal, and a user.

[0307] System Program

[0308] The program mainly consists of the following measures:

[0309] 1. Means of providing information through an interactive user interface:

[0310] The server provides an interactive user interface that accepts user requests. This interface supports voice and text input. The user interface is designed to allow users to enter the desired information in a natural, interactive manner.

[0311] 2. A method to analyze user requests and extract relevant keywords:

[0312] The server receives requests from users and analyzes them using a natural language processing (NLP) module. The analysis process involves extracting important keywords from the input sentence. The NLP module uses the Transformers library from Huggingface.

[0313] 3. Means of retrieving relevant information from internal and external sources:

[0314] The server searches for relevant information based on the extracted keywords from an internal database, which includes documents, memos, and minutes from within the company or organization, and from external sources, such as databases and libraries on the Internet.

[0315] 4. A means of providing users with interactive messages generated based on search results:

[0316] After searching for relevant information, the results are sent to the generation AI, which then generates a conversational message to be provided to the user based on the search results. The generation AI uses OpenAI's text-davinci-003 model.

[0317] 5. A way to learn additional documents and notes registered by the user and reflect them in subsequent request responses:

[0318] The server passes documents and notes uploaded by users to the machine learning module for additional learning, allowing the system to improve accuracy in subsequent information provision.

[0319] 6. Means of providing product information in virtual and physical stores:

[0320] As a means of providing users with the product information they require in real time in a virtual physical store, the server displays product details, reviews, usage examples, etc. based on user requests.

[0321] Hardware and Software

[0322] Server: The server acts as the central processing unit and performs all data calculations and processing.

[0323] Terminal: A device that provides a user interface, including smartphones and head-mounted displays (HMDs).

[0324] Software: Huggingface's Transformers library, OpenAI's text-davinci-003 model.

[0325] Specific examples

[0326] User Request: "I want to see the latest smartphones"

[0327] System behavior:

[0328] 1. The user makes a voice or text request on their smartphone or HMD saying, "I want to see the latest smartphones."

[0329] 2. The request is sent to the server and parsed by the NLP module.

[0330] 3. Keywords such as "latest" and "smartphone" are extracted.

[0331] 4. The server retrieves relevant product information from internal databases and external sources.

[0332] 5. Generative AI creates conversational messages based on search results.

[0333] 6. A message such as "Here are the latest smartphones: Model X, Model Y" will appear on the user's device.

[0334] Example of input prompt for generative AI model:

[0335] User request: "I want to see the latest smartphones."

[0336] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0337] Step 1:

[0338] The user sends a request from the device by voice or text, for example, "I want to see the latest smartphones." This request is entered into the device and sent to the server.

[0339] Step 2:

[0340] The server receives requests from the device and analyzes them using a natural language processing (NLP) module. Specifically, it extracts important keywords such as "latest" and "smartphone" from the request text. This process uses the Transformers library from Huggingface. The input is the user's request text, and the output is a list of extracted keywords.

[0341] Step 3:

[0342] The server searches for relevant information from internal databases and external information sources based on the extracted keywords. The internal database stores company documents, memos, minutes, etc. The external information sources include databases and libraries on the Internet. The input is a list of keywords, and the output is the search results data.

[0343] Step 4:

[0344] The server passes the search results to a generation AI, which creates an interactive message to be provided to the user. This generation AI uses OpenAI's text-davinci-003 model. Based on the search results, the generation AI generates an appropriate message to provide information to the user. The input is the search result data, and the output is an interactive message.

[0345] Step 5:

[0346] The server sends the generated interactive message to the terminal, which displays this message to the user. For example, a message like "Here are the latest smartphones: Model X, Model Y" is displayed. The input is the generated message, and the output is the message displayed to the user.

[0347] Step 6:

[0348] New documents and notes registered by users are uploaded to the server via their devices. The server passes these to the machine learning module as additional learning data, which improves the accuracy of subsequent information provision. The input is the uploaded documents and notes, and the output is updated learning data.

[0349] Step 7:

[0350] When a user requests product information in a virtual store, the server provides product details, reviews, usage examples, etc. in real time. Information can be easily delivered to the user via the terminal. The input is the user's request, and the output is the provision of product information.

[0351] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0352] This invention relates to an AI system that integrates all notes and documents, providing information through an interactive user interface and streamlining user work. This system not only analyzes user requests and quickly provides necessary information from relevant information sources, but also recognizes the user's emotions and responds accordingly. Specifically, by additionally learning from documents and notes that the user has looked up and using an emotion engine, the system provides optimal information and responses for the user.

[0353] Providing information through an interactive UI

[0354] Example: A scene where you request the minutes of an internal meeting

[0355] 1. User:

[0356] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[0357] 2. Terminal:

[0358] The terminal sends a request to the server.

[0359] 3. Server:

[0360] The server receives the request and parses it through a natural language processing (NLP) module, extracting keywords such as "last month," "management meeting," and "minutes."

[0361] 4. Server:

[0362] Based on the extracted keywords, search for relevant meeting minutes documents from internal databases or external libraries.

[0363] 5. Server:

[0364] The search results are passed to the generation AI.

[0365] 6. Generation AI:

[0366] Based on the search results, the generative AI generates a conversational message to be delivered to the user, such as, "The minutes of last month's management meeting can be viewed at the link below."

[0367] 7. Server:

[0368] Send the generated message to the terminal.

[0369] 8. Terminal:

[0370] Display received messages to the user.

[0371] 9. Users:

[0372] The user can view the minutes of the meeting by clicking the provided link.

[0373] Assistance in creating notes and documents

[0374] Example: Support for taking notes on presentation materials

[0375] 1. User:

[0376] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[0377] 2. Terminal:

[0378] The terminal sends a request to the server.

[0379] 3. Server:

[0380] The server receives the request and analyzes it using an NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[0381] 4. Server:

[0382] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[0383] 5. Generation AI:

[0384] Based on the search results, the generative AI creates a memo template for presentation materials to be provided to the user. For example, it creates a memo that states, "In the next customer briefing, it is important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[0385] 6. Server:

[0386] The generated memo is sent to the terminal.

[0387] 7. Terminal:

[0388] Display received notes to the user.

[0389] 8. Users:

[0390] The user can check the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[0391] Additional learning of registered information

[0392] Example: Adding a new internal guideline

[0393] 1. User:

[0394] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[0395] 2. Terminal:

[0396] The terminal sends the instruction to the server.

[0397] 3. Server:

[0398] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[0399] 4. User:

[0400] The user uploads a guideline document.

[0401] 5. Terminal:

[0402] Sends the uploaded document to the server.

[0403] 6. Server:

[0404] The server receives the uploaded document and passes it to the generative AI's learning module.

[0405] 7. Generation AI:

[0406] The generative AI analyzes new documents and adds their contents as learning data, so that new guideline information is reflected in future information provision.

[0407] 8. Server:

[0408] Sends a message to the terminal indicating that the update is complete.

[0409] 9. Terminal:

[0410] The terminal will display a message to the user indicating that the update is complete.

[0411] 10. Users:

[0412] The user checks the displayed message and checks the operation of the system as necessary.

[0413] Utilizing the Emotion Engine

[0414] Example: Responding based on user emotions

[0415] 1. User:

[0416] The user sends an emotional request via text or voice from the device.

[0417] 2. Terminal:

[0418] The terminal receives the request and sends it to the server.

[0419] 3. Server:

[0420] The server receives the request and uses an emotion engine to analyze the user's emotional state.

[0421] 4. Server:

[0422] Based on the emotional information analyzed by the emotion engine, the generative AI adjusts the tone and content of the response. For example, if the user is feeling stressed, it will generate a more friendly response that emphasizes solutions.

[0423] 5. Generation AI:

[0424] Generative AI generates messages based on emotions, such as, "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with."

[0425] 6. Server:

[0426] Send the generated message to the terminal.

[0427] 7. Terminal:

[0428] Display received messages to the user.

[0429] 8. Users:

[0430] The user sees the friendly message and makes further requests if necessary.

[0431] This system enables the provision of information that takes into consideration the user's feelings, contributing to improved business efficiency and user satisfaction.

[0432] The processing flow will be explained below.

[0433] Providing information through an interactive UI

[0434] Example: A scene where you request the minutes of an internal meeting

[0435] Processing Steps

[0436] Step 1:

[0437] User:

[0438] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[0439] Step 2:

[0440] Device:

[0441] The terminal receives the user's request and sends it to the server.

[0442] Step 3:

[0443] server:

[0444] The server receives the request and calls a natural language processing (NLP) module to analyze the request, extracting keywords such as "last month," "management meeting," and "minutes."

[0445] Step 4:

[0446] server:

[0447] Based on the extracted keywords, search for relevant meeting minutes documents from internal databases or external libraries.

[0448] Step 5:

[0449] server:

[0450] The search results are passed to the generation AI.

[0451] Step 6:

[0452] Generation AI:

[0453] Based on the search results, the generative AI generates a conversational message to be delivered to the user, such as, "The minutes of last month's management meeting can be viewed at the link below."

[0454] Step 7:

[0455] server:

[0456] Send the generated message to the terminal.

[0457] Step 8:

[0458] Device:

[0459] Display received messages to the user.

[0460] Step 9:

[0461] User:

[0462] The user can view the minutes of the meeting by clicking the provided link.

[0463] Assistance in creating notes and documents

[0464] Example: Support for taking notes on presentation materials

[0465] Processing Steps

[0466] Step 1:

[0467] User:

[0468] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[0469] Step 2:

[0470] Device:

[0471] The terminal sends a request to the server.

[0472] Step 3:

[0473] server:

[0474] The server receives the request and analyzes it using the NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[0475] Step 4:

[0476] server:

[0477] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[0478] Step 5:

[0479] Generation AI:

[0480] Based on the search results, the generative AI creates a memo template for presentation materials to be provided to the user. For example, it creates a memo that states, "In the next customer briefing, it is important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[0481] Step 6:

[0482] server:

[0483] The generated memo is sent to the terminal.

[0484] Step 7:

[0485] Device:

[0486] Display received notes to the user.

[0487] Step 8:

[0488] User:

[0489] The user can check the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[0490] Additional learning of registered information

[0491] Example: Adding a new internal guideline

[0492] Processing Steps

[0493] Step 1:

[0494] User:

[0495] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[0496] Step 2:

[0497] Device:

[0498] The terminal sends the instruction to the server.

[0499] Step 3:

[0500] server:

[0501] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[0502] Step 4:

[0503] User:

[0504] The user uploads a guideline document.

[0505] Step 5:

[0506] Device:

[0507] Sends the uploaded document to the server.

[0508] Step 6:

[0509] server:

[0510] The server receives the uploaded document and passes it to the generative AI's learning module.

[0511] Step 7:

[0512] Generation AI:

[0513] The generative AI analyzes new documents and adds their contents as learning data, so that new guideline information is reflected in future information provision.

[0514] Step 8:

[0515] server:

[0516] Sends a message to the terminal indicating that the update is complete.

[0517] Step 9:

[0518] Device:

[0519] The terminal will display a message to the user indicating that the update is complete.

[0520] Step 10:

[0521] User:

[0522] The user checks the displayed message and checks the operation of the system as necessary.

[0523] Utilizing the Emotion Engine

[0524] Example: Responding based on user emotions

[0525] Processing Steps

[0526] Step 1:

[0527] User:

[0528] The user sends an emotional request via text or voice from the device.

[0529] Step 2:

[0530] Device:

[0531] The terminal sends a request to the server.

[0532] Step 3:

[0533] server:

[0534] The server receives the request and uses an emotion engine to analyze the user's emotional state.

[0535] Step 4:

[0536] server:

[0537] Based on the emotional information analyzed by the emotion engine, the generative AI adjusts the tone and content of the response. For example, if the user is feeling stressed, it will generate a more friendly response that emphasizes solutions.

[0538] Step 5:

[0539] Generation AI:

[0540] Generative AI generates messages based on emotions, such as, "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with."

[0541] Step 6:

[0542] server:

[0543] Send the generated message to the terminal.

[0544] Step 7:

[0545] Device:

[0546] Display received messages to the user.

[0547] Step 8:

[0548] User:

[0549] The user sees the friendly message and makes further requests if necessary.

[0550] This system enables the provision of information that takes into consideration the user's feelings, contributing to improved business efficiency and user satisfaction.

[0551] Example 2

[0552] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0553] In today's business environment, users are required to quickly and accurately obtain the information they need from multiple sources. It is also important to reduce fatigue and stress by providing appropriate responses based on the user's emotions and circumstances, thereby improving work efficiency and user satisfaction. However, conventional systems have not been able to adequately meet these needs, resulting in reduced user convenience and cumbersome work.

[0554] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for providing information through an interactive user interface, means for analyzing a user request and extracting related keywords, means for searching for related information from an internal database or an external information source, means for providing the user with an interactive message generated based on the extracted information, means for additionally learning documents and notes registered by the user and reflecting them in subsequent request responses, and means for analyzing the user's emotions and generating a response according to the emotions. This allows the user to not only quickly and accurately obtain the information they need, but also receive a response according to their emotions, thereby reducing the complexity of their work and improving work efficiency and user satisfaction.

[0555] An "interactive user interface" refers to a user interface that allows two-way communication through user operations.

[0556] "Means for analyzing user requests and extracting relevant keywords" refers to a function that analyzes requests sent by users using technologies such as natural language processing and extracts important language elements.

[0557] "Means of searching for relevant information from internal databases and external information sources" refers to the function of searching for relevant information from databases inside and outside the system based on extracted keywords.

[0558] The "means for providing a generated interactive message to the user" refers to a function for providing a response to the user in the form of a message generated based on the search results.

[0559] "Means for additional learning of documents and notes registered by the user and reflecting them in subsequent request responses" refers to the function by which the system learns newly registered information by the user and uses it in subsequent responses.

[0560] "Means for analyzing the user's emotions and generating a response that corresponds to those emotions" refers to a function that determines the emotions from the user's request and generates a response that is appropriate to those emotions.

[0561] This invention relates to an AI system that integrates memos and documents, providing information through an interactive user interface to improve the efficiency of users' work. This system has the ability to analyze user requests and quickly provide necessary information from relevant information sources, as well as recognize the user's emotions and respond accordingly.

[0562] Providing information through an interactive UI

[0563] Hardware and Software Use

[0564] User device: personal computer, smartphone, tablet, etc. Uses a microphone for voice input and a keyboard for text input.

[0565] Server: A server equipped with a high-performance processor and a large amount of memory. RDBMS or NoSQL databases are used.

[0566] NLP Modules: Software modules for natural language processing, such as SpaCy and NLTK.

[0567] Generative AI: High-performance generative AI models, such as OpenAI's GPT-3.

[0568] Specific examples

[0569] 1. User: The user makes a request from their terminal saying, "I want to see the minutes of last month's management meeting."

[0570] 2. Terminal: The terminal sends a request to the server.

[0571] 3. Server: The server analyzes the request using a natural language processing module and extracts keywords such as "last month," "management meeting," and "minutes."

[0572] 4. Server: The server uses these keywords to search for relevant information from internal databases and external sources, using a search engine such as Elasticsearch.

[0573] 5. Server: Passes search results to the generation AI.

[0574] 6. Generative AI: The Generative AI generates a conversational message saying, "The minutes of last month's management meeting can be viewed at the link below."

[0575] 7. Server: Sends the generated message to the terminal.

[0576] 8. Terminal: displays received messages to the user.

[0577] Prompt Sentence Examples

[0578] "Show me the minutes of last month's management meeting."

[0579] Assistance in creating notes and documents

[0580] Specific examples

[0581] 1. User: The user requests from the terminal, "I want to create notes for the presentation materials for the next customer briefing."

[0582] 2. Terminal: The terminal sends a request to the server.

[0583] 3. Server: The server analyzes the request using a natural language processing module and extracts keywords such as "customer briefing," "presentation materials," and "memo."

[0584] 4. Server: The server searches the user's past documents and notes from its internal database.

[0585] 5. Generative AI: Generative AI creates memo templates for presentation materials based on search results. For example, it generates a memo saying, "In the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[0586] 6. Server: Sends the generated memo to the device.

[0587] 7. Terminal: Displays the received memo to the user.

[0588] 8. User: The user checks the displayed notes and makes edits or additions as necessary.

[0589] Prompt Sentence Examples

[0590] "Please prepare notes for the next client briefing."

[0591] Additional learning of registered information

[0592] Specific examples

[0593] 1. User: The user issues a command from their device saying, "I want to add new internal guidelines to the AI ​​and have it learn them."

[0594] 2. Terminal: The terminal sends instructions to the server.

[0595] 3. Server: The server receives the instructions and sends a message to the user prompting them to upload their guidelines.

[0596] 4. User: The user uploads the guideline document.

[0597] 5. Terminal: Sends the uploaded document to the server.

[0598] 6. Server: The server receives the document and passes it to the generative AI's learning module.

[0599] 7. Generative AI: Generative AI analyzes new documents and adds their contents to the learning data. This allows new guideline information to be reflected in future information provision.

[0600] 8. Server: Sends a message to the device indicating that the update is complete.

[0601] 9. Terminal: The terminal displays a message to the user that the update is complete.

[0602] Prompt Sentence Examples

[0603] "Learn these new guidelines and incorporate them into your next search."

[0604] Utilizing the Emotion Engine

[0605] Specific examples

[0606] 1. User: The user sends an emotional request from the device, for example, with an elevated tone of voice or emotional text.

[0607] 2. Terminal: The terminal sends a request to the server.

[0608] 3. Server: The server receives the request and uses an emotion engine to analyze the user's emotional state, for example determining that the user is tired.

[0609] 4. Server: Based on the emotional information analyzed by the emotion engine, the generative AI adjusts the tone and content of the response, for example, generating a friendly message that emphasizes solutions.

[0610] 5. Generative AI: Generative AI generates messages based on emotions, such as "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with."

[0611] 6. Server: Sends the generated message to the terminal.

[0612] 7. Terminal: Displays received messages to the user.

[0613] Prompt Sentence Examples

[0614] "I'm very busy, but could you please give me a quick summary of the recent meeting?"

[0615] With these functions, the system streamlines users' work and provides necessary information quickly and accurately. It also responds with consideration for the user's feelings, improving work efficiency and user satisfaction.

[0616] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0617] Providing information through an interactive UI

[0618] Example: A scene where you request the minutes of an internal meeting

[0619] Step 1:

[0620] User: The user sends a request from their device saying, "I want to see the minutes of last month's management meeting." This operation is performed by the user either typing text or making a request by voice. Input is done using the device's keyboard and microphone.

[0621] Step 2:

[0622] Terminal: The terminal sends the received request to the server using an HTTP POST request or WebSocket. The data is encoded in JSON format.

[0623] Step 3:

[0624] Server: The server receives the request and analyzes it using a natural language processing (NLP) module. This process extracts keywords such as "last month," "management meeting," and "minutes" from the request text. The input is the request data in JSON format, and the output is a list of keywords. The specific software used is SpaCy and NLTK.

[0625] Step 4:

[0626] Server: The server searches for relevant information from internal databases and external sources based on the extracted keywords. This search is performed using a search engine such as Elasticsearch. The input is a list of keywords, and the output is a set of related information.

[0627] Step 5:

[0628] Server: The server passes the search results to the generative AI. The input is a set of related information, and the output is a response message generated by the generative AI. Specifically, OpenAI's GPT-3 is used.

[0629] Step 6:

[0630] Generative AI: Based on the search results, the generative AI generates a conversational message such as, "The minutes of last month's management meeting can be viewed at the link below." The input is a set of relevant information, and the output is a response message.

[0631] Step 7:

[0632] Server: The server sends the generated message to the terminal. HTTP or WebSocket is used for transmission. The input is the response message, and the output is the transmission completion status.

[0633] Step 8:

[0634] Terminal: A terminal displays received messages to the user, such as pop-up notifications or chat windows on the UI. Input is a response message, and output is displayed in the user interface.

[0635] Step 9:

[0636] User: The user is presented with a link to view the minutes. When the user clicks on the link, their browser opens a new tab or window with the relevant document.

[0637] Assistance in creating notes and documents

[0638] Example: Support for taking notes on presentation materials

[0639] Step 1:

[0640] User: The user requests, "I want to create notes for the presentation materials for the next client briefing." This can be done by typing text or by making a voice request.

[0641] Step 2:

[0642] Terminal: The terminal sends the received request to the server using an HTTP POST request or WebSocket. The data is encoded in JSON format.

[0643] Step 3:

[0644] Server: The server receives the request and analyzes it using the NLP module. Keywords such as "customer briefing," "presentation materials," and "memo" are extracted through the analysis. The input is the request data in JSON format, and the output is a list of keywords.

[0645] Step 4:

[0646] Server: The server searches the user's past documents and notes from an internal database based on the extracted keywords. The search engine used is Elasticsearch. The input is a list of keywords, and the output is a set of related documents and notes.

[0647] Step 5:

[0648] Generative AI: Generative AI creates memo templates for presentation materials based on search results. For example, it generates memos such as, "At the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session." The input is related materials and memos, and the output is a memo template.

[0649] Step 6:

[0650] Server: Sends the generated memo to the device. HTTP POST request or WebSocket is used for sending. The input is the memo template, and the output is the sending completion status.

[0651] Step 7:

[0652] Terminal: Displays received memos to the user. Specifically, it displays them in the memo field on the UI or in notifications. The input is the memo template, and the output is displayed in the user interface.

[0653] Step 8:

[0654] User: The user checks the displayed notes, edits and supplements them as necessary, and creates the final presentation materials.

[0655] Additional learning of registered information

[0656] Example: Adding a new internal guideline

[0657] Step 1:

[0658] User: The user issues a command from the device saying, "I want to add new internal guidelines to the AI ​​and have it learn them." This operation is performed by the user entering text or giving voice instructions.

[0659] Step 2:

[0660] Terminal: The terminal sends instructions to the server using HTTP POST requests or WebSockets. The data is encoded in JSON format.

[0661] Step 3:

[0662] Server: The server receives the instructions and sends a message to the user prompting them to upload the guideline document. The input is the instructions, and the output is the upload prompt message.

[0663] Step 4:

[0664] User: The user uploads a guideline document. The operation is performed by selecting the file through a file selection dialog. The input is the document file, and the output is the uploaded file.

[0665] Step 5:

[0666] Terminal: Sends the uploaded document to the server using an HTTP POST request. The input is the uploaded file, and the output is the status of the successful submission.

[0667] Step 6:

[0668] Server: The server passes the received document to the generative AI learning module. The input is the document file, and the output is the learning data.

[0669] Step 7:

[0670] Generative AI: The generative AI analyzes new documents and adds their contents as training data. This allows new guideline information to be reflected in subsequent information provision. The input is training data, and the output is updated training data.

[0671] Step 8:

[0672] Server: Sends an update completion message to the terminal. The input is the updated training data, and the output is the update completion message.

[0673] Step 9:

[0674] Terminal: The terminal displays a message to the user that the update is complete. The input is the message, and the output is what is displayed in the user interface.

[0675] Step 10:

[0676] User: The user checks the displayed messages and checks the system operation if necessary.

[0677] Utilizing the Emotion Engine

[0678] Example: Responding based on user emotions

[0679] Step 1:

[0680] User: The user sends an emotional request from the device, for example, by typing in a voice with a high tone or by typing in a text that expresses emotion. The input is done using the device's keyboard and microphone.

[0681] Step 2:

[0682] Terminal: The terminal receives the request and sends it to the server using an HTTP POST request or WebSocket. The data is encoded in JSON format.

[0683] Step 3:

[0684] Server: The server receives the request and uses an emotion engine to analyze the user's emotional state. Emotion analysis uses techniques to extract emotions from text and speech. The input is the request data in JSON format, and the output is the emotional status.

[0685] Step 4:

[0686] Server: Based on the emotional information analyzed by the emotion engine, the generation AI adjusts the tone and content of the response. For example, if the user is tired, it generates a message in a friendly tone that emphasizes solutions. The input is the emotional status, and the output is the response message.

[0687] Step 5:

[0688] Generative AI: Generative AI generates messages based on emotions. For example, it generates a message like, "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with." The input is the emotional status, and the output is a response message.

[0689] Step 6:

[0690] Server: Sends the generated message to the terminal. HTTP POST request or WebSocket is used for sending. The input is the response message, and the output is the status of sending completion.

[0691] Step 7:

[0692] Terminal: A terminal displays received messages to the user, such as in a popup notification or a chat window. Input is a response message, and output is displayed in the user interface.

[0693] Step 8:

[0694] User: The user reviews the friendly message and makes a follow-up request if necessary, by retyping the request through the chat interface.

[0695] The specific operations, data processing, and data calculations in each processing step have been described above.

[0696] (Application example 2)

[0697] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0698] Conventional information provision systems often found it difficult to properly analyze user requests and quickly provide relevant information. Furthermore, they were unable to respond in a way that took the user's emotions into account, limiting the improvement of user satisfaction. Furthermore, the system's ability to effectively add and learn new information registered by the user and reflect it in subsequent requests was incomplete, creating a need for efficient customer service in physical stores.

[0699] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing information through an interactive operation screen, means for analyzing a user request and extracting related keywords, means for searching for related information from internal data or an external database, means for providing the user with an interactive message generated based on the extracted information, means for additionally learning information registered by the user and reflecting the information in subsequent requests and responses, and means for analyzing the user's emotional state and adjusting the tone and content of the response based on the learning. This enables a quick and accurate response to a user request, and further enables a response that takes the user's emotions into consideration, thereby improving user satisfaction.

[0700] An "interactive operation screen" is an interface that allows a user to interact with the system through the screen and receive information.

[0701] "Means of providing information" refers to the function that the all-memo / document integrated AI system uses to provide the necessary information to users.

[0702] The "means for analyzing a user request and extracting relevant keywords" refers to a method and device for analyzing the content of a user request and identifying important keywords.

[0703] "Means for searching for relevant information from internal data and external databases" is a function for searching for appropriate information from sources inside and outside the system based on identified keywords.

[0704] The "means for providing a user with a message in an interactive format" is a function for providing information to the user in an easy-to-understand message format based on the extracted and searched information.

[0705] The "means for additionally learning information registered by the user and reflecting it in subsequent request responses" is a function for learning information newly added by the user and using it to provide future information.

[0706] "Means for analyzing a user's emotional state and adjusting the tone and content of a response based thereon" refers to a method and apparatus for analyzing a user's emotions and providing a response with the optimal tone and content according to those emotions.

[0707] An "interactive user interface" is an interface that allows a user to operate and obtain information through natural interaction with a system.

[0708] This invention relates to an AI system that integrates all notes and documents, providing information through an interactive user interface and streamlining user work. The system mainly includes the following hardware and software:

[0709] Required Hardware and Software

[0710] Smart glasses: Devices worn by store staff that capture customers' questions and facial expressions in real time.

[0711] Server: A computer server that processes data, performs generative AI, and performs natural language processing (NLP).

[0712] Python: A programming language.

[0713] Transformers Library: A library for NLP tasks.

[0714] cv2 (OpenCV): Computer vision library.

[0715] requests: A library for sending HTTP requests.

[0716] System Operation

[0717] 1. Analyzing user requests and providing information

[0718] The server receives requests from users (store staff) through the smart glasses. The requests are input in voice or text format. The server then analyzes the requests using natural language processing (NLP) and extracts relevant keywords. It then searches for relevant information in internal and external databases and uses generative AI to generate conversational messages. This allows users to easily answer customer questions.

[0719] For example, if a customer asks, "What color variations of this product are there?", the server retrieves the product information and generates the answer, "This product comes in red, blue, and green." An example prompt sentence is as follows:

[0720] Question: "What color variations are available for this item?"

[0721] Context: "Product ABC is available in red, blue, and green colors. Details for each color are as follows..."

[0722] 2. Sentiment Analysis and Response Adjustment

[0723] The server captures the customer's facial expressions and tone of voice through the smart glasses and inputs them into an emotion analysis engine. Based on the analysis results, the customer's emotional state is identified. The generative AI then adjusts the tone and content of the response according to the identified emotional state. This allows the response to take the customer's emotions into consideration.

[0724] For example, if a customer says, "I'm a little disappointed with this product," the sentiment analysis engine will identify the "disappointed" emotion, and the generative AI will generate a response such as, "I'm sorry you're disappointed. Would you like us to find another product that meets your expectations?"

[0725] Examples of concrete examples and prompts

[0726] Below is a concrete example in which a user responds to a customer question using a prompt generated by the server.

[0727] Question: "What material is this item made of?"

[0728] Context: "Product XYZ is made from high-quality cotton and polyester. Its specific composition is as follows..."

[0729] In this way, the present invention enables store staff to respond to customer questions quickly and accurately, and also enables responses that are in line with the customer's feelings, which is expected to improve user satisfaction.

[0730] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0731] Step 1:

[0732] The server receives user requests from the smart glasses. The user inputs the request in voice or text format, and the device sends it to the server. The input data is the user's question or instruction.

[0733] Step 2:

[0734] After receiving the request, the server analyzes it using a natural language processing (NLP) module. Specifically, it breaks down the sentences and extracts keywords. As a result of this analysis, important keywords are output.

[0735] Step 3:

[0736] The server searches for relevant information from internal and external databases based on the extracted keywords, such as product information, instructions, and related notes. The server searches this data through API requests and obtains search results.

[0737] Step 4:

[0738] The server generates a dialogue-style message using a generative AI model based on the search results. The generative AI model uses the context information and extracted keywords as input. The generative AI model generates a response message to be provided to the user and returns the output to the server.

[0739] Step 5:

[0740] The server sends the generated message to the terminal, which receives the message and provides it to the user by display or voice. The content of the message is a specific answer to the user's question.

[0741] Step 6:

[0742] The server captures the emotional state of the user or customer through the smart glasses. The emotion analysis engine analyzes the data based on voice input and facial expression capture data. The input data is audio and video capture data.

[0743] Step 7:

[0744] The server uses a generative AI model to adjust the tone and content of the response based on the results of the emotion analysis engine. Specifically, it generates a response appropriate for when the user is feeling stressed. The input is the analysis result, and the output is the adjusted response message.

[0745] Step 8:

[0746] The server sends the tailored response message to the terminal, which displays the received message and provides a specific response to the user, who then responds to the customer based on the response.

[0747] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0748] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0749] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0750] [Second embodiment]

[0751] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0752] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0753] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0754] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0755] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0756] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0757] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0758] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0759] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0760] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0761] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0762] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0763] This invention relates to an AI system that integrates all notes and documents, providing information through an interactive user interface and streamlining user work. This system has the ability to analyze user requests and provide relevant information quickly and accurately. It can also learn additional information from documents and notes registered by the user, improving the accuracy of the information provided.

[0764] Providing information through an interactive UI

[0765] Example: A scene where you request the minutes of an internal meeting

[0766] 1. User:

[0767] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[0768] 2. Terminal:

[0769] The terminal receives this request and sends it to the server.

[0770] 3. Server:

[0771] The server analyzes the received request through a natural language processing (NLP) module, extracting keywords such as "last month," "management meeting," and "minutes."

[0772] 4. Server:

[0773] Based on the extracted keywords, the system searches for relevant meeting minutes documents from internal databases and external libraries. When relevant documents are found, the information is passed to the generation AI.

[0774] 5. Generation AI:

[0775] Based on the search results, the AI ​​generates a conversational message to be provided to the user, such as "The minutes of last month's management meeting can be viewed at the link below."

[0776] 6. Server:

[0777] Send the generated message to the terminal.

[0778] 7. Terminal:

[0779] Display received messages to the user.

[0780] 8. Users:

[0781] The user can view the minutes of the meeting in question by clicking the provided link.

[0782] Assistance in creating notes and documents

[0783] Example: Support for taking notes on presentation materials

[0784] 1. User:

[0785] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[0786] 2. Terminal:

[0787] The terminal sends this request to the server.

[0788] 3. Server:

[0789] The server receives the request and analyzes it using the NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[0790] 4. Server:

[0791] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[0792] 5. Generation AI:

[0793] Based on the search results, the AI ​​creates a memo template for presentation materials to be provided to the user. It creates specific memos such as, "At the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[0794] 6. Server:

[0795] The generated memo is sent to the terminal.

[0796] 7. Terminal:

[0797] Display received notes to the user.

[0798] 8. Users:

[0799] Users can review the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[0800] Additional learning of registered information

[0801] Example: Adding a new internal guideline

[0802] 1. User:

[0803] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[0804] 2. Terminal:

[0805] The terminal transmits the user's instructions to the server.

[0806] 3. Server:

[0807] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[0808] 4. User:

[0809] The user uploads a guideline document.

[0810] 5. Terminal:

[0811] The uploaded document is sent to the server.

[0812] 6. Server:

[0813] The server receives the uploaded document and passes it to the generative AI's learning module.

[0814] 7. Generation AI:

[0815] The generative AI analyzes new documents and adds their contents as learning data, so that the new guideline information will be reflected in future information provision.

[0816] 8. Server:

[0817] Sends a message to the terminal indicating that the update is complete.

[0818] 9. Terminal:

[0819] The terminal will display a message to the user indicating that the update is complete.

[0820] 10. Users:

[0821] The user can check the displayed messages and check the operation of the system as needed.

[0822] This enables the system to provide users with information that is optimized for them, greatly contributing to improved business efficiency.

[0823] The processing flow will be explained below.

[0824] Providing information through an interactive UI

[0825] Example: A scene where you request the minutes of an internal meeting

[0826] Processing Steps

[0827] Step 1:

[0828] User:

[0829] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[0830] Step 2:

[0831] Device:

[0832] The terminal receives the user's request and sends it to the server.

[0833] Step 3:

[0834] server:

[0835] The server receives the request and calls a natural language processing (NLP) module to parse the request, extracting keywords such as "last month," "management meeting," and "minutes."

[0836] Step 4:

[0837] server:

[0838] Based on the extracted keywords, search for relevant meeting minutes documents from internal databases or external libraries.

[0839] Step 5:

[0840] server:

[0841] The search results are passed to the generation AI.

[0842] Step 6:

[0843] Generation AI:

[0844] Based on the search results, the generative AI generates a conversational message to be delivered to the user, such as, "The minutes of last month's management meeting can be viewed at the link below."

[0845] Step 7:

[0846] server:

[0847] Send the generated message to the terminal.

[0848] Step 8:

[0849] Device:

[0850] Display received messages to the user.

[0851] Step 9:

[0852] User:

[0853] The user can view the minutes of the meeting by clicking the provided link.

[0854] Assistance in creating notes and documents

[0855] Example: Support for taking notes on presentation materials

[0856] Processing Steps

[0857] Step 1:

[0858] User:

[0859] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[0860] Step 2:

[0861] Device:

[0862] The terminal sends this request to the server.

[0863] Step 3:

[0864] server:

[0865] The server receives the request and analyzes it using the NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[0866] Step 4:

[0867] server:

[0868] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[0869] Step 5:

[0870] Generation AI:

[0871] Based on the search results, the AI ​​creates a memo template for presentation materials to be provided to the user. For example, it might create a memo such as, "At the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[0872] Step 6:

[0873] server:

[0874] The generated memo is sent to the terminal.

[0875] Step 7:

[0876] Device:

[0877] Display received notes to the user.

[0878] Step 8:

[0879] User:

[0880] The user can check the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[0881] Additional learning of registered information

[0882] Example: Adding a new internal guideline

[0883] Processing Steps

[0884] Step 1:

[0885] User:

[0886] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[0887] Step 2:

[0888] Device:

[0889] The terminal transmits the user's instructions to the server.

[0890] Step 3:

[0891] server:

[0892] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[0893] Step 4:

[0894] User:

[0895] The user uploads a guideline document.

[0896] Step 5:

[0897] Device:

[0898] Sends the uploaded document to the server.

[0899] Step 6:

[0900] server:

[0901] The server receives the uploaded document and passes it to the generative AI's learning module.

[0902] Step 7:

[0903] Generation AI:

[0904] The generative AI analyzes new documents and adds their contents as learning data, so that new guideline information is reflected in future information provision.

[0905] Step 8:

[0906] server:

[0907] A message indicating that learning is complete is sent to the device.

[0908] Step 9:

[0909] Device:

[0910] The terminal displays a message to the user indicating that learning is complete.

[0911] Step 10:

[0912] User:

[0913] The user checks the displayed message and checks the operation of the system as necessary.

[0914] Example 1

[0915] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0916] Existing information provision systems often have delays in responding to user requests or provide inaccurate information. Furthermore, even when a user adds new information, it takes time for that information to be reflected in the system. Furthermore, few systems can also respond to voice commands, which limits user convenience.

[0917] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0918] In this invention, the server includes a means for analyzing the request through a natural language processing module and extracting relevant keywords, a means for searching for relevant information from an internal database or external information source based on the extracted keywords, and a means for passing the search results to a generative AI model to generate a dialogue-style message. This enables a quick and accurate response to a user's request. Furthermore, even if new information is added, the generative AI model can instantly learn from it and reflect it in subsequent request responses. Furthermore, support for voice instructions improves user convenience.

[0919] "User" means an individual or organization that uses the system.

[0920] A "terminal" is a device used by a user to send requests or receive information.

[0921] A "request" is a request for information made by a user to the system.

[0922] A "natural language processing module" is a software component that analyzes the text of a request and extracts meaning and keywords.

[0923] "Keywords" are important words or phrases extracted from a user's request.

[0924] An "internal database" is a database within the system where information such as user documents and notes is stored.

[0925] "External sources" refers to information obtained from libraries or databases outside the system.

[0926] A "generative AI model" is an artificial intelligence model that automatically generates messages to be provided to users based on search results and learning data.

[0927] "Interactive messaging" refers to messaging that is in a natural format for interacting with a user.

[0928] A "new document or note" is new information that a user adds to the system.

[0929] "Learning" is the process by which a generative AI model takes in new information and uses it to improve its responses in future scenarios.

[0930] "Voice instructions" are instructions sent by the user to the system through voice.

[0931] This invention relates to an AI system that integrates all memos and documents, providing information through an interactive user interface and streamlining user work. This system has the ability to analyze user requests and provide relevant information quickly and accurately. It can also learn from documents and memos registered by users to improve the accuracy of the information provided.

[0932] The following hardware and software are used to implement the present invention.

[0933] 1. Hardware

[0934] Device: The device through which the user enters the request (e.g., PC, smartphone, tablet).

[0935] Server: A server that analyzes requests, searches for information, generates messages, etc.

[0936] 2. Software

[0937] Natural Language Processing Module: Software for analyzing requests. Specifically, it uses the Google Cloud Natural Language API.

[0938] Internal database: A database that stores information such as user notes and documents. Specifically, MongoDB is used.

[0939] Generative AI model: An artificial intelligence model that generates messages to be provided to users. Specifically, we use OpenAI's GPT-4.

[0940] Examples of concrete examples and prompts

[0941] 1. Requesting minutes of an internal meeting

[0942] User: The user sends a request from their terminal saying, "I want to see the minutes of last month's management meeting."

[0943] Terminal: The terminal receives the user's request and sends it to the server.

[0944] Server: The server parses the request using the Google Cloud Natural Language API and extracts keywords such as "last month," "management meeting," and "minutes."

[0945] The server searches for relevant meeting minutes from MongoDB based on keywords and passes them to the generative AI model.

[0946] Based on the search results, a generative AI model (OpenAI GPT-4) generates a message such as, "The minutes of last month's management meeting can be viewed at the link below."

[0947] Server: Sends the generated message to the device.

[0948] Terminal: Displays received messages to the user.

[0949] User: View the minutes via the provided link.

[0950] Example prompt sentence:

[0951] "I'd like to see the minutes of last month's management meeting."

[0952] 2. Support for taking notes on presentation materials

[0953] User: The user makes a request from their device saying, "I want to create notes for the presentation materials for the next customer briefing."

[0954] Device: The device sends a request to the server.

[0955] Server: The server analyzes the request using the Google Cloud Natural Language API and extracts keywords such as "customer briefing," "presentation materials," and "memo."

[0956] The server searches MongoDB for relevant past documents and notes based on the extracted keywords and passes them to the generative AI model.

[0957] A generative AI model (OpenAI GPT-4) generates notes such as, "At your next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[0958] Server: Sends the generated memo to the device.

[0959] Terminal: Displays received notes to the user.

[0960] User: Review, edit and supplement notes, and create final presentation materials.

[0961] Example prompt sentence:

[0962] "I want to make notes for the presentation materials for the next client briefing."

[0963] 3. Addition of new internal guidelines

[0964] User: The user submits a request to add a new internal guideline document to the AI ​​and have it learn it.

[0965] Device: The device sends a request to the server.

[0966] Server: The server receives the request and sends a message to the user prompting them to upload a guideline document.

[0967] User: Upload a guideline document.

[0968] Terminal: Sends the document to the server.

[0969] Server: The server receives the document and passes it to the learning module of the generative AI model, which then learns from it and applies it to future responses.

[0970] Server: Sends a message to the device indicating that the update is complete.

[0971] Terminal: Display a message to the user that the update is complete.

[0972] User: Check the displayed messages and check the system operation.

[0973] Example prompt sentence:

[0974] "We want to add new internal guideline documents to the AI ​​and have it learn them."

[0975] As described above, the present invention is a system that combines a natural language processing module, an internal database, and a generative AI model to provide efficient information provision, enabling quick and accurate responses to user requests and improving business efficiency.

[0976] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0977] Step 1:

[0978] The user sends a request from the terminal.

[0979] Example: A user types or voice requests, "I'd like to see the minutes of last month's management meeting."

[0980] Input: User's text or voice request.

[0981] Output: Received request on terminal.

[0982] Step 2:

[0983] The terminal receives the user's request and sends it to the server.

[0984] Example: A device sends a request to the server saying, "I want to see the minutes of last month's management meeting."

[0985] Input: The user request.

[0986] Output: The request sent to the server.

[0987] Step 3:

[0988] The server analyzes the received request through a natural language processing module.

[0989] Example: The server passes the request to the Google Cloud Natural Language API for parsing.

[0990] Input: The user request received from the device.

[0991] Output: Keywords such as "last month", "management meeting", and "minutes".

[0992] Step 4:

[0993] The server searches for relevant information from internal databases and external information sources based on the extracted keywords.

[0994] Example: The server searches MongoDB for documents matching "last month", "management meeting", and "minutes".

[0995] Input: Extracted keywords.

[0996] Output: A document of the relevant minutes.

[0997] Step 5:

[0998] The server passes the search results to a generative AI model, which generates conversational messages.

[0999] Example: The server uses the search results to generate a message using a generative AI model (OpenAI GPT-4).

[1000] Input: Relevant minutes document.

[1001] Output: The message "The minutes of last month's management meeting can be found at the link below."

[1002] Step 6:

[1003] The server sends the generated message to the terminal.

[1004] Example: The server sends a generated message to the user terminal.

[1005] Input: Message from the generative AI model.

[1006] Output: Messages sent to the terminal.

[1007] Step 7:

[1008] The terminal displays the received message to the user.

[1009] Example: The device displays the message "The minutes of last month's management meeting can be viewed at the link below."

[1010] Input: The message sent by the server.

[1011] Output: The message displayed to the user.

[1012] Step 8:

[1013] The user checks the minutes of the meeting from the provided link.

[1014] Example: A user clicks on a displayed link to view the meeting minutes.

[1015] Input: The link displayed on the terminal.

[1016] Output: The minutes to be viewed.

[1017] (Application example 1)

[1018] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1019] Conventional information provision systems using interactive user interfaces have had difficulty in providing prompt and appropriate information in response to user requests. In particular, in virtual and physical stores, the lack of a function to provide product information desired by users in real time limited the user experience. Therefore, a system that allows users to quickly and accurately obtain the information they desire is needed.

[1020] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1021] In this invention, the server includes means for providing information through an interactive user interface, means for analyzing a user request and extracting related keywords, means for searching for related information from internal and external information sources, means for providing the user with an interactive message generated based on the search results, means for additionally learning documents and notes registered by the user and reflecting them in subsequent request responses, and means for providing product information in a virtual physical store. This allows users to quickly and accurately obtain product information in real time even in a virtual physical store.

[1022] An "interactive user interface" refers to a mechanism that allows two-way communication between the user and the system.

[1023] "Means for extracting relevant keywords" refers to a method for identifying and extracting important words and phrases from the user's request.

[1024] "Internal sources" refers to information stored within a company or organization, such as databases, documents, and notes.

[1025] "External sources" refer to information that exists outside the organization, such as external libraries or databases on the Internet.

[1026] "Conversational messages" refer to text that provides information to users in the form of natural conversation.

[1027] "Virtual brick-and-mortar store" refers to a virtual shopping environment operated on the Internet.

[1028] "Means for additional learning from registered documents and notes" refers to a method by which the system incorporates new information provided by the user and uses this information to improve the accuracy of subsequent answers.

[1029] "Means for providing product information" refers to a method for displaying data such as product details, reviews, and use examples to a user in response to a user's request.

[1030] "Generative AI" refers to artificial intelligence technology that generates new data and sentences based on learned data.

[1031] To implement this invention, the following system program is required: This system is composed of the following elements: a server, a terminal, and a user.

[1032] System Program

[1033] The program mainly consists of the following measures:

[1034] 1. Means of providing information through an interactive user interface:

[1035] The server provides an interactive user interface that accepts user requests. This interface supports voice and text input. The user interface is designed to allow users to enter the desired information in a natural, interactive manner.

[1036] 2. A method to analyze user requests and extract relevant keywords:

[1037] The server receives requests from users and analyzes them using a natural language processing (NLP) module. The analysis process involves extracting important keywords from the input sentence. The NLP module uses the Transformers library from Huggingface.

[1038] 3. Means of retrieving relevant information from internal and external sources:

[1039] The server searches for relevant information based on the extracted keywords from an internal database, which includes documents, memos, and minutes from within the company or organization, and from external sources, such as databases and libraries on the Internet.

[1040] 4. A means of providing users with interactive messages generated based on search results:

[1041] After searching for relevant information, the results are sent to the generation AI, which then generates a conversational message to be provided to the user based on the search results. The generation AI uses OpenAI's text-davinci-003 model.

[1042] 5. A way to learn additional documents and notes registered by the user and reflect them in subsequent request responses:

[1043] The server passes documents and notes uploaded by users to the machine learning module for additional learning, allowing the system to improve accuracy in subsequent information provision.

[1044] 6. Means of providing product information in virtual and physical stores:

[1045] As a means of providing users with the product information they require in real time in a virtual physical store, the server displays product details, reviews, usage examples, etc. based on user requests.

[1046] Hardware and Software

[1047] Server: The server acts as the central processing unit and performs all data calculations and processing.

[1048] Terminal: A device that provides a user interface, including smartphones and head-mounted displays (HMDs).

[1049] Software: Huggingface's Transformers library, OpenAI's text-davinci-003 model.

[1050] Specific examples

[1051] User Request: "I want to see the latest smartphones"

[1052] System behavior:

[1053] 1. The user makes a voice or text request on their smartphone or HMD saying, "I want to see the latest smartphones."

[1054] 2. The request is sent to the server and parsed by the NLP module.

[1055] 3. Keywords such as "latest" and "smartphone" are extracted.

[1056] 4. The server retrieves relevant product information from internal databases and external sources.

[1057] 5. Generative AI creates conversational messages based on search results.

[1058] 6. A message such as "Here are the latest smartphones: Model X, Model Y" will appear on the user's device.

[1059] Example of input prompt for generative AI model:

[1060] User request: "I want to see the latest smartphones."

[1061] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1062] Step 1:

[1063] The user sends a request from the device by voice or text, for example, "I want to see the latest smartphones." This request is entered into the device and sent to the server.

[1064] Step 2:

[1065] The server receives requests from the device and analyzes them using a natural language processing (NLP) module. Specifically, it extracts important keywords such as "latest" and "smartphone" from the request text. This process uses the Transformers library from Huggingface. The input is the user's request text, and the output is a list of extracted keywords.

[1066] Step 3:

[1067] The server searches for relevant information from internal databases and external information sources based on the extracted keywords. The internal database stores company documents, memos, minutes, etc. The external information sources include databases and libraries on the Internet. The input is a list of keywords, and the output is the search results data.

[1068] Step 4:

[1069] The server passes the search results to a generation AI, which creates an interactive message to be provided to the user. This generation AI uses OpenAI's text-davinci-003 model. Based on the search results, the generation AI generates an appropriate message to provide information to the user. The input is the search result data, and the output is an interactive message.

[1070] Step 5:

[1071] The server sends the generated interactive message to the terminal, which displays this message to the user. For example, a message like "Here are the latest smartphones: Model X, Model Y" is displayed. The input is the generated message, and the output is the message displayed to the user.

[1072] Step 6:

[1073] New documents and notes registered by users are uploaded to the server via their devices. The server passes these to the machine learning module as additional learning data, which improves the accuracy of subsequent information provision. The input is the uploaded documents and notes, and the output is updated learning data.

[1074] Step 7:

[1075] When a user requests product information in a virtual store, the server provides product details, reviews, usage examples, etc. in real time. Information can be easily delivered to the user via the terminal. The input is the user's request, and the output is the provision of product information.

[1076] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1077] This invention relates to an AI system that integrates all notes and documents, providing information through an interactive user interface and streamlining user work. This system not only analyzes user requests and quickly provides necessary information from relevant information sources, but also recognizes the user's emotions and responds accordingly. Specifically, by additionally learning from documents and notes that the user has looked up and using an emotion engine, the system provides optimal information and responses for the user.

[1078] Providing information through an interactive UI

[1079] Example: A scene where you request the minutes of an internal meeting

[1080] 1. User:

[1081] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[1082] 2. Terminal:

[1083] The terminal sends a request to the server.

[1084] 3. Server:

[1085] The server receives the request and parses it through a natural language processing (NLP) module, extracting keywords such as "last month," "management meeting," and "minutes."

[1086] 4. Server:

[1087] Based on the extracted keywords, search for relevant meeting minutes documents from internal databases or external libraries.

[1088] 5. Server:

[1089] The search results are passed to the generation AI.

[1090] 6. Generation AI:

[1091] Based on the search results, the generative AI generates a conversational message to be delivered to the user, such as, "The minutes of last month's management meeting can be viewed at the link below."

[1092] 7. Server:

[1093] Send the generated message to the terminal.

[1094] 8. Terminal:

[1095] Display received messages to the user.

[1096] 9. Users:

[1097] The user can view the minutes of the meeting by clicking the provided link.

[1098] Assistance in creating notes and documents

[1099] Example: Support for taking notes on presentation materials

[1100] 1. User:

[1101] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[1102] 2. Terminal:

[1103] The terminal sends a request to the server.

[1104] 3. Server:

[1105] The server receives the request and analyzes it using an NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[1106] 4. Server:

[1107] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[1108] 5. Generation AI:

[1109] Based on the search results, the generative AI creates a memo template for presentation materials to be provided to the user. For example, it creates a memo that states, "In the next customer briefing, it is important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[1110] 6. Server:

[1111] The generated memo is sent to the terminal.

[1112] 7. Terminal:

[1113] Display received notes to the user.

[1114] 8. Users:

[1115] The user can check the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[1116] Additional learning of registered information

[1117] Example: Adding a new internal guideline

[1118] 1. User:

[1119] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[1120] 2. Terminal:

[1121] The terminal sends the instruction to the server.

[1122] 3. Server:

[1123] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[1124] 4. User:

[1125] The user uploads a guideline document.

[1126] 5. Terminal:

[1127] Sends the uploaded document to the server.

[1128] 6. Server:

[1129] The server receives the uploaded document and passes it to the generative AI's learning module.

[1130] 7. Generation AI:

[1131] The generative AI analyzes new documents and adds their contents as learning data, so that new guideline information is reflected in future information provision.

[1132] 8. Server:

[1133] Sends a message to the terminal indicating that the update is complete.

[1134] 9. Terminal:

[1135] The terminal will display a message to the user indicating that the update is complete.

[1136] 10. Users:

[1137] The user checks the displayed message and checks the operation of the system as necessary.

[1138] Utilizing the Emotion Engine

[1139] Example: Responding based on user emotions

[1140] 1. User:

[1141] The user sends an emotional request via text or voice from the device.

[1142] 2. Terminal:

[1143] The terminal receives the request and sends it to the server.

[1144] 3. Server:

[1145] The server receives the request and uses an emotion engine to analyze the user's emotional state.

[1146] 4. Server:

[1147] Based on the emotional information analyzed by the emotion engine, the generative AI adjusts the tone and content of the response. For example, if the user is feeling stressed, it will generate a more friendly response that emphasizes solutions.

[1148] 5. Generation AI:

[1149] Generative AI generates messages based on emotions, such as, "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with."

[1150] 6. Server:

[1151] Send the generated message to the terminal.

[1152] 7. Terminal:

[1153] Display received messages to the user.

[1154] 8. Users:

[1155] The user sees the friendly message and makes further requests if necessary.

[1156] This system enables the provision of information that takes into consideration the user's feelings, contributing to improved business efficiency and user satisfaction.

[1157] The processing flow will be explained below.

[1158] Providing information through an interactive UI

[1159] Example: A scene where you request the minutes of an internal meeting

[1160] Processing Steps

[1161] Step 1:

[1162] User:

[1163] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[1164] Step 2:

[1165] Device:

[1166] The terminal receives the user's request and sends it to the server.

[1167] Step 3:

[1168] server:

[1169] The server receives the request and calls a natural language processing (NLP) module to analyze the request, extracting keywords such as "last month," "management meeting," and "minutes."

[1170] Step 4:

[1171] server:

[1172] Based on the extracted keywords, search for relevant meeting minutes documents from internal databases or external libraries.

[1173] Step 5:

[1174] server:

[1175] The search results are passed to the generation AI.

[1176] Step 6:

[1177] Generation AI:

[1178] Based on the search results, the generative AI generates a conversational message to be delivered to the user, such as, "The minutes of last month's management meeting can be viewed at the link below."

[1179] Step 7:

[1180] server:

[1181] Send the generated message to the terminal.

[1182] Step 8:

[1183] Device:

[1184] Display received messages to the user.

[1185] Step 9:

[1186] User:

[1187] The user can view the minutes of the meeting by clicking the provided link.

[1188] Assistance in creating notes and documents

[1189] Example: Support for taking notes on presentation materials

[1190] Processing Steps

[1191] Step 1:

[1192] User:

[1193] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[1194] Step 2:

[1195] Device:

[1196] The terminal sends a request to the server.

[1197] Step 3:

[1198] server:

[1199] The server receives the request and analyzes it using the NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[1200] Step 4:

[1201] server:

[1202] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[1203] Step 5:

[1204] Generation AI:

[1205] Based on the search results, the generative AI creates a memo template for presentation materials to be provided to the user. For example, it creates a memo that states, "In the next customer briefing, it is important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[1206] Step 6:

[1207] server:

[1208] The generated memo is sent to the terminal.

[1209] Step 7:

[1210] Device:

[1211] Display received notes to the user.

[1212] Step 8:

[1213] User:

[1214] The user can check the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[1215] Additional learning of registered information

[1216] Example: Adding a new internal guideline

[1217] Processing Steps

[1218] Step 1:

[1219] User:

[1220] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[1221] Step 2:

[1222] Device:

[1223] The terminal sends the instruction to the server.

[1224] Step 3:

[1225] server:

[1226] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[1227] Step 4:

[1228] User:

[1229] The user uploads a guideline document.

[1230] Step 5:

[1231] Device:

[1232] Sends the uploaded document to the server.

[1233] Step 6:

[1234] server:

[1235] The server receives the uploaded document and passes it to the generative AI's learning module.

[1236] Step 7:

[1237] Generation AI:

[1238] The generative AI analyzes new documents and adds their contents as learning data, so that new guideline information is reflected in future information provision.

[1239] Step 8:

[1240] server:

[1241] Sends a message to the terminal indicating that the update is complete.

[1242] Step 9:

[1243] Device:

[1244] The terminal will display a message to the user indicating that the update is complete.

[1245] Step 10:

[1246] User:

[1247] The user checks the displayed message and checks the operation of the system as necessary.

[1248] Utilizing the Emotion Engine

[1249] Example: Responding based on user emotions

[1250] Processing Steps

[1251] Step 1:

[1252] User:

[1253] The user sends an emotional request via text or voice from the device.

[1254] Step 2:

[1255] Device:

[1256] The terminal sends a request to the server.

[1257] Step 3:

[1258] server:

[1259] The server receives the request and uses an emotion engine to analyze the user's emotional state.

[1260] Step 4:

[1261] server:

[1262] Based on the emotional information analyzed by the emotion engine, the generative AI adjusts the tone and content of the response. For example, if the user is feeling stressed, it will generate a more friendly response that emphasizes solutions.

[1263] Step 5:

[1264] Generation AI:

[1265] Generative AI generates messages based on emotions, such as, "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with."

[1266] Step 6:

[1267] server:

[1268] Send the generated message to the terminal.

[1269] Step 7:

[1270] Device:

[1271] Display received messages to the user.

[1272] Step 8:

[1273] User:

[1274] The user sees the friendly message and makes further requests if necessary.

[1275] This system enables the provision of information that takes into consideration the user's feelings, contributing to improved business efficiency and user satisfaction.

[1276] Example 2

[1277] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1278] In today's business environment, users are required to quickly and accurately obtain the information they need from multiple sources. It is also important to reduce fatigue and stress by providing appropriate responses based on the user's emotions and circumstances, thereby improving work efficiency and user satisfaction. However, conventional systems have not been able to adequately meet these needs, resulting in reduced user convenience and cumbersome work.

[1279] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for providing information through an interactive user interface, means for analyzing a user request and extracting related keywords, means for searching for related information from an internal database or an external information source, means for providing the user with an interactive message generated based on the extracted information, means for additionally learning documents and notes registered by the user and reflecting them in subsequent request responses, and means for analyzing the user's emotions and generating a response according to the emotions. This allows the user to not only quickly and accurately obtain the information they need, but also receive a response according to their emotions, thereby reducing the complexity of their work and improving work efficiency and user satisfaction.

[1280] An "interactive user interface" refers to a user interface that allows two-way communication through user operations.

[1281] "Means for analyzing user requests and extracting relevant keywords" refers to a function that analyzes requests sent by users using technologies such as natural language processing and extracts important language elements.

[1282] "Means of searching for relevant information from internal databases and external information sources" refers to the function of searching for relevant information from databases inside and outside the system based on extracted keywords.

[1283] The "means for providing a generated interactive message to the user" refers to a function for providing a response to the user in the form of a message generated based on the search results.

[1284] "Means for additional learning of documents and notes registered by the user and reflecting them in subsequent request responses" refers to the function by which the system learns newly registered information by the user and uses it in subsequent responses.

[1285] "Means for analyzing the user's emotions and generating a response that corresponds to those emotions" refers to a function that determines the emotions from the user's request and generates a response that is appropriate to those emotions.

[1286] This invention relates to an AI system that integrates memos and documents, providing information through an interactive user interface to improve the efficiency of users' work. This system has the ability to analyze user requests and quickly provide necessary information from relevant information sources, as well as recognize the user's emotions and respond accordingly.

[1287] Providing information through an interactive UI

[1288] Hardware and Software Use

[1289] User device: personal computer, smartphone, tablet, etc. Uses a microphone for voice input and a keyboard for text input.

[1290] Server: A server equipped with a high-performance processor and a large amount of memory. RDBMS or NoSQL databases are used.

[1291] NLP Modules: Software modules for natural language processing, such as SpaCy and NLTK.

[1292] Generative AI: High-performance generative AI models, such as OpenAI's GPT-3.

[1293] Specific examples

[1294] 1. User: The user makes a request from their terminal saying, "I want to see the minutes of last month's management meeting."

[1295] 2. Terminal: The terminal sends a request to the server.

[1296] 3. Server: The server analyzes the request using a natural language processing module and extracts keywords such as "last month," "management meeting," and "minutes."

[1297] 4. Server: The server uses these keywords to search for relevant information from internal databases and external sources, using a search engine such as Elasticsearch.

[1298] 5. Server: Passes search results to the generation AI.

[1299] 6. Generative AI: The Generative AI generates a conversational message saying, "The minutes of last month's management meeting can be viewed at the link below."

[1300] 7. Server: Sends the generated message to the terminal.

[1301] 8. Terminal: displays received messages to the user.

[1302] Prompt Sentence Examples

[1303] "Show me the minutes of last month's management meeting."

[1304] Assistance in creating notes and documents

[1305] Specific examples

[1306] 1. User: The user requests from the terminal, "I want to create notes for the presentation materials for the next customer briefing."

[1307] 2. Terminal: The terminal sends a request to the server.

[1308] 3. Server: The server analyzes the request using a natural language processing module and extracts keywords such as "customer briefing," "presentation materials," and "memo."

[1309] 4. Server: The server searches the user's past documents and notes from its internal database.

[1310] 5. Generative AI: Generative AI creates memo templates for presentation materials based on search results. For example, it generates a memo saying, "In the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[1311] 6. Server: Sends the generated memo to the device.

[1312] 7. Terminal: Displays the received memo to the user.

[1313] 8. User: The user checks the displayed notes and makes edits or additions as necessary.

[1314] Prompt Sentence Examples

[1315] "Please prepare notes for the next client briefing."

[1316] Additional learning of registered information

[1317] Specific examples

[1318] 1. User: The user issues a command from their device saying, "I want to add new internal guidelines to the AI ​​and have it learn them."

[1319] 2. Terminal: The terminal sends instructions to the server.

[1320] 3. Server: The server receives the instructions and sends a message to the user prompting them to upload their guidelines.

[1321] 4. User: The user uploads the guideline document.

[1322] 5. Terminal: Sends the uploaded document to the server.

[1323] 6. Server: The server receives the document and passes it to the generative AI's learning module.

[1324] 7. Generative AI: Generative AI analyzes new documents and adds their contents to the learning data. This allows new guideline information to be reflected in future information provision.

[1325] 8. Server: Sends a message to the device indicating that the update is complete.

[1326] 9. Terminal: The terminal displays a message to the user that the update is complete.

[1327] Prompt Sentence Examples

[1328] "Learn these new guidelines and incorporate them into your next search."

[1329] Utilizing the Emotion Engine

[1330] Specific examples

[1331] 1. User: The user sends an emotional request from the device, for example, with an elevated tone of voice or emotional text.

[1332] 2. Terminal: The terminal sends a request to the server.

[1333] 3. Server: The server receives the request and uses an emotion engine to analyze the user's emotional state, for example determining that the user is tired.

[1334] 4. Server: Based on the emotional information analyzed by the emotion engine, the generative AI adjusts the tone and content of the response, for example, generating a friendly message that emphasizes solutions.

[1335] 5. Generative AI: Generative AI generates messages based on emotions, such as "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with."

[1336] 6. Server: Sends the generated message to the terminal.

[1337] 7. Terminal: Displays received messages to the user.

[1338] Prompt Sentence Examples

[1339] "I'm very busy, but could you please give me a quick summary of the recent meeting?"

[1340] With these functions, the system streamlines users' work and provides necessary information quickly and accurately. It also responds with consideration for the user's feelings, improving work efficiency and user satisfaction.

[1341] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1342] Providing information through an interactive UI

[1343] Example: A scene where you request the minutes of an internal meeting

[1344] Step 1:

[1345] User: The user sends a request from their device saying, "I want to see the minutes of last month's management meeting." This operation is performed by the user either typing text or making a request by voice. Input is done using the device's keyboard and microphone.

[1346] Step 2:

[1347] Terminal: The terminal sends the received request to the server using an HTTP POST request or WebSocket. The data is encoded in JSON format.

[1348] Step 3:

[1349] Server: The server receives the request and analyzes it using a natural language processing (NLP) module. This process extracts keywords such as "last month," "management meeting," and "minutes" from the request text. The input is the request data in JSON format, and the output is a list of keywords. The specific software used is SpaCy and NLTK.

[1350] Step 4:

[1351] Server: The server searches for relevant information from internal databases and external sources based on the extracted keywords. This search is performed using a search engine such as Elasticsearch. The input is a list of keywords, and the output is a set of related information.

[1352] Step 5:

[1353] Server: The server passes the search results to the generative AI. The input is a set of related information, and the output is a response message generated by the generative AI. Specifically, OpenAI's GPT-3 is used.

[1354] Step 6:

[1355] Generative AI: Based on the search results, the generative AI generates a conversational message such as, "The minutes of last month's management meeting can be viewed at the link below." The input is a set of relevant information, and the output is a response message.

[1356] Step 7:

[1357] Server: The server sends the generated message to the terminal. HTTP or WebSocket is used for transmission. The input is the response message, and the output is the transmission completion status.

[1358] Step 8:

[1359] Terminal: A terminal displays received messages to the user, such as pop-up notifications or chat windows on the UI. Input is a response message, and output is displayed in the user interface.

[1360] Step 9:

[1361] User: The user is presented with a link to view the minutes. When the user clicks on the link, their browser opens a new tab or window with the relevant document.

[1362] Assistance in creating notes and documents

[1363] Example: Support for taking notes on presentation materials

[1364] Step 1:

[1365] User: The user requests, "I want to create notes for the presentation materials for the next client briefing." This can be done by typing text or by making a voice request.

[1366] Step 2:

[1367] Terminal: The terminal sends the received request to the server using an HTTP POST request or WebSocket. The data is encoded in JSON format.

[1368] Step 3:

[1369] Server: The server receives the request and analyzes it using the NLP module. Keywords such as "customer briefing," "presentation materials," and "memo" are extracted through the analysis. The input is the request data in JSON format, and the output is a list of keywords.

[1370] Step 4:

[1371] Server: The server searches the user's past documents and notes from an internal database based on the extracted keywords. The search engine used is Elasticsearch. The input is a list of keywords, and the output is a set of related documents and notes.

[1372] Step 5:

[1373] Generative AI: Generative AI creates memo templates for presentation materials based on search results. For example, it generates memos such as, "At the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session." The input is related materials and memos, and the output is a memo template.

[1374] Step 6:

[1375] Server: Sends the generated memo to the device. HTTP POST request or WebSocket is used for sending. The input is the memo template, and the output is the sending completion status.

[1376] Step 7:

[1377] Terminal: Displays received memos to the user. Specifically, it displays them in the memo field on the UI or in notifications. The input is the memo template, and the output is displayed in the user interface.

[1378] Step 8:

[1379] User: The user checks the displayed notes, edits and supplements them as necessary, and creates the final presentation materials.

[1380] Additional learning of registered information

[1381] Example: Adding a new internal guideline

[1382] Step 1:

[1383] User: The user issues a command from the device saying, "I want to add new internal guidelines to the AI ​​and have it learn them." This operation is performed by the user entering text or giving voice instructions.

[1384] Step 2:

[1385] Terminal: The terminal sends instructions to the server using HTTP POST requests or WebSockets. The data is encoded in JSON format.

[1386] Step 3:

[1387] Server: The server receives the instructions and sends a message to the user prompting them to upload the guideline document. The input is the instructions, and the output is the upload prompt message.

[1388] Step 4:

[1389] User: The user uploads a guideline document. The operation is performed by selecting the file through a file selection dialog. The input is the document file, and the output is the uploaded file.

[1390] Step 5:

[1391] Terminal: Sends the uploaded document to the server using an HTTP POST request. The input is the uploaded file, and the output is the status of the successful submission.

[1392] Step 6:

[1393] Server: The server passes the received document to the generative AI learning module. The input is the document file, and the output is the learning data.

[1394] Step 7:

[1395] Generative AI: The generative AI analyzes new documents and adds their contents as training data. This allows new guideline information to be reflected in subsequent information provision. The input is training data, and the output is updated training data.

[1396] Step 8:

[1397] Server: Sends an update completion message to the terminal. The input is the updated training data, and the output is the update completion message.

[1398] Step 9:

[1399] Terminal: The terminal displays a message to the user that the update is complete. The input is the message, and the output is what is displayed in the user interface.

[1400] Step 10:

[1401] User: The user checks the displayed messages and checks the system operation if necessary.

[1402] Utilizing the Emotion Engine

[1403] Example: Responding based on user emotions

[1404] Step 1:

[1405] User: The user sends an emotional request from the device, for example, by typing in a voice with a high tone or by typing in a text that expresses emotion. The input is done using the device's keyboard and microphone.

[1406] Step 2:

[1407] Terminal: The terminal receives the request and sends it to the server using an HTTP POST request or WebSocket. The data is encoded in JSON format.

[1408] Step 3:

[1409] Server: The server receives the request and uses an emotion engine to analyze the user's emotional state. Emotion analysis uses techniques to extract emotions from text and speech. The input is the request data in JSON format, and the output is the emotional status.

[1410] Step 4:

[1411] Server: Based on the emotional information analyzed by the emotion engine, the generation AI adjusts the tone and content of the response. For example, if the user is tired, it generates a message in a friendly tone that emphasizes solutions. The input is the emotional status, and the output is the response message.

[1412] Step 5:

[1413] Generative AI: Generative AI generates messages based on emotions. For example, it generates a message like, "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with." The input is the emotional status, and the output is a response message.

[1414] Step 6:

[1415] Server: Sends the generated message to the terminal. HTTP POST request or WebSocket is used for sending. The input is the response message, and the output is the status of sending completion.

[1416] Step 7:

[1417] Terminal: A terminal displays received messages to the user, such as in a popup notification or a chat window. Input is a response message, and output is displayed in the user interface.

[1418] Step 8:

[1419] User: The user reviews the friendly message and makes a follow-up request if necessary, by retyping the request through the chat interface.

[1420] The specific operations, data processing, and data calculations in each processing step have been described above.

[1421] (Application example 2)

[1422] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1423] Conventional information provision systems often found it difficult to properly analyze user requests and quickly provide relevant information. Furthermore, they were unable to respond in a way that took the user's emotions into account, limiting the improvement of user satisfaction. Furthermore, the system's ability to effectively add and learn new information registered by the user and reflect it in subsequent requests was incomplete, creating a need for efficient customer service in physical stores.

[1424] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing information through an interactive operation screen, means for analyzing a user request and extracting related keywords, means for searching for related information from internal data or an external database, means for providing the user with an interactive message generated based on the extracted information, means for additionally learning information registered by the user and reflecting the information in subsequent requests and responses, and means for analyzing the user's emotional state and adjusting the tone and content of the response based on the learning. This enables a quick and accurate response to a user request, and further enables a response that takes the user's emotions into consideration, thereby improving user satisfaction.

[1425] An "interactive operation screen" is an interface that allows a user to interact with the system through the screen and receive information.

[1426] "Means of providing information" refers to the function that the all-memo / document integrated AI system uses to provide the necessary information to users.

[1427] The "means for analyzing a user request and extracting relevant keywords" refers to a method and device for analyzing the content of a user request and identifying important keywords.

[1428] "Means for searching for relevant information from internal data and external databases" is a function for searching for appropriate information from sources inside and outside the system based on identified keywords.

[1429] The "means for providing a user with a message in an interactive format" is a function for providing information to the user in an easy-to-understand message format based on the extracted and searched information.

[1430] The "means for additionally learning information registered by the user and reflecting it in subsequent request responses" is a function for learning information newly added by the user and using it to provide future information.

[1431] "Means for analyzing a user's emotional state and adjusting the tone and content of a response based thereon" refers to a method and apparatus for analyzing a user's emotions and providing a response with the optimal tone and content according to those emotions.

[1432] An "interactive user interface" is an interface that allows a user to operate and obtain information through natural interaction with a system.

[1433] This invention relates to an AI system that integrates all notes and documents, providing information through an interactive user interface and streamlining user work. The system mainly includes the following hardware and software:

[1434] Required Hardware and Software

[1435] Smart glasses: Devices worn by store staff that capture customers' questions and facial expressions in real time.

[1436] Server: A computer server that processes data, performs generative AI, and performs natural language processing (NLP).

[1437] Python: A programming language.

[1438] Transformers Library: A library for NLP tasks.

[1439] cv2 (OpenCV): Computer vision library.

[1440] requests: A library for sending HTTP requests.

[1441] System Operation

[1442] 1. Analyzing user requests and providing information

[1443] The server receives requests from users (store staff) through the smart glasses. The requests are input in voice or text format. The server then analyzes the requests using natural language processing (NLP) and extracts relevant keywords. It then searches for relevant information in internal and external databases and uses generative AI to generate conversational messages. This allows users to easily answer customer questions.

[1444] For example, if a customer asks, "What color variations of this product are there?", the server retrieves the product information and generates the answer, "This product comes in red, blue, and green." An example prompt sentence is as follows:

[1445] Question: "What color variations are available for this item?"

[1446] Context: "Product ABC is available in red, blue, and green colors. Details for each color are as follows..."

[1447] 2. Sentiment Analysis and Response Adjustment

[1448] The server captures the customer's facial expressions and tone of voice through the smart glasses and inputs them into an emotion analysis engine. Based on the analysis results, the customer's emotional state is identified. The generative AI then adjusts the tone and content of the response according to the identified emotional state. This allows the response to take the customer's emotions into consideration.

[1449] For example, if a customer says, "I'm a little disappointed with this product," the sentiment analysis engine will identify the "disappointed" emotion, and the generative AI will generate a response such as, "I'm sorry you're disappointed. Would you like us to find another product that meets your expectations?"

[1450] Examples of concrete examples and prompts

[1451] Below is a concrete example in which a user responds to a customer question using a prompt generated by the server.

[1452] Question: "What material is this item made of?"

[1453] Context: "Product XYZ is made from high-quality cotton and polyester. Its specific composition is as follows..."

[1454] In this way, the present invention enables store staff to respond to customer questions quickly and accurately, and also enables responses that are in line with the customer's feelings, which is expected to improve user satisfaction.

[1455] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1456] Step 1:

[1457] The server receives user requests from the smart glasses. The user inputs the request in voice or text format, and the device sends it to the server. The input data is the user's question or instruction.

[1458] Step 2:

[1459] After receiving the request, the server analyzes it using a natural language processing (NLP) module. Specifically, it breaks down the sentences and extracts keywords. As a result of this analysis, important keywords are output.

[1460] Step 3:

[1461] The server searches for relevant information from internal and external databases based on the extracted keywords, such as product information, instructions, and related notes. The server searches this data through API requests and obtains search results.

[1462] Step 4:

[1463] The server generates a dialogue-style message using a generative AI model based on the search results. The generative AI model uses the context information and extracted keywords as input. The generative AI model generates a response message to be provided to the user and returns the output to the server.

[1464] Step 5:

[1465] The server sends the generated message to the terminal, which receives the message and provides it to the user by display or voice. The content of the message is a specific answer to the user's question.

[1466] Step 6:

[1467] The server captures the emotional state of the user or customer through the smart glasses. The emotion analysis engine analyzes the data based on voice input and facial expression capture data. The input data is audio and video capture data.

[1468] Step 7:

[1469] The server uses a generative AI model to adjust the tone and content of the response based on the results of the emotion analysis engine. Specifically, it generates a response appropriate for when the user is feeling stressed. The input is the analysis result, and the output is the adjusted response message.

[1470] Step 8:

[1471] The server sends the tailored response message to the terminal, which displays the received message and provides a specific response to the user, who then responds to the customer based on the response.

[1472] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1473] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1474] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1475] [Third embodiment]

[1476] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1477] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1478] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1479] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1480] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1481] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1482] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1483] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1484] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1485] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1486] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1487] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1488] This invention relates to an AI system that integrates all notes and documents, providing information through an interactive user interface and streamlining user work. This system has the ability to analyze user requests and provide relevant information quickly and accurately. It can also learn additional information from documents and notes registered by the user, improving the accuracy of the information provided.

[1489] Providing information through an interactive UI

[1490] Example: A scene where you request the minutes of an internal meeting

[1491] 1. User:

[1492] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[1493] 2. Terminal:

[1494] The terminal receives this request and sends it to the server.

[1495] 3. Server:

[1496] The server analyzes the received request through a natural language processing (NLP) module, extracting keywords such as "last month," "management meeting," and "minutes."

[1497] 4. Server:

[1498] Based on the extracted keywords, the system searches for relevant meeting minutes documents from internal databases and external libraries. When relevant documents are found, the information is passed to the generation AI.

[1499] 5. Generation AI:

[1500] Based on the search results, the AI ​​generates a conversational message to be provided to the user, such as "The minutes of last month's management meeting can be viewed at the link below."

[1501] 6. Server:

[1502] Send the generated message to the terminal.

[1503] 7. Terminal:

[1504] Display received messages to the user.

[1505] 8. Users:

[1506] The user can view the minutes of the meeting in question by clicking the provided link.

[1507] Assistance in creating notes and documents

[1508] Example: Support for taking notes on presentation materials

[1509] 1. User:

[1510] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[1511] 2. Terminal:

[1512] The terminal sends this request to the server.

[1513] 3. Server:

[1514] The server receives the request and analyzes it using the NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[1515] 4. Server:

[1516] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[1517] 5. Generation AI:

[1518] Based on the search results, the AI ​​creates a memo template for presentation materials to be provided to the user. It creates specific memos such as, "At the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[1519] 6. Server:

[1520] The generated memo is sent to the terminal.

[1521] 7. Terminal:

[1522] Display received notes to the user.

[1523] 8. Users:

[1524] Users can review the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[1525] Additional learning of registered information

[1526] Example: Adding a new internal guideline

[1527] 1. User:

[1528] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[1529] 2. Terminal:

[1530] The terminal transmits the user's instructions to the server.

[1531] 3. Server:

[1532] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[1533] 4. User:

[1534] The user uploads a guideline document.

[1535] 5. Terminal:

[1536] The uploaded document is sent to the server.

[1537] 6. Server:

[1538] The server receives the uploaded document and passes it to the generative AI's learning module.

[1539] 7. Generation AI:

[1540] The generative AI analyzes new documents and adds their contents as learning data, so that the new guideline information will be reflected in future information provision.

[1541] 8. Server:

[1542] Sends a message to the terminal indicating that the update is complete.

[1543] 9. Terminal:

[1544] The terminal will display a message to the user indicating that the update is complete.

[1545] 10. Users:

[1546] The user can check the displayed messages and check the operation of the system as needed.

[1547] This enables the system to provide users with information that is optimized for them, greatly contributing to improved business efficiency.

[1548] The processing flow will be explained below.

[1549] Providing information through an interactive UI

[1550] Example: A scene where you request the minutes of an internal meeting

[1551] Processing Steps

[1552] Step 1:

[1553] User:

[1554] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[1555] Step 2:

[1556] Device:

[1557] The terminal receives the user's request and sends it to the server.

[1558] Step 3:

[1559] server:

[1560] The server receives the request and calls a natural language processing (NLP) module to parse the request, extracting keywords such as "last month," "management meeting," and "minutes."

[1561] Step 4:

[1562] server:

[1563] Based on the extracted keywords, search for relevant meeting minutes documents from internal databases or external libraries.

[1564] Step 5:

[1565] server:

[1566] The search results are passed to the generation AI.

[1567] Step 6:

[1568] Generation AI:

[1569] Based on the search results, the generative AI generates a conversational message to be delivered to the user, such as, "The minutes of last month's management meeting can be viewed at the link below."

[1570] Step 7:

[1571] server:

[1572] Send the generated message to the terminal.

[1573] Step 8:

[1574] Device:

[1575] Display received messages to the user.

[1576] Step 9:

[1577] User:

[1578] The user can view the minutes of the meeting by clicking the provided link.

[1579] Assistance in creating notes and documents

[1580] Example: Support for taking notes on presentation materials

[1581] Processing Steps

[1582] Step 1:

[1583] User:

[1584] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[1585] Step 2:

[1586] Device:

[1587] The terminal sends this request to the server.

[1588] Step 3:

[1589] server:

[1590] The server receives the request and analyzes it using the NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[1591] Step 4:

[1592] server:

[1593] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[1594] Step 5:

[1595] Generation AI:

[1596] Based on the search results, the AI ​​creates a memo template for presentation materials to be provided to the user. For example, it might create a memo such as, "At the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[1597] Step 6:

[1598] server:

[1599] The generated memo is sent to the terminal.

[1600] Step 7:

[1601] Device:

[1602] Display received notes to the user.

[1603] Step 8:

[1604] User:

[1605] The user can check the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[1606] Additional learning of registered information

[1607] Example: Adding a new internal guideline

[1608] Processing Steps

[1609] Step 1:

[1610] User:

[1611] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[1612] Step 2:

[1613] Device:

[1614] The terminal transmits the user's instructions to the server.

[1615] Step 3:

[1616] server:

[1617] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[1618] Step 4:

[1619] User:

[1620] The user uploads a guideline document.

[1621] Step 5:

[1622] Device:

[1623] Sends the uploaded document to the server.

[1624] Step 6:

[1625] server:

[1626] The server receives the uploaded document and passes it to the generative AI's learning module.

[1627] Step 7:

[1628] Generation AI:

[1629] The generative AI analyzes new documents and adds their contents as learning data, so that new guideline information is reflected in future information provision.

[1630] Step 8:

[1631] server:

[1632] A message indicating that learning is complete is sent to the device.

[1633] Step 9:

[1634] Device:

[1635] The terminal displays a message to the user indicating that learning is complete.

[1636] Step 10:

[1637] User:

[1638] The user checks the displayed message and checks the operation of the system as necessary.

[1639] Example 1

[1640] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1641] Existing information provision systems often have delays in responding to user requests or provide inaccurate information. Furthermore, even when a user adds new information, it takes time for that information to be reflected in the system. Furthermore, few systems can also respond to voice commands, which limits user convenience.

[1642] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1643] In this invention, the server includes a means for analyzing the request through a natural language processing module and extracting relevant keywords, a means for searching for relevant information from an internal database or external information source based on the extracted keywords, and a means for passing the search results to a generative AI model to generate a dialogue-style message. This enables a quick and accurate response to a user's request. Furthermore, even if new information is added, the generative AI model can instantly learn from it and reflect it in subsequent request responses. Furthermore, support for voice instructions improves user convenience.

[1644] "User" means an individual or organization that uses the system.

[1645] A "terminal" is a device used by a user to send requests or receive information.

[1646] A "request" is a request for information made by a user to the system.

[1647] A "natural language processing module" is a software component that analyzes the text of a request and extracts meaning and keywords.

[1648] "Keywords" are important words or phrases extracted from a user's request.

[1649] An "internal database" is a database within the system where information such as user documents and notes is stored.

[1650] "External sources" refers to information obtained from libraries or databases outside the system.

[1651] A "generative AI model" is an artificial intelligence model that automatically generates messages to be provided to users based on search results and learning data.

[1652] "Interactive messaging" refers to messaging that is in a natural format for interacting with a user.

[1653] A "new document or note" is new information that a user adds to the system.

[1654] "Learning" is the process by which a generative AI model takes in new information and uses it to improve its responses in future scenarios.

[1655] "Voice instructions" are instructions sent by the user to the system through voice.

[1656] This invention relates to an AI system that integrates all memos and documents, providing information through an interactive user interface and streamlining user work. This system has the ability to analyze user requests and provide relevant information quickly and accurately. It can also learn from documents and memos registered by users to improve the accuracy of the information provided.

[1657] The following hardware and software are used to implement the present invention.

[1658] 1. Hardware

[1659] Device: The device through which the user enters the request (e.g., PC, smartphone, tablet).

[1660] Server: A server that analyzes requests, searches for information, generates messages, etc.

[1661] 2. Software

[1662] Natural Language Processing Module: Software for analyzing requests. Specifically, it uses the Google Cloud Natural Language API.

[1663] Internal database: A database that stores information such as user notes and documents. Specifically, MongoDB is used.

[1664] Generative AI model: An artificial intelligence model that generates messages to be provided to users. Specifically, we use OpenAI's GPT-4.

[1665] Examples of concrete examples and prompts

[1666] 1. Requesting minutes of an internal meeting

[1667] User: The user sends a request from their terminal saying, "I want to see the minutes of last month's management meeting."

[1668] Terminal: The terminal receives the user's request and sends it to the server.

[1669] Server: The server parses the request using the Google Cloud Natural Language API and extracts keywords such as "last month," "management meeting," and "minutes."

[1670] The server searches for relevant meeting minutes from MongoDB based on keywords and passes them to the generative AI model.

[1671] Based on the search results, a generative AI model (OpenAI GPT-4) generates a message such as, "The minutes of last month's management meeting can be viewed at the link below."

[1672] Server: Sends the generated message to the device.

[1673] Terminal: Displays received messages to the user.

[1674] User: View the minutes via the provided link.

[1675] Example prompt sentence:

[1676] "I'd like to see the minutes of last month's management meeting."

[1677] 2. Support for taking notes on presentation materials

[1678] User: The user makes a request from their device saying, "I want to create notes for the presentation materials for the next customer briefing."

[1679] Device: The device sends a request to the server.

[1680] Server: The server analyzes the request using the Google Cloud Natural Language API and extracts keywords such as "customer briefing," "presentation materials," and "memo."

[1681] The server searches MongoDB for relevant past documents and notes based on the extracted keywords and passes them to the generative AI model.

[1682] A generative AI model (OpenAI GPT-4) generates notes such as, "At your next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[1683] Server: Sends the generated memo to the device.

[1684] Terminal: Displays received notes to the user.

[1685] User: Review, edit and supplement notes, and create final presentation materials.

[1686] Example prompt sentence:

[1687] "I want to make notes for the presentation materials for the next client briefing."

[1688] 3. Addition of new internal guidelines

[1689] User: The user submits a request to add a new internal guideline document to the AI ​​and have it learn it.

[1690] Device: The device sends a request to the server.

[1691] Server: The server receives the request and sends a message to the user prompting them to upload a guideline document.

[1692] User: Upload a guideline document.

[1693] Terminal: Sends the document to the server.

[1694] Server: The server receives the document and passes it to the learning module of the generative AI model, which then learns from it and applies it to future responses.

[1695] Server: Sends a message to the device indicating that the update is complete.

[1696] Terminal: Display a message to the user that the update is complete.

[1697] User: Check the displayed messages and check the system operation.

[1698] Example prompt sentence:

[1699] "We want to add new internal guideline documents to the AI ​​and have it learn them."

[1700] As described above, the present invention is a system that combines a natural language processing module, an internal database, and a generative AI model to provide efficient information provision, enabling quick and accurate responses to user requests and improving business efficiency.

[1701] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1702] Step 1:

[1703] The user sends a request from the terminal.

[1704] Example: A user types or voice requests, "I'd like to see the minutes of last month's management meeting."

[1705] Input: User's text or voice request.

[1706] Output: Received request on terminal.

[1707] Step 2:

[1708] The terminal receives the user's request and sends it to the server.

[1709] Example: A device sends a request to the server saying, "I want to see the minutes of last month's management meeting."

[1710] Input: The user request.

[1711] Output: The request sent to the server.

[1712] Step 3:

[1713] The server analyzes the received request through a natural language processing module.

[1714] Example: The server passes the request to the Google Cloud Natural Language API for parsing.

[1715] Input: The user request received from the device.

[1716] Output: Keywords such as "last month", "management meeting", and "minutes".

[1717] Step 4:

[1718] The server searches for relevant information from internal databases and external information sources based on the extracted keywords.

[1719] Example: The server searches MongoDB for documents matching "last month", "management meeting", and "minutes".

[1720] Input: Extracted keywords.

[1721] Output: A document of the relevant minutes.

[1722] Step 5:

[1723] The server passes the search results to a generative AI model, which generates conversational messages.

[1724] Example: The server uses the search results to generate a message using a generative AI model (OpenAI GPT-4).

[1725] Input: Relevant minutes document.

[1726] Output: The message "The minutes of last month's management meeting can be found at the link below."

[1727] Step 6:

[1728] The server sends the generated message to the terminal.

[1729] Example: The server sends a generated message to the user terminal.

[1730] Input: Message from the generative AI model.

[1731] Output: Messages sent to the terminal.

[1732] Step 7:

[1733] The terminal displays the received message to the user.

[1734] Example: The device displays the message "The minutes of last month's management meeting can be viewed at the link below."

[1735] Input: The message sent by the server.

[1736] Output: The message displayed to the user.

[1737] Step 8:

[1738] The user checks the minutes of the meeting from the provided link.

[1739] Example: A user clicks on a displayed link to view the meeting minutes.

[1740] Input: The link displayed on the terminal.

[1741] Output: The minutes to be viewed.

[1742] (Application example 1)

[1743] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1744] Conventional information provision systems using interactive user interfaces have had difficulty in providing prompt and appropriate information in response to user requests. In particular, in virtual and physical stores, the lack of a function to provide product information desired by users in real time limited the user experience. Therefore, a system that allows users to quickly and accurately obtain the information they desire is needed.

[1745] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1746] In this invention, the server includes means for providing information through an interactive user interface, means for analyzing a user request and extracting related keywords, means for searching for related information from internal and external information sources, means for providing the user with an interactive message generated based on the search results, means for additionally learning documents and notes registered by the user and reflecting them in subsequent request responses, and means for providing product information in a virtual physical store. This allows users to quickly and accurately obtain product information in real time even in a virtual physical store.

[1747] An "interactive user interface" refers to a mechanism that allows two-way communication between the user and the system.

[1748] "Means for extracting relevant keywords" refers to a method for identifying and extracting important words and phrases from the user's request.

[1749] "Internal sources" refers to information stored within a company or organization, such as databases, documents, and notes.

[1750] "External sources" refer to information that exists outside the organization, such as external libraries or databases on the Internet.

[1751] "Conversational messages" refer to text that provides information to users in the form of natural conversation.

[1752] "Virtual brick-and-mortar store" refers to a virtual shopping environment operated on the Internet.

[1753] "Means for additional learning from registered documents and notes" refers to a method by which the system incorporates new information provided by the user and uses this information to improve the accuracy of subsequent answers.

[1754] "Means for providing product information" refers to a method for displaying data such as product details, reviews, and use examples to a user in response to a user's request.

[1755] "Generative AI" refers to artificial intelligence technology that generates new data and sentences based on learned data.

[1756] To implement this invention, the following system program is required: This system is composed of the following elements: a server, a terminal, and a user.

[1757] System Program

[1758] The program mainly consists of the following measures:

[1759] 1. Means of providing information through an interactive user interface:

[1760] The server provides an interactive user interface that accepts user requests. This interface supports voice and text input. The user interface is designed to allow users to enter the desired information in a natural, interactive manner.

[1761] 2. A method to analyze user requests and extract relevant keywords:

[1762] The server receives requests from users and analyzes them using a natural language processing (NLP) module. The analysis process involves extracting important keywords from the input sentence. The NLP module uses the Transformers library from Huggingface.

[1763] 3. Means of retrieving relevant information from internal and external sources:

[1764] The server searches for relevant information based on the extracted keywords from an internal database, which includes documents, memos, and minutes from within the company or organization, and from external sources, such as databases and libraries on the Internet.

[1765] 4. A means of providing users with interactive messages generated based on search results:

[1766] After searching for relevant information, the results are sent to the generation AI, which then generates a conversational message to be provided to the user based on the search results. The generation AI uses OpenAI's text-davinci-003 model.

[1767] 5. A way to learn additional documents and notes registered by the user and reflect them in subsequent request responses:

[1768] The server passes documents and notes uploaded by users to the machine learning module for additional learning, allowing the system to improve accuracy in subsequent information provision.

[1769] 6. Means of providing product information in virtual and physical stores:

[1770] As a means of providing users with the product information they require in real time in a virtual physical store, the server displays product details, reviews, usage examples, etc. based on user requests.

[1771] Hardware and Software

[1772] Server: The server acts as the central processing unit and performs all data calculations and processing.

[1773] Terminal: A device that provides a user interface, including smartphones and head-mounted displays (HMDs).

[1774] Software: Huggingface's Transformers library, OpenAI's text-davinci-003 model.

[1775] Specific examples

[1776] User Request: "I want to see the latest smartphones"

[1777] System behavior:

[1778] 1. The user makes a voice or text request on their smartphone or HMD saying, "I want to see the latest smartphones."

[1779] 2. The request is sent to the server and parsed by the NLP module.

[1780] 3. Keywords such as "latest" and "smartphone" are extracted.

[1781] 4. The server retrieves relevant product information from internal databases and external sources.

[1782] 5. Generative AI creates conversational messages based on search results.

[1783] 6. A message such as "Here are the latest smartphones: Model X, Model Y" will appear on the user's device.

[1784] Example of input prompt for generative AI model:

[1785] User request: "I want to see the latest smartphones."

[1786] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1787] Step 1:

[1788] The user sends a request from the device by voice or text, for example, "I want to see the latest smartphones." This request is entered into the device and sent to the server.

[1789] Step 2:

[1790] The server receives requests from the device and analyzes them using a natural language processing (NLP) module. Specifically, it extracts important keywords such as "latest" and "smartphone" from the request text. This process uses the Transformers library from Huggingface. The input is the user's request text, and the output is a list of extracted keywords.

[1791] Step 3:

[1792] The server searches for relevant information from internal databases and external information sources based on the extracted keywords. The internal database stores company documents, memos, minutes, etc. The external information sources include databases and libraries on the Internet. The input is a list of keywords, and the output is the search results data.

[1793] Step 4:

[1794] The server passes the search results to a generation AI, which creates an interactive message to be provided to the user. This generation AI uses OpenAI's text-davinci-003 model. Based on the search results, the generation AI generates an appropriate message to provide information to the user. The input is the search result data, and the output is an interactive message.

[1795] Step 5:

[1796] The server sends the generated interactive message to the terminal, which displays this message to the user. For example, a message like "Here are the latest smartphones: Model X, Model Y" is displayed. The input is the generated message, and the output is the message displayed to the user.

[1797] Step 6:

[1798] New documents and notes registered by users are uploaded to the server via their devices. The server passes these to the machine learning module as additional learning data, which improves the accuracy of subsequent information provision. The input is the uploaded documents and notes, and the output is updated learning data.

[1799] Step 7:

[1800] When a user requests product information in a virtual store, the server provides product details, reviews, usage examples, etc. in real time. Information can be easily delivered to the user via the terminal. The input is the user's request, and the output is the provision of product information.

[1801] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1802] This invention relates to an AI system that integrates all notes and documents, providing information through an interactive user interface and streamlining user work. This system not only analyzes user requests and quickly provides necessary information from relevant information sources, but also recognizes the user's emotions and responds accordingly. Specifically, by additionally learning from documents and notes that the user has looked up and using an emotion engine, the system provides optimal information and responses for the user.

[1803] Providing information through an interactive UI

[1804] Example: A scene where you request the minutes of an internal meeting

[1805] 1. User:

[1806] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[1807] 2. Terminal:

[1808] The terminal sends a request to the server.

[1809] 3. Server:

[1810] The server receives the request and parses it through a natural language processing (NLP) module, extracting keywords such as "last month," "management meeting," and "minutes."

[1811] 4. Server:

[1812] Based on the extracted keywords, search for relevant meeting minutes documents from internal databases or external libraries.

[1813] 5. Server:

[1814] The search results are passed to the generation AI.

[1815] 6. Generation AI:

[1816] Based on the search results, the generative AI generates a conversational message to be delivered to the user, such as, "The minutes of last month's management meeting can be viewed at the link below."

[1817] 7. Server:

[1818] Send the generated message to the terminal.

[1819] 8. Terminal:

[1820] Display received messages to the user.

[1821] 9. Users:

[1822] The user can view the minutes of the meeting by clicking the provided link.

[1823] Assistance in creating notes and documents

[1824] Example: Support for taking notes on presentation materials

[1825] 1. User:

[1826] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[1827] 2. Terminal:

[1828] The terminal sends a request to the server.

[1829] 3. Server:

[1830] The server receives the request and analyzes it using an NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[1831] 4. Server:

[1832] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[1833] 5. Generation AI:

[1834] Based on the search results, the generative AI creates a memo template for presentation materials to be provided to the user. For example, it creates a memo that states, "In the next customer briefing, it is important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[1835] 6. Server:

[1836] The generated memo is sent to the terminal.

[1837] 7. Terminal:

[1838] Display received notes to the user.

[1839] 8. Users:

[1840] The user can check the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[1841] Additional learning of registered information

[1842] Example: Adding a new internal guideline

[1843] 1. User:

[1844] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[1845] 2. Terminal:

[1846] The terminal sends the instruction to the server.

[1847] 3. Server:

[1848] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[1849] 4. User:

[1850] The user uploads a guideline document.

[1851] 5. Terminal:

[1852] Sends the uploaded document to the server.

[1853] 6. Server:

[1854] The server receives the uploaded document and passes it to the generative AI's learning module.

[1855] 7. Generation AI:

[1856] The generative AI analyzes new documents and adds their contents as learning data, so that new guideline information is reflected in future information provision.

[1857] 8. Server:

[1858] Sends a message to the terminal indicating that the update is complete.

[1859] 9. Terminal:

[1860] The terminal will display a message to the user indicating that the update is complete.

[1861] 10. Users:

[1862] The user checks the displayed message and checks the operation of the system as necessary.

[1863] Utilizing the Emotion Engine

[1864] Example: Responding based on user emotions

[1865] 1. User:

[1866] The user sends an emotional request via text or voice from the device.

[1867] 2. Terminal:

[1868] The terminal receives the request and sends it to the server.

[1869] 3. Server:

[1870] The server receives the request and uses an emotion engine to analyze the user's emotional state.

[1871] 4. Server:

[1872] Based on the emotional information analyzed by the emotion engine, the generative AI adjusts the tone and content of the response. For example, if the user is feeling stressed, it will generate a more friendly response that emphasizes solutions.

[1873] 5. Generation AI:

[1874] Generative AI generates messages based on emotions, such as, "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with."

[1875] 6. Server:

[1876] Send the generated message to the terminal.

[1877] 7. Terminal:

[1878] Display received messages to the user.

[1879] 8. Users:

[1880] The user sees the friendly message and makes further requests if necessary.

[1881] This system enables the provision of information that takes into consideration the user's feelings, contributing to improved business efficiency and user satisfaction.

[1882] The processing flow will be explained below.

[1883] Providing information through an interactive UI

[1884] Example: A scene where you request the minutes of an internal meeting

[1885] Processing Steps

[1886] Step 1:

[1887] User:

[1888] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[1889] Step 2:

[1890] Device:

[1891] The terminal receives the user's request and sends it to the server.

[1892] Step 3:

[1893] server:

[1894] The server receives the request and calls a natural language processing (NLP) module to analyze the request, extracting keywords such as "last month," "management meeting," and "minutes."

[1895] Step 4:

[1896] server:

[1897] Based on the extracted keywords, search for relevant meeting minutes documents from internal databases or external libraries.

[1898] Step 5:

[1899] server:

[1900] The search results are passed to the generation AI.

[1901] Step 6:

[1902] Generation AI:

[1903] Based on the search results, the generative AI generates a conversational message to be delivered to the user, such as, "The minutes of last month's management meeting can be viewed at the link below."

[1904] Step 7:

[1905] server:

[1906] Send the generated message to the terminal.

[1907] Step 8:

[1908] Device:

[1909] Display received messages to the user.

[1910] Step 9:

[1911] User:

[1912] The user can view the minutes of the meeting by clicking the provided link.

[1913] Assistance in creating notes and documents

[1914] Example: Support for taking notes on presentation materials

[1915] Processing Steps

[1916] Step 1:

[1917] User:

[1918] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[1919] Step 2:

[1920] Device:

[1921] The terminal sends a request to the server.

[1922] Step 3:

[1923] server:

[1924] The server receives the request and analyzes it using the NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[1925] Step 4:

[1926] server:

[1927] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[1928] Step 5:

[1929] Generation AI:

[1930] Based on the search results, the generative AI creates a memo template for presentation materials to be provided to the user. For example, it creates a memo that states, "In the next customer briefing, it is important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[1931] Step 6:

[1932] server:

[1933] The generated memo is sent to the terminal.

[1934] Step 7:

[1935] Device:

[1936] Display received notes to the user.

[1937] Step 8:

[1938] User:

[1939] The user can check the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[1940] Additional learning of registered information

[1941] Example: Adding a new internal guideline

[1942] Processing Steps

[1943] Step 1:

[1944] User:

[1945] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[1946] Step 2:

[1947] Device:

[1948] The terminal sends the instruction to the server.

[1949] Step 3:

[1950] server:

[1951] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[1952] Step 4:

[1953] User:

[1954] The user uploads a guideline document.

[1955] Step 5:

[1956] Device:

[1957] Sends the uploaded document to the server.

[1958] Step 6:

[1959] server:

[1960] The server receives the uploaded document and passes it to the generative AI's learning module.

[1961] Step 7:

[1962] Generation AI:

[1963] The generative AI analyzes new documents and adds their contents as learning data, so that new guideline information is reflected in future information provision.

[1964] Step 8:

[1965] server:

[1966] Sends a message to the terminal indicating that the update is complete.

[1967] Step 9:

[1968] Device:

[1969] The terminal will display a message to the user indicating that the update is complete.

[1970] Step 10:

[1971] User:

[1972] The user checks the displayed message and checks the operation of the system as necessary.

[1973] Utilizing the Emotion Engine

[1974] Example: Responding based on user emotions

[1975] Processing Steps

[1976] Step 1:

[1977] User:

[1978] The user sends an emotional request via text or voice from the device.

[1979] Step 2:

[1980] Device:

[1981] The terminal sends a request to the server.

[1982] Step 3:

[1983] server:

[1984] The server receives the request and uses an emotion engine to analyze the user's emotional state.

[1985] Step 4:

[1986] server:

[1987] Based on the emotional information analyzed by the emotion engine, the generative AI adjusts the tone and content of the response. For example, if the user is feeling stressed, it will generate a more friendly response that emphasizes solutions.

[1988] Step 5:

[1989] Generation AI:

[1990] Generative AI generates messages based on emotions, such as, "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with."

[1991] Step 6:

[1992] server:

[1993] Send the generated message to the terminal.

[1994] Step 7:

[1995] Device:

[1996] Display received messages to the user.

[1997] Step 8:

[1998] User:

[1999] The user sees the friendly message and makes further requests if necessary.

[2000] This system enables the provision of information that takes into consideration the user's feelings, contributing to improved business efficiency and user satisfaction.

[2001] Example 2

[2002] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2003] In today's business environment, users are required to quickly and accurately obtain the information they need from multiple sources. It is also important to reduce fatigue and stress by providing appropriate responses based on the user's emotions and circumstances, thereby improving work efficiency and user satisfaction. However, conventional systems have not been able to adequately meet these needs, resulting in reduced user convenience and cumbersome work.

[2004] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for providing information through an interactive user interface, means for analyzing a user request and extracting related keywords, means for searching for related information from an internal database or an external information source, means for providing the user with an interactive message generated based on the extracted information, means for additionally learning documents and notes registered by the user and reflecting them in subsequent request responses, and means for analyzing the user's emotions and generating a response according to the emotions. This allows the user to not only quickly and accurately obtain the information they need, but also receive a response according to their emotions, thereby reducing the complexity of their work and improving work efficiency and user satisfaction.

[2005] An "interactive user interface" refers to a user interface that allows two-way communication through user operations.

[2006] "Means for analyzing user requests and extracting relevant keywords" refers to a function that analyzes requests sent by users using technologies such as natural language processing and extracts important language elements.

[2007] "Means of searching for relevant information from internal databases and external information sources" refers to the function of searching for relevant information from databases inside and outside the system based on extracted keywords.

[2008] The "means for providing a generated interactive message to the user" refers to a function for providing a response to the user in the form of a message generated based on the search results.

[2009] "Means for additional learning of documents and notes registered by the user and reflecting them in subsequent request responses" refers to the function by which the system learns newly registered information by the user and uses it in subsequent responses.

[2010] "Means for analyzing the user's emotions and generating a response that corresponds to those emotions" refers to a function that determines the emotions from the user's request and generates a response that is appropriate to those emotions.

[2011] This invention relates to an AI system that integrates memos and documents, providing information through an interactive user interface to improve the efficiency of users' work. This system has the ability to analyze user requests and quickly provide necessary information from relevant information sources, as well as recognize the user's emotions and respond accordingly.

[2012] Providing information through an interactive UI

[2013] Hardware and Software Use

[2014] User device: personal computer, smartphone, tablet, etc. Uses a microphone for voice input and a keyboard for text input.

[2015] Server: A server equipped with a high-performance processor and a large amount of memory. RDBMS or NoSQL databases are used.

[2016] NLP Modules: Software modules for natural language processing, such as SpaCy and NLTK.

[2017] Generative AI: High-performance generative AI models, such as OpenAI's GPT-3.

[2018] Specific examples

[2019] 1. User: The user makes a request from their terminal saying, "I want to see the minutes of last month's management meeting."

[2020] 2. Terminal: The terminal sends a request to the server.

[2021] 3. Server: The server analyzes the request using a natural language processing module and extracts keywords such as "last month," "management meeting," and "minutes."

[2022] 4. Server: The server uses these keywords to search for relevant information from internal databases and external sources, using a search engine such as Elasticsearch.

[2023] 5. Server: Passes search results to the generation AI.

[2024] 6. Generative AI: The Generative AI generates a conversational message saying, "The minutes of last month's management meeting can be viewed at the link below."

[2025] 7. Server: Sends the generated message to the terminal.

[2026] 8. Terminal: displays received messages to the user.

[2027] Prompt Sentence Examples

[2028] "Show me the minutes of last month's management meeting."

[2029] Assistance in creating notes and documents

[2030] Specific examples

[2031] 1. User: The user requests from the terminal, "I want to create notes for the presentation materials for the next customer briefing."

[2032] 2. Terminal: The terminal sends a request to the server.

[2033] 3. Server: The server analyzes the request using a natural language processing module and extracts keywords such as "customer briefing," "presentation materials," and "memo."

[2034] 4. Server: The server searches the user's past documents and notes from its internal database.

[2035] 5. Generative AI: Generative AI creates memo templates for presentation materials based on search results. For example, it generates a memo saying, "In the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[2036] 6. Server: Sends the generated memo to the device.

[2037] 7. Terminal: Displays the received memo to the user.

[2038] 8. User: The user checks the displayed notes and makes edits or additions as necessary.

[2039] Prompt Sentence Examples

[2040] "Please prepare notes for the next client briefing."

[2041] Additional learning of registered information

[2042] Specific examples

[2043] 1. User: The user issues a command from their device saying, "I want to add new internal guidelines to the AI ​​and have it learn them."

[2044] 2. Terminal: The terminal sends instructions to the server.

[2045] 3. Server: The server receives the instructions and sends a message to the user prompting them to upload their guidelines.

[2046] 4. User: The user uploads the guideline document.

[2047] 5. Terminal: Sends the uploaded document to the server.

[2048] 6. Server: The server receives the document and passes it to the generative AI's learning module.

[2049] 7. Generative AI: Generative AI analyzes new documents and adds their contents to the learning data. This allows new guideline information to be reflected in future information provision.

[2050] 8. Server: Sends a message to the device indicating that the update is complete.

[2051] 9. Terminal: The terminal displays a message to the user that the update is complete.

[2052] Prompt Sentence Examples

[2053] "Learn these new guidelines and incorporate them into your next search."

[2054] Utilizing the Emotion Engine

[2055] Specific examples

[2056] 1. User: The user sends an emotional request from the device, for example, with an elevated tone of voice or emotional text.

[2057] 2. Terminal: The terminal sends a request to the server.

[2058] 3. Server: The server receives the request and uses an emotion engine to analyze the user's emotional state, for example determining that the user is tired.

[2059] 4. Server: Based on the emotional information analyzed by the emotion engine, the generative AI adjusts the tone and content of the response, for example, generating a friendly message that emphasizes solutions.

[2060] 5. Generative AI: Generative AI generates messages based on emotions, such as "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with."

[2061] 6. Server: Sends the generated message to the terminal.

[2062] 7. Terminal: Displays received messages to the user.

[2063] Prompt Sentence Examples

[2064] "I'm very busy, but could you please give me a quick summary of the recent meeting?"

[2065] With these functions, the system streamlines users' work and provides necessary information quickly and accurately. It also responds with consideration for the user's feelings, improving work efficiency and user satisfaction.

[2066] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2067] Providing information through an interactive UI

[2068] Example: A scene where you request the minutes of an internal meeting

[2069] Step 1:

[2070] User: The user sends a request from their device saying, "I want to see the minutes of last month's management meeting." This operation is performed by the user either typing text or making a request by voice. Input is done using the device's keyboard and microphone.

[2071] Step 2:

[2072] Terminal: The terminal sends the received request to the server using an HTTP POST request or WebSocket. The data is encoded in JSON format.

[2073] Step 3:

[2074] Server: The server receives the request and analyzes it using a natural language processing (NLP) module. This process extracts keywords such as "last month," "management meeting," and "minutes" from the request text. The input is the request data in JSON format, and the output is a list of keywords. The specific software used is SpaCy and NLTK.

[2075] Step 4:

[2076] Server: The server searches for relevant information from internal databases and external sources based on the extracted keywords. This search is performed using a search engine such as Elasticsearch. The input is a list of keywords, and the output is a set of related information.

[2077] Step 5:

[2078] Server: The server passes the search results to the generative AI. The input is a set of related information, and the output is a response message generated by the generative AI. Specifically, OpenAI's GPT-3 is used.

[2079] Step 6:

[2080] Generative AI: Based on the search results, the generative AI generates a conversational message such as, "The minutes of last month's management meeting can be viewed at the link below." The input is a set of relevant information, and the output is a response message.

[2081] Step 7:

[2082] Server: The server sends the generated message to the terminal. HTTP or WebSocket is used for transmission. The input is the response message, and the output is the transmission completion status.

[2083] Step 8:

[2084] Terminal: A terminal displays received messages to the user, such as pop-up notifications or chat windows on the UI. Input is a response message, and output is displayed in the user interface.

[2085] Step 9:

[2086] User: The user is presented with a link to view the minutes. When the user clicks on the link, their browser opens a new tab or window with the relevant document.

[2087] Assistance in creating notes and documents

[2088] Example: Support for taking notes on presentation materials

[2089] Step 1:

[2090] User: The user requests, "I want to create notes for the presentation materials for the next client briefing." This can be done by typing text or by making a voice request.

[2091] Step 2:

[2092] Terminal: The terminal sends the received request to the server using an HTTP POST request or WebSocket. The data is encoded in JSON format.

[2093] Step 3:

[2094] Server: The server receives the request and analyzes it using the NLP module. Keywords such as "customer briefing," "presentation materials," and "memo" are extracted through the analysis. The input is the request data in JSON format, and the output is a list of keywords.

[2095] Step 4:

[2096] Server: The server searches the user's past documents and notes from an internal database based on the extracted keywords. The search engine used is Elasticsearch. The input is a list of keywords, and the output is a set of related documents and notes.

[2097] Step 5:

[2098] Generative AI: Generative AI creates memo templates for presentation materials based on search results. For example, it generates memos such as, "At the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session." The input is related materials and memos, and the output is a memo template.

[2099] Step 6:

[2100] Server: Sends the generated memo to the device. HTTP POST request or WebSocket is used for sending. The input is the memo template, and the output is the sending completion status.

[2101] Step 7:

[2102] Terminal: Displays received memos to the user. Specifically, it displays them in the memo field on the UI or in notifications. The input is the memo template, and the output is displayed in the user interface.

[2103] Step 8:

[2104] User: The user checks the displayed notes, edits and supplements them as necessary, and creates the final presentation materials.

[2105] Additional learning of registered information

[2106] Example: Adding a new internal guideline

[2107] Step 1:

[2108] User: The user issues a command from the device saying, "I want to add new internal guidelines to the AI ​​and have it learn them." This operation is performed by the user entering text or giving voice instructions.

[2109] Step 2:

[2110] Terminal: The terminal sends instructions to the server using HTTP POST requests or WebSockets. The data is encoded in JSON format.

[2111] Step 3:

[2112] Server: The server receives the instructions and sends a message to the user prompting them to upload the guideline document. The input is the instructions, and the output is the upload prompt message.

[2113] Step 4:

[2114] User: The user uploads a guideline document. The operation is performed by selecting the file through a file selection dialog. The input is the document file, and the output is the uploaded file.

[2115] Step 5:

[2116] Terminal: Sends the uploaded document to the server using an HTTP POST request. The input is the uploaded file, and the output is the status of the successful submission.

[2117] Step 6:

[2118] Server: The server passes the received document to the generative AI learning module. The input is the document file, and the output is the learning data.

[2119] Step 7:

[2120] Generative AI: The generative AI analyzes new documents and adds their contents as training data. This allows new guideline information to be reflected in subsequent information provision. The input is training data, and the output is updated training data.

[2121] Step 8:

[2122] Server: Sends an update completion message to the terminal. The input is the updated training data, and the output is the update completion message.

[2123] Step 9:

[2124] Terminal: The terminal displays a message to the user that the update is complete. The input is the message, and the output is what is displayed in the user interface.

[2125] Step 10:

[2126] User: The user checks the displayed messages and checks the system operation if necessary.

[2127] Utilizing the Emotion Engine

[2128] Example: Responding based on user emotions

[2129] Step 1:

[2130] User: The user sends an emotional request from the device, for example, by typing in a voice with a high tone or by typing in a text that expresses emotion. The input is done using the device's keyboard and microphone.

[2131] Step 2:

[2132] Terminal: The terminal receives the request and sends it to the server using an HTTP POST request or WebSocket. The data is encoded in JSON format.

[2133] Step 3:

[2134] Server: The server receives the request and uses an emotion engine to analyze the user's emotional state. Emotion analysis uses techniques to extract emotions from text and speech. The input is the request data in JSON format, and the output is the emotional status.

[2135] Step 4:

[2136] Server: Based on the emotional information analyzed by the emotion engine, the generation AI adjusts the tone and content of the response. For example, if the user is tired, it generates a message in a friendly tone that emphasizes solutions. The input is the emotional status, and the output is the response message.

[2137] Step 5:

[2138] Generative AI: Generative AI generates messages based on emotions. For example, it generates a message like, "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with." The input is the emotional status, and the output is a response message.

[2139] Step 6:

[2140] Server: Sends the generated message to the terminal. HTTP POST request or WebSocket is used for sending. The input is the response message, and the output is the status of sending completion.

[2141] Step 7:

[2142] Terminal: A terminal displays received messages to the user, such as in a popup notification or a chat window. Input is a response message, and output is displayed in the user interface.

[2143] Step 8:

[2144] User: The user reviews the friendly message and makes a follow-up request if necessary, by retyping the request through the chat interface.

[2145] The specific operations, data processing, and data calculations in each processing step have been described above.

[2146] (Application example 2)

[2147] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[2148] Conventional information provision systems often found it difficult to properly analyze user requests and quickly provide relevant information. Furthermore, they were unable to respond in a way that took the user's emotions into account, limiting the improvement of user satisfaction. Furthermore, the system's ability to effectively add and learn new information registered by the user and reflect it in subsequent requests was incomplete, creating a need for efficient customer service in physical stores.

[2149] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for providing information through an interactive operation screen, means for analyzing a user request and extracting related keywords, means for searching for related information from internal data or an external database, means for providing the user with an interactive message generated based on the extracted information, means for additionally learning information registered by the user and reflecting the information in subsequent requests and responses, and means for analyzing the user's emotional state and adjusting the tone and content of the response based on the learning. This enables a quick and accurate response to a user request, and further enables a response that takes the user's emotions into consideration, thereby improving user satisfaction.

[2150] An "interactive operation screen" is an interface that allows a user to interact with the system through the screen and receive information.

[2151] "Means of providing information" refers to the function that the all-memo / document integrated AI system uses to provide the necessary information to users.

[2152] The "means for analyzing a user request and extracting relevant keywords" refers to a method and device for analyzing the content of a user request and identifying important keywords.

[2153] "Means for searching for relevant information from internal data and external databases" is a function for searching for appropriate information from sources inside and outside the system based on identified keywords.

[2154] The "means for providing a user with a message in an interactive format" is a function for providing information to the user in an easy-to-understand message format based on the extracted and searched information.

[2155] The "means for additionally learning information registered by the user and reflecting it in subsequent request responses" is a function for learning information newly added by the user and using it to provide future information.

[2156] "Means for analyzing a user's emotional state and adjusting the tone and content of a response based thereon" refers to a method and apparatus for analyzing a user's emotions and providing a response with the optimal tone and content according to those emotions.

[2157] An "interactive user interface" is an interface that allows a user to operate and obtain information through natural interaction with a system.

[2158] This invention relates to an AI system that integrates all notes and documents, providing information through an interactive user interface and streamlining user work. The system mainly includes the following hardware and software:

[2159] Required Hardware and Software

[2160] Smart glasses: Devices worn by store staff that capture customers' questions and facial expressions in real time.

[2161] Server: A computer server that processes data, performs generative AI, and performs natural language processing (NLP).

[2162] Python: A programming language.

[2163] Transformers Library: A library for NLP tasks.

[2164] cv2 (OpenCV): Computer vision library.

[2165] requests: A library for sending HTTP requests.

[2166] System Operation

[2167] 1. Analyzing user requests and providing information

[2168] The server receives requests from users (store staff) through the smart glasses. The requests are input in voice or text format. The server then analyzes the requests using natural language processing (NLP) and extracts relevant keywords. It then searches for relevant information in internal and external databases and uses generative AI to generate conversational messages. This allows users to easily answer customer questions.

[2169] For example, if a customer asks, "What color variations of this product are there?", the server retrieves the product information and generates the answer, "This product comes in red, blue, and green." An example prompt sentence is as follows:

[2170] Question: "What color variations are available for this item?"

[2171] Context: "Product ABC is available in red, blue, and green colors. Details for each color are as follows..."

[2172] 2. Sentiment Analysis and Response Adjustment

[2173] The server captures the customer's facial expressions and tone of voice through the smart glasses and inputs them into an emotion analysis engine. Based on the analysis results, the customer's emotional state is identified. The generative AI then adjusts the tone and content of the response according to the identified emotional state. This allows the response to take the customer's emotions into consideration.

[2174] For example, if a customer says, "I'm a little disappointed with this product," the sentiment analysis engine will identify the "disappointed" emotion, and the generative AI will generate a response such as, "I'm sorry you're disappointed. Would you like us to find another product that meets your expectations?"

[2175] Examples of concrete examples and prompts

[2176] Below is a concrete example in which a user responds to a customer question using a prompt generated by the server.

[2177] Question: "What material is this item made of?"

[2178] Context: "Product XYZ is made from high-quality cotton and polyester. Its specific composition is as follows..."

[2179] In this way, the present invention enables store staff to respond to customer questions quickly and accurately, and also enables responses that are in line with the customer's feelings, which is expected to improve user satisfaction.

[2180] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2181] Step 1:

[2182] The server receives user requests from the smart glasses. The user inputs the request in voice or text format, and the device sends it to the server. The input data is the user's question or instruction.

[2183] Step 2:

[2184] After receiving the request, the server analyzes it using a natural language processing (NLP) module. Specifically, it breaks down the sentences and extracts keywords. As a result of this analysis, important keywords are output.

[2185] Step 3:

[2186] The server searches for relevant information from internal and external databases based on the extracted keywords, such as product information, instructions, and related notes. The server searches this data through API requests and obtains search results.

[2187] Step 4:

[2188] The server generates a dialogue-style message using a generative AI model based on the search results. The generative AI model uses the context information and extracted keywords as input. The generative AI model generates a response message to be provided to the user and returns the output to the server.

[2189] Step 5:

[2190] The server sends the generated message to the terminal, which receives the message and provides it to the user by display or voice. The content of the message is a specific answer to the user's question.

[2191] Step 6:

[2192] The server captures the emotional state of the user or customer through the smart glasses. The emotion analysis engine analyzes the data based on voice input and facial expression capture data. The input data is audio and video capture data.

[2193] Step 7:

[2194] The server uses a generative AI model to adjust the tone and content of the response based on the results of the emotion analysis engine. Specifically, it generates a response appropriate for when the user is feeling stressed. The input is the analysis result, and the output is the adjusted response message.

[2195] Step 8:

[2196] The server sends the tailored response message to the terminal, which displays the received message and provides a specific response to the user, who then responds to the customer based on the response.

[2197] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2198] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2199] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2200] [Fourth embodiment]

[2201] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2202] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2203] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2204] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2205] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2206] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2207] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2208] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2209] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2210] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2211] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2212] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2213] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2214] This invention relates to an AI system that integrates all notes and documents, providing information through an interactive user interface and streamlining user work. This system has the ability to analyze user requests and provide relevant information quickly and accurately. It can also learn additional information from documents and notes registered by the user, improving the accuracy of the information provided.

[2215] Providing information through an interactive UI

[2216] Example: A scene where you request the minutes of an internal meeting

[2217] 1. User:

[2218] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[2219] 2. Terminal:

[2220] The terminal receives this request and sends it to the server.

[2221] 3. Server:

[2222] The server analyzes the received request through a natural language processing (NLP) module, extracting keywords such as "last month," "management meeting," and "minutes."

[2223] 4. Server:

[2224] Based on the extracted keywords, the system searches for relevant meeting minutes documents from internal databases and external libraries. When relevant documents are found, the information is passed to the generation AI.

[2225] 5. Generation AI:

[2226] Based on the search results, the AI ​​generates a conversational message to be provided to the user, such as "The minutes of last month's management meeting can be viewed at the link below."

[2227] 6. Server:

[2228] Send the generated message to the terminal.

[2229] 7. Terminal:

[2230] Display received messages to the user.

[2231] 8. Users:

[2232] The user can view the minutes of the meeting in question by clicking the provided link.

[2233] Assistance in creating notes and documents

[2234] Example: Support for taking notes on presentation materials

[2235] 1. User:

[2236] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[2237] 2. Terminal:

[2238] The terminal sends this request to the server.

[2239] 3. Server:

[2240] The server receives the request and analyzes it using the NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[2241] 4. Server:

[2242] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[2243] 5. Generation AI:

[2244] Based on the search results, the AI ​​creates a memo template for presentation materials to be provided to the user. It creates specific memos such as, "At the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[2245] 6. Server:

[2246] The generated memo is sent to the terminal.

[2247] 7. Terminal:

[2248] Display received notes to the user.

[2249] 8. Users:

[2250] Users can review the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[2251] Additional learning of registered information

[2252] Example: Adding a new internal guideline

[2253] 1. User:

[2254] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[2255] 2. Terminal:

[2256] The terminal transmits the user's instructions to the server.

[2257] 3. Server:

[2258] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[2259] 4. User:

[2260] The user uploads a guideline document.

[2261] 5. Terminal:

[2262] The uploaded document is sent to the server.

[2263] 6. Server:

[2264] The server receives the uploaded document and passes it to the generative AI's learning module.

[2265] 7. Generation AI:

[2266] The generative AI analyzes new documents and adds their contents as learning data, so that the new guideline information will be reflected in future information provision.

[2267] 8. Server:

[2268] Sends a message to the terminal indicating that the update is complete.

[2269] 9. Terminal:

[2270] The terminal will display a message to the user indicating that the update is complete.

[2271] 10. Users:

[2272] The user can check the displayed messages and check the operation of the system as needed.

[2273] This enables the system to provide users with information that is optimized for them, greatly contributing to improved business efficiency.

[2274] The processing flow will be explained below.

[2275] Providing information through an interactive UI

[2276] Example: A scene where you request the minutes of an internal meeting

[2277] Processing Steps

[2278] Step 1:

[2279] User:

[2280] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[2281] Step 2:

[2282] Device:

[2283] The terminal receives the user's request and sends it to the server.

[2284] Step 3:

[2285] server:

[2286] The server receives the request and calls a natural language processing (NLP) module to parse the request, extracting keywords such as "last month," "management meeting," and "minutes."

[2287] Step 4:

[2288] server:

[2289] Based on the extracted keywords, search for relevant meeting minutes documents from internal databases or external libraries.

[2290] Step 5:

[2291] server:

[2292] The search results are passed to the generation AI.

[2293] Step 6:

[2294] Generation AI:

[2295] Based on the search results, the generative AI generates a conversational message to be delivered to the user, such as, "The minutes of last month's management meeting can be viewed at the link below."

[2296] Step 7:

[2297] server:

[2298] Send the generated message to the terminal.

[2299] Step 8:

[2300] Device:

[2301] Display received messages to the user.

[2302] Step 9:

[2303] User:

[2304] The user can view the minutes of the meeting by clicking the provided link.

[2305] Assistance in creating notes and documents

[2306] Example: Support for taking notes on presentation materials

[2307] Processing Steps

[2308] Step 1:

[2309] User:

[2310] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[2311] Step 2:

[2312] Device:

[2313] The terminal sends this request to the server.

[2314] Step 3:

[2315] server:

[2316] The server receives the request and analyzes it using the NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[2317] Step 4:

[2318] server:

[2319] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[2320] Step 5:

[2321] Generation AI:

[2322] Based on the search results, the AI ​​creates a memo template for presentation materials to be provided to the user. For example, it might create a memo such as, "At the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[2323] Step 6:

[2324] server:

[2325] The generated memo is sent to the terminal.

[2326] Step 7:

[2327] Device:

[2328] Display received notes to the user.

[2329] Step 8:

[2330] User:

[2331] The user can check the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[2332] Additional learning of registered information

[2333] Example: Adding a new internal guideline

[2334] Processing Steps

[2335] Step 1:

[2336] User:

[2337] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[2338] Step 2:

[2339] Device:

[2340] The terminal transmits the user's instructions to the server.

[2341] Step 3:

[2342] server:

[2343] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[2344] Step 4:

[2345] User:

[2346] The user uploads a guideline document.

[2347] Step 5:

[2348] Device:

[2349] Sends the uploaded document to the server.

[2350] Step 6:

[2351] server:

[2352] The server receives the uploaded document and passes it to the generative AI's learning module.

[2353] Step 7:

[2354] Generation AI:

[2355] The generative AI analyzes new documents and adds their contents as learning data, so that new guideline information is reflected in future information provision.

[2356] Step 8:

[2357] server:

[2358] A message indicating that learning is complete is sent to the device.

[2359] Step 9:

[2360] Device:

[2361] The terminal displays a message to the user indicating that learning is complete.

[2362] Step 10:

[2363] User:

[2364] The user checks the displayed message and checks the operation of the system as necessary.

[2365] Example 1

[2366] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2367] Existing information provision systems often have delays in responding to user requests or provide inaccurate information. Furthermore, even when a user adds new information, it takes time for that information to be reflected in the system. Furthermore, few systems can also respond to voice commands, which limits user convenience.

[2368] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2369] In this invention, the server includes a means for analyzing the request through a natural language processing module and extracting relevant keywords, a means for searching for relevant information from an internal database or external information source based on the extracted keywords, and a means for passing the search results to a generative AI model to generate a dialogue-style message. This enables a quick and accurate response to a user's request. Furthermore, even if new information is added, the generative AI model can instantly learn from it and reflect it in subsequent request responses. Furthermore, support for voice instructions improves user convenience.

[2370] "User" means an individual or organization that uses the system.

[2371] A "terminal" is a device used by a user to send requests or receive information.

[2372] A "request" is a request for information made by a user to the system.

[2373] A "natural language processing module" is a software component that analyzes the text of a request and extracts meaning and keywords.

[2374] "Keywords" are important words or phrases extracted from a user's request.

[2375] An "internal database" is a database within the system where information such as user documents and notes is stored.

[2376] "External sources" refers to information obtained from libraries or databases outside the system.

[2377] A "generative AI model" is an artificial intelligence model that automatically generates messages to be provided to users based on search results and learning data.

[2378] "Interactive messaging" refers to messaging that is in a natural format for interacting with a user.

[2379] A "new document or note" is new information that a user adds to the system.

[2380] "Learning" is the process by which a generative AI model takes in new information and uses it to improve its responses in future scenarios.

[2381] "Voice instructions" are instructions sent by the user to the system through voice.

[2382] This invention relates to an AI system that integrates all memos and documents, providing information through an interactive user interface and streamlining user work. This system has the ability to analyze user requests and provide relevant information quickly and accurately. It can also learn from documents and memos registered by users to improve the accuracy of the information provided.

[2383] The following hardware and software are used to implement the present invention.

[2384] 1. Hardware

[2385] Device: The device through which the user enters the request (e.g., PC, smartphone, tablet).

[2386] Server: A server that analyzes requests, searches for information, generates messages, etc.

[2387] 2. Software

[2388] Natural Language Processing Module: Software for analyzing requests. Specifically, it uses the Google Cloud Natural Language API.

[2389] Internal database: A database that stores information such as user notes and documents. Specifically, MongoDB is used.

[2390] Generative AI model: An artificial intelligence model that generates messages to be provided to users. Specifically, we use OpenAI's GPT-4.

[2391] Examples of concrete examples and prompts

[2392] 1. Requesting minutes of an internal meeting

[2393] User: The user sends a request from their terminal saying, "I want to see the minutes of last month's management meeting."

[2394] Terminal: The terminal receives the user's request and sends it to the server.

[2395] Server: The server parses the request using the Google Cloud Natural Language API and extracts keywords such as "last month," "management meeting," and "minutes."

[2396] The server searches for relevant meeting minutes from MongoDB based on keywords and passes them to the generative AI model.

[2397] Based on the search results, a generative AI model (OpenAI GPT-4) generates a message such as, "The minutes of last month's management meeting can be viewed at the link below."

[2398] Server: Sends the generated message to the device.

[2399] Terminal: Displays received messages to the user.

[2400] User: View the minutes via the provided link.

[2401] Example prompt sentence:

[2402] "I'd like to see the minutes of last month's management meeting."

[2403] 2. Support for taking notes on presentation materials

[2404] User: The user makes a request from their device saying, "I want to create notes for the presentation materials for the next customer briefing."

[2405] Device: The device sends a request to the server.

[2406] Server: The server analyzes the request using the Google Cloud Natural Language API and extracts keywords such as "customer briefing," "presentation materials," and "memo."

[2407] The server searches MongoDB for relevant past documents and notes based on the extracted keywords and passes them to the generative AI model.

[2408] A generative AI model (OpenAI GPT-4) generates notes such as, "At your next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[2409] Server: Sends the generated memo to the device.

[2410] Terminal: Displays received notes to the user.

[2411] User: Review, edit and supplement notes, and create final presentation materials.

[2412] Example prompt sentence:

[2413] "I want to make notes for the presentation materials for the next client briefing."

[2414] 3. Addition of new internal guidelines

[2415] User: The user submits a request to add a new internal guideline document to the AI ​​and have it learn it.

[2416] Device: The device sends a request to the server.

[2417] Server: The server receives the request and sends a message to the user prompting them to upload a guideline document.

[2418] User: Upload a guideline document.

[2419] Terminal: Sends the document to the server.

[2420] Server: The server receives the document and passes it to the learning module of the generative AI model, which then learns from it and applies it to future responses.

[2421] Server: Sends a message to the device indicating that the update is complete.

[2422] Terminal: Display a message to the user that the update is complete.

[2423] User: Check the displayed messages and check the system operation.

[2424] Example prompt sentence:

[2425] "We want to add new internal guideline documents to the AI ​​and have it learn them."

[2426] As described above, the present invention is a system that combines a natural language processing module, an internal database, and a generative AI model to provide efficient information provision, enabling quick and accurate responses to user requests and improving business efficiency.

[2427] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2428] Step 1:

[2429] The user sends a request from the terminal.

[2430] Example: A user types or voice requests, "I'd like to see the minutes of last month's management meeting."

[2431] Input: User's text or voice request.

[2432] Output: Received request on terminal.

[2433] Step 2:

[2434] The terminal receives the user's request and sends it to the server.

[2435] Example: A device sends a request to the server saying, "I want to see the minutes of last month's management meeting."

[2436] Input: The user request.

[2437] Output: The request sent to the server.

[2438] Step 3:

[2439] The server analyzes the received request through a natural language processing module.

[2440] Example: The server passes the request to the Google Cloud Natural Language API for parsing.

[2441] Input: The user request received from the device.

[2442] Output: Keywords such as "last month", "management meeting", and "minutes".

[2443] Step 4:

[2444] The server searches for relevant information from internal databases and external information sources based on the extracted keywords.

[2445] Example: The server searches MongoDB for documents matching "last month", "management meeting", and "minutes".

[2446] Input: Extracted keywords.

[2447] Output: A document of the relevant minutes.

[2448] Step 5:

[2449] The server passes the search results to a generative AI model, which generates conversational messages.

[2450] Example: The server uses the search results to generate a message using a generative AI model (OpenAI GPT-4).

[2451] Input: Relevant minutes document.

[2452] Output: The message "The minutes of last month's management meeting can be found at the link below."

[2453] Step 6:

[2454] The server sends the generated message to the terminal.

[2455] Example: The server sends a generated message to the user terminal.

[2456] Input: Message from the generative AI model.

[2457] Output: Messages sent to the terminal.

[2458] Step 7:

[2459] The terminal displays the received message to the user.

[2460] Example: The device displays the message "The minutes of last month's management meeting can be viewed at the link below."

[2461] Input: The message sent by the server.

[2462] Output: The message displayed to the user.

[2463] Step 8:

[2464] The user checks the minutes of the meeting from the provided link.

[2465] Example: A user clicks on a displayed link to view the meeting minutes.

[2466] Input: The link displayed on the terminal.

[2467] Output: The minutes to be viewed.

[2468] (Application example 1)

[2469] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2470] Conventional information provision systems using interactive user interfaces have had difficulty in providing prompt and appropriate information in response to user requests. In particular, in virtual and physical stores, the lack of a function to provide product information desired by users in real time limited the user experience. Therefore, a system that allows users to quickly and accurately obtain the information they desire is needed.

[2471] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2472] In this invention, the server includes means for providing information through an interactive user interface, means for analyzing a user request and extracting related keywords, means for searching for related information from internal and external information sources, means for providing the user with an interactive message generated based on the search results, means for additionally learning documents and notes registered by the user and reflecting them in subsequent request responses, and means for providing product information in a virtual physical store. This allows users to quickly and accurately obtain product information in real time even in a virtual physical store.

[2473] An "interactive user interface" refers to a mechanism that allows two-way communication between the user and the system.

[2474] "Means for extracting relevant keywords" refers to a method for identifying and extracting important words and phrases from the user's request.

[2475] "Internal sources" refers to information stored within a company or organization, such as databases, documents, and notes.

[2476] "External sources" refer to information that exists outside the organization, such as external libraries or databases on the Internet.

[2477] "Conversational messages" refer to text that provides information to users in the form of natural conversation.

[2478] "Virtual brick-and-mortar store" refers to a virtual shopping environment operated on the Internet.

[2479] "Means for additional learning from registered documents and notes" refers to a method by which the system incorporates new information provided by the user and uses this information to improve the accuracy of subsequent answers.

[2480] "Means for providing product information" refers to a method for displaying data such as product details, reviews, and use examples to a user in response to a user's request.

[2481] "Generative AI" refers to artificial intelligence technology that generates new data and sentences based on learned data.

[2482] To implement this invention, the following system program is required: This system is composed of the following elements: a server, a terminal, and a user.

[2483] System Program

[2484] The program mainly consists of the following measures:

[2485] 1. Means of providing information through an interactive user interface:

[2486] The server provides an interactive user interface that accepts user requests. This interface supports voice and text input. The user interface is designed to allow users to enter the desired information in a natural, interactive manner.

[2487] 2. A method to analyze user requests and extract relevant keywords:

[2488] The server receives requests from users and analyzes them using a natural language processing (NLP) module. The analysis process involves extracting important keywords from the input sentence. The NLP module uses the Transformers library from Huggingface.

[2489] 3. Means of retrieving relevant information from internal and external sources:

[2490] The server searches for relevant information based on the extracted keywords from an internal database, which includes documents, memos, and minutes from within the company or organization, and from external sources, such as databases and libraries on the Internet.

[2491] 4. A means of providing users with interactive messages generated based on search results:

[2492] After searching for relevant information, the results are sent to the generation AI, which then generates a conversational message to be provided to the user based on the search results. The generation AI uses OpenAI's text-davinci-003 model.

[2493] 5. A way to learn additional documents and notes registered by the user and reflect them in subsequent request responses:

[2494] The server passes documents and notes uploaded by users to the machine learning module for additional learning, allowing the system to improve accuracy in subsequent information provision.

[2495] 6. Means of providing product information in virtual and physical stores:

[2496] As a means of providing users with the product information they require in real time in a virtual physical store, the server displays product details, reviews, usage examples, etc. based on user requests.

[2497] Hardware and Software

[2498] Server: The server acts as the central processing unit and performs all data calculations and processing.

[2499] Terminal: A device that provides a user interface, including smartphones and head-mounted displays (HMDs).

[2500] Software: Huggingface's Transformers library, OpenAI's text-davinci-003 model.

[2501] Specific examples

[2502] User Request: "I want to see the latest smartphones"

[2503] System behavior:

[2504] 1. The user makes a voice or text request on their smartphone or HMD saying, "I want to see the latest smartphones."

[2505] 2. The request is sent to the server and parsed by the NLP module.

[2506] 3. Keywords such as "latest" and "smartphone" are extracted.

[2507] 4. The server retrieves relevant product information from internal databases and external sources.

[2508] 5. Generative AI creates conversational messages based on search results.

[2509] 6. A message such as "Here are the latest smartphones: Model X, Model Y" will appear on the user's device.

[2510] Example of input prompt for generative AI model:

[2511] User request: "I want to see the latest smartphones."

[2512] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2513] Step 1:

[2514] The user sends a request from the device by voice or text, for example, "I want to see the latest smartphones." This request is entered into the device and sent to the server.

[2515] Step 2:

[2516] The server receives requests from the device and analyzes them using a natural language processing (NLP) module. Specifically, it extracts important keywords such as "latest" and "smartphone" from the request text. This process uses the Transformers library from Huggingface. The input is the user's request text, and the output is a list of extracted keywords.

[2517] Step 3:

[2518] The server searches for relevant information from internal databases and external information sources based on the extracted keywords. The internal database stores company documents, memos, minutes, etc. The external information sources include databases and libraries on the Internet. The input is a list of keywords, and the output is the search results data.

[2519] Step 4:

[2520] The server passes the search results to a generation AI, which creates an interactive message to be provided to the user. This generation AI uses OpenAI's text-davinci-003 model. Based on the search results, the generation AI generates an appropriate message to provide information to the user. The input is the search result data, and the output is an interactive message.

[2521] Step 5:

[2522] The server sends the generated interactive message to the terminal, which displays this message to the user. For example, a message like "Here are the latest smartphones: Model X, Model Y" is displayed. The input is the generated message, and the output is the message displayed to the user.

[2523] Step 6:

[2524] New documents and notes registered by users are uploaded to the server via their devices. The server passes these to the machine learning module as additional learning data, which improves the accuracy of subsequent information provision. The input is the uploaded documents and notes, and the output is updated learning data.

[2525] Step 7:

[2526] When a user requests product information in a virtual store, the server provides product details, reviews, usage examples, etc. in real time. Information can be easily delivered to the user via the terminal. The input is the user's request, and the output is the provision of product information.

[2527] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2528] This invention relates to an AI system that integrates all notes and documents, providing information through an interactive user interface and streamlining user work. This system not only analyzes user requests and quickly provides necessary information from relevant information sources, but also recognizes the user's emotions and responds accordingly. Specifically, by additionally learning from documents and notes that the user has looked up and using an emotion engine, the system provides optimal information and responses for the user.

[2529] Providing information through an interactive UI

[2530] Example: A scene where you request the minutes of an internal meeting

[2531] 1. User:

[2532] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[2533] 2. Terminal:

[2534] The terminal sends a request to the server.

[2535] 3. Server:

[2536] The server receives the request and parses it through a natural language processing (NLP) module, extracting keywords such as "last month," "management meeting," and "minutes."

[2537] 4. Server:

[2538] Based on the extracted keywords, search for relevant meeting minutes documents from internal databases or external libraries.

[2539] 5. Server:

[2540] The search results are passed to the generation AI.

[2541] 6. Generation AI:

[2542] Based on the search results, the generative AI generates a conversational message to be delivered to the user, such as, "The minutes of last month's management meeting can be viewed at the link below."

[2543] 7. Server:

[2544] Send the generated message to the terminal.

[2545] 8. Terminal:

[2546] Display received messages to the user.

[2547] 9. Users:

[2548] The user can view the minutes of the meeting by clicking the provided link.

[2549] Assistance in creating notes and documents

[2550] Example: Support for taking notes on presentation materials

[2551] 1. User:

[2552] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[2553] 2. Terminal:

[2554] The terminal sends a request to the server.

[2555] 3. Server:

[2556] The server receives the request and analyzes it using an NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[2557] 4. Server:

[2558] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[2559] 5. Generation AI:

[2560] Based on the search results, the generative AI creates a memo template for presentation materials to be provided to the user. For example, it creates a memo that states, "In the next customer briefing, it is important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[2561] 6. Server:

[2562] The generated memo is sent to the terminal.

[2563] 7. Terminal:

[2564] Display received notes to the user.

[2565] 8. Users:

[2566] The user can check the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[2567] Additional learning of registered information

[2568] Example: Adding a new internal guideline

[2569] 1. User:

[2570] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[2571] 2. Terminal:

[2572] The terminal sends the instruction to the server.

[2573] 3. Server:

[2574] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[2575] 4. User:

[2576] The user uploads a guideline document.

[2577] 5. Terminal:

[2578] Sends the uploaded document to the server.

[2579] 6. Server:

[2580] The server receives the uploaded document and passes it to the generative AI's learning module.

[2581] 7. Generation AI:

[2582] The generative AI analyzes new documents and adds their contents as learning data, so that new guideline information is reflected in future information provision.

[2583] 8. Server:

[2584] Sends a message to the terminal indicating that the update is complete.

[2585] 9. Terminal:

[2586] The terminal will display a message to the user indicating that the update is complete.

[2587] 10. Users:

[2588] The user checks the displayed message and checks the operation of the system as necessary.

[2589] Utilizing the Emotion Engine

[2590] Example: Responding based on user emotions

[2591] 1. User:

[2592] The user sends an emotional request via text or voice from the device.

[2593] 2. Terminal:

[2594] The terminal receives the request and sends it to the server.

[2595] 3. Server:

[2596] The server receives the request and uses an emotion engine to analyze the user's emotional state.

[2597] 4. Server:

[2598] Based on the emotional information analyzed by the emotion engine, the generative AI adjusts the tone and content of the response. For example, if the user is feeling stressed, it will generate a more friendly response that emphasizes solutions.

[2599] 5. Generation AI:

[2600] Generative AI generates messages based on emotions, such as, "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with."

[2601] 6. Server:

[2602] Send the generated message to the terminal.

[2603] 7. Terminal:

[2604] Display received messages to the user.

[2605] 8. Users:

[2606] The user sees the friendly message and makes further requests if necessary.

[2607] This system enables the provision of information that takes into consideration the user's feelings, contributing to improved business efficiency and user satisfaction.

[2608] The processing flow will be explained below.

[2609] Providing information through an interactive UI

[2610] Example: A scene where you request the minutes of an internal meeting

[2611] Processing Steps

[2612] Step 1:

[2613] User:

[2614] The user sends a text or voice request from the device saying, "I'd like to see the minutes of last month's management meeting."

[2615] Step 2:

[2616] Device:

[2617] The terminal receives the user's request and sends it to the server.

[2618] Step 3:

[2619] server:

[2620] The server receives the request and calls a natural language processing (NLP) module to analyze the request, extracting keywords such as "last month," "management meeting," and "minutes."

[2621] Step 4:

[2622] server:

[2623] Based on the extracted keywords, search for relevant meeting minutes documents from internal databases or external libraries.

[2624] Step 5:

[2625] server:

[2626] The search results are passed to the generation AI.

[2627] Step 6:

[2628] Generation AI:

[2629] Based on the search results, the generative AI generates a conversational message to be delivered to the user, such as, "The minutes of last month's management meeting can be viewed at the link below."

[2630] Step 7:

[2631] server:

[2632] Send the generated message to the terminal.

[2633] Step 8:

[2634] Device:

[2635] Display received messages to the user.

[2636] Step 9:

[2637] User:

[2638] The user can view the minutes of the meeting by clicking the provided link.

[2639] Assistance in creating notes and documents

[2640] Example: Support for taking notes on presentation materials

[2641] Processing Steps

[2642] Step 1:

[2643] User:

[2644] The user makes a request from the terminal saying, "I would like to create notes for the presentation materials for the next customer briefing."

[2645] Step 2:

[2646] Device:

[2647] The terminal sends a request to the server.

[2648] Step 3:

[2649] server:

[2650] The server receives the request and analyzes it using the NLP module, extracting keywords such as "customer briefing," "presentation materials," and "memo."

[2651] Step 4:

[2652] server:

[2653] Based on the extracted keywords, the user's past documents and notes are searched from an internal database.

[2654] Step 5:

[2655] Generation AI:

[2656] Based on the search results, the generative AI creates a memo template for presentation materials to be provided to the user. For example, it creates a memo that states, "In the next customer briefing, it is important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[2657] Step 6:

[2658] server:

[2659] The generated memo is sent to the terminal.

[2660] Step 7:

[2661] Device:

[2662] Display received notes to the user.

[2663] Step 8:

[2664] User:

[2665] The user can check the displayed notes, edit and supplement them as necessary, and create the final presentation materials.

[2666] Additional learning of registered information

[2667] Example: Adding a new internal guideline

[2668] Processing Steps

[2669] Step 1:

[2670] User:

[2671] The user instructs the device, "I want to add a new internal guideline document to the AI ​​and have it learn it."

[2672] Step 2:

[2673] Device:

[2674] The terminal sends the instruction to the server.

[2675] Step 3:

[2676] server:

[2677] The server receives the instruction and sends a message to the user prompting them to upload the guideline document.

[2678] Step 4:

[2679] User:

[2680] The user uploads a guideline document.

[2681] Step 5:

[2682] Device:

[2683] Sends the uploaded document to the server.

[2684] Step 6:

[2685] server:

[2686] The server receives the uploaded document and passes it to the generative AI's learning module.

[2687] Step 7:

[2688] Generation AI:

[2689] The generative AI analyzes new documents and adds their contents as learning data, so that new guideline information is reflected in future information provision.

[2690] Step 8:

[2691] server:

[2692] Sends a message to the terminal indicating that the update is complete.

[2693] Step 9:

[2694] Device:

[2695] The terminal will display a message to the user indicating that the update is complete.

[2696] Step 10:

[2697] User:

[2698] The user checks the displayed message and checks the operation of the system as necessary.

[2699] Utilizing the Emotion Engine

[2700] Example: Responding based on user emotions

[2701] Processing Steps

[2702] Step 1:

[2703] User:

[2704] The user sends an emotional request via text or voice from the device.

[2705] Step 2:

[2706] Device:

[2707] The terminal sends a request to the server.

[2708] Step 3:

[2709] server:

[2710] The server receives the request and uses an emotion engine to analyze the user's emotional state.

[2711] Step 4:

[2712] server:

[2713] Based on the emotional information analyzed by the emotion engine, the generative AI adjusts the tone and content of the response. For example, if the user is feeling stressed, it will generate a more friendly response that emphasizes solutions.

[2714] Step 5:

[2715] Generation AI:

[2716] Generative AI generates messages based on emotions, such as, "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with."

[2717] Step 6:

[2718] server:

[2719] Send the generated message to the terminal.

[2720] Step 7:

[2721] Device:

[2722] Display received messages to the user.

[2723] Step 8:

[2724] User:

[2725] The user sees the friendly message and makes further requests if necessary.

[2726] This system enables the provision of information that takes into consideration the user's feelings, contributing to improved business efficiency and user satisfaction.

[2727] Example 2

[2728] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2729] In today's business environment, users are required to quickly and accurately obtain the information they need from multiple sources. It is also important to reduce fatigue and stress by providing appropriate responses based on the user's emotions and circumstances, thereby improving work efficiency and user satisfaction. However, conventional systems have not been able to adequately meet these needs, resulting in reduced user convenience and cumbersome work.

[2730] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes means for providing information through an interactive user interface, means for analyzing a user request and extracting related keywords, means for searching for related information from an internal database or an external information source, means for providing the user with an interactive message generated based on the extracted information, means for additionally learning documents and notes registered by the user and reflecting them in subsequent request responses, and means for analyzing the user's emotions and generating a response according to the emotions. This allows the user to not only quickly and accurately obtain the information they need, but also receive a response according to their emotions, thereby reducing the complexity of their work and improving work efficiency and user satisfaction.

[2731] An "interactive user interface" refers to a user interface that allows two-way communication through user operations.

[2732] "Means for analyzing user requests and extracting relevant keywords" refers to a function that analyzes requests sent by users using technologies such as natural language processing and extracts important language elements.

[2733] "Means of searching for relevant information from internal databases and external information sources" refers to the function of searching for relevant information from databases inside and outside the system based on extracted keywords.

[2734] The "means for providing a generated interactive message to the user" refers to a function for providing a response to the user in the form of a message generated based on the search results.

[2735] "Means for additional learning of documents and notes registered by the user and reflecting them in subsequent request responses" refers to the function by which the system learns newly registered information by the user and uses it in subsequent responses.

[2736] "Means for analyzing the user's emotions and generating a response that corresponds to those emotions" refers to a function that determines the emotions from the user's request and generates a response that is appropriate to those emotions.

[2737] This invention relates to an AI system that integrates memos and documents, providing information through an interactive user interface to improve the efficiency of users' work. This system has the ability to analyze user requests and quickly provide necessary information from relevant information sources, as well as recognize the user's emotions and respond accordingly.

[2738] Providing information through an interactive UI

[2739] Hardware and Software Use

[2740] User device: personal computer, smartphone, tablet, etc. Uses a microphone for voice input and a keyboard for text input.

[2741] Server: A server equipped with a high-performance processor and a large amount of memory. RDBMS or NoSQL databases are used.

[2742] NLP Modules: Software modules for natural language processing, such as SpaCy and NLTK.

[2743] Generative AI: High-performance generative AI models, such as OpenAI's GPT-3.

[2744] Specific examples

[2745] 1. User: The user makes a request from their terminal saying, "I want to see the minutes of last month's management meeting."

[2746] 2. Terminal: The terminal sends a request to the server.

[2747] 3. Server: The server analyzes the request using a natural language processing module and extracts keywords such as "last month," "management meeting," and "minutes."

[2748] 4. Server: The server uses these keywords to search for relevant information from internal databases and external sources, using a search engine such as Elasticsearch.

[2749] 5. Server: Passes search results to the generation AI.

[2750] 6. Generative AI: The Generative AI generates a conversational message saying, "The minutes of last month's management meeting can be viewed at the link below."

[2751] 7. Server: Sends the generated message to the terminal.

[2752] 8. Terminal: displays received messages to the user.

[2753] Prompt Sentence Examples

[2754] "Show me the minutes of last month's management meeting."

[2755] Assistance in creating notes and documents

[2756] Specific examples

[2757] 1. User: The user requests from the terminal, "I want to create notes for the presentation materials for the next customer briefing."

[2758] 2. Terminal: The terminal sends a request to the server.

[2759] 3. Server: The server analyzes the request using a natural language processing module and extracts keywords such as "customer briefing," "presentation materials," and "memo."

[2760] 4. Server: The server searches the user's past documents and notes from its internal database.

[2761] 5. Generative AI: Generative AI creates memo templates for presentation materials based on search results. For example, it generates a memo saying, "In the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session."

[2762] 6. Server: Sends the generated memo to the device.

[2763] 7. Terminal: Displays the received memo to the user.

[2764] 8. User: The user checks the displayed notes and makes edits or additions as necessary.

[2765] Prompt Sentence Examples

[2766] "Please prepare notes for the next client briefing."

[2767] Additional learning of registered information

[2768] Specific examples

[2769] 1. User: The user issues a command from their device saying, "I want to add new internal guidelines to the AI ​​and have it learn them."

[2770] 2. Terminal: The terminal sends instructions to the server.

[2771] 3. Server: The server receives the instructions and sends a message to the user prompting them to upload their guidelines.

[2772] 4. User: The user uploads the guideline document.

[2773] 5. Terminal: Sends the uploaded document to the server.

[2774] 6. Server: The server receives the document and passes it to the generative AI's learning module.

[2775] 7. Generative AI: Generative AI analyzes new documents and adds their contents to the learning data. This allows new guideline information to be reflected in future information provision.

[2776] 8. Server: Sends a message to the device indicating that the update is complete.

[2777] 9. Terminal: The terminal displays a message to the user that the update is complete.

[2778] Prompt Sentence Examples

[2779] "Learn these new guidelines and incorporate them into your next search."

[2780] Utilizing the Emotion Engine

[2781] Specific examples

[2782] 1. User: The user sends an emotional request from the device, for example, with an elevated tone of voice or emotional text.

[2783] 2. Terminal: The terminal sends a request to the server.

[2784] 3. Server: The server receives the request and uses an emotion engine to analyze the user's emotional state, for example determining that the user is tired.

[2785] 4. Server: Based on the emotional information analyzed by the emotion engine, the generative AI adjusts the tone and content of the response, for example, generating a friendly message that emphasizes solutions.

[2786] 5. Generative AI: Generative AI generates messages based on emotions, such as "Thank you for your hard work. You can find the minutes of last month's management meeting at this link. Let me know if there's anything I can help you with."

[2787] 6. Server: Sends the generated message to the terminal.

[2788] 7. Terminal: Displays received messages to the user.

[2789] Prompt Sentence Examples

[2790] "I'm very busy, but could you please give me a quick summary of the recent meeting?"

[2791] With these functions, the system streamlines users' work and provides necessary information quickly and accurately. It also responds with consideration for the user's feelings, improving work efficiency and user satisfaction.

[2792] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2793] Providing information through an interactive UI

[2794] Example: A scene where you request the minutes of an internal meeting

[2795] Step 1:

[2796] User: The user sends a request from their device saying, "I want to see the minutes of last month's management meeting." This operation is performed by the user either typing text or making a request by voice. Input is done using the device's keyboard and microphone.

[2797] Step 2:

[2798] Terminal: The terminal sends the received request to the server using an HTTP POST request or WebSocket. The data is encoded in JSON format.

[2799] Step 3:

[2800] Server: The server receives the request and analyzes it using a natural language processing (NLP) module. This process extracts keywords such as "last month," "management meeting," and "minutes" from the request text. The input is the request data in JSON format, and the output is a list of keywords. The specific software used is SpaCy and NLTK.

[2801] Step 4:

[2802] Server: The server searches for relevant information from internal databases and external sources based on the extracted keywords. This search is performed using a search engine such as Elasticsearch. The input is a list of keywords, and the output is a set of related information.

[2803] Step 5:

[2804] Server: The server passes the search results to the generative AI. The input is a set of related information, and the output is a response message generated by the generative AI. Specifically, OpenAI's GPT-3 is used.

[2805] Step 6:

[2806] Generative AI: Based on the search results, the generative AI generates a conversational message such as, "The minutes of last month's management meeting can be viewed at the link below." The input is a set of relevant information, and the output is a response message.

[2807] Step 7:

[2808] Server: The server sends the generated message to the terminal. HTTP or WebSocket is used for transmission. The input is the response message, and the output is the transmission completion status.

[2809] Step 8:

[2810] Terminal: A terminal displays received messages to the user, such as pop-up notifications or chat windows on the UI. Input is a response message, and output is displayed in the user interface.

[2811] Step 9:

[2812] User: The user is presented with a link to view the minutes. When the user clicks on the link, their browser opens a new tab or window with the relevant document.

[2813] Assistance in creating notes and documents

[2814] Example: Support for taking notes on presentation materials

[2815] Step 1:

[2816] User: The user requests, "I want to create notes for the presentation materials for the next client briefing." This can be done by typing text or by making a voice request.

[2817] Step 2:

[2818] Terminal: The terminal sends the received request to the server using an HTTP POST request or WebSocket. The data is encoded in JSON format.

[2819] Step 3:

[2820] Server: The server receives the request and analyzes it using the NLP module. Keywords such as "customer briefing," "presentation materials," and "memo" are extracted through the analysis. The input is the request data in JSON format, and the output is a list of keywords.

[2821] Step 4:

[2822] Server: The server searches the user's past documents and notes from an internal database based on the extracted keywords. The search engine used is Elasticsearch. The input is a list of keywords, and the output is a set of related documents and notes.

[2823] Step 5:

[2824] Generative AI: Generative AI creates memo templates for presentation materials based on search results. For example, it generates memos such as, "At the next customer briefing, it's important to cover the following points: 1. New product features, 2. Cost reduction proposals, 3. Q&A session." The input is related materials and memos, and the output is a memo template.

[2825] Step 6:

[2826] Server: Sends the generated memo to the device. HTTP POST request or WebSocket is used for sending. The input is the memo template, and the output is the sending completion status.

[2827] Step 7:

[2828] Terminal: Displays received memos to the user. Specifically, it displays them in the memo field on the UI or in notifications. The input is the memo template, and the output is displayed in the user interface.

[2829] Step 8:

[2830] User: The user checks the displayed notes, edits and supplements them as necessary, and creates the final presentation materials.

[2831] Additional learning of registered information

[2832] Example: Adding a new internal guideline

[2833] Step 1:

[2834] User: The user issues a command from the device saying, "I want to add new internal guidelines to the AI ​​and have it learn them." This operation is performed by the user entering text or giving voice instructions.

[2835] Step 2:

[2836] Terminal: The terminal sends instructions to the server using HTTP POST requests or WebSockets. The data is encoded in JSON format.

[2837] Step 3:

[2838] Server: The server receives the instructions and sends a message to the user prompting them to upload the guideline document. The input is the instructions, and the output is the upload prompt message.

[2839] Step 4:

[2840] User: The user uploads a guideline document. The operation is performed by selecting the file through a file selection dialog. The input is the document file, and the output is the uploaded file.

[2841] Step 5:

[2842] Terminal: Sends the uploaded document to the server using an HTTP POST request. The input is the uploaded file, and the output is the status of the successful submission.

[2843] Step 6:

[2844] Server: The server passes the received document to the generative AI learning module. The input is the document file, and the output is the learning data.

[2845] Step 7:

[2846] Generative AI: The generative AI analyzes new documents and adds their contents as training data. This allows new guideline information to be reflected in subsequent information provision. The input is training data, and the output is updated training data.

[2847] Step 8...

Claims

1. a means for providing information through an interactive user interface; A means for analyzing a user request and extracting related keywords; A means of searching for relevant information from sources such as internal documents, meeting minutes, personal notes, documents in external libraries, etc. means for providing a user with an interactive message generated based on the extracted information; A means to learn additional documents and notes registered by users and reflect them in future request responses. A system including:

2. 2. The system according to claim 1, further comprising means for generating an easy-to-read template of notes or materials based on the contents of a user's request.

3. 10. The system of claim 1, further comprising means for receiving and analyzing voice instructions from a user.

4. 2. The system according to claim 1, further comprising means for learning the user's past usage history and improving the accuracy of the information provided.

Images