System

Abstract

A system is provided.SOLUTION: A system comprising: means for monitoring generated data; means for collecting details of the generated data; means for storing the collected details in a database; means for generating a note based on the details stored in the database; means for providing the generated note through a user interface; means for integrating the collected details in conjunction with other artificial intelligence services; means for analyzing the integrated details to find an optimal work pattern; and means for improving a process of the generated data based on the optimal work pattern.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Artificial intelligence (AI) technology has evolved rapidly in recent years, but many users remain concerned about AI's black-box nature and potential misjudgments. As a result, improving the transparency and reliability of AI has become an urgent issue. Another problem is that current generative AI cannot refer to past work history, making it difficult to perform work of consistent quality. Furthermore, when multiple AI services work together, it is difficult to integrate and analyze each service's work history, and there is a lack of methods for identifying optimal work patterns. Furthermore, AI's ability to adapt to individual user needs is limited. To address these issues, the present invention aims to provide a system that records AI work history in detail and provides transparency. [Means for solving the problem]

[0005] The present invention provides a system including: means for monitoring generated data; means for collecting details of the generated data; means for storing the collected details in a database; means for generating memos based on the details stored in the database; means for providing the generated memos through a user interface; means for integrating the collected details in cooperation with other artificial intelligence services; means for analyzing the integrated details to find optimal work patterns; means for improving the process of the generated data based on the optimal work patterns; means for evaluating performance of the generated data by combining the collected details with user feedback obtained through the user interface; and means for learning user preferences and work styles from the collected details and customizing the process of the generated data based on the learning results. This system enables detailed recording of AI work history and provides transparency, thereby improving the reliability and quality of AI.

[0006] "Generated data" refers to the output generated by artificial intelligence using specific algorithms or models.

[0007] "Means of monitoring" refers to the ability to monitor all tasks and processes performed by artificial intelligence in real time.

[0008] "Means of collecting details" refers to the ability of the AI ​​to systematically obtain data (such as inputs, processing steps, and outputs) relevant to the task it is to perform.

[0009] "Database" refers to a system that structures and stores collected details in a queryable format.

[0010] "Means of generating notes" refers to the ability to document summaries and extract key information from the details collected.

[0011] "User interface" refers to a visual or interactive operating screen that allows a user to refer to notes and data generated by artificial intelligence.

[0012] "Other artificial intelligence services" refers to independent artificial intelligence systems or platforms other than this system.

[0013] "Means of integration" refers to the function of centralizing and managing data obtained from multiple artificial intelligence services.

[0014] "Optimal work patterns" refer to the most efficient and effective processes and procedures identified based on collected data and analysis results.

[0015] "Means to improve processes" refers to the ability to adjust and optimize the operations and algorithms of artificial intelligence based on the optimal work patterns discovered.

[0016] "User feedback" refers to opinions and evaluations from users regarding the work history and generated results.

[0017] "Means for evaluating performance" refers to the ability to measure and evaluate the performance of artificial intelligence based on user feedback and work history data.

[0018] "User preferences and working style" refers to the output format and operation method desired by a particular user.

[0019] "Means for customization" refers to the ability to adjust the settings and output of the artificial intelligence according to the user's preferences and work style. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The AI ​​MemoSphere system of the present invention aims to record the history of tasks performed by an artificial intelligence, generate memos based on that history, and provide them to users in a transparent and reliable manner. Below, we will explain in natural language the program processing of a specific embodiment of this system, and provide detailed examples.

[0042] Recording work history

[0043] The server monitors the generated data in real time. For example, when a generative AI generates an answer to a user's question, the server continuously monitors this process.

[0044] The server collects details of each task (input data, algorithm used, output results, etc.) and stores them in a database. For example, if a user asks a question about a product, the generative AI's answer to that question, the process leading to that answer, and the version of the model and algorithm used are recorded.

[0045] Create and manage notes

[0046] The server generates memos based on the collected work history, including task summaries and key points.

[0047] Example: The server compiles important information about the answer generation tasks performed by the AI ​​over the past week and saves it as a summary memo. This summary memo includes information such as what types of questions were most popular, the trends in the answers, and which algorithms were most frequently used.

[0048] Viewing work history

[0049] Users can view their work history notes through a dedicated user interface (UI), which is available as a browser-based or desktop application.

[0050] Example: A user can access the UI to see the AI's answer process for their question and view detailed notes, making it easier to understand how the answer was generated.

[0051] Integration with other AI services

[0052] The server connects with other AI services via APIs and integrates the work history of each AI service.

[0053] Example: By linking text generation AI and image recognition AI, when a user asks a text question, they can also provide an image, and the process of generating an answer based on that is recorded and integrated. Users can then refer to this link history in a unified manner.

[0054] Analysis of optimal work patterns

[0055] The server analyzes the integrated work history data and finds optimal work patterns.

[0056] Example: The server finds the most efficient algorithms and processes across multiple tasks and automatically applies those patterns to the next task, improving the accuracy and speed of AI responses and significantly improving the user experience.

[0057] Adapting to user needs

[0058] The device collects user feedback and uses this to customize the AI's behavior.

[0059] Example: If a user requests a specific expression or format, that feedback will be reflected and the next AI output will be provided in a format that matches that preference.

[0060] Creating metrics and evaluating performance

[0061] The server creates new performance metrics based on collected details and user feedback to evaluate the AI's performance.

[0062] Example: Evaluations are conducted based on a combination of multiple metrics, including response speed and accuracy, as well as user satisfaction, to identify areas for improvement in AI and improve its quality.

[0063] Adapts to user preferences and work styles

[0064] The device analyzes work history data and learns the user's preferences and work style.

[0065] Example: It learns the tone and style of responses preferred by a particular user and adjusts output accordingly, allowing it to best meet the user's individual needs.

[0066] In this way, the AI ​​MemoSphere system records, manages, and analyzes the work history of the generating AI in detail, and provides users with a highly transparent service based on that information, which improves the reliability of the AI ​​and enables efficient, high-quality work.

[0067] The processing flow will be explained below.

[0068] Step 1:

[0069] The server monitors the generated data in real time. Specifically, when the generating AI starts a task, it obtains the task's identification information and triggers the monitoring process.

[0070] Step 2:

[0071] The server collects details of each task, including the input data, the algorithms used, each step of the processing, and the output results, and then organizes and structures the data.

[0072] Step 3:

[0073] The server stores the collected details in a database, which stores the data according to an appropriate schema in a format that can be queried later.

[0074] Step 4:

[0075] The server generates notes based on details stored in the database, summarizing the key points of the data and automatically creating easy-to-understand notes in natural language.

[0076] Step 5:

[0077] The server provides the generated notes through a dedicated user interface (UI), which is designed as a browser-based or desktop application.

[0078] Step 6:

[0079] Users can access work history notes through the user interface, allowing them to search for the history of a specific task or check the contents of the notes.

[0080] Step 7:

[0081] The server will connect with other AI services via API, allowing work history data to be sent and received between the services.

[0082] Step 8:

[0083] The server integrates the work history collected from the linked AI services, thereby centrally managing the historical data of multiple services.

[0084] Step 9:

[0085] The server analyzes the integrated data to identify optimal work patterns, applying machine learning algorithms and statistical models to identify the most effective processes and procedures.

[0086] Step 10:

[0087] The server then uses the analytical results to improve the processing of the generated data, specifically by automatically applying optimized algorithm settings and parameters.

[0088] Step 11:

[0089] The terminal provides an interface for collecting user feedback, allowing the user to input their opinions on the user interface.

[0090] Step 12:

[0091] The server analyzes the collected feedback and calculates metrics to evaluate the performance of the generated data. It combines the feedback content and work history to derive various evaluation indicators.

[0092] Step 13:

[0093] The server learns the user's preferences and working style, which involves analyzing past usage data and feedback to extract the user's specific needs and tendencies.

[0094] Step 14:

[0095] The server then customizes the processing of the generated data based on the learning results, providing responses and functionality that match the user's preferences and working style.

[0096] Example 1

[0097] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0098] There is a growing need to improve the transparency and reliability of information generated in modern automation systems. However, current systems have difficulty effectively managing and analyzing the detailed history of generated information, making it difficult for users to understand the generation process. Furthermore, it is difficult to efficiently link different automation services and discover and apply the optimal work format. For these reasons, there is a need for a system that can improve the reliability and transparency of generated information and effectively manage and analyze it.

[0099] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0100] In this invention, the server includes means for monitoring generated information, means for collecting details of the generated information, means for storing the collected details in a storage device, means for generating a summary based on the details stored in the storage device, means for providing the generated summary through a user interface, means for integrating the collected details in cooperation with other automated services, means for analyzing the integrated details to find an optimal work format, and means for improving the process of the generated information based on the optimal work format. This makes it possible to manage and analyze the history of generated information in detail and provide highly transparent services to users.

[0101] "Generated Information" means output or data generated by an automated system based on user input.

[0102] "Monitoring means" refers to a device or program that tracks and records the process and results of the information generated in real time.

[0103] "Means of collection" refers to a device or program that systematically acquires and stores details of the generated information and related data.

[0104] "Storage" refers to physical and logical storage for storing collected detailed data.

[0105] "Means for generating a summary" refers to a device or program that generates a compact, easily understandable summary from the detailed data collected.

[0106] "User interface" refers to the means, including screens and applications, by which a user interacts with a system.

[0107] "Automated services" refer to programs or systems that autonomously perform specific tasks.

[0108] "Integration means" refers to a device or program that centralizes the collected detailed data with data from other automated services and manages it in a unified format.

[0109] The "optimal work format" refers to the most efficient and accurate method among multiple work processes.

[0110] "Means for improving" refers to a device or program for improving the process of generating information based on the optimal working format.

[0111] MODE FOR CARRYING OUT THE INVENTION

[0112] The AI ​​MemoSphere system of the present invention is a system designed to increase the transparency and reliability of generated information. The following describes how the present invention is specifically implemented. The specific names of the hardware and software used are also provided.

[0113] System Configuration

[0114] The system of the present invention consists of the following main components:

[0115] 1. Server

[0116] Hardware used: High-performance servers (e.g., AWS EC2 instances)

[0117] Software used: log collection tools (e.g., Elasticsearch), databases (e.g., PostgreSQL), automatic summary generation tools (e.g., NLTK), data analysis tools (e.g., pandas, scikit-learn), performance measurement tools (e.g., TensorBoard), REST API (e.g., Flask)

[0118] 2. Terminal

[0119] Hardware used: User device (e.g., PC, smartphone)

[0120] Software used: web browser (e.g., Chrome), feedback collection tool (e.g., Google Forms), machine learning model (e.g., scikit-learn)

[0121] 3. User Interface (UI)

[0122] Software used: Browser-based or desktop application

[0123] Program processing explanation

[0124] The specific operation and processing of this system will be explained in natural language below.

[0125] Recording work history

[0126] The server monitors in real time the process by which the generative AI model (e.g., GPT-4) generates an answer based on the user's input. For example, when a user inputs "What's the weather like?", the server collects data until the generative AI model replies "Today's weather is sunny." The collected data includes the input data, the algorithm used, the generated output, etc. This data is stored in a storage device by the server.

[0127] Create and manage notes

[0128] The server generates a summary memo based on the collected work history. This summary includes a summary of the task and key points. For example, the server compiles the answer history of the generation AI over the past week and generates a summary memo that includes frequently asked questions, their trends, and information about the version of the algorithm used.

[0129] Viewing work history

[0130] Users can refer to the work history notes through the user interface (UI). For example, users can access the UI to check the past question history and the answering process of the generation AI, and view detailed notes.

[0131] Integration with other AI services

[0132] The server connects with other automation services, such as image recognition AI, via API and integrates the work history data of each AI service. For example, a user can provide an image along with a text question, and the server records and integrates the answer process based on that.

[0133] Analysis of optimal work patterns

[0134] The server analyzes the integrated work history data and finds the optimal work format. For example, the server can identify the most efficient algorithms and processes across multiple tasks and automatically apply them to future tasks, improving the accuracy and speed of answers.

[0135] Adapting to user needs

[0136] The device collects user feedback and uses it to customize the generative AI model's responses. For example, if a user sends feedback such as "I like this expression," that preference will be reflected in the next answer generated.

[0137] Adapts to user preferences and work styles

[0138] The device learns user preferences and styles based on historical work data, for example, learning the tone and style of responses preferred by a particular user, and adjusts the output of the generative AI model accordingly.

[0139] Specific examples

[0140] Here are some examples of prompts:

[0141] "Please tell me the weather."

[0142] When a user enters this prompt, the server uses a generative AI model to generate an answer and records the process in detail. A summary memo is generated based on this record, and the user can view the details through the UI. By linking with other image recognition AI, the user can provide additional information and obtain a more detailed answer. The integrated data is analyzed to identify the optimal work format and apply it to the next task. The AI's responses are continuously improved based on user feedback and historical data.

[0143] In this way, the AI ​​MemoSphere system can manage and analyze the history of generated information in detail, providing users with transparent and reliable services.

[0144] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0145] Step 1:

[0146] The user inputs a prompt sentence into the user interface (UI), for example, "Please tell me the weather."

[0147] Input: prompt statement

[0148] Output: User input data

[0149] What happens: A user enters a prompt sentence using a browser or desktop application.

[0150] Step 2:

[0151] The server receives the prompt sentence sent by the user and inputs it into a generative AI model, which then generates an answer (e.g., GPT-4).

[0152] Input: User-entered data

[0153] Output: Answer data generated by the generative AI model

[0154] How it works: The server receives the input "What is the weather like?", submits it to a generative AI model, and generates the answer "It's sunny today."

[0155] Step 3:

[0156] The server monitors the generated answers and the process in real time, collecting data, including the algorithms and model versions used.

[0157] Input: Answer data generated by a generative AI model

[0158] Output: Detailed data on the generation process

[0159] How it works: The server monitors the entire generation process in real time and collects detailed data such as "Question: What's the weather like?", "Answer: It's sunny today," "Model used: GPT-4," and "Time: XX seconds."

[0160] Step 4:

[0161] The server stores the collected detailed data in a storage device (database).

[0162] Input: Detailed data of the generation process

[0163] Output: Detailed data stored in a database

[0164] Specific operation: The server stores the collected detailed data in a database (e.g., PostgreSQL).

[0165] Step 5:

[0166] The server generates a summary memo based on the detailed data stored in the storage device, which includes a summary of the task and important points.

[0167] Input: Detailed data stored in the database

[0168] Output: Generated summary notes

[0169] Specific operation: The server uses an automatic summary generation tool (e.g., NLTK) to summarize the generation AI's answer history for the past week and create a summary memo.

[0170] Step 6:

[0171] The server provides the generated summary memo through a user interface (UI).

[0172] Input: Generated summary note

[0173] Output: A user-visible summary note

[0174] Specific operation: The server displays the summary memo through the UI (browser or desktop application) so that the user can view it.

[0175] Step 7:

[0176] The server connects with other automation services via APIs and integrates the work history data of each service.

[0177] Input: Work history data from other automated services

[0178] Output: Integrated work history data

[0179] Specific operation: The server uses a REST API (e.g., Flask) to integrate data from image recognition AI, etc.

[0180] Step 8:

[0181] The server analyzes the integrated work history data and finds the optimal work format.

[0182] Input: Integrated work history data

[0183] Output: Data in a format that works best for you

[0184] Specific behavior: The server uses data analysis tools (e.g., pandas, scikit-learn) to identify efficient algorithms and processes.

[0185] Step 9:

[0186] The server improves the process of generating the generated information based on the most suitable working format.

[0187] Input: Data in the format that works best for you

[0188] Output: Improved generation process

[0189] Specific operation: The server automatically applies the identified optimal work format to the next task, improving the accuracy and speed of the generative AI model.

[0190] Step 10:

[0191] The device collects user feedback and uses it to customize the responses of the generative AI model.

[0192] Input: User feedback

[0193] Output: The customized generative AI model response

[0194] Specific operation: The device collects feedback from users using a feedback collection tool (e.g., Google Forms) and reflects it in the output of the generative AI model.

[0195] Step 11:

[0196] The device learns the user's preferences and style based on work history data and adjusts the output of the generative AI model accordingly.

[0197] Input: Work history data

[0198] Output: The output of the tuned generative AI model.

[0199] What it does: The device uses a machine learning model (e.g., scikit-learn) to learn the response format and tone preferred by a particular user and reflects that in the next output.

[0200] (Application example 1)

[0201] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0202] In modern logistics centers, maximizing work efficiency and minimizing errors are important. However, recording and analyzing work history, and then flexibly modifying work plans based on that information, is not easy. Current systems lack the tools to ensure data transparency and reliability while working efficiently. A major problem is the lack of a function that allows on-site staff to input information in real time and generate optimal notes based on that information. A new system is needed to solve these issues.

[0203] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0204] In this invention, the server includes means for monitoring generated data, means for collecting details of the generated data, means for storing the collected details in a database, means for generating memos based on the details stored in the database, means for providing the generated memos through a user interface, means for integrating the collected details in cooperation with other artificial intelligence services, means for analyzing the integrated details to find optimal work patterns, means for improving the process of the generated data based on the optimal work patterns, means for recording work history, generating memos, and improving work efficiency in the logistics center, means for inputting work information via a smartphone application and sending it to the server, and means for sending generated memos from the server to the smartphone application, thereby improving work efficiency in the logistics center, reducing work errors, and ensuring data transparency and reliability.

[0205] "Generated data" is data generated by artificial intelligence systems or other information systems.

[0206] "Monitoring means" refers to a means for observing the generated data in real time and detecting fluctuations or abnormalities.

[0207] "Collection means" refers to the means for systematically capturing and recording detailed information about the data generated.

[0208] A "database" is an area where detailed information on collected data is systematically stored and can be retrieved as needed.

[0209] The "means for generating memos" refers to a means for organizing the main points based on the collected detailed information and generating concisely written memos.

[0210] A "user interface" is an interface that provides a screen and operating means for a user to interact with a system.

[0211] "Artificial Intelligence Services" are services based on artificial intelligence technology that are utilized to perform specific tasks.

[0212] An "optimal work pattern" is an efficient and effective work procedure or method discovered by analyzing past work history.

[0213] "Process improvement measures" are measures to improve current work procedures and methods based on optimal work patterns, thereby increasing efficiency and effectiveness.

[0214] A "smartphone application" is software that runs on a smartphone and provides specific functions.

[0215] "Work information" is information relating to the specific content and status of work being carried out at the logistics center.

[0216] A "server" is a computer system for processing, storing, and managing data.

[0217] The purpose of this invention, the "AI MemoSphere System," is to improve work efficiency at logistics centers, reduce work errors, and ensure data transparency and reliability. Specific implementations of this system are described below.

[0218] Hardware and software used

[0219] Hardware:

[0220] Server: AWS server (EC2)

[0221] Device: Smartphone (iOS / Android)

[0222] software:

[0223] Server side: Python (Django framework)

[0224] Database: PostgreSQL

[0225] Frontend: React Native

[0226] AI model: GPT-4 (API provided by OpenAI)

[0227] API integration: AWS API Gateway

[0228] System Program

[0229] Recording work history

[0230] The server monitors information about each operation performed at the distribution center in real time and collects the generated data. Specifically, staff enter operation information (e.g., item ID, quantity, picking time) through a smartphone application, and the data is sent to the server, which then stores it in a database.

[0231] Generate notes

[0232] Based on the collected work data, the server uses an AI model (GPT-4) to generate memos, which concisely summarize the main points and important information of the work. These memos are then sent from the server to a smartphone application where they can be viewed by staff.

[0233] Viewing work history

[0234] Users (logistics center staff and managers) can view work history and generated notes through a smartphone application, making it easy to understand how each task was performed and what results were achieved.

[0235] Analysis of optimal work patterns

[0236] The server analyzes the collected comprehensive work data to find the optimal work pattern, and the results of this analysis are reflected in the next work, allowing more efficient methods to be automatically applied.

[0237] Specific examples

[0238] At a distribution center, staff have a task of picking items from shelves. When they input the task data (e.g., item ID, quantity, picking time) into a smartphone application, the history is sent to the server and stored in a database. GPT-4 creates notes from the generated history, and these notes can be viewed by staff and managers through the app.

[0239] Prompt Sentence Examples

[0240] Below are some example prompts sent to the AI ​​model:

[0241] Task ID: 123 Details:

[0242] Input data: Item ID: A001, Quantity: 10, Picking time: 15 minutes

[0243] Algorithm used: Picking algorithm version 2.0

[0244] Output: Success, all items picked correctly

[0245] Generated note:

[0246] In this task, 10 units of item ID A001 were accurately picked. The picking algorithm used was version 2.0, which took 15 minutes to complete. This algorithm was effective for this task.

[0247] In this way, the AI ​​MemoSphere system can support skilled personnel in logistics centers and provide a highly transparent and efficient working environment.

[0248] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0249] Step 1:

[0250] The server monitors the generated data. Staff at the logistics center use a smartphone application to input work information. This input (e.g., item ID, quantity, picking time) is sent to the server. The server receives and monitors this data in real time. The input data is work information such as item ID, quantity, and picking time, and the output is the data received by the server.

[0251] Step 2:

[0252] The server collects details of the generated data. The server analyzes the task data sent from the smartphone application and extracts detailed information. The collected data includes the execution time of each task, the version of the algorithm used, and the success or failure of the task. These detailed information are stored in a database. The input data is the task information from the smartphone application, and the output is the detailed information stored in the database.

[0253] Step 3:

[0254] The server generates a memo based on the detailed information stored in the database. The server sends the collected work data details to the AI ​​model (GPT-4) and receives the generated memo. The prompt sentence to be sent to the AI ​​model is composed as follows:

[0255] Task ID: 123 Details:

[0256] Input data: Item ID: A001, Quantity: 10, Picking time: 15 minutes

[0257] Algorithm used: Picking algorithm version 2.0

[0258] Output: Success, all items picked correctly

[0259] Generated note:

[0260] In this task, 10 units of item ID A001 were accurately picked. The picking algorithm used was version 2.0, which took 15 minutes to complete. This algorithm was effective for this task.

[0261] The input data are the collected details, and the output are the notes generated by the AI ​​model.

[0262] Step 4:

[0263] The server provides the generated notes through a user interface. The generated notes are sent to a smartphone application so that the user can view them. The user can check the notes for each task through the smartphone application and understand the details of the task. The input data are the notes generated by the AI ​​model, and the output is the notes displayed in the user interface.

[0264] Step 5:

[0265] The server integrates the collected details in cooperation with other AI services. For example, by collaborating with an image recognition AI, it also collects image data and integrates detailed information based on that. This generates data that includes not only text information but also image information. The input data is detailed information from other AI services, and the output is the integrated detailed information.

[0266] Step 6:

[0267] The server analyzes the integrated details to find the optimal work pattern. It compares multiple work histories and extracts the most efficient work procedures and methods. This allows the optimal pattern to be applied in future work. The input data is the integrated details, and the output is the optimal work pattern.

[0268] Step 7:

[0269] The server improves the process of the generated data based on the optimal work pattern. Based on the discovered optimal pattern, it automatically corrects the current work procedures and methods to improve efficiency. The input data is the optimal work pattern, and the output is the improved work process.

[0270] Step 8:

[0271] The device collects user feedback and uses it to customize the AI's behavior. Feedback provided by the user through a smartphone application improves the AI ​​model's output and adapts to the user's needs. The input data is the user's feedback, and the output is the customized AI model's behavior.

[0272] In this way, the AI ​​MemoSphere System can improve work efficiency and ensure data transparency in logistics centers, providing a more reliable work environment.

[0273] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0274] The AI ​​MemoSphere system of the present invention aims to record the history of tasks performed by an artificial intelligence, generate memos based on that history, and provide them to users in a transparent and reliable manner. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions, and by taking emotion data into consideration, it is possible to provide more personalized services. Below, a specific embodiment of this system will be described in detail, with program processing explained in natural language and examples provided.

[0275] Recording work history

[0276] The server monitors the generated data in real time. For example, when a generative AI generates an answer to a user's question, the server continuously monitors this process.

[0277] The server collects details of each task (input data, algorithm used, output results, user emotion data, etc.) and stores them in a database. When a user asks a question about a product, the generative AI's answer, the process leading to that answer, the model and algorithm version used, and the user's emotion are recorded.

[0278] Use of emotion engine

[0279] The device analyzes the user's facial expressions and tone of voice as they type, and uses an emotion engine to recognize the user's emotions.

[0280] The server stores the emotion data recognized by the emotion engine in a database. For example, if the user is dissatisfied, the emotion data is also recorded.

[0281] Create and manage notes

[0282] The server generates memos based on the collected work history and emotional data. The generated memos include a summary of the task, important points, and the user's emotional data.

[0283] Example: The server generates a summary memo containing important information about the answer generation tasks performed by the AI ​​over the past week, as well as changes in user sentiment. This summary memo includes information such as what types of questions were most popular, the trends in their answers, and the user's emotional reactions.

[0284] Viewing work history

[0285] Users can access their work history notes through a dedicated user interface (UI), which can be provided as a browser-based or desktop application.

[0286] Example: A user can access the UI to see the AI's answer process to their question and the emotional data at the time, and view detailed notes, making it easier to understand how the answer was generated and how it reflected their emotional state at the time.

[0287] Integration with other AI services

[0288] The server connects with other AI services via APIs and integrates the work history and emotional data of each AI service.

[0289] Example: By linking text generation AI and image recognition AI, users can simultaneously provide images when asking text questions, and the process of generating answers based on these images is recorded and integrated. Users can then refer to this link history and emotion data in a unified manner.

[0290] Analysis of optimal work patterns

[0291] The server analyzes the integrated work history data and emotion data to find optimal work patterns.

[0292] Example: The server finds the most efficient algorithms and processes across multiple tasks and automatically applies those patterns to the next task. This takes emotional data into account, improving the accuracy of AI responses and user satisfaction.

[0293] Adapting to user needs

[0294] The device collects user feedback and uses this to customize the AI's behavior.

[0295] Example: If a user requests a particular style, format, or even a desired emotional tone, that feedback will be incorporated and the next AI output will be tailored to those preferences.

[0296] Creating metrics and evaluating performance

[0297] The server creates new performance indicators based on collected work history, emotional data, and user feedback to evaluate the AI's performance.

[0298] Example: In addition to response speed and accuracy, evaluations are conducted based on a combination of multiple metrics, such as user satisfaction and emotional response, to identify areas for improvement in AI and improve its quality.

[0299] Improving processes by taking emotional data into account

[0300] The server then uses the analysis results to improve the processing of the generated data, specifically by automatically applying algorithm settings and parameters that reflect the user's emotional data.

[0301] Example: For tasks that are likely to frustrate users, the AI ​​is tuned to generate more polite responses.

[0302] Adapts to user preferences and work styles

[0303] The device analyzes work history data and emotional data to tailor responses to best suit the user's preferences and work style.

[0304] Example: Learning the tone and style of responses preferred by a particular user and tailoring output accordingly, allowing it to best meet the user's individual needs and emotions.

[0305] In this way, the AI ​​MemoSphere system records, manages, and analyzes the work history of the generating AI and user emotional data in detail, and provides users with highly transparent services based on this information. This improves the reliability and personalization of the AI, enabling efficient and high-quality work.

[0306] The processing flow will be explained below.

[0307] Step 1:

[0308] The server monitors the generated data in real time. Specifically, when the generation AI starts a task, it acquires the task's identification information and triggers the monitoring process. For example, when a user enters a question, it generates a question ID and records it as a monitoring target.

[0309] Step 2:

[0310] The device analyzes the user's facial expressions and tone of voice when inputting and uses an emotion engine to recognize the user's emotions. Specifically, it uses a camera and microphone to detect the user's emotional state in real time and sends that data to the emotion engine.

[0311] Step 3:

[0312] The server stores the user's emotion data recognized by the emotion engine in a database. For example, if the user makes a dissatisfied expression, the emotion data is recorded and associated with the question ID.

[0313] Step 4:

[0314] The server collects details of each task (input data, algorithm used, processing steps, output results, and user emotional data), including the question entered by the user, the process of the generative AI, and the final answer.

[0315] Step 5:

[0316] The server stores the collected details in a database, which stores the data according to an appropriate schema in a format that can be queried later.

[0317] Step 6:

[0318] The server generates notes based on the details stored in the database, including a summary of the task, key points, and the user's emotional data.

[0319] Step 7:

[0320] The server provides the generated notes through a dedicated user interface (UI), which is designed as a browser-based or desktop application.

[0321] Step 8:

[0322] The user can view the work history notes through the user interface. The user can check the history of a specific task, the contents of the notes, and the emotional data at the time.

[0323] Step 9:

[0324] The server will connect with other AI services via API, allowing for the sending and receiving of work history data and emotion data between the connected services.

[0325] Step 10:

[0326] The server integrates the work history and emotion data collected from the linked AI services, thereby centrally managing the history data from multiple services.

[0327] Step 11:

[0328] The server analyzes the integrated data to identify optimal work patterns, applying machine learning algorithms and statistical models to identify the most effective processes, procedures, and emotional responses.

[0329] Step 12:

[0330] The server then uses the analysis results to improve the processing of the generated data, specifically by automatically applying optimized algorithm settings and parameters that also take emotion data into account.

[0331] Step 13:

[0332] The terminal provides an interface for collecting user feedback, allowing users to input their opinions and thoughts on the user interface.

[0333] Step 14:

[0334] The server analyzes the collected feedback and calculates metrics to evaluate the performance of the generated data. It combines the feedback content, work history, and emotional data to derive various evaluation indicators.

[0335] Step 15:

[0336] The server learns the user's preferences and work style, which involves analyzing past usage data, feedback, and sentiment data to extract the user's specific needs and tendencies.

[0337] Step 16:

[0338] The server then customizes the processing of the generated data based on the learning results, delivering responses and functionality in a format that best suits the user's preferences, work style, and emotions.

[0339] Example 2

[0340] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0341] Conventional AI systems focus on generating answers to user questions, but the process often lacks transparency and reliability. They also lack the ability to recognize user emotions and provide responses that take these into account. This makes it difficult to provide truly useful and personalized services to users. Furthermore, there are also issues with inconsistencies in analyzing optimal work patterns and integrating with other AI services.

[0342] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0343] In this invention, the server includes means for monitoring the generated data, means for collecting details of the generated data, means for storing the collected details in a database, means for generating memos based on the details stored in the database, means for providing the generated memos through a user interface, means for integrating the collected details in cooperation with other AI services, means for analyzing the integrated details to find optimal work patterns, means for improving the process of the generated data based on the optimal work patterns, means for recognizing user emotion data, means for collecting and storing the recognized emotion data in a database, and means for generating memos based on the collected and stored emotion data. This makes it possible to record, manage, and analyze the work history of the generative AI model and the user emotion data in detail, and to provide users with a transparent and reliable personalized service.

[0344] "Generated Data" refers to the responses or results generated by a generative AI model in response to a user's question or request.

[0345] "Monitoring means" refers to a system including hardware and software for monitoring the generated data in real time and understanding its contents.

[0346] "Collection means" refers to the set of processes or functions used to compile and record details of the data obtained through surveillance.

[0347] "Database" refers to a system of information collection that stores details of collected data in an organized manner so that they can be easily retrieved and used later.

[0348] "Means for generating notes" refers to algorithms or programs that create summary information based on the details and emotional data collected.

[0349] "User Interface" refers to the screens and applications that allow a user to access and manipulate generated notes and data details.

[0350] "Other artificial intelligence services" refers to artificial intelligence technologies and services that exist other than the system of the present invention, and refers to systems that can be linked with these.

[0351] "Means of integration" refers to the ability to bring together data and information obtained from other artificial intelligence services and manage them in a consistent manner.

[0352] "Optimal work patterns" refer to efficient and effective work procedures and processes discovered through detailed analysis of collected data.

[0353] "Measures for improvement" refers to methods and systems for improving the current data generation process based on the optimal work patterns identified.

[0354] "Emotion data" refers to data that indicates the emotional state recognized from the user's statements and actions.

[0355] "Means for recognizing emotions" refers to a system that includes hardware and software for analyzing data such as a user's facial expressions and voice and identifying the user's emotions.

[0356] "Collected and stored emotion data" refers to the state in which the recognized emotion data is stored in a database and can be used for later interpretation and analysis.

[0357] The AI ​​MemoSphere system of the present invention records, manages, and analyzes the work history of the generative AI model and the user's emotional data in detail, and provides users with a transparent, reliable, and personalized service based on the data. Specific embodiments of the system are described below.

[0358] Basic configuration

[0359] This system consists of a server, a terminal, and a user. The server mainly monitors, collects, stores, and analyzes data, while the terminal provides a user interface and collects emotion data.

[0360] Hardware and software used

[0361] Generative AI models: For example, using GPT-3 for natural language processing.

[0362] Server: A high-performance computer server that collects, stores, analyzes, and generates notes on data.

[0363] Database: For example, using a relational database management system (RDBMS).

[0364] Emotion Engine: Uses Microsoft Azure's Emotion API to recognize user emotions.

[0365] User interface: Provided as a web browser or desktop application.

[0366] Processing flow and specific examples

[0367] 1. Monitoring generated data:

[0368] The server monitors the process in real time of the generative AI model generating answers to user questions. For example, if a user asks, "What is the price of a new product?", the request is passed to the generative AI model (GPT-3).

[0369] 2. Collect task details:

[0370] The server collects detailed information about the algorithms used by the generative AI, input data, output results, and user emotional data, and stores it in a database. For example, it records the generative AI model used, the prompt (user question), the output result (product price), and the user's emotional data (dissatisfaction, satisfaction, etc.).

[0371] 3. User emotion recognition:

[0372] The device recognizes the user's facial expression and tone of voice when they input a question and uses an emotion engine to recognize their emotion. For example, if a user says "Is this really the right price?" in a dissatisfied tone, their facial expression and tone of voice are captured.

[0373] 4. Emotional Data Recording:

[0374] The server stores the recognized emotion data in a database. Specifically, the server receives emotion data (e.g., dissatisfaction) sent from the device and adds it to the database.

[0375] 5. Automatic note generation:

[0376] The server generates memos based on the collected work history and emotion data, including task summaries and key points. For example, it summarizes the tasks performed by the AI ​​over the past week and creates a summary memo that includes the user's emotions regarding each task.

[0377] 6. Accessing the User Interface:

[0378] Users can access their work history notes through a dedicated user interface, such as a dashboard in a web browser, to view details of past interactions and sentiment data.

[0379] 7. Data integration with other AI services:

[0380] The server also works with image recognition AI (e.g., an image analysis system) to integrate the entire work history and emotional data. For example, a user can upload a product image and record the answer generation process based on it.

[0381] 8. Extracting optimal work patterns:

[0382] The server then extracts the optimal work pattern based on the integrated work history and emotional data and applies it to the next task, for example by analyzing multiple response patterns and selecting the fastest and most effective algorithm to apply.

[0383] 9. Collaboration with Feedback:

[0384] The device collects user feedback and customizes the AI's behavior based on that feedback. For example, if the user provides feedback such as "Please provide more detailed explanation next time," that feedback is sent to the server and reflected in the next response.

[0385] 10. Performance Evaluation:

[0386] The server creates new evaluation indicators based on work history, emotional data, and user feedback to evaluate AI performance. For example, it creates an indicator that combines response speed, accuracy, and user satisfaction.

[0387] Examples and prompts

[0388] For example, a user inputs a prompt such as "Please tell me the price of the new product," and the answer generation process and emotional data based on that are processed. The AI's answer to the user's question (e.g., the price of the new product) and the user's emotion in response (e.g., dissatisfaction) are recorded.

[0389] In this way, the AI ​​MemoSphere system records and analyzes the work history of the generative AI model and the user's emotional data in detail, providing a transparent, reliable and personalized service.

[0390] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0391] Step 1:

[0392] Monitoring generated data

[0393] Input: A prompt from the user (e.g., "What is the price of the new product?")

[0394] Output: Details of the answer generation process by the generative AI model

[0395] Specific operation: The server monitors the prompt sentences entered by the user and tracks how the generative AI model (e.g., GPT-3) responds to the prompt. When a user enters "Please tell me the price of the new product," the server sends this prompt sentence to the generative AI model and records its response process.

[0396] Step 2:

[0397] Collecting task details

[0398] Input: Responses output by the generative AI model, prompts, algorithms, and user emotional data

[0399] Output: Collected task details data

[0400] What it does: The server collects the answer generated by the generative AI model (e.g., "The price of the new product is $100"), the algorithm used (e.g., the version and specific settings of GPT-3), the input prompt, and the user's emotional data. These details are then compiled and stored in a database.

[0401] Step 3:

[0402] Recognizing user emotions

[0403] Input: User's facial expression data, voice data

[0404] Output: Emotion recognition result (e.g., dissatisfied, satisfied)

[0405] Specific operation: The device captures the user's facial expressions and tone of voice when entering prompts using a camera and microphone, and analyzes them using an emotion engine (e.g., Microsoft Azure's Emotion API). The user may say, with a dissatisfied tone, "Is this really the right price?", and their facial and voice data is captured.

[0406] Step 4:

[0407] Emotional data recording

[0408] Input: Emotion recognition results

[0409] Output: Emotion data stored in a database

[0410] Specific operation: The server receives the emotion data sent from the device and stores it in a database. For example, it stores the emotion data "dissatisfied" in the database.

[0411] Step 5:

[0412] Auto-generate notes

[0413] Input: Collected task details data, emotion data

[0414] Output: Auto-generated notes

[0415] Specific operation: The server generates a memo based on the collected task details and emotion data. This includes a brief summary of the task and key points. For example, it may summarize all tasks performed by the AI ​​in the past week and create a summary memo including the user's associated emotion data.

[0416] Step 6:

[0417] Accessing the User Interface

[0418] Input: User's reference request

[0419] Output: Display of created note

[0420] Specific operation: The user refers to the work history memo through a dedicated user interface (e.g., a web browser). The user accesses the dashboard on the browser to view details of past interactions and emotion data.

[0421] Step 7:

[0422] Data integration with other AI services

[0423] Input: Data from other AI services (e.g., image recognition results)

[0424] Output: Integrated work history and emotion data

[0425] Specific operation: The server works with image recognition AI (e.g., image analysis system) in addition to the generation AI, and integrates their work history and emotional data. For example, a user uploads a product image and records the answer generation process based on it.

[0426] Step 8:

[0427] Extracting optimal work patterns

[0428] Input: Integrated work history, emotional data

[0429] Output: The optimal work pattern found

[0430] Specific operation: The server performs detailed analysis of the integrated work history and emotion data to extract the optimal work pattern. For example, it analyzes multiple response patterns to find the fastest and most effective algorithm and apply it to the next task.

[0431] Step 9:

[0432] Collaboration with Feedback

[0433] Input: User feedback

[0434] Output: Customized AI response

[0435] Specific operation: The device collects feedback from the user and sends it to the server. For example, if the user says, "I'd like more detailed explanation next time," the device customizes the AI's response based on this feedback and reflects it in the next response.

[0436] Step 10:

[0437] Performance Evaluation

[0438] Input: Work history, emotion data, feedback

[0439] Output: Generated performance metrics

[0440] Specific operation: The server creates new evaluation indicators based on work history, emotional data, and user feedback to evaluate the AI's performance. For example, it creates an indicator that combines response speed, accuracy of generated results, and user satisfaction, and evaluates the AI's performance based on that.

[0441] In this way, the AI ​​MemoSphere system records, manages, and analyzes the work history of the generative AI model and user emotional data in detail, providing transparent, reliable, and personalized services.

[0442] (Application example 2)

[0443] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0444] Modern customer service requires accurately understanding customer emotions and needs and providing appropriate responses in real time. However, traditional systems make it difficult to provide services that fully reflect customer emotions, and as a result, they are unable to provide responses that increase customer satisfaction. Furthermore, it is difficult to record detailed customer interaction history and refer to it later, resulting in a lack of data to continuously improve service quality.

[0445] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring generated data, means for collecting details, means for storing details in a database, means for generating memos, means for providing the memos through a user interface, means for integrating the collected details in cooperation with other AI services, means for analyzing the integrated details to find optimal work patterns, means for improving processes based on the optimal work patterns, means for capturing visual and audio data and recognizing user emotions, means for storing the emotion data in a database and incorporating it into memos, and means for real-time recording and emotion recognition using a smart device. This enables accurate understanding of customer emotions and appropriate responses in real time. Furthermore, by recording details of interactions with customers and referencing them later, service quality can be continuously improved and customer satisfaction can be increased.

[0446] "Generated data" is information that results from tasks performed by artificial intelligence.

[0447] "Details" are specific information including input data to the generated data, algorithms used, output results, and other relevant information.

[0448] A "database" is a system for storing collected details and emotional data for later reference and analysis.

[0449] A "memo" is a summary document generated based on collected details and emotion data, and is information provided to the user.

[0450] "User interface" refers to the interaction environment through which a user accesses the system and views and inputs notes and data.

[0451] An "artificial intelligence service" is a collection of AI systems designed to perform a specific task, such as text generation or image recognition.

[0452] An "optimal work pattern" refers to the most efficient and accurate algorithm or process for multiple tasks.

[0453] "Visual data" refers to image and video data captured by a camera or other imaging device.

[0454] "Audio data" refers to recorded audio data captured by an audio device such as a microphone.

[0455] "Emotion data" refers to data relating to the user's emotional state analyzed from facial expressions, tone of voice, and the like.

[0456] A "smart device" is an interactive device such as glasses equipped with a camera and microphone.

[0457] "Recording" is the act of saving interaction details and emotional data in a database.

[0458] "Emotion recognition" is the process of analyzing and identifying a user's emotions from visual and audio data.

[0459] The present invention is a system for capturing visual and audio data in real time, recognizing user emotions, and providing high-quality customer service based on the data. To implement the present invention, the following technical configuration and steps are required.

[0460] First, a smart device (e.g., smart glasses equipped with a camera and microphone) captures customer interactions in real time. The device then sends the captured video and audio data to the EmotionEngine (an emotion recognition library) to recognize the user's emotions. The recognized emotion data is then immediately sent to a server and stored in the MemoSphere system's database.

[0461] The server then monitors the data generated by the AI ​​model and collects details (such as input data, algorithms used, output results, and emotional data). The collected data is stored in a database for future reference. For example, if a customer asks about the price of a particular product, the customer's facial expression and tone of voice can be analyzed to record emotional data such as "interest" or "anxiety."

[0462] The server generates appropriate notes for users based on real-time interactions and emotional data. These notes reflect the customer's questions and emotional changes and are provided as specific, personalized information. The generated notes are provided to store clerks through the user interface, who then refer to them when assisting the customer.

[0463] In addition, the server will also work with other artificial intelligence services, integrating functions such as text generation and image recognition, enabling more advanced information provision. For example, when a customer presents a product image, the image can be recognized and relevant information can be instantly provided.

[0464] The system also analyzes optimal work patterns, identifying the most efficient algorithms and processes across multiple tasks and automatically applying them to the next task. Based on collected data and sentiment data, the system automatically improves processes and enhances customer service.

[0465] In this way, smart devices can be used to understand customer sentiment in real time and take appropriate action based on that sentiment. Furthermore, detailed records and analysis of the collected data can be used to continuously improve the quality of service.

[0466] For example, when a store clerk wearing smart glasses interacts with a customer, the customer asks about the price of a product. The smart glasses' camera captures the customer's facial expressions, and the Emotion Engine analyzes their expressions for interest and anxiety. The MemoSphere system then records the details of the interaction along with their emotions and generates a summary memo that can be referenced later.

[0467] An example prompt is:

[0468] "Analyze the emotions customers express when asking about product prices. Record the emotion data and details of the interaction."

[0469] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0470] Step 1:

[0471] The device captures visual and audio data. Specifically, it uses the camera and microphone of the smart glasses to record real-time interactions with customers. The input is the video and audio data obtained from the camera and microphone, and the output is the process of sending this data to the EmotionEngine.

[0472] Step 2:

[0473] The device uses the EmotionEngine to recognize customer emotions from captured visual and audio data. Specifically, it analyzes facial expressions from video and tone from audio. The input is visual and audio data, and the output is recognized emotional data.

[0474] Step 3:

[0475] The device sends the generated emotion data to the server. Specifically, the output data from the Emotion Engine is transferred to the server in real time. The input is the emotion data, and the output is the data reaching the server.

[0476] Step 4:

[0477] The server stores the received emotion data and interaction details in the MemoSphere system database. Specifically, it records detailed information such as the interaction content, the algorithm used, and the output results. The input is emotion data and detailed interaction information, and the output is storing this in the database.

[0478] Step 5:

[0479] The server generates memos based on the collected data. Specifically, it integrates the content of the interaction with emotional data to create memos containing key points and summaries. The input is the details and emotional data stored in the database, and the output is the generated memo.

[0480] Step 6:

[0481] The server provides the generated memo to the terminal through a user interface. Specifically, the memo can be viewed by the store clerk through smart glasses. The input is the generated memo, and the output is the memo as information provided to the user.

[0482] Step 7:

[0483] The server works with other AI services to integrate collected data and emotional data. Specifically, it integrates data from other AI services such as text generation and image recognition, enabling centralized information acquisition. The input is data from other AI services, and the output is the integrated detailed data.

[0484] Step 8:

[0485] The server analyzes the integrated detailed data and finds the optimal work pattern. Specifically, it identifies the most efficient algorithms and processes for multiple tasks. The input is the integrated detailed data, and the output is the optimal work pattern.

[0486] Step 9:

[0487] The server improves the process based on the optimal work pattern. Specifically, it automatically applies the newly identified work pattern to the next task, improving efficiency and quality. The input is the optimal work pattern, and the output is the improved process.

[0488] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0489] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0490] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0491] [Second embodiment]

[0492] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0493] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0494] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0495] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0496] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0497] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0498] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0499] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0500] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0501] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0502] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0503] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0504] The AI ​​MemoSphere system of the present invention aims to record the history of tasks performed by an artificial intelligence, generate memos based on that history, and provide them to users in a transparent and reliable manner. Below, we will explain in natural language the program processing of a specific embodiment of this system, and provide detailed examples.

[0505] Recording work history

[0506] The server monitors the generated data in real time. For example, when a generative AI generates an answer to a user's question, the server continuously monitors this process.

[0507] The server collects details of each task (input data, algorithm used, output results, etc.) and stores them in a database. For example, if a user asks a question about a product, the generative AI's answer to that question, the process leading to that answer, and the version of the model and algorithm used are recorded.

[0508] Create and manage notes

[0509] The server generates memos based on the collected work history, including task summaries and key points.

[0510] Example: The server compiles important information about the answer generation tasks performed by the AI ​​over the past week and saves it as a summary memo. This summary memo includes information such as what types of questions were most popular, the trends in the answers, and which algorithms were most frequently used.

[0511] Viewing work history

[0512] Users can view their work history notes through a dedicated user interface (UI), which is available as a browser-based or desktop application.

[0513] Example: A user can access the UI to see the AI's answer process for their question and view detailed notes, making it easier to understand how the answer was generated.

[0514] Integration with other AI services

[0515] The server connects with other AI services via APIs and integrates the work history of each AI service.

[0516] Example: By linking text generation AI and image recognition AI, when a user asks a text question, they can also provide an image, and the process of generating an answer based on that is recorded and integrated. Users can then refer to this link history in a unified manner.

[0517] Analysis of optimal work patterns

[0518] The server analyzes the integrated work history data and finds optimal work patterns.

[0519] Example: The server finds the most efficient algorithms and processes across multiple tasks and automatically applies those patterns to the next task, improving the accuracy and speed of AI responses and significantly improving the user experience.

[0520] Adapting to user needs

[0521] The device collects user feedback and uses this to customize the AI's behavior.

[0522] Example: If a user requests a specific expression or format, that feedback will be reflected and the next AI output will be provided in a format that matches that preference.

[0523] Creating metrics and evaluating performance

[0524] The server creates new performance metrics based on collected details and user feedback to evaluate the AI's performance.

[0525] Example: Evaluations are conducted based on a combination of multiple metrics, including response speed and accuracy, as well as user satisfaction, to identify areas for improvement in AI and improve its quality.

[0526] Adapts to user preferences and work styles

[0527] The device analyzes work history data and learns the user's preferences and work style.

[0528] Example: It learns the tone and style of responses preferred by a particular user and adjusts output accordingly, allowing it to best meet the user's individual needs.

[0529] In this way, the AI ​​MemoSphere system records, manages, and analyzes the work history of the generating AI in detail, and provides users with a highly transparent service based on that information, which improves the reliability of the AI ​​and enables efficient, high-quality work.

[0530] The processing flow will be explained below.

[0531] Step 1:

[0532] The server monitors the generated data in real time. Specifically, when the generating AI starts a task, it obtains the task's identification information and triggers the monitoring process.

[0533] Step 2:

[0534] The server collects details of each task, including the input data, the algorithms used, each step of the processing, and the output results, and then organizes and structures the data.

[0535] Step 3:

[0536] The server stores the collected details in a database, which stores the data according to an appropriate schema in a format that can be queried later.

[0537] Step 4:

[0538] The server generates notes based on details stored in the database, summarizing the key points of the data and automatically creating easy-to-understand notes in natural language.

[0539] Step 5:

[0540] The server provides the generated notes through a dedicated user interface (UI), which is designed as a browser-based or desktop application.

[0541] Step 6:

[0542] Users can access work history notes through the user interface, allowing them to search for the history of a specific task or check the contents of the notes.

[0543] Step 7:

[0544] The server will connect with other AI services via API, allowing work history data to be sent and received between the services.

[0545] Step 8:

[0546] The server integrates the work history collected from the linked AI services, thereby centrally managing the historical data of multiple services.

[0547] Step 9:

[0548] The server analyzes the integrated data to identify optimal work patterns, applying machine learning algorithms and statistical models to identify the most effective processes and procedures.

[0549] Step 10:

[0550] The server then uses the analytical results to improve the processing of the generated data, specifically by automatically applying optimized algorithm settings and parameters.

[0551] Step 11:

[0552] The terminal provides an interface for collecting user feedback, allowing the user to input their opinions on the user interface.

[0553] Step 12:

[0554] The server analyzes the collected feedback and calculates metrics to evaluate the performance of the generated data. It combines the feedback content and work history to derive various evaluation indicators.

[0555] Step 13:

[0556] The server learns the user's preferences and working style, which involves analyzing past usage data and feedback to extract the user's specific needs and tendencies.

[0557] Step 14:

[0558] The server then customizes the processing of the generated data based on the learning results, providing responses and functionality that match the user's preferences and working style.

[0559] Example 1

[0560] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0561] There is a growing need to improve the transparency and reliability of information generated in modern automation systems. However, current systems have difficulty effectively managing and analyzing the detailed history of generated information, making it difficult for users to understand the generation process. Furthermore, it is difficult to efficiently link different automation services and discover and apply the optimal work format. For these reasons, there is a need for a system that can improve the reliability and transparency of generated information and effectively manage and analyze it.

[0562] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0563] In this invention, the server includes means for monitoring generated information, means for collecting details of the generated information, means for storing the collected details in a storage device, means for generating a summary based on the details stored in the storage device, means for providing the generated summary through a user interface, means for integrating the collected details in cooperation with other automated services, means for analyzing the integrated details to find an optimal work format, and means for improving the process of the generated information based on the optimal work format. This makes it possible to manage and analyze the history of generated information in detail and provide highly transparent services to users.

[0564] "Generated Information" means output or data generated by an automated system based on user input.

[0565] "Monitoring means" refers to a device or program that tracks and records the process and results of the information generated in real time.

[0566] "Means of collection" refers to a device or program that systematically acquires and stores details of the generated information and related data.

[0567] "Storage" refers to physical and logical storage for storing collected detailed data.

[0568] "Means for generating a summary" refers to a device or program that generates a compact, easily understandable summary from the detailed data collected.

[0569] "User interface" refers to the means, including screens and applications, by which a user interacts with a system.

[0570] "Automated services" refer to programs or systems that autonomously perform specific tasks.

[0571] "Integration means" refers to a device or program that centralizes the collected detailed data with data from other automated services and manages it in a unified format.

[0572] The "optimal work format" refers to the most efficient and accurate method among multiple work processes.

[0573] "Means for improving" refers to a device or program for improving the process of generating information based on the optimal working format.

[0574] MODE FOR CARRYING OUT THE INVENTION

[0575] The AI ​​MemoSphere system of the present invention is a system designed to increase the transparency and reliability of generated information. The following describes how the present invention is specifically implemented. The specific names of the hardware and software used are also provided.

[0576] System Configuration

[0577] The system of the present invention consists of the following main components:

[0578] 1. Server

[0579] Hardware used: High-performance servers (e.g., AWS EC2 instances)

[0580] Software used: log collection tools (e.g., Elasticsearch), databases (e.g., PostgreSQL), automatic summary generation tools (e.g., NLTK), data analysis tools (e.g., pandas, scikit-learn), performance measurement tools (e.g., TensorBoard), REST API (e.g., Flask)

[0581] 2. Terminal

[0582] Hardware used: User device (e.g., PC, smartphone)

[0583] Software used: web browser (e.g., Chrome), feedback collection tool (e.g., Google Forms), machine learning model (e.g., scikit-learn)

[0584] 3. User Interface (UI)

[0585] Software used: Browser-based or desktop application

[0586] Program processing explanation

[0587] The specific operation and processing of this system will be explained in natural language below.

[0588] Recording work history

[0589] The server monitors in real time the process by which the generative AI model (e.g., GPT-4) generates an answer based on the user's input. For example, when a user inputs "What's the weather like?", the server collects data until the generative AI model replies "Today's weather is sunny." The collected data includes the input data, the algorithm used, the generated output, etc. This data is stored in a storage device by the server.

[0590] Create and manage notes

[0591] The server generates a summary memo based on the collected work history. This summary includes a summary of the task and key points. For example, the server compiles the answer history of the generation AI over the past week and generates a summary memo that includes frequently asked questions, their trends, and information about the version of the algorithm used.

[0592] Viewing work history

[0593] Users can refer to the work history notes through the user interface (UI). For example, users can access the UI to check the past question history and the answering process of the generation AI, and view detailed notes.

[0594] Integration with other AI services

[0595] The server connects with other automation services, such as image recognition AI, via API and integrates the work history data of each AI service. For example, a user can provide an image along with a text question, and the server records and integrates the answer process based on that.

[0596] Analysis of optimal work patterns

[0597] The server analyzes the integrated work history data and finds the optimal work format. For example, the server can identify the most efficient algorithms and processes across multiple tasks and automatically apply them to future tasks, improving the accuracy and speed of answers.

[0598] Adapting to user needs

[0599] The device collects user feedback and uses it to customize the generative AI model's responses. For example, if a user sends feedback such as "I like this expression," that preference will be reflected in the next answer generated.

[0600] Adapts to user preferences and work styles

[0601] The device learns user preferences and styles based on historical work data, for example, learning the tone and style of responses preferred by a particular user, and adjusts the output of the generative AI model accordingly.

[0602] Specific examples

[0603] Here are some examples of prompts:

[0604] "Please tell me the weather."

[0605] When a user enters this prompt, the server uses a generative AI model to generate an answer and records the process in detail. A summary memo is generated based on this record, and the user can view the details through the UI. By linking with other image recognition AI, the user can provide additional information and obtain a more detailed answer. The integrated data is analyzed to identify the optimal work format and apply it to the next task. The AI's responses are continuously improved based on user feedback and historical data.

[0606] In this way, the AI ​​MemoSphere system can manage and analyze the history of generated information in detail, providing users with transparent and reliable services.

[0607] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0608] Step 1:

[0609] The user inputs a prompt sentence into the user interface (UI), for example, "Please tell me the weather."

[0610] Input: prompt statement

[0611] Output: User input data

[0612] What happens: A user enters a prompt sentence using a browser or desktop application.

[0613] Step 2:

[0614] The server receives the prompt sentence sent by the user and inputs it into a generative AI model, which then generates an answer (e.g., GPT-4).

[0615] Input: User-entered data

[0616] Output: Answer data generated by the generative AI model

[0617] How it works: The server receives the input "What is the weather like?", submits it to a generative AI model, and generates the answer "It's sunny today."

[0618] Step 3:

[0619] The server monitors the generated answers and the process in real time, collecting data, including the algorithms and model versions used.

[0620] Input: Answer data generated by a generative AI model

[0621] Output: Detailed data on the generation process

[0622] How it works: The server monitors the entire generation process in real time and collects detailed data such as "Question: What's the weather like?", "Answer: It's sunny today," "Model used: GPT-4," and "Time: XX seconds."

[0623] Step 4:

[0624] The server stores the collected detailed data in a storage device (database).

[0625] Input: Detailed data of the generation process

[0626] Output: Detailed data stored in a database

[0627] Specific operation: The server stores the collected detailed data in a database (e.g., PostgreSQL).

[0628] Step 5:

[0629] The server generates a summary memo based on the detailed data stored in the storage device, which includes a summary of the task and important points.

[0630] Input: Detailed data stored in the database

[0631] Output: Generated summary notes

[0632] Specific operation: The server uses an automatic summary generation tool (e.g., NLTK) to summarize the generation AI's answer history for the past week and create a summary memo.

[0633] Step 6:

[0634] The server provides the generated summary memo through a user interface (UI).

[0635] Input: Generated summary note

[0636] Output: A user-visible summary note

[0637] Specific operation: The server displays the summary memo through the UI (browser or desktop application) so that the user can view it.

[0638] Step 7:

[0639] The server connects with other automation services via APIs and integrates the work history data of each service.

[0640] Input: Work history data from other automated services

[0641] Output: Integrated work history data

[0642] Specific operation: The server uses a REST API (e.g., Flask) to integrate data from image recognition AI, etc.

[0643] Step 8:

[0644] The server analyzes the integrated work history data and finds the optimal work format.

[0645] Input: Integrated work history data

[0646] Output: Data in a format that works best for you

[0647] Specific behavior: The server uses data analysis tools (e.g., pandas, scikit-learn) to identify efficient algorithms and processes.

[0648] Step 9:

[0649] The server improves the process of generating the generated information based on the most suitable working format.

[0650] Input: Data in the format that works best for you

[0651] Output: Improved generation process

[0652] Specific operation: The server automatically applies the identified optimal work format to the next task, improving the accuracy and speed of the generative AI model.

[0653] Step 10:

[0654] The device collects user feedback and uses it to customize the responses of the generative AI model.

[0655] Input: User feedback

[0656] Output: The customized generative AI model response

[0657] Specific operation: The device collects feedback from users using a feedback collection tool (e.g., Google Forms) and reflects it in the output of the generative AI model.

[0658] Step 11:

[0659] The device learns the user's preferences and style based on work history data and adjusts the output of the generative AI model accordingly.

[0660] Input: Work history data

[0661] Output: The output of the tuned generative AI model.

[0662] What it does: The device uses a machine learning model (e.g., scikit-learn) to learn the response format and tone preferred by a particular user and reflects that in the next output.

[0663] (Application example 1)

[0664] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0665] In modern logistics centers, maximizing work efficiency and minimizing errors are important. However, recording and analyzing work history, and then flexibly modifying work plans based on that information, is not easy. Current systems lack the tools to ensure data transparency and reliability while working efficiently. A major problem is the lack of a function that allows on-site staff to input information in real time and generate optimal notes based on that information. A new system is needed to solve these issues.

[0666] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0667] In this invention, the server includes means for monitoring generated data, means for collecting details of the generated data, means for storing the collected details in a database, means for generating memos based on the details stored in the database, means for providing the generated memos through a user interface, means for integrating the collected details in cooperation with other artificial intelligence services, means for analyzing the integrated details to find optimal work patterns, means for improving the process of the generated data based on the optimal work patterns, means for recording work history, generating memos, and improving work efficiency in the logistics center, means for inputting work information via a smartphone application and sending it to the server, and means for sending generated memos from the server to the smartphone application, thereby improving work efficiency in the logistics center, reducing work errors, and ensuring data transparency and reliability.

[0668] "Generated data" is data generated by artificial intelligence systems or other information systems.

[0669] "Monitoring means" refers to a means for observing the generated data in real time and detecting fluctuations or abnormalities.

[0670] "Collection means" refers to the means for systematically capturing and recording detailed information about the data generated.

[0671] A "database" is an area where detailed information on collected data is systematically stored and can be retrieved as needed.

[0672] The "means for generating memos" refers to a means for organizing the main points based on the collected detailed information and generating concisely written memos.

[0673] A "user interface" is an interface that provides a screen and operating means for a user to interact with a system.

[0674] "Artificial Intelligence Services" are services based on artificial intelligence technology that are utilized to perform specific tasks.

[0675] An "optimal work pattern" is an efficient and effective work procedure or method discovered by analyzing past work history.

[0676] "Process improvement measures" are measures to improve current work procedures and methods based on optimal work patterns, thereby increasing efficiency and effectiveness.

[0677] A "smartphone application" is software that runs on a smartphone and provides specific functions.

[0678] "Work information" is information relating to the specific content and status of work being carried out at the logistics center.

[0679] A "server" is a computer system for processing, storing, and managing data.

[0680] The purpose of this invention, the "AI MemoSphere System," is to improve work efficiency at logistics centers, reduce work errors, and ensure data transparency and reliability. Specific implementations of this system are described below.

[0681] Hardware and software used

[0682] Hardware:

[0683] Server: AWS server (EC2)

[0684] Device: Smartphone (iOS / Android)

[0685] software:

[0686] Server side: Python (Django framework)

[0687] Database: PostgreSQL

[0688] Frontend: React Native

[0689] AI model: GPT-4 (API provided by OpenAI)

[0690] API integration: AWS API Gateway

[0691] System Program

[0692] Recording work history

[0693] The server monitors information about each operation performed at the distribution center in real time and collects the generated data. Specifically, staff enter operation information (e.g., item ID, quantity, picking time) through a smartphone application, and the data is sent to the server, which then stores it in a database.

[0694] Generate notes

[0695] Based on the collected work data, the server uses an AI model (GPT-4) to generate memos, which concisely summarize the main points and important information of the work. These memos are then sent from the server to a smartphone application where they can be viewed by staff.

[0696] Viewing work history

[0697] Users (logistics center staff and managers) can view work history and generated notes through a smartphone application, making it easy to understand how each task was performed and what results were achieved.

[0698] Analysis of optimal work patterns

[0699] The server analyzes the collected comprehensive work data to find the optimal work pattern, and the results of this analysis are reflected in the next work, allowing more efficient methods to be automatically applied.

[0700] Specific examples

[0701] At a distribution center, staff have a task of picking items from shelves. When they input the task data (e.g., item ID, quantity, picking time) into a smartphone application, the history is sent to the server and stored in a database. GPT-4 creates notes from the generated history, and these notes can be viewed by staff and managers through the app.

[0702] Prompt Sentence Examples

[0703] Below are some example prompts sent to the AI ​​model:

[0704] Task ID: 123 Details:

[0705] Input data: Item ID: A001, Quantity: 10, Picking time: 15 minutes

[0706] Algorithm used: Picking algorithm version 2.0

[0707] Output: Success, all items picked correctly

[0708] Generated note:

[0709] In this task, 10 units of item ID A001 were accurately picked. The picking algorithm used was version 2.0, which took 15 minutes to complete. This algorithm was effective for this task.

[0710] In this way, the AI ​​MemoSphere system can support skilled personnel in logistics centers and provide a highly transparent and efficient working environment.

[0711] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0712] Step 1:

[0713] The server monitors the generated data. Staff at the logistics center use a smartphone application to input work information. This input (e.g., item ID, quantity, picking time) is sent to the server. The server receives and monitors this data in real time. The input data is work information such as item ID, quantity, and picking time, and the output is the data received by the server.

[0714] Step 2:

[0715] The server collects details of the generated data. The server analyzes the task data sent from the smartphone application and extracts detailed information. The collected data includes the execution time of each task, the version of the algorithm used, and the success or failure of the task. These detailed information are stored in a database. The input data is the task information from the smartphone application, and the output is the detailed information stored in the database.

[0716] Step 3:

[0717] The server generates a memo based on the detailed information stored in the database. The server sends the collected work data details to the AI ​​model (GPT-4) and receives the generated memo. The prompt sentence to be sent to the AI ​​model is composed as follows:

[0718] Task ID: 123 Details:

[0719] Input data: Item ID: A001, Quantity: 10, Picking time: 15 minutes

[0720] Algorithm used: Picking algorithm version 2.0

[0721] Output: Success, all items picked correctly

[0722] Generated note:

[0723] In this task, 10 units of item ID A001 were accurately picked. The picking algorithm used was version 2.0, which took 15 minutes to complete. This algorithm was effective for this task.

[0724] The input data are the collected details, and the output are the notes generated by the AI ​​model.

[0725] Step 4:

[0726] The server provides the generated notes through a user interface. The generated notes are sent to a smartphone application so that the user can view them. The user can check the notes for each task through the smartphone application and understand the details of the task. The input data are the notes generated by the AI ​​model, and the output is the notes displayed in the user interface.

[0727] Step 5:

[0728] The server integrates the collected details in cooperation with other AI services. For example, by collaborating with an image recognition AI, it also collects image data and integrates detailed information based on that. This generates data that includes not only text information but also image information. The input data is detailed information from other AI services, and the output is the integrated detailed information.

[0729] Step 6:

[0730] The server analyzes the integrated details to find the optimal work pattern. It compares multiple work histories and extracts the most efficient work procedures and methods. This allows the optimal pattern to be applied in future work. The input data is the integrated details, and the output is the optimal work pattern.

[0731] Step 7:

[0732] The server improves the process of the generated data based on the optimal work pattern. Based on the discovered optimal pattern, it automatically corrects the current work procedures and methods to improve efficiency. The input data is the optimal work pattern, and the output is the improved work process.

[0733] Step 8:

[0734] The device collects user feedback and uses it to customize the AI's behavior. Feedback provided by the user through a smartphone application improves the AI ​​model's output and adapts to the user's needs. The input data is the user's feedback, and the output is the customized AI model's behavior.

[0735] In this way, the AI ​​MemoSphere System can improve work efficiency and ensure data transparency in logistics centers, providing a more reliable work environment.

[0736] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0737] The AI ​​MemoSphere system of the present invention aims to record the history of tasks performed by an artificial intelligence, generate memos based on that history, and provide them to users in a transparent and reliable manner. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions, and by taking emotion data into consideration, it is possible to provide more personalized services. Below, a specific embodiment of this system will be described in detail, with program processing explained in natural language and examples provided.

[0738] Recording work history

[0739] The server monitors the generated data in real time. For example, when a generative AI generates an answer to a user's question, the server continuously monitors this process.

[0740] The server collects details of each task (input data, algorithm used, output results, user emotion data, etc.) and stores them in a database. When a user asks a question about a product, the generative AI's answer, the process leading to that answer, the model and algorithm version used, and the user's emotion are recorded.

[0741] Use of emotion engine

[0742] The device analyzes the user's facial expressions and tone of voice as they type, and uses an emotion engine to recognize the user's emotions.

[0743] The server stores the emotion data recognized by the emotion engine in a database. For example, if the user is dissatisfied, the emotion data is also recorded.

[0744] Create and manage notes

[0745] The server generates memos based on the collected work history and emotional data. The generated memos include a summary of the task, important points, and the user's emotional data.

[0746] Example: The server generates a summary memo containing important information about the answer generation tasks performed by the AI ​​over the past week, as well as changes in user sentiment. This summary memo includes information such as what types of questions were most popular, the trends in their answers, and the user's emotional reactions.

[0747] Viewing work history

[0748] Users can access their work history notes through a dedicated user interface (UI), which can be provided as a browser-based or desktop application.

[0749] Example: A user can access the UI to see the AI's answer process to their question and the emotional data at the time, and view detailed notes, making it easier to understand how the answer was generated and how it reflected their emotional state at the time.

[0750] Integration with other AI services

[0751] The server connects with other AI services via APIs and integrates the work history and emotional data of each AI service.

[0752] Example: By linking text generation AI and image recognition AI, users can simultaneously provide images when asking text questions, and the process of generating answers based on these images is recorded and integrated. Users can then refer to this link history and emotion data in a unified manner.

[0753] Analysis of optimal work patterns

[0754] The server analyzes the integrated work history data and emotion data to find optimal work patterns.

[0755] Example: The server finds the most efficient algorithms and processes across multiple tasks and automatically applies those patterns to the next task. This takes emotional data into account, improving the accuracy of AI responses and user satisfaction.

[0756] Adapting to user needs

[0757] The device collects user feedback and uses this to customize the AI's behavior.

[0758] Example: If a user requests a particular style, format, or even a desired emotional tone, that feedback will be incorporated and the next AI output will be tailored to those preferences.

[0759] Creating metrics and evaluating performance

[0760] The server creates new performance indicators based on collected work history, emotional data, and user feedback to evaluate the AI's performance.

[0761] Example: In addition to response speed and accuracy, evaluations are conducted based on a combination of multiple metrics, such as user satisfaction and emotional response, to identify areas for improvement in AI and improve its quality.

[0762] Improving processes by taking emotional data into account

[0763] The server then uses the analysis results to improve the processing of the generated data, specifically by automatically applying algorithm settings and parameters that reflect the user's emotional data.

[0764] Example: For tasks that are likely to frustrate users, the AI ​​is tuned to generate more polite responses.

[0765] Adapts to user preferences and work styles

[0766] The device analyzes work history data and emotional data to tailor responses to best suit the user's preferences and work style.

[0767] Example: Learning the tone and style of responses preferred by a particular user and tailoring output accordingly, allowing it to best meet the user's individual needs and emotions.

[0768] In this way, the AI ​​MemoSphere system records, manages, and analyzes the work history of the generating AI and user emotional data in detail, and provides users with highly transparent services based on this information. This improves the reliability and personalization of the AI, enabling efficient and high-quality work.

[0769] The processing flow will be explained below.

[0770] Step 1:

[0771] The server monitors the generated data in real time. Specifically, when the generation AI starts a task, it acquires the task's identification information and triggers the monitoring process. For example, when a user enters a question, it generates a question ID and records it as a monitoring target.

[0772] Step 2:

[0773] The device analyzes the user's facial expressions and tone of voice when inputting and uses an emotion engine to recognize the user's emotions. Specifically, it uses a camera and microphone to detect the user's emotional state in real time and sends that data to the emotion engine.

[0774] Step 3:

[0775] The server stores the user's emotion data recognized by the emotion engine in a database. For example, if the user makes a dissatisfied expression, the emotion data is recorded and associated with the question ID.

[0776] Step 4:

[0777] The server collects details of each task (input data, algorithm used, processing steps, output results, and user emotional data), including the question entered by the user, the process of the generative AI, and the final answer.

[0778] Step 5:

[0779] The server stores the collected details in a database, which stores the data according to an appropriate schema in a format that can be queried later.

[0780] Step 6:

[0781] The server generates notes based on the details stored in the database, including a summary of the task, key points, and the user's emotional data.

[0782] Step 7:

[0783] The server provides the generated notes through a dedicated user interface (UI), which is designed as a browser-based or desktop application.

[0784] Step 8:

[0785] The user can view the work history notes through the user interface. The user can check the history of a specific task, the contents of the notes, and the emotional data at the time.

[0786] Step 9:

[0787] The server will connect with other AI services via API, allowing for the sending and receiving of work history data and emotion data between the connected services.

[0788] Step 10:

[0789] The server integrates the work history and emotion data collected from the linked AI services, thereby centrally managing the history data from multiple services.

[0790] Step 11:

[0791] The server analyzes the integrated data to identify optimal work patterns, applying machine learning algorithms and statistical models to identify the most effective processes, procedures, and emotional responses.

[0792] Step 12:

[0793] The server then uses the analysis results to improve the processing of the generated data, specifically by automatically applying optimized algorithm settings and parameters that also take emotion data into account.

[0794] Step 13:

[0795] The terminal provides an interface for collecting user feedback, allowing users to input their opinions and thoughts on the user interface.

[0796] Step 14:

[0797] The server analyzes the collected feedback and calculates metrics to evaluate the performance of the generated data. It combines the feedback content, work history, and emotional data to derive various evaluation indicators.

[0798] Step 15:

[0799] The server learns the user's preferences and work style, which involves analyzing past usage data, feedback, and sentiment data to extract the user's specific needs and tendencies.

[0800] Step 16:

[0801] The server then customizes the processing of the generated data based on the learning results, delivering responses and functionality in a format that best suits the user's preferences, work style, and emotions.

[0802] Example 2

[0803] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0804] Conventional AI systems focus on generating answers to user questions, but the process often lacks transparency and reliability. They also lack the ability to recognize user emotions and provide responses that take these into account. This makes it difficult to provide truly useful and personalized services to users. Furthermore, there are also issues with inconsistencies in analyzing optimal work patterns and integrating with other AI services.

[0805] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0806] In this invention, the server includes means for monitoring the generated data, means for collecting details of the generated data, means for storing the collected details in a database, means for generating memos based on the details stored in the database, means for providing the generated memos through a user interface, means for integrating the collected details in cooperation with other AI services, means for analyzing the integrated details to find optimal work patterns, means for improving the process of the generated data based on the optimal work patterns, means for recognizing user emotion data, means for collecting and storing the recognized emotion data in a database, and means for generating memos based on the collected and stored emotion data. This makes it possible to record, manage, and analyze the work history of the generative AI model and the user emotion data in detail, and to provide users with a transparent and reliable personalized service.

[0807] "Generated Data" refers to the responses or results generated by a generative AI model in response to a user's question or request.

[0808] "Monitoring means" refers to a system including hardware and software for monitoring the generated data in real time and understanding its contents.

[0809] "Collection means" refers to the set of processes or functions used to compile and record details of the data obtained through surveillance.

[0810] "Database" refers to a system of information collection that stores details of collected data in an organized manner so that they can be easily retrieved and used later.

[0811] "Means for generating notes" refers to algorithms or programs that create summary information based on the details and emotional data collected.

[0812] "User Interface" refers to the screens and applications that allow a user to access and manipulate generated notes and data details.

[0813] "Other artificial intelligence services" refers to artificial intelligence technologies and services that exist other than the system of the present invention, and refers to systems that can be linked with these.

[0814] "Means of integration" refers to the ability to bring together data and information obtained from other artificial intelligence services and manage them in a consistent manner.

[0815] "Optimal work patterns" refer to efficient and effective work procedures and processes discovered through detailed analysis of collected data.

[0816] "Measures for improvement" refers to methods and systems for improving the current data generation process based on the optimal work patterns identified.

[0817] "Emotion data" refers to data that indicates the emotional state recognized from the user's statements and actions.

[0818] "Means for recognizing emotions" refers to a system that includes hardware and software for analyzing data such as a user's facial expressions and voice and identifying the user's emotions.

[0819] "Collected and stored emotion data" refers to the state in which the recognized emotion data is stored in a database and can be used for later interpretation and analysis.

[0820] The AI ​​MemoSphere system of the present invention records, manages, and analyzes the work history of the generative AI model and the user's emotional data in detail, and provides users with a transparent, reliable, and personalized service based on the data. Specific embodiments of the system are described below.

[0821] Basic configuration

[0822] This system consists of a server, a terminal, and a user. The server mainly monitors, collects, stores, and analyzes data, while the terminal provides a user interface and collects emotion data.

[0823] Hardware and software used

[0824] Generative AI models: For example, using GPT-3 for natural language processing.

[0825] Server: A high-performance computer server that collects, stores, analyzes, and generates notes on data.

[0826] Database: For example, using a relational database management system (RDBMS).

[0827] Emotion Engine: Uses Microsoft Azure's Emotion API to recognize user emotions.

[0828] User interface: Provided as a web browser or desktop application.

[0829] Processing flow and specific examples

[0830] 1. Monitoring generated data:

[0831] The server monitors the process in real time of the generative AI model generating answers to user questions. For example, if a user asks, "What is the price of a new product?", the request is passed to the generative AI model (GPT-3).

[0832] 2. Collect task details:

[0833] The server collects detailed information about the algorithms used by the generative AI, input data, output results, and user emotional data, and stores it in a database. For example, it records the generative AI model used, the prompt (user question), the output result (product price), and the user's emotional data (dissatisfaction, satisfaction, etc.).

[0834] 3. User emotion recognition:

[0835] The device recognizes the user's facial expression and tone of voice when they input a question and uses an emotion engine to recognize their emotion. For example, if a user says "Is this really the right price?" in a dissatisfied tone, their facial expression and tone of voice are captured.

[0836] 4. Emotional Data Recording:

[0837] The server stores the recognized emotion data in a database. Specifically, the server receives emotion data (e.g., dissatisfaction) sent from the device and adds it to the database.

[0838] 5. Automatic note generation:

[0839] The server generates memos based on the collected work history and emotion data, including task summaries and key points. For example, it summarizes the tasks performed by the AI ​​over the past week and creates a summary memo that includes the user's emotions regarding each task.

[0840] 6. Accessing the User Interface:

[0841] Users can access their work history notes through a dedicated user interface, such as a dashboard in a web browser, to view details of past interactions and sentiment data.

[0842] 7. Data integration with other AI services:

[0843] The server also works with image recognition AI (e.g., an image analysis system) to integrate the entire work history and emotional data. For example, a user can upload a product image and record the answer generation process based on it.

[0844] 8. Extracting optimal work patterns:

[0845] The server then extracts the optimal work pattern based on the integrated work history and emotional data and applies it to the next task, for example by analyzing multiple response patterns and selecting the fastest and most effective algorithm to apply.

[0846] 9. Collaboration with Feedback:

[0847] The device collects user feedback and customizes the AI's behavior based on that feedback. For example, if the user provides feedback such as "Please provide more detailed explanation next time," that feedback is sent to the server and reflected in the next response.

[0848] 10. Performance Evaluation:

[0849] The server creates new evaluation indicators based on work history, emotional data, and user feedback to evaluate AI performance. For example, it creates an indicator that combines response speed, accuracy, and user satisfaction.

[0850] Examples and prompts

[0851] For example, a user inputs a prompt such as "Please tell me the price of the new product," and the answer generation process and emotional data based on that are processed. The AI's answer to the user's question (e.g., the price of the new product) and the user's emotion in response (e.g., dissatisfaction) are recorded.

[0852] In this way, the AI ​​MemoSphere system records and analyzes the work history of the generative AI model and the user's emotional data in detail, providing a transparent, reliable and personalized service.

[0853] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0854] Step 1:

[0855] Monitoring generated data

[0856] Input: A prompt from the user (e.g., "What is the price of the new product?")

[0857] Output: Details of the answer generation process by the generative AI model

[0858] Specific operation: The server monitors the prompt sentences entered by the user and tracks how the generative AI model (e.g., GPT-3) responds to the prompt. When a user enters "Please tell me the price of the new product," the server sends this prompt sentence to the generative AI model and records its response process.

[0859] Step 2:

[0860] Collecting task details

[0861] Input: Responses output by the generative AI model, prompts, algorithms, and user emotional data

[0862] Output: Collected task details data

[0863] What it does: The server collects the answer generated by the generative AI model (e.g., "The price of the new product is $100"), the algorithm used (e.g., the version and specific settings of GPT-3), the input prompt, and the user's emotional data. These details are then compiled and stored in a database.

[0864] Step 3:

[0865] Recognizing user emotions

[0866] Input: User's facial expression data, voice data

[0867] Output: Emotion recognition result (e.g., dissatisfied, satisfied)

[0868] Specific operation: The device captures the user's facial expressions and tone of voice when entering prompts using a camera and microphone, and analyzes them using an emotion engine (e.g., Microsoft Azure's Emotion API). The user may say, with a dissatisfied tone, "Is this really the right price?", and their facial and voice data is captured.

[0869] Step 4:

[0870] Emotional data recording

[0871] Input: Emotion recognition results

[0872] Output: Emotion data stored in a database

[0873] Specific operation: The server receives the emotion data sent from the device and stores it in a database. For example, it stores the emotion data "dissatisfied" in the database.

[0874] Step 5:

[0875] Auto-generate notes

[0876] Input: Collected task details data, emotion data

[0877] Output: Auto-generated notes

[0878] Specific operation: The server generates a memo based on the collected task details and emotion data. This includes a brief summary of the task and key points. For example, it may summarize all tasks performed by the AI ​​in the past week and create a summary memo including the user's associated emotion data.

[0879] Step 6:

[0880] Accessing the User Interface

[0881] Input: User's reference request

[0882] Output: Display of created note

[0883] Specific operation: The user refers to the work history memo through a dedicated user interface (e.g., a web browser). The user accesses the dashboard on the browser to view details of past interactions and emotion data.

[0884] Step 7:

[0885] Data integration with other AI services

[0886] Input: Data from other AI services (e.g., image recognition results)

[0887] Output: Integrated work history and emotion data

[0888] Specific operation: The server works with image recognition AI (e.g., image analysis system) in addition to the generation AI, and integrates their work history and emotional data. For example, a user uploads a product image and records the answer generation process based on it.

[0889] Step 8:

[0890] Extracting optimal work patterns

[0891] Input: Integrated work history, emotional data

[0892] Output: The optimal work pattern found

[0893] Specific operation: The server performs detailed analysis of the integrated work history and emotion data to extract the optimal work pattern. For example, it analyzes multiple response patterns to find the fastest and most effective algorithm and apply it to the next task.

[0894] Step 9:

[0895] Collaboration with Feedback

[0896] Input: User feedback

[0897] Output: Customized AI response

[0898] Specific operation: The device collects feedback from the user and sends it to the server. For example, if the user says, "I'd like more detailed explanation next time," the device customizes the AI's response based on this feedback and reflects it in the next response.

[0899] Step 10:

[0900] Performance Evaluation

[0901] Input: Work history, emotion data, feedback

[0902] Output: Generated performance metrics

[0903] Specific operation: The server creates new evaluation indicators based on work history, emotional data, and user feedback to evaluate the AI's performance. For example, it creates an indicator that combines response speed, accuracy of generated results, and user satisfaction, and evaluates the AI's performance based on that.

[0904] In this way, the AI ​​MemoSphere system records, manages, and analyzes the work history of the generative AI model and user emotional data in detail, providing transparent, reliable, and personalized services.

[0905] (Application example 2)

[0906] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0907] Modern customer service requires accurately understanding customer emotions and needs and providing appropriate responses in real time. However, traditional systems make it difficult to provide services that fully reflect customer emotions, and as a result, they are unable to provide responses that increase customer satisfaction. Furthermore, it is difficult to record detailed customer interaction history and refer to it later, resulting in a lack of data to continuously improve service quality.

[0908] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring generated data, means for collecting details, means for storing details in a database, means for generating memos, means for providing the memos through a user interface, means for integrating the collected details in cooperation with other AI services, means for analyzing the integrated details to find optimal work patterns, means for improving processes based on the optimal work patterns, means for capturing visual and audio data and recognizing user emotions, means for storing the emotion data in a database and incorporating it into memos, and means for real-time recording and emotion recognition using a smart device. This enables accurate understanding of customer emotions and appropriate responses in real time. Furthermore, by recording details of interactions with customers and referencing them later, service quality can be continuously improved and customer satisfaction can be increased.

[0909] "Generated data" is information that results from tasks performed by artificial intelligence.

[0910] "Details" are specific information including input data to the generated data, algorithms used, output results, and other relevant information.

[0911] A "database" is a system for storing collected details and emotional data for later reference and analysis.

[0912] A "memo" is a summary document generated based on collected details and emotion data, and is information provided to the user.

[0913] "User interface" refers to the interaction environment through which a user accesses the system and views and inputs notes and data.

[0914] An "artificial intelligence service" is a collection of AI systems designed to perform a specific task, such as text generation or image recognition.

[0915] An "optimal work pattern" refers to the most efficient and accurate algorithm or process for multiple tasks.

[0916] "Visual data" refers to image and video data captured by a camera or other imaging device.

[0917] "Audio data" refers to recorded audio data captured by an audio device such as a microphone.

[0918] "Emotion data" refers to data relating to the user's emotional state analyzed from facial expressions, tone of voice, and the like.

[0919] A "smart device" is an interactive device such as glasses equipped with a camera and microphone.

[0920] "Recording" is the act of saving interaction details and emotional data in a database.

[0921] "Emotion recognition" is the process of analyzing and identifying a user's emotions from visual and audio data.

[0922] The present invention is a system for capturing visual and audio data in real time, recognizing user emotions, and providing high-quality customer service based on the data. To implement the present invention, the following technical configuration and steps are required.

[0923] First, a smart device (e.g., smart glasses equipped with a camera and microphone) captures customer interactions in real time. The device then sends the captured video and audio data to the EmotionEngine (an emotion recognition library) to recognize the user's emotions. The recognized emotion data is then immediately sent to a server and stored in the MemoSphere system's database.

[0924] The server then monitors the data generated by the AI ​​model and collects details (such as input data, algorithms used, output results, and emotional data). The collected data is stored in a database for future reference. For example, if a customer asks about the price of a particular product, the customer's facial expression and tone of voice can be analyzed to record emotional data such as "interest" or "anxiety."

[0925] The server generates appropriate notes for users based on real-time interactions and emotional data. These notes reflect the customer's questions and emotional changes and are provided as specific, personalized information. The generated notes are provided to store clerks through the user interface, who then refer to them when assisting the customer.

[0926] In addition, the server will also work with other artificial intelligence services, integrating functions such as text generation and image recognition, enabling more advanced information provision. For example, when a customer presents a product image, the image can be recognized and relevant information can be instantly provided.

[0927] The system also analyzes optimal work patterns, identifying the most efficient algorithms and processes across multiple tasks and automatically applying them to the next task. Based on collected data and sentiment data, the system automatically improves processes and enhances customer service.

[0928] In this way, smart devices can be used to understand customer sentiment in real time and take appropriate action based on that sentiment. Furthermore, detailed records and analysis of the collected data can be used to continuously improve the quality of service.

[0929] For example, when a store clerk wearing smart glasses interacts with a customer, the customer asks about the price of a product. The smart glasses' camera captures the customer's facial expressions, and the Emotion Engine analyzes their expressions for interest and anxiety. The MemoSphere system then records the details of the interaction along with their emotions and generates a summary memo that can be referenced later.

[0930] An example prompt is:

[0931] "Analyze the emotions customers express when asking about product prices. Record the emotion data and details of the interaction."

[0932] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0933] Step 1:

[0934] The device captures visual and audio data. Specifically, it uses the camera and microphone of the smart glasses to record real-time interactions with customers. The input is the video and audio data obtained from the camera and microphone, and the output is the process of sending this data to the EmotionEngine.

[0935] Step 2:

[0936] The device uses the EmotionEngine to recognize customer emotions from captured visual and audio data. Specifically, it analyzes facial expressions from video and tone from audio. The input is visual and audio data, and the output is recognized emotional data.

[0937] Step 3:

[0938] The device sends the generated emotion data to the server. Specifically, the output data from the Emotion Engine is transferred to the server in real time. The input is the emotion data, and the output is the data reaching the server.

[0939] Step 4:

[0940] The server stores the received emotion data and interaction details in the MemoSphere system database. Specifically, it records detailed information such as the interaction content, the algorithm used, and the output results. The input is emotion data and detailed interaction information, and the output is storing this in the database.

[0941] Step 5:

[0942] The server generates memos based on the collected data. Specifically, it integrates the content of the interaction with emotional data to create memos containing key points and summaries. The input is the details and emotional data stored in the database, and the output is the generated memo.

[0943] Step 6:

[0944] The server provides the generated memo to the terminal through a user interface. Specifically, the memo can be viewed by the store clerk through smart glasses. The input is the generated memo, and the output is the memo as information provided to the user.

[0945] Step 7:

[0946] The server works with other AI services to integrate collected data and emotional data. Specifically, it integrates data from other AI services such as text generation and image recognition, enabling centralized information acquisition. The input is data from other AI services, and the output is the integrated detailed data.

[0947] Step 8:

[0948] The server analyzes the integrated detailed data and finds the optimal work pattern. Specifically, it identifies the most efficient algorithms and processes for multiple tasks. The input is the integrated detailed data, and the output is the optimal work pattern.

[0949] Step 9:

[0950] The server improves the process based on the optimal work pattern. Specifically, it automatically applies the newly identified work pattern to the next task, improving efficiency and quality. The input is the optimal work pattern, and the output is the improved process.

[0951] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0952] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0953] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0954] [Third embodiment]

[0955] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0956] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0957] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0958] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0959] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0960] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0961] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0962] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0963] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0964] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0965] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0966] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0967] The AI ​​MemoSphere system of the present invention aims to record the history of tasks performed by an artificial intelligence, generate memos based on that history, and provide them to users in a transparent and reliable manner. Below, we will explain in natural language the program processing of a specific embodiment of this system, and provide detailed examples.

[0968] Recording work history

[0969] The server monitors the generated data in real time. For example, when a generative AI generates an answer to a user's question, the server continuously monitors this process.

[0970] The server collects details of each task (input data, algorithm used, output results, etc.) and stores them in a database. For example, if a user asks a question about a product, the generative AI's answer to that question, the process leading to that answer, and the version of the model and algorithm used are recorded.

[0971] Create and manage notes

[0972] The server generates memos based on the collected work history, including task summaries and key points.

[0973] Example: The server compiles important information about the answer generation tasks performed by the AI ​​over the past week and saves it as a summary memo. This summary memo includes information such as what types of questions were most popular, the trends in the answers, and which algorithms were most frequently used.

[0974] Viewing work history

[0975] Users can view their work history notes through a dedicated user interface (UI), which is available as a browser-based or desktop application.

[0976] Example: A user can access the UI to see the AI's answer process for their question and view detailed notes, making it easier to understand how the answer was generated.

[0977] Integration with other AI services

[0978] The server connects with other AI services via APIs and integrates the work history of each AI service.

[0979] Example: By linking text generation AI and image recognition AI, when a user asks a text question, they can also provide an image, and the process of generating an answer based on that is recorded and integrated. Users can then refer to this link history in a unified manner.

[0980] Analysis of optimal work patterns

[0981] The server analyzes the integrated work history data and finds optimal work patterns.

[0982] Example: The server finds the most efficient algorithms and processes across multiple tasks and automatically applies those patterns to the next task, improving the accuracy and speed of AI responses and significantly improving the user experience.

[0983] Adapting to user needs

[0984] The device collects user feedback and uses this to customize the AI's behavior.

[0985] Example: If a user requests a specific expression or format, that feedback will be reflected and the next AI output will be provided in a format that matches that preference.

[0986] Creating metrics and evaluating performance

[0987] The server creates new performance metrics based on collected details and user feedback to evaluate the AI's performance.

[0988] Example: Evaluations are conducted based on a combination of multiple metrics, including response speed and accuracy, as well as user satisfaction, to identify areas for improvement in AI and improve its quality.

[0989] Adapts to user preferences and work styles

[0990] The device analyzes work history data and learns the user's preferences and work style.

[0991] Example: It learns the tone and style of responses preferred by a particular user and adjusts output accordingly, allowing it to best meet the user's individual needs.

[0992] In this way, the AI ​​MemoSphere system records, manages, and analyzes the work history of the generating AI in detail, and provides users with a highly transparent service based on that information, which improves the reliability of the AI ​​and enables efficient, high-quality work.

[0993] The processing flow will be explained below.

[0994] Step 1:

[0995] The server monitors the generated data in real time. Specifically, when the generating AI starts a task, it obtains the task's identification information and triggers the monitoring process.

[0996] Step 2:

[0997] The server collects details of each task, including the input data, the algorithms used, each step of the processing, and the output results, and then organizes and structures the data.

[0998] Step 3:

[0999] The server stores the collected details in a database, which stores the data according to an appropriate schema in a format that can be queried later.

[1000] Step 4:

[1001] The server generates notes based on details stored in the database, summarizing the key points of the data and automatically creating easy-to-understand notes in natural language.

[1002] Step 5:

[1003] The server provides the generated notes through a dedicated user interface (UI), which is designed as a browser-based or desktop application.

[1004] Step 6:

[1005] Users can access work history notes through the user interface, allowing them to search for the history of a specific task or check the contents of the notes.

[1006] Step 7:

[1007] The server will connect with other AI services via API, allowing work history data to be sent and received between the services.

[1008] Step 8:

[1009] The server integrates the work history collected from the linked AI services, thereby centrally managing the historical data of multiple services.

[1010] Step 9:

[1011] The server analyzes the integrated data to identify optimal work patterns, applying machine learning algorithms and statistical models to identify the most effective processes and procedures.

[1012] Step 10:

[1013] The server then uses the analytical results to improve the processing of the generated data, specifically by automatically applying optimized algorithm settings and parameters.

[1014] Step 11:

[1015] The terminal provides an interface for collecting user feedback, allowing the user to input their opinions on the user interface.

[1016] Step 12:

[1017] The server analyzes the collected feedback and calculates metrics to evaluate the performance of the generated data. It combines the feedback content and work history to derive various evaluation indicators.

[1018] Step 13:

[1019] The server learns the user's preferences and working style, which involves analyzing past usage data and feedback to extract the user's specific needs and tendencies.

[1020] Step 14:

[1021] The server then customizes the processing of the generated data based on the learning results, providing responses and functionality that match the user's preferences and working style.

[1022] Example 1

[1023] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1024] There is a growing need to improve the transparency and reliability of information generated in modern automation systems. However, current systems have difficulty effectively managing and analyzing the detailed history of generated information, making it difficult for users to understand the generation process. Furthermore, it is difficult to efficiently link different automation services and discover and apply the optimal work format. For these reasons, there is a need for a system that can improve the reliability and transparency of generated information and effectively manage and analyze it.

[1025] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1026] In this invention, the server includes means for monitoring generated information, means for collecting details of the generated information, means for storing the collected details in a storage device, means for generating a summary based on the details stored in the storage device, means for providing the generated summary through a user interface, means for integrating the collected details in cooperation with other automated services, means for analyzing the integrated details to find an optimal work format, and means for improving the process of the generated information based on the optimal work format. This makes it possible to manage and analyze the history of generated information in detail and provide highly transparent services to users.

[1027] "Generated Information" means output or data generated by an automated system based on user input.

[1028] "Monitoring means" refers to a device or program that tracks and records the process and results of the information generated in real time.

[1029] "Means of collection" refers to a device or program that systematically acquires and stores details of the generated information and related data.

[1030] "Storage" refers to physical and logical storage for storing collected detailed data.

[1031] "Means for generating a summary" refers to a device or program that generates a compact, easily understandable summary from the detailed data collected.

[1032] "User interface" refers to the means, including screens and applications, by which a user interacts with a system.

[1033] "Automated services" refer to programs or systems that autonomously perform specific tasks.

[1034] "Integration means" refers to a device or program that centralizes the collected detailed data with data from other automated services and manages it in a unified format.

[1035] The "optimal work format" refers to the most efficient and accurate method among multiple work processes.

[1036] "Means for improving" refers to a device or program for improving the process of generating information based on the optimal working format.

[1037] MODE FOR CARRYING OUT THE INVENTION

[1038] The AI ​​MemoSphere system of the present invention is a system designed to increase the transparency and reliability of generated information. The following describes how the present invention is specifically implemented. The specific names of the hardware and software used are also provided.

[1039] System Configuration

[1040] The system of the present invention consists of the following main components:

[1041] 1. Server

[1042] Hardware used: High-performance servers (e.g., AWS EC2 instances)

[1043] Software used: log collection tools (e.g., Elasticsearch), databases (e.g., PostgreSQL), automatic summary generation tools (e.g., NLTK), data analysis tools (e.g., pandas, scikit-learn), performance measurement tools (e.g., TensorBoard), REST API (e.g., Flask)

[1044] 2. Terminal

[1045] Hardware used: User device (e.g., PC, smartphone)

[1046] Software used: web browser (e.g., Chrome), feedback collection tool (e.g., Google Forms), machine learning model (e.g., scikit-learn)

[1047] 3. User Interface (UI)

[1048] Software used: Browser-based or desktop application

[1049] Program processing explanation

[1050] The specific operation and processing of this system will be explained in natural language below.

[1051] Recording work history

[1052] The server monitors in real time the process by which the generative AI model (e.g., GPT-4) generates an answer based on the user's input. For example, when a user inputs "What's the weather like?", the server collects data until the generative AI model replies "Today's weather is sunny." The collected data includes the input data, the algorithm used, the generated output, etc. This data is stored in a storage device by the server.

[1053] Create and manage notes

[1054] The server generates a summary memo based on the collected work history. This summary includes a summary of the task and key points. For example, the server compiles the answer history of the generation AI over the past week and generates a summary memo that includes frequently asked questions, their trends, and information about the version of the algorithm used.

[1055] Viewing work history

[1056] Users can refer to the work history notes through the user interface (UI). For example, users can access the UI to check the past question history and the answering process of the generation AI, and view detailed notes.

[1057] Integration with other AI services

[1058] The server connects with other automation services, such as image recognition AI, via API and integrates the work history data of each AI service. For example, a user can provide an image along with a text question, and the server records and integrates the answer process based on that.

[1059] Analysis of optimal work patterns

[1060] The server analyzes the integrated work history data and finds the optimal work format. For example, the server can identify the most efficient algorithms and processes across multiple tasks and automatically apply them to future tasks, improving the accuracy and speed of answers.

[1061] Adapting to user needs

[1062] The device collects user feedback and uses it to customize the generative AI model's responses. For example, if a user sends feedback such as "I like this expression," that preference will be reflected in the next answer generated.

[1063] Adapts to user preferences and work styles

[1064] The device learns user preferences and styles based on historical work data, for example, learning the tone and style of responses preferred by a particular user, and adjusts the output of the generative AI model accordingly.

[1065] Specific examples

[1066] Here are some examples of prompts:

[1067] "Please tell me the weather."

[1068] When a user enters this prompt, the server uses a generative AI model to generate an answer and records the process in detail. A summary memo is generated based on this record, and the user can view the details through the UI. By linking with other image recognition AI, the user can provide additional information and obtain a more detailed answer. The integrated data is analyzed to identify the optimal work format and apply it to the next task. The AI's responses are continuously improved based on user feedback and historical data.

[1069] In this way, the AI ​​MemoSphere system can manage and analyze the history of generated information in detail, providing users with transparent and reliable services.

[1070] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1071] Step 1:

[1072] The user inputs a prompt sentence into the user interface (UI), for example, "Please tell me the weather."

[1073] Input: prompt statement

[1074] Output: User input data

[1075] What happens: A user enters a prompt sentence using a browser or desktop application.

[1076] Step 2:

[1077] The server receives the prompt sentence sent by the user and inputs it into a generative AI model, which then generates an answer (e.g., GPT-4).

[1078] Input: User-entered data

[1079] Output: Answer data generated by the generative AI model

[1080] How it works: The server receives the input "What is the weather like?", submits it to a generative AI model, and generates the answer "It's sunny today."

[1081] Step 3:

[1082] The server monitors the generated answers and the process in real time, collecting data, including the algorithms and model versions used.

[1083] Input: Answer data generated by a generative AI model

[1084] Output: Detailed data on the generation process

[1085] How it works: The server monitors the entire generation process in real time and collects detailed data such as "Question: What's the weather like?", "Answer: It's sunny today," "Model used: GPT-4," and "Time: XX seconds."

[1086] Step 4:

[1087] The server stores the collected detailed data in a storage device (database).

[1088] Input: Detailed data of the generation process

[1089] Output: Detailed data stored in a database

[1090] Specific operation: The server stores the collected detailed data in a database (e.g., PostgreSQL).

[1091] Step 5:

[1092] The server generates a summary memo based on the detailed data stored in the storage device, which includes a summary of the task and important points.

[1093] Input: Detailed data stored in the database

[1094] Output: Generated summary notes

[1095] Specific operation: The server uses an automatic summary generation tool (e.g., NLTK) to summarize the generation AI's answer history for the past week and create a summary memo.

[1096] Step 6:

[1097] The server provides the generated summary memo through a user interface (UI).

[1098] Input: Generated summary note

[1099] Output: A user-visible summary note

[1100] Specific operation: The server displays the summary memo through the UI (browser or desktop application) so that the user can view it.

[1101] Step 7:

[1102] The server connects with other automation services via APIs and integrates the work history data of each service.

[1103] Input: Work history data from other automated services

[1104] Output: Integrated work history data

[1105] Specific operation: The server uses a REST API (e.g., Flask) to integrate data from image recognition AI, etc.

[1106] Step 8:

[1107] The server analyzes the integrated work history data and finds the optimal work format.

[1108] Input: Integrated work history data

[1109] Output: Data in a format that works best for you

[1110] Specific behavior: The server uses data analysis tools (e.g., pandas, scikit-learn) to identify efficient algorithms and processes.

[1111] Step 9:

[1112] The server improves the process of generating the generated information based on the most suitable working format.

[1113] Input: Data in the format that works best for you

[1114] Output: Improved generation process

[1115] Specific operation: The server automatically applies the identified optimal work format to the next task, improving the accuracy and speed of the generative AI model.

[1116] Step 10:

[1117] The device collects user feedback and uses it to customize the responses of the generative AI model.

[1118] Input: User feedback

[1119] Output: The customized generative AI model response

[1120] Specific operation: The device collects feedback from users using a feedback collection tool (e.g., Google Forms) and reflects it in the output of the generative AI model.

[1121] Step 11:

[1122] The device learns the user's preferences and style based on work history data and adjusts the output of the generative AI model accordingly.

[1123] Input: Work history data

[1124] Output: The output of the tuned generative AI model.

[1125] What it does: The device uses a machine learning model (e.g., scikit-learn) to learn the response format and tone preferred by a particular user and reflects that in the next output.

[1126] (Application example 1)

[1127] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1128] In modern logistics centers, maximizing work efficiency and minimizing errors are important. However, recording and analyzing work history, and then flexibly modifying work plans based on that information, is not easy. Current systems lack the tools to ensure data transparency and reliability while working efficiently. A major problem is the lack of a function that allows on-site staff to input information in real time and generate optimal notes based on that information. A new system is needed to solve these issues.

[1129] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1130] In this invention, the server includes means for monitoring generated data, means for collecting details of the generated data, means for storing the collected details in a database, means for generating memos based on the details stored in the database, means for providing the generated memos through a user interface, means for integrating the collected details in cooperation with other artificial intelligence services, means for analyzing the integrated details to find optimal work patterns, means for improving the process of the generated data based on the optimal work patterns, means for recording work history, generating memos, and improving work efficiency in the logistics center, means for inputting work information via a smartphone application and sending it to the server, and means for sending generated memos from the server to the smartphone application, thereby improving work efficiency in the logistics center, reducing work errors, and ensuring data transparency and reliability.

[1131] "Generated data" is data generated by artificial intelligence systems or other information systems.

[1132] "Monitoring means" refers to a means for observing the generated data in real time and detecting fluctuations or abnormalities.

[1133] "Collection means" refers to the means for systematically capturing and recording detailed information about the data generated.

[1134] A "database" is an area where detailed information on collected data is systematically stored and can be retrieved as needed.

[1135] The "means for generating memos" refers to a means for organizing the main points based on the collected detailed information and generating concisely written memos.

[1136] A "user interface" is an interface that provides a screen and operating means for a user to interact with a system.

[1137] "Artificial Intelligence Services" are services based on artificial intelligence technology that are utilized to perform specific tasks.

[1138] An "optimal work pattern" is an efficient and effective work procedure or method discovered by analyzing past work history.

[1139] "Process improvement measures" are measures to improve current work procedures and methods based on optimal work patterns, thereby increasing efficiency and effectiveness.

[1140] A "smartphone application" is software that runs on a smartphone and provides specific functions.

[1141] "Work information" is information relating to the specific content and status of work being carried out at the logistics center.

[1142] A "server" is a computer system for processing, storing, and managing data.

[1143] The purpose of this invention, the "AI MemoSphere System," is to improve work efficiency at logistics centers, reduce work errors, and ensure data transparency and reliability. Specific implementations of this system are described below.

[1144] Hardware and software used

[1145] Hardware:

[1146] Server: AWS server (EC2)

[1147] Device: Smartphone (iOS / Android)

[1148] software:

[1149] Server side: Python (Django framework)

[1150] Database: PostgreSQL

[1151] Frontend: React Native

[1152] AI model: GPT-4 (API provided by OpenAI)

[1153] API integration: AWS API Gateway

[1154] System Program

[1155] Recording work history

[1156] The server monitors information about each operation performed at the distribution center in real time and collects the generated data. Specifically, staff enter operation information (e.g., item ID, quantity, picking time) through a smartphone application, and the data is sent to the server, which then stores it in a database.

[1157] Generate notes

[1158] Based on the collected work data, the server uses an AI model (GPT-4) to generate memos, which concisely summarize the main points and important information of the work. These memos are then sent from the server to a smartphone application where they can be viewed by staff.

[1159] Viewing work history

[1160] Users (logistics center staff and managers) can view work history and generated notes through a smartphone application, making it easy to understand how each task was performed and what results were achieved.

[1161] Analysis of optimal work patterns

[1162] The server analyzes the collected comprehensive work data to find the optimal work pattern, and the results of this analysis are reflected in the next work, allowing more efficient methods to be automatically applied.

[1163] Specific examples

[1164] At a distribution center, staff have a task of picking items from shelves. When they input the task data (e.g., item ID, quantity, picking time) into a smartphone application, the history is sent to the server and stored in a database. GPT-4 creates notes from the generated history, and these notes can be viewed by staff and managers through the app.

[1165] Prompt Sentence Examples

[1166] Below are some example prompts sent to the AI ​​model:

[1167] Task ID: 123 Details:

[1168] Input data: Item ID: A001, Quantity: 10, Picking time: 15 minutes

[1169] Algorithm used: Picking algorithm version 2.0

[1170] Output: Success, all items picked correctly

[1171] Generated note:

[1172] In this task, 10 units of item ID A001 were accurately picked. The picking algorithm used was version 2.0, which took 15 minutes to complete. This algorithm was effective for this task.

[1173] In this way, the AI ​​MemoSphere system can support skilled personnel in logistics centers and provide a highly transparent and efficient working environment.

[1174] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1175] Step 1:

[1176] The server monitors the generated data. Staff at the logistics center use a smartphone application to input work information. This input (e.g., item ID, quantity, picking time) is sent to the server. The server receives and monitors this data in real time. The input data is work information such as item ID, quantity, and picking time, and the output is the data received by the server.

[1177] Step 2:

[1178] The server collects details of the generated data. The server analyzes the task data sent from the smartphone application and extracts detailed information. The collected data includes the execution time of each task, the version of the algorithm used, and the success or failure of the task. These detailed information are stored in a database. The input data is the task information from the smartphone application, and the output is the detailed information stored in the database.

[1179] Step 3:

[1180] The server generates a memo based on the detailed information stored in the database. The server sends the collected work data details to the AI ​​model (GPT-4) and receives the generated memo. The prompt sentence to be sent to the AI ​​model is composed as follows:

[1181] Task ID: 123 Details:

[1182] Input data: Item ID: A001, Quantity: 10, Picking time: 15 minutes

[1183] Algorithm used: Picking algorithm version 2.0

[1184] Output: Success, all items picked correctly

[1185] Generated note:

[1186] In this task, 10 units of item ID A001 were accurately picked. The picking algorithm used was version 2.0, which took 15 minutes to complete. This algorithm was effective for this task.

[1187] The input data are the collected details, and the output are the notes generated by the AI ​​model.

[1188] Step 4:

[1189] The server provides the generated notes through a user interface. The generated notes are sent to a smartphone application so that the user can view them. The user can check the notes for each task through the smartphone application and understand the details of the task. The input data are the notes generated by the AI ​​model, and the output is the notes displayed in the user interface.

[1190] Step 5:

[1191] The server integrates the collected details in cooperation with other AI services. For example, by collaborating with an image recognition AI, it also collects image data and integrates detailed information based on that. This generates data that includes not only text information but also image information. The input data is detailed information from other AI services, and the output is the integrated detailed information.

[1192] Step 6:

[1193] The server analyzes the integrated details to find the optimal work pattern. It compares multiple work histories and extracts the most efficient work procedures and methods. This allows the optimal pattern to be applied in future work. The input data is the integrated details, and the output is the optimal work pattern.

[1194] Step 7:

[1195] The server improves the process of the generated data based on the optimal work pattern. Based on the discovered optimal pattern, it automatically corrects the current work procedures and methods to improve efficiency. The input data is the optimal work pattern, and the output is the improved work process.

[1196] Step 8:

[1197] The device collects user feedback and uses it to customize the AI's behavior. Feedback provided by the user through a smartphone application improves the AI ​​model's output and adapts to the user's needs. The input data is the user's feedback, and the output is the customized AI model's behavior.

[1198] In this way, the AI ​​MemoSphere System can improve work efficiency and ensure data transparency in logistics centers, providing a more reliable work environment.

[1199] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1200] The AI ​​MemoSphere system of the present invention aims to record the history of tasks performed by an artificial intelligence, generate memos based on that history, and provide them to users in a transparent and reliable manner. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions, and by taking emotion data into consideration, it is possible to provide more personalized services. Below, a specific embodiment of this system will be described in detail, with program processing explained in natural language and examples provided.

[1201] Recording work history

[1202] The server monitors the generated data in real time. For example, when a generative AI generates an answer to a user's question, the server continuously monitors this process.

[1203] The server collects details of each task (input data, algorithm used, output results, user emotion data, etc.) and stores them in a database. When a user asks a question about a product, the generative AI's answer, the process leading to that answer, the model and algorithm version used, and the user's emotion are recorded.

[1204] Use of emotion engine

[1205] The device analyzes the user's facial expressions and tone of voice as they type, and uses an emotion engine to recognize the user's emotions.

[1206] The server stores the emotion data recognized by the emotion engine in a database. For example, if the user is dissatisfied, the emotion data is also recorded.

[1207] Create and manage notes

[1208] The server generates memos based on the collected work history and emotional data. The generated memos include a summary of the task, important points, and the user's emotional data.

[1209] Example: The server generates a summary memo containing important information about the answer generation tasks performed by the AI ​​over the past week, as well as changes in user sentiment. This summary memo includes information such as what types of questions were most popular, the trends in their answers, and the user's emotional reactions.

[1210] Viewing work history

[1211] Users can access their work history notes through a dedicated user interface (UI), which can be provided as a browser-based or desktop application.

[1212] Example: A user can access the UI to see the AI's answer process to their question and the emotional data at the time, and view detailed notes, making it easier to understand how the answer was generated and how it reflected their emotional state at the time.

[1213] Integration with other AI services

[1214] The server connects with other AI services via APIs and integrates the work history and emotional data of each AI service.

[1215] Example: By linking text generation AI and image recognition AI, users can simultaneously provide images when asking text questions, and the process of generating answers based on these images is recorded and integrated. Users can then refer to this link history and emotion data in a unified manner.

[1216] Analysis of optimal work patterns

[1217] The server analyzes the integrated work history data and emotion data to find optimal work patterns.

[1218] Example: The server finds the most efficient algorithms and processes across multiple tasks and automatically applies those patterns to the next task. This takes emotional data into account, improving the accuracy of AI responses and user satisfaction.

[1219] Adapting to user needs

[1220] The device collects user feedback and uses this to customize the AI's behavior.

[1221] Example: If a user requests a particular style, format, or even a desired emotional tone, that feedback will be incorporated and the next AI output will be tailored to those preferences.

[1222] Creating metrics and evaluating performance

[1223] The server creates new performance indicators based on collected work history, emotional data, and user feedback to evaluate the AI's performance.

[1224] Example: In addition to response speed and accuracy, evaluations are conducted based on a combination of multiple metrics, such as user satisfaction and emotional response, to identify areas for improvement in AI and improve its quality.

[1225] Improving processes by taking emotional data into account

[1226] The server then uses the analysis results to improve the processing of the generated data, specifically by automatically applying algorithm settings and parameters that reflect the user's emotional data.

[1227] Example: For tasks that are likely to frustrate users, the AI ​​is tuned to generate more polite responses.

[1228] Adapts to user preferences and work styles

[1229] The device analyzes work history data and emotional data to tailor responses to best suit the user's preferences and work style.

[1230] Example: Learning the tone and style of responses preferred by a particular user and tailoring output accordingly, allowing it to best meet the user's individual needs and emotions.

[1231] In this way, the AI ​​MemoSphere system records, manages, and analyzes the work history of the generating AI and user emotional data in detail, and provides users with highly transparent services based on this information. This improves the reliability and personalization of the AI, enabling efficient and high-quality work.

[1232] The processing flow will be explained below.

[1233] Step 1:

[1234] The server monitors the generated data in real time. Specifically, when the generation AI starts a task, it acquires the task's identification information and triggers the monitoring process. For example, when a user enters a question, it generates a question ID and records it as a monitoring target.

[1235] Step 2:

[1236] The device analyzes the user's facial expressions and tone of voice when inputting and uses an emotion engine to recognize the user's emotions. Specifically, it uses a camera and microphone to detect the user's emotional state in real time and sends that data to the emotion engine.

[1237] Step 3:

[1238] The server stores the user's emotion data recognized by the emotion engine in a database. For example, if the user makes a dissatisfied expression, the emotion data is recorded and associated with the question ID.

[1239] Step 4:

[1240] The server collects details of each task (input data, algorithm used, processing steps, output results, and user emotional data), including the question entered by the user, the process of the generative AI, and the final answer.

[1241] Step 5:

[1242] The server stores the collected details in a database, which stores the data according to an appropriate schema in a format that can be queried later.

[1243] Step 6:

[1244] The server generates notes based on the details stored in the database, including a summary of the task, key points, and the user's emotional data.

[1245] Step 7:

[1246] The server provides the generated notes through a dedicated user interface (UI), which is designed as a browser-based or desktop application.

[1247] Step 8:

[1248] The user can view the work history notes through the user interface. The user can check the history of a specific task, the contents of the notes, and the emotional data at the time.

[1249] Step 9:

[1250] The server will connect with other AI services via API, allowing for the sending and receiving of work history data and emotion data between the connected services.

[1251] Step 10:

[1252] The server integrates the work history and emotion data collected from the linked AI services, thereby centrally managing the history data from multiple services.

[1253] Step 11:

[1254] The server analyzes the integrated data to identify optimal work patterns, applying machine learning algorithms and statistical models to identify the most effective processes, procedures, and emotional responses.

[1255] Step 12:

[1256] The server then uses the analysis results to improve the processing of the generated data, specifically by automatically applying optimized algorithm settings and parameters that also take emotion data into account.

[1257] Step 13:

[1258] The terminal provides an interface for collecting user feedback, allowing users to input their opinions and thoughts on the user interface.

[1259] Step 14:

[1260] The server analyzes the collected feedback and calculates metrics to evaluate the performance of the generated data. It combines the feedback content, work history, and emotional data to derive various evaluation indicators.

[1261] Step 15:

[1262] The server learns the user's preferences and work style, which involves analyzing past usage data, feedback, and sentiment data to extract the user's specific needs and tendencies.

[1263] Step 16:

[1264] The server then customizes the processing of the generated data based on the learning results, delivering responses and functionality in a format that best suits the user's preferences, work style, and emotions.

[1265] Example 2

[1266] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1267] Conventional AI systems focus on generating answers to user questions, but the process often lacks transparency and reliability. They also lack the ability to recognize user emotions and provide responses that take these into account. This makes it difficult to provide truly useful and personalized services to users. Furthermore, there are also issues with inconsistencies in analyzing optimal work patterns and integrating with other AI services.

[1268] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1269] In this invention, the server includes means for monitoring the generated data, means for collecting details of the generated data, means for storing the collected details in a database, means for generating memos based on the details stored in the database, means for providing the generated memos through a user interface, means for integrating the collected details in cooperation with other AI services, means for analyzing the integrated details to find optimal work patterns, means for improving the process of the generated data based on the optimal work patterns, means for recognizing user emotion data, means for collecting and storing the recognized emotion data in a database, and means for generating memos based on the collected and stored emotion data. This makes it possible to record, manage, and analyze the work history of the generative AI model and the user emotion data in detail, and to provide users with a transparent and reliable personalized service.

[1270] "Generated Data" refers to the responses or results generated by a generative AI model in response to a user's question or request.

[1271] "Monitoring means" refers to a system including hardware and software for monitoring the generated data in real time and understanding its contents.

[1272] "Collection means" refers to the set of processes or functions used to compile and record details of the data obtained through surveillance.

[1273] "Database" refers to a system of information collection that stores details of collected data in an organized manner so that they can be easily retrieved and used later.

[1274] "Means for generating notes" refers to algorithms or programs that create summary information based on the details and emotional data collected.

[1275] "User Interface" refers to the screens and applications that allow a user to access and manipulate generated notes and data details.

[1276] "Other artificial intelligence services" refers to artificial intelligence technologies and services that exist other than the system of the present invention, and refers to systems that can be linked with these.

[1277] "Means of integration" refers to the ability to bring together data and information obtained from other artificial intelligence services and manage them in a consistent manner.

[1278] "Optimal work patterns" refer to efficient and effective work procedures and processes discovered through detailed analysis of collected data.

[1279] "Measures for improvement" refers to methods and systems for improving the current data generation process based on the optimal work patterns identified.

[1280] "Emotion data" refers to data that indicates the emotional state recognized from the user's statements and actions.

[1281] "Means for recognizing emotions" refers to a system that includes hardware and software for analyzing data such as a user's facial expressions and voice and identifying the user's emotions.

[1282] "Collected and stored emotion data" refers to the state in which the recognized emotion data is stored in a database and can be used for later interpretation and analysis.

[1283] The AI ​​MemoSphere system of the present invention records, manages, and analyzes the work history of the generative AI model and the user's emotional data in detail, and provides users with a transparent, reliable, and personalized service based on the data. Specific embodiments of the system are described below.

[1284] Basic configuration

[1285] This system consists of a server, a terminal, and a user. The server mainly monitors, collects, stores, and analyzes data, while the terminal provides a user interface and collects emotion data.

[1286] Hardware and software used

[1287] Generative AI models: For example, using GPT-3 for natural language processing.

[1288] Server: A high-performance computer server that collects, stores, analyzes, and generates notes on data.

[1289] Database: For example, using a relational database management system (RDBMS).

[1290] Emotion Engine: Uses Microsoft Azure's Emotion API to recognize user emotions.

[1291] User interface: Provided as a web browser or desktop application.

[1292] Processing flow and specific examples

[1293] 1. Monitoring generated data:

[1294] The server monitors the process in real time of the generative AI model generating answers to user questions. For example, if a user asks, "What is the price of a new product?", the request is passed to the generative AI model (GPT-3).

[1295] 2. Collect task details:

[1296] The server collects detailed information about the algorithms used by the generative AI, input data, output results, and user emotional data, and stores it in a database. For example, it records the generative AI model used, the prompt (user question), the output result (product price), and the user's emotional data (dissatisfaction, satisfaction, etc.).

[1297] 3. User emotion recognition:

[1298] The device recognizes the user's facial expression and tone of voice when they input a question and uses an emotion engine to recognize their emotion. For example, if a user says "Is this really the right price?" in a dissatisfied tone, their facial expression and tone of voice are captured.

[1299] 4. Emotional Data Recording:

[1300] The server stores the recognized emotion data in a database. Specifically, the server receives emotion data (e.g., dissatisfaction) sent from the device and adds it to the database.

[1301] 5. Automatic note generation:

[1302] The server generates memos based on the collected work history and emotion data, including task summaries and key points. For example, it summarizes the tasks performed by the AI ​​over the past week and creates a summary memo that includes the user's emotions regarding each task.

[1303] 6. Accessing the User Interface:

[1304] Users can access their work history notes through a dedicated user interface, such as a dashboard in a web browser, to view details of past interactions and sentiment data.

[1305] 7. Data integration with other AI services:

[1306] The server also works with image recognition AI (e.g., an image analysis system) to integrate the entire work history and emotional data. For example, a user can upload a product image and record the answer generation process based on it.

[1307] 8. Extracting optimal work patterns:

[1308] The server then extracts the optimal work pattern based on the integrated work history and emotional data and applies it to the next task, for example by analyzing multiple response patterns and selecting the fastest and most effective algorithm to apply.

[1309] 9. Collaboration with Feedback:

[1310] The device collects user feedback and customizes the AI's behavior based on that feedback. For example, if the user provides feedback such as "Please provide more detailed explanation next time," that feedback is sent to the server and reflected in the next response.

[1311] 10. Performance Evaluation:

[1312] The server creates new evaluation indicators based on work history, emotional data, and user feedback to evaluate AI performance. For example, it creates an indicator that combines response speed, accuracy, and user satisfaction.

[1313] Examples and prompts

[1314] For example, a user inputs a prompt such as "Please tell me the price of the new product," and the answer generation process and emotional data based on that are processed. The AI's answer to the user's question (e.g., the price of the new product) and the user's emotion in response (e.g., dissatisfaction) are recorded.

[1315] In this way, the AI ​​MemoSphere system records and analyzes the work history of the generative AI model and the user's emotional data in detail, providing a transparent, reliable and personalized service.

[1316] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1317] Step 1:

[1318] Monitoring generated data

[1319] Input: A prompt from the user (e.g., "What is the price of the new product?")

[1320] Output: Details of the answer generation process by the generative AI model

[1321] Specific operation: The server monitors the prompt sentences entered by the user and tracks how the generative AI model (e.g., GPT-3) responds to the prompt. When a user enters "Please tell me the price of the new product," the server sends this prompt sentence to the generative AI model and records its response process.

[1322] Step 2:

[1323] Collecting task details

[1324] Input: Responses output by the generative AI model, prompts, algorithms, and user emotional data

[1325] Output: Collected task details data

[1326] What it does: The server collects the answer generated by the generative AI model (e.g., "The price of the new product is $100"), the algorithm used (e.g., the version and specific settings of GPT-3), the input prompt, and the user's emotional data. These details are then compiled and stored in a database.

[1327] Step 3:

[1328] Recognizing user emotions

[1329] Input: User's facial expression data, voice data

[1330] Output: Emotion recognition result (e.g., dissatisfied, satisfied)

[1331] Specific operation: The device captures the user's facial expressions and tone of voice when entering prompts using a camera and microphone, and analyzes them using an emotion engine (e.g., Microsoft Azure's Emotion API). The user may say, with a dissatisfied tone, "Is this really the right price?", and their facial and voice data is captured.

[1332] Step 4:

[1333] Emotional data recording

[1334] Input: Emotion recognition results

[1335] Output: Emotion data stored in a database

[1336] Specific operation: The server receives the emotion data sent from the device and stores it in a database. For example, it stores the emotion data "dissatisfied" in the database.

[1337] Step 5:

[1338] Auto-generate notes

[1339] Input: Collected task details data, emotion data

[1340] Output: Auto-generated notes

[1341] Specific operation: The server generates a memo based on the collected task details and emotion data. This includes a brief summary of the task and key points. For example, it may summarize all tasks performed by the AI ​​in the past week and create a summary memo including the user's associated emotion data.

[1342] Step 6:

[1343] Accessing the User Interface

[1344] Input: User's reference request

[1345] Output: Display of created note

[1346] Specific operation: The user refers to the work history memo through a dedicated user interface (e.g., a web browser). The user accesses the dashboard on the browser to view details of past interactions and emotion data.

[1347] Step 7:

[1348] Data integration with other AI services

[1349] Input: Data from other AI services (e.g., image recognition results)

[1350] Output: Integrated work history and emotion data

[1351] Specific operation: The server works with image recognition AI (e.g., image analysis system) in addition to the generation AI, and integrates their work history and emotional data. For example, a user uploads a product image and records the answer generation process based on it.

[1352] Step 8:

[1353] Extracting optimal work patterns

[1354] Input: Integrated work history, emotional data

[1355] Output: The optimal work pattern found

[1356] Specific operation: The server performs detailed analysis of the integrated work history and emotion data to extract the optimal work pattern. For example, it analyzes multiple response patterns to find the fastest and most effective algorithm and apply it to the next task.

[1357] Step 9:

[1358] Collaboration with Feedback

[1359] Input: User feedback

[1360] Output: Customized AI response

[1361] Specific operation: The device collects feedback from the user and sends it to the server. For example, if the user says, "I'd like more detailed explanation next time," the device customizes the AI's response based on this feedback and reflects it in the next response.

[1362] Step 10:

[1363] Performance Evaluation

[1364] Input: Work history, emotion data, feedback

[1365] Output: Generated performance metrics

[1366] Specific operation: The server creates new evaluation indicators based on work history, emotional data, and user feedback to evaluate the AI's performance. For example, it creates an indicator that combines response speed, accuracy of generated results, and user satisfaction, and evaluates the AI's performance based on that.

[1367] In this way, the AI ​​MemoSphere system records, manages, and analyzes the work history of the generative AI model and user emotional data in detail, providing transparent, reliable, and personalized services.

[1368] (Application example 2)

[1369] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1370] Modern customer service requires accurately understanding customer emotions and needs and providing appropriate responses in real time. However, traditional systems make it difficult to provide services that fully reflect customer emotions, and as a result, they are unable to provide responses that increase customer satisfaction. Furthermore, it is difficult to record detailed customer interaction history and refer to it later, resulting in a lack of data to continuously improve service quality.

[1371] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring generated data, means for collecting details, means for storing details in a database, means for generating memos, means for providing the memos through a user interface, means for integrating the collected details in cooperation with other AI services, means for analyzing the integrated details to find optimal work patterns, means for improving processes based on the optimal work patterns, means for capturing visual and audio data and recognizing user emotions, means for storing the emotion data in a database and incorporating it into memos, and means for real-time recording and emotion recognition using a smart device. This enables accurate understanding of customer emotions and appropriate responses in real time. Furthermore, by recording details of interactions with customers and referencing them later, service quality can be continuously improved and customer satisfaction can be increased.

[1372] "Generated data" is information that results from tasks performed by artificial intelligence.

[1373] "Details" are specific information including input data to the generated data, algorithms used, output results, and other relevant information.

[1374] A "database" is a system for storing collected details and emotional data for later reference and analysis.

[1375] A "memo" is a summary document generated based on collected details and emotion data, and is information provided to the user.

[1376] "User interface" refers to the interaction environment through which a user accesses the system and views and inputs notes and data.

[1377] An "artificial intelligence service" is a collection of AI systems designed to perform a specific task, such as text generation or image recognition.

[1378] An "optimal work pattern" refers to the most efficient and accurate algorithm or process for multiple tasks.

[1379] "Visual data" refers to image and video data captured by a camera or other imaging device.

[1380] "Audio data" refers to recorded audio data captured by an audio device such as a microphone.

[1381] "Emotion data" refers to data relating to the user's emotional state analyzed from facial expressions, tone of voice, and the like.

[1382] A "smart device" is an interactive device such as glasses equipped with a camera and microphone.

[1383] "Recording" is the act of saving interaction details and emotional data in a database.

[1384] "Emotion recognition" is the process of analyzing and identifying a user's emotions from visual and audio data.

[1385] The present invention is a system for capturing visual and audio data in real time, recognizing user emotions, and providing high-quality customer service based on the data. To implement the present invention, the following technical configuration and steps are required.

[1386] First, a smart device (e.g., smart glasses equipped with a camera and microphone) captures customer interactions in real time. The device then sends the captured video and audio data to the EmotionEngine (an emotion recognition library) to recognize the user's emotions. The recognized emotion data is then immediately sent to a server and stored in the MemoSphere system's database.

[1387] The server then monitors the data generated by the AI ​​model and collects details (such as input data, algorithms used, output results, and emotional data). The collected data is stored in a database for future reference. For example, if a customer asks about the price of a particular product, the customer's facial expression and tone of voice can be analyzed to record emotional data such as "interest" or "anxiety."

[1388] The server generates appropriate notes for users based on real-time interactions and emotional data. These notes reflect the customer's questions and emotional changes and are provided as specific, personalized information. The generated notes are provided to store clerks through the user interface, who then refer to them when assisting the customer.

[1389] In addition, the server will also work with other artificial intelligence services, integrating functions such as text generation and image recognition, enabling more advanced information provision. For example, when a customer presents a product image, the image can be recognized and relevant information can be instantly provided.

[1390] The system also analyzes optimal work patterns, identifying the most efficient algorithms and processes across multiple tasks and automatically applying them to the next task. Based on collected data and sentiment data, the system automatically improves processes and enhances customer service.

[1391] In this way, smart devices can be used to understand customer sentiment in real time and take appropriate action based on that sentiment. Furthermore, detailed records and analysis of the collected data can be used to continuously improve the quality of service.

[1392] For example, when a store clerk wearing smart glasses interacts with a customer, the customer asks about the price of a product. The smart glasses' camera captures the customer's facial expressions, and the Emotion Engine analyzes their expressions for interest and anxiety. The MemoSphere system then records the details of the interaction along with their emotions and generates a summary memo that can be referenced later.

[1393] An example prompt is:

[1394] "Analyze the emotions customers express when asking about product prices. Record the emotion data and details of the interaction."

[1395] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1396] Step 1:

[1397] The device captures visual and audio data. Specifically, it uses the camera and microphone of the smart glasses to record real-time interactions with customers. The input is the video and audio data obtained from the camera and microphone, and the output is the process of sending this data to the EmotionEngine.

[1398] Step 2:

[1399] The device uses the EmotionEngine to recognize customer emotions from captured visual and audio data. Specifically, it analyzes facial expressions from video and tone from audio. The input is visual and audio data, and the output is recognized emotional data.

[1400] Step 3:

[1401] The device sends the generated emotion data to the server. Specifically, the output data from the Emotion Engine is transferred to the server in real time. The input is the emotion data, and the output is the data reaching the server.

[1402] Step 4:

[1403] The server stores the received emotion data and interaction details in the MemoSphere system database. Specifically, it records detailed information such as the interaction content, the algorithm used, and the output results. The input is emotion data and detailed interaction information, and the output is storing this in the database.

[1404] Step 5:

[1405] The server generates memos based on the collected data. Specifically, it integrates the content of the interaction with emotional data to create memos containing key points and summaries. The input is the details and emotional data stored in the database, and the output is the generated memo.

[1406] Step 6:

[1407] The server provides the generated memo to the terminal through a user interface. Specifically, the memo can be viewed by the store clerk through smart glasses. The input is the generated memo, and the output is the memo as information provided to the user.

[1408] Step 7:

[1409] The server works with other AI services to integrate collected data and emotional data. Specifically, it integrates data from other AI services such as text generation and image recognition, enabling centralized information acquisition. The input is data from other AI services, and the output is the integrated detailed data.

[1410] Step 8:

[1411] The server analyzes the integrated detailed data and finds the optimal work pattern. Specifically, it identifies the most efficient algorithms and processes for multiple tasks. The input is the integrated detailed data, and the output is the optimal work pattern.

[1412] Step 9:

[1413] The server improves the process based on the optimal work pattern. Specifically, it automatically applies the newly identified work pattern to the next task, improving efficiency and quality. The input is the optimal work pattern, and the output is the improved process.

[1414] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1415] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1416] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1417] [Fourth embodiment]

[1418] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1419] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1420] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1421] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1422] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1423] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1424] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1425] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1426] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1427] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1428] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1429] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1430] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1431] The AI ​​MemoSphere system of the present invention aims to record the history of tasks performed by an artificial intelligence, generate memos based on that history, and provide them to users in a transparent and reliable manner. Below, we will explain in natural language the program processing of a specific embodiment of this system, and provide detailed examples.

[1432] Recording work history

[1433] The server monitors the generated data in real time. For example, when a generative AI generates an answer to a user's question, the server continuously monitors this process.

[1434] The server collects details of each task (input data, algorithm used, output results, etc.) and stores them in a database. For example, if a user asks a question about a product, the generative AI's answer to that question, the process leading to that answer, and the version of the model and algorithm used are recorded.

[1435] Create and manage notes

[1436] The server generates memos based on the collected work history, including task summaries and key points.

[1437] Example: The server compiles important information about the answer generation tasks performed by the AI ​​over the past week and saves it as a summary memo. This summary memo includes information such as what types of questions were most popular, the trends in the answers, and which algorithms were most frequently used.

[1438] Viewing work history

[1439] Users can view their work history notes through a dedicated user interface (UI), which is available as a browser-based or desktop application.

[1440] Example: A user can access the UI to see the AI's answer process for their question and view detailed notes, making it easier to understand how the answer was generated.

[1441] Integration with other AI services

[1442] The server connects with other AI services via APIs and integrates the work history of each AI service.

[1443] Example: By linking text generation AI and image recognition AI, when a user asks a text question, they can also provide an image, and the process of generating an answer based on that is recorded and integrated. Users can then refer to this link history in a unified manner.

[1444] Analysis of optimal work patterns

[1445] The server analyzes the integrated work history data and finds optimal work patterns.

[1446] Example: The server finds the most efficient algorithms and processes across multiple tasks and automatically applies those patterns to the next task, improving the accuracy and speed of AI responses and significantly improving the user experience.

[1447] Adapting to user needs

[1448] The device collects user feedback and uses this to customize the AI's behavior.

[1449] Example: If a user requests a specific expression or format, that feedback will be reflected and the next AI output will be provided in a format that matches that preference.

[1450] Creating metrics and evaluating performance

[1451] The server creates new performance metrics based on collected details and user feedback to evaluate the AI's performance.

[1452] Example: Evaluations are conducted based on a combination of multiple metrics, including response speed and accuracy, as well as user satisfaction, to identify areas for improvement in AI and improve its quality.

[1453] Adapts to user preferences and work styles

[1454] The device analyzes work history data and learns the user's preferences and work style.

[1455] Example: It learns the tone and style of responses preferred by a particular user and adjusts output accordingly, allowing it to best meet the user's individual needs.

[1456] In this way, the AI ​​MemoSphere system records, manages, and analyzes the work history of the generating AI in detail, and provides users with a highly transparent service based on that information, which improves the reliability of the AI ​​and enables efficient, high-quality work.

[1457] The processing flow will be explained below.

[1458] Step 1:

[1459] The server monitors the generated data in real time. Specifically, when the generating AI starts a task, it obtains the task's identification information and triggers the monitoring process.

[1460] Step 2:

[1461] The server collects details of each task, including the input data, the algorithms used, each step of the processing, and the output results, and then organizes and structures the data.

[1462] Step 3:

[1463] The server stores the collected details in a database, which stores the data according to an appropriate schema in a format that can be queried later.

[1464] Step 4:

[1465] The server generates notes based on details stored in the database, summarizing the key points of the data and automatically creating easy-to-understand notes in natural language.

[1466] Step 5:

[1467] The server provides the generated notes through a dedicated user interface (UI), which is designed as a browser-based or desktop application.

[1468] Step 6:

[1469] Users can access work history notes through the user interface, allowing them to search for the history of a specific task or check the contents of the notes.

[1470] Step 7:

[1471] The server will connect with other AI services via API, allowing work history data to be sent and received between the services.

[1472] Step 8:

[1473] The server integrates the work history collected from the linked AI services, thereby centrally managing the historical data of multiple services.

[1474] Step 9:

[1475] The server analyzes the integrated data to identify optimal work patterns, applying machine learning algorithms and statistical models to identify the most effective processes and procedures.

[1476] Step 10:

[1477] The server then uses the analytical results to improve the processing of the generated data, specifically by automatically applying optimized algorithm settings and parameters.

[1478] Step 11:

[1479] The terminal provides an interface for collecting user feedback, allowing the user to input their opinions on the user interface.

[1480] Step 12:

[1481] The server analyzes the collected feedback and calculates metrics to evaluate the performance of the generated data. It combines the feedback content and work history to derive various evaluation indicators.

[1482] Step 13:

[1483] The server learns the user's preferences and working style, which involves analyzing past usage data and feedback to extract the user's specific needs and tendencies.

[1484] Step 14:

[1485] The server then customizes the processing of the generated data based on the learning results, providing responses and functionality that match the user's preferences and working style.

[1486] Example 1

[1487] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1488] There is a growing need to improve the transparency and reliability of information generated in modern automation systems. However, current systems have difficulty effectively managing and analyzing the detailed history of generated information, making it difficult for users to understand the generation process. Furthermore, it is difficult to efficiently link different automation services and discover and apply the optimal work format. For these reasons, there is a need for a system that can improve the reliability and transparency of generated information and effectively manage and analyze it.

[1489] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1490] In this invention, the server includes means for monitoring generated information, means for collecting details of the generated information, means for storing the collected details in a storage device, means for generating a summary based on the details stored in the storage device, means for providing the generated summary through a user interface, means for integrating the collected details in cooperation with other automated services, means for analyzing the integrated details to find an optimal work format, and means for improving the process of the generated information based on the optimal work format. This makes it possible to manage and analyze the history of generated information in detail and provide highly transparent services to users.

[1491] "Generated Information" means output or data generated by an automated system based on user input.

[1492] "Monitoring means" refers to a device or program that tracks and records the process and results of the information generated in real time.

[1493] "Means of collection" refers to a device or program that systematically acquires and stores details of the generated information and related data.

[1494] "Storage" refers to physical and logical storage for storing collected detailed data.

[1495] "Means for generating a summary" refers to a device or program that generates a compact, easily understandable summary from the detailed data collected.

[1496] "User interface" refers to the means, including screens and applications, by which a user interacts with a system.

[1497] "Automated services" refer to programs or systems that autonomously perform specific tasks.

[1498] "Integration means" refers to a device or program that centralizes the collected detailed data with data from other automated services and manages it in a unified format.

[1499] The "optimal work format" refers to the most efficient and accurate method among multiple work processes.

[1500] "Means for improving" refers to a device or program for improving the process of generating information based on the optimal working format.

[1501] MODE FOR CARRYING OUT THE INVENTION

[1502] The AI ​​MemoSphere system of the present invention is a system designed to increase the transparency and reliability of generated information. The following describes how the present invention is specifically implemented. The specific names of the hardware and software used are also provided.

[1503] System Configuration

[1504] The system of the present invention consists of the following main components:

[1505] 1. Server

[1506] Hardware used: High-performance servers (e.g., AWS EC2 instances)

[1507] Software used: log collection tools (e.g., Elasticsearch), databases (e.g., PostgreSQL), automatic summary generation tools (e.g., NLTK), data analysis tools (e.g., pandas, scikit-learn), performance measurement tools (e.g., TensorBoard), REST API (e.g., Flask)

[1508] 2. Terminal

[1509] Hardware used: User device (e.g., PC, smartphone)

[1510] Software used: web browser (e.g., Chrome), feedback collection tool (e.g., Google Forms), machine learning model (e.g., scikit-learn)

[1511] 3. User Interface (UI)

[1512] Software used: Browser-based or desktop application

[1513] Program processing explanation

[1514] The specific operation and processing of this system will be explained in natural language below.

[1515] Recording work history

[1516] The server monitors in real time the process by which the generative AI model (e.g., GPT-4) generates an answer based on the user's input. For example, when a user inputs "What's the weather like?", the server collects data until the generative AI model replies "Today's weather is sunny." The collected data includes the input data, the algorithm used, the generated output, etc. This data is stored in a storage device by the server.

[1517] Create and manage notes

[1518] The server generates a summary memo based on the collected work history. This summary includes a summary of the task and key points. For example, the server compiles the answer history of the generation AI over the past week and generates a summary memo that includes frequently asked questions, their trends, and information about the version of the algorithm used.

[1519] Viewing work history

[1520] Users can refer to the work history notes through the user interface (UI). For example, users can access the UI to check the past question history and the answering process of the generation AI, and view detailed notes.

[1521] Integration with other AI services

[1522] The server connects with other automation services, such as image recognition AI, via API and integrates the work history data of each AI service. For example, a user can provide an image along with a text question, and the server records and integrates the answer process based on that.

[1523] Analysis of optimal work patterns

[1524] The server analyzes the integrated work history data and finds the optimal work format. For example, the server can identify the most efficient algorithms and processes across multiple tasks and automatically apply them to future tasks, improving the accuracy and speed of answers.

[1525] Adapting to user needs

[1526] The device collects user feedback and uses it to customize the generative AI model's responses. For example, if a user sends feedback such as "I like this expression," that preference will be reflected in the next answer generated.

[1527] Adapts to user preferences and work styles

[1528] The device learns user preferences and styles based on historical work data, for example, learning the tone and style of responses preferred by a particular user, and adjusts the output of the generative AI model accordingly.

[1529] Specific examples

[1530] Here are some examples of prompts:

[1531] "Please tell me the weather."

[1532] When a user enters this prompt, the server uses a generative AI model to generate an answer and records the process in detail. A summary memo is generated based on this record, and the user can view the details through the UI. By linking with other image recognition AI, the user can provide additional information and obtain a more detailed answer. The integrated data is analyzed to identify the optimal work format and apply it to the next task. The AI's responses are continuously improved based on user feedback and historical data.

[1533] In this way, the AI ​​MemoSphere system can manage and analyze the history of generated information in detail, providing users with transparent and reliable services.

[1534] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1535] Step 1:

[1536] The user inputs a prompt sentence into the user interface (UI), for example, "Please tell me the weather."

[1537] Input: prompt statement

[1538] Output: User input data

[1539] What happens: A user enters a prompt sentence using a browser or desktop application.

[1540] Step 2:

[1541] The server receives the prompt sentence sent by the user and inputs it into a generative AI model, which then generates an answer (e.g., GPT-4).

[1542] Input: User-entered data

[1543] Output: Answer data generated by the generative AI model

[1544] How it works: The server receives the input "What is the weather like?", submits it to a generative AI model, and generates the answer "It's sunny today."

[1545] Step 3:

[1546] The server monitors the generated answers and the process in real time, collecting data, including the algorithms and model versions used.

[1547] Input: Answer data generated by a generative AI model

[1548] Output: Detailed data on the generation process

[1549] How it works: The server monitors the entire generation process in real time and collects detailed data such as "Question: What's the weather like?", "Answer: It's sunny today," "Model used: GPT-4," and "Time: XX seconds."

[1550] Step 4:

[1551] The server stores the collected detailed data in a storage device (database).

[1552] Input: Detailed data of the generation process

[1553] Output: Detailed data stored in a database

[1554] Specific operation: The server stores the collected detailed data in a database (e.g., PostgreSQL).

[1555] Step 5:

[1556] The server generates a summary memo based on the detailed data stored in the storage device, which includes a summary of the task and important points.

[1557] Input: Detailed data stored in the database

[1558] Output: Generated summary notes

[1559] Specific operation: The server uses an automatic summary generation tool (e.g., NLTK) to summarize the generation AI's answer history for the past week and create a summary memo.

[1560] Step 6:

[1561] The server provides the generated summary memo through a user interface (UI).

[1562] Input: Generated summary note

[1563] Output: A user-visible summary note

[1564] Specific operation: The server displays the summary memo through the UI (browser or desktop application) so that the user can view it.

[1565] Step 7:

[1566] The server connects with other automation services via APIs and integrates the work history data of each service.

[1567] Input: Work history data from other automated services

[1568] Output: Integrated work history data

[1569] Specific operation: The server uses a REST API (e.g., Flask) to integrate data from image recognition AI, etc.

[1570] Step 8:

[1571] The server analyzes the integrated work history data and finds the optimal work format.

[1572] Input: Integrated work history data

[1573] Output: Data in a format that works best for you

[1574] Specific behavior: The server uses data analysis tools (e.g., pandas, scikit-learn) to identify efficient algorithms and processes.

[1575] Step 9:

[1576] The server improves the process of generating the generated information based on the most suitable working format.

[1577] Input: Data in the format that works best for you

[1578] Output: Improved generation process

[1579] Specific operation: The server automatically applies the identified optimal work format to the next task, improving the accuracy and speed of the generative AI model.

[1580] Step 10:

[1581] The device collects user feedback and uses it to customize the responses of the generative AI model.

[1582] Input: User feedback

[1583] Output: The customized generative AI model response

[1584] Specific operation: The device collects feedback from users using a feedback collection tool (e.g., Google Forms) and reflects it in the output of the generative AI model.

[1585] Step 11:

[1586] The device learns the user's preferences and style based on work history data and adjusts the output of the generative AI model accordingly.

[1587] Input: Work history data

[1588] Output: The output of the tuned generative AI model.

[1589] What it does: The device uses a machine learning model (e.g., scikit-learn) to learn the response format and tone preferred by a particular user and reflects that in the next output.

[1590] (Application example 1)

[1591] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1592] In modern logistics centers, maximizing work efficiency and minimizing errors are important. However, recording and analyzing work history, and then flexibly modifying work plans based on that information, is not easy. Current systems lack the tools to ensure data transparency and reliability while working efficiently. A major problem is the lack of a function that allows on-site staff to input information in real time and generate optimal notes based on that information. A new system is needed to solve these issues.

[1593] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1594] In this invention, the server includes means for monitoring generated data, means for collecting details of the generated data, means for storing the collected details in a database, means for generating memos based on the details stored in the database, means for providing the generated memos through a user interface, means for integrating the collected details in cooperation with other artificial intelligence services, means for analyzing the integrated details to find optimal work patterns, means for improving the process of the generated data based on the optimal work patterns, means for recording work history, generating memos, and improving work efficiency in the logistics center, means for inputting work information via a smartphone application and sending it to the server, and means for sending generated memos from the server to the smartphone application, thereby improving work efficiency in the logistics center, reducing work errors, and ensuring data transparency and reliability.

[1595] "Generated data" is data generated by artificial intelligence systems or other information systems.

[1596] "Monitoring means" refers to a means for observing the generated data in real time and detecting fluctuations or abnormalities.

[1597] "Collection means" refers to the means for systematically capturing and recording detailed information about the data generated.

[1598] A "database" is an area where detailed information on collected data is systematically stored and can be retrieved as needed.

[1599] The "means for generating memos" refers to a means for organizing the main points based on the collected detailed information and generating concisely written memos.

[1600] A "user interface" is an interface that provides a screen and operating means for a user to interact with a system.

[1601] "Artificial Intelligence Services" are services based on artificial intelligence technology that are utilized to perform specific tasks.

[1602] An "optimal work pattern" is an efficient and effective work procedure or method discovered by analyzing past work history.

[1603] "Process improvement measures" are measures to improve current work procedures and methods based on optimal work patterns, thereby increasing efficiency and effectiveness.

[1604] A "smartphone application" is software that runs on a smartphone and provides specific functions.

[1605] "Work information" is information relating to the specific content and status of work being carried out at the logistics center.

[1606] A "server" is a computer system for processing, storing, and managing data.

[1607] The purpose of this invention, the "AI MemoSphere System," is to improve work efficiency at logistics centers, reduce work errors, and ensure data transparency and reliability. Specific implementations of this system are described below.

[1608] Hardware and software used

[1609] Hardware:

[1610] Server: AWS server (EC2)

[1611] Device: Smartphone (iOS / Android)

[1612] software:

[1613] Server side: Python (Django framework)

[1614] Database: PostgreSQL

[1615] Frontend: React Native

[1616] AI model: GPT-4 (API provided by OpenAI)

[1617] API integration: AWS API Gateway

[1618] System Program

[1619] Recording work history

[1620] The server monitors information about each operation performed at the distribution center in real time and collects the generated data. Specifically, staff enter operation information (e.g., item ID, quantity, picking time) through a smartphone application, and the data is sent to the server, which then stores it in a database.

[1621] Generate notes

[1622] Based on the collected work data, the server uses an AI model (GPT-4) to generate memos, which concisely summarize the main points and important information of the work. These memos are then sent from the server to a smartphone application where they can be viewed by staff.

[1623] Viewing work history

[1624] Users (logistics center staff and managers) can view work history and generated notes through a smartphone application, making it easy to understand how each task was performed and what results were achieved.

[1625] Analysis of optimal work patterns

[1626] The server analyzes the collected comprehensive work data to find the optimal work pattern, and the results of this analysis are reflected in the next work, allowing more efficient methods to be automatically applied.

[1627] Specific examples

[1628] At a distribution center, staff have a task of picking items from shelves. When they input the task data (e.g., item ID, quantity, picking time) into a smartphone application, the history is sent to the server and stored in a database. GPT-4 creates notes from the generated history, and these notes can be viewed by staff and managers through the app.

[1629] Prompt Sentence Examples

[1630] Below are some example prompts sent to the AI ​​model:

[1631] Task ID: 123 Details:

[1632] Input data: Item ID: A001, Quantity: 10, Picking time: 15 minutes

[1633] Algorithm used: Picking algorithm version 2.0

[1634] Output: Success, all items picked correctly

[1635] Generated note:

[1636] In this task, 10 units of item ID A001 were accurately picked. The picking algorithm used was version 2.0, which took 15 minutes to complete. This algorithm was effective for this task.

[1637] In this way, the AI ​​MemoSphere system can support skilled personnel in logistics centers and provide a highly transparent and efficient working environment.

[1638] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1639] Step 1:

[1640] The server monitors the generated data. Staff at the logistics center use a smartphone application to input work information. This input (e.g., item ID, quantity, picking time) is sent to the server. The server receives and monitors this data in real time. The input data is work information such as item ID, quantity, and picking time, and the output is the data received by the server.

[1641] Step 2:

[1642] The server collects details of the generated data. The server analyzes the task data sent from the smartphone application and extracts detailed information. The collected data includes the execution time of each task, the version of the algorithm used, and the success or failure of the task. These detailed information are stored in a database. The input data is the task information from the smartphone application, and the output is the detailed information stored in the database.

[1643] Step 3:

[1644] The server generates a memo based on the detailed information stored in the database. The server sends the collected work data details to the AI ​​model (GPT-4) and receives the generated memo. The prompt sentence to be sent to the AI ​​model is composed as follows:

[1645] Task ID: 123 Details:

[1646] Input data: Item ID: A001, Quantity: 10, Picking time: 15 minutes

[1647] Algorithm used: Picking algorithm version 2.0

[1648] Output: Success, all items picked correctly

[1649] Generated note:

[1650] In this task, 10 units of item ID A001 were accurately picked. The picking algorithm used was version 2.0, which took 15 minutes to complete. This algorithm was effective for this task.

[1651] The input data are the collected details, and the output are the notes generated by the AI ​​model.

[1652] Step 4:

[1653] The server provides the generated notes through a user interface. The generated notes are sent to a smartphone application so that the user can view them. The user can check the notes for each task through the smartphone application and understand the details of the task. The input data are the notes generated by the AI ​​model, and the output is the notes displayed in the user interface.

[1654] Step 5:

[1655] The server integrates the collected details in cooperation with other AI services. For example, by collaborating with an image recognition AI, it also collects image data and integrates detailed information based on that. This generates data that includes not only text information but also image information. The input data is detailed information from other AI services, and the output is the integrated detailed information.

[1656] Step 6:

[1657] The server analyzes the integrated details to find the optimal work pattern. It compares multiple work histories and extracts the most efficient work procedures and methods. This allows the optimal pattern to be applied in future work. The input data is the integrated details, and the output is the optimal work pattern.

[1658] Step 7:

[1659] The server improves the process of the generated data based on the optimal work pattern. Based on the discovered optimal pattern, it automatically corrects the current work procedures and methods to improve efficiency. The input data is the optimal work pattern, and the output is the improved work process.

[1660] Step 8:

[1661] The device collects user feedback and uses it to customize the AI's behavior. Feedback provided by the user through a smartphone application improves the AI ​​model's output and adapts to the user's needs. The input data is the user's feedback, and the output is the customized AI model's behavior.

[1662] In this way, the AI ​​MemoSphere System can improve work efficiency and ensure data transparency in logistics centers, providing a more reliable work environment.

[1663] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1664] The AI ​​MemoSphere system of the present invention aims to record the history of tasks performed by an artificial intelligence, generate memos based on that history, and provide them to users in a transparent and reliable manner. Furthermore, the present invention incorporates an emotion engine that recognizes the user's emotions, and by taking emotion data into consideration, it is possible to provide more personalized services. Below, a specific embodiment of this system will be described in detail, with program processing explained in natural language and examples provided.

[1665] Recording work history

[1666] The server monitors the generated data in real time. For example, when a generative AI generates an answer to a user's question, the server continuously monitors this process.

[1667] The server collects details of each task (input data, algorithm used, output results, user emotion data, etc.) and stores them in a database. When a user asks a question about a product, the generative AI's answer, the process leading to that answer, the model and algorithm version used, and the user's emotion are recorded.

[1668] Use of emotion engine

[1669] The device analyzes the user's facial expressions and tone of voice as they type, and uses an emotion engine to recognize the user's emotions.

[1670] The server stores the emotion data recognized by the emotion engine in a database. For example, if the user is dissatisfied, the emotion data is also recorded.

[1671] Create and manage notes

[1672] The server generates memos based on the collected work history and emotional data. The generated memos include a summary of the task, important points, and the user's emotional data.

[1673] Example: The server generates a summary memo containing important information about the answer generation tasks performed by the AI ​​over the past week, as well as changes in user sentiment. This summary memo includes information such as what types of questions were most popular, the trends in their answers, and the user's emotional reactions.

[1674] Viewing work history

[1675] Users can access their work history notes through a dedicated user interface (UI), which can be provided as a browser-based or desktop application.

[1676] Example: A user can access the UI to see the AI's answer process to their question and the emotional data at the time, and view detailed notes, making it easier to understand how the answer was generated and how it reflected their emotional state at the time.

[1677] Integration with other AI services

[1678] The server connects with other AI services via APIs and integrates the work history and emotional data of each AI service.

[1679] Example: By linking text generation AI and image recognition AI, users can simultaneously provide images when asking text questions, and the process of generating answers based on these images is recorded and integrated. Users can then refer to this link history and emotion data in a unified manner.

[1680] Analysis of optimal work patterns

[1681] The server analyzes the integrated work history data and emotion data to find optimal work patterns.

[1682] Example: The server finds the most efficient algorithms and processes across multiple tasks and automatically applies those patterns to the next task. This takes emotional data into account, improving the accuracy of AI responses and user satisfaction.

[1683] Adapting to user needs

[1684] The device collects user feedback and uses this to customize the AI's behavior.

[1685] Example: If a user requests a particular style, format, or even a desired emotional tone, that feedback will be incorporated and the next AI output will be tailored to those preferences.

[1686] Creating metrics and evaluating performance

[1687] The server creates new performance indicators based on collected work history, emotional data, and user feedback to evaluate the AI's performance.

[1688] Example: In addition to response speed and accuracy, evaluations are conducted based on a combination of multiple metrics, such as user satisfaction and emotional response, to identify areas for improvement in AI and improve its quality.

[1689] Improving processes by taking emotional data into account

[1690] The server then uses the analysis results to improve the processing of the generated data, specifically by automatically applying algorithm settings and parameters that reflect the user's emotional data.

[1691] Example: For tasks that are likely to frustrate users, the AI ​​is tuned to generate more polite responses.

[1692] Adapts to user preferences and work styles

[1693] The device analyzes work history data and emotional data to tailor responses to best suit the user's preferences and work style.

[1694] Example: Learning the tone and style of responses preferred by a particular user and tailoring output accordingly, allowing it to best meet the user's individual needs and emotions.

[1695] In this way, the AI ​​MemoSphere system records, manages, and analyzes the work history of the generating AI and user emotional data in detail, and provides users with highly transparent services based on this information. This improves the reliability and personalization of the AI, enabling efficient and high-quality work.

[1696] The processing flow will be explained below.

[1697] Step 1:

[1698] The server monitors the generated data in real time. Specifically, when the generation AI starts a task, it acquires the task's identification information and triggers the monitoring process. For example, when a user enters a question, it generates a question ID and records it as a monitoring target.

[1699] Step 2:

[1700] The device analyzes the user's facial expressions and tone of voice when inputting and uses an emotion engine to recognize the user's emotions. Specifically, it uses a camera and microphone to detect the user's emotional state in real time and sends that data to the emotion engine.

[1701] Step 3:

[1702] The server stores the user's emotion data recognized by the emotion engine in a database. For example, if the user makes a dissatisfied expression, the emotion data is recorded and associated with the question ID.

[1703] Step 4:

[1704] The server collects details of each task (input data, algorithm used, processing steps, output results, and user emotional data), including the question entered by the user, the process of the generative AI, and the final answer.

[1705] Step 5:

[1706] The server stores the collected details in a database, which stores the data according to an appropriate schema in a format that can be queried later.

[1707] Step 6:

[1708] The server generates notes based on the details stored in the database, including a summary of the task, key points, and the user's emotional data.

[1709] Step 7:

[1710] The server provides the generated notes through a dedicated user interface (UI), which is designed as a browser-based or desktop application.

[1711] Step 8:

[1712] The user can view the work history notes through the user interface. The user can check the history of a specific task, the contents of the notes, and the emotional data at the time.

[1713] Step 9:

[1714] The server will connect with other AI services via API, allowing for the sending and receiving of work history data and emotion data between the connected services.

[1715] Step 10:

[1716] The server integrates the work history and emotion data collected from the linked AI services, thereby centrally managing the history data from multiple services.

[1717] Step 11:

[1718] The server analyzes the integrated data to identify optimal work patterns, applying machine learning algorithms and statistical models to identify the most effective processes, procedures, and emotional responses.

[1719] Step 12:

[1720] The server then uses the analysis results to improve the processing of the generated data, specifically by automatically applying optimized algorithm settings and parameters that also take emotion data into account.

[1721] Step 13:

[1722] The terminal provides an interface for collecting user feedback, allowing users to input their opinions and thoughts on the user interface.

[1723] Step 14:

[1724] The server analyzes the collected feedback and calculates metrics to evaluate the performance of the generated data. It combines the feedback content, work history, and emotional data to derive various evaluation indicators.

[1725] Step 15:

[1726] The server learns the user's preferences and work style, which involves analyzing past usage data, feedback, and sentiment data to extract the user's specific needs and tendencies.

[1727] Step 16:

[1728] The server then customizes the processing of the generated data based on the learning results, delivering responses and functionality in a format that best suits the user's preferences, work style, and emotions.

[1729] Example 2

[1730] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1731] Conventional AI systems focus on generating answers to user questions, but the process often lacks transparency and reliability. They also lack the ability to recognize user emotions and provide responses that take these into account. This makes it difficult to provide truly useful and personalized services to users. Furthermore, there are also issues with inconsistencies in analyzing optimal work patterns and integrating with other AI services.

[1732] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1733] In this invention, the server includes means for monitoring the generated data, means for collecting details of the generated data, means for storing the collected details in a database, means for generating memos based on the details stored in the database, means for providing the generated memos through a user interface, means for integrating the collected details in cooperation with other AI services, means for analyzing the integrated details to find optimal work patterns, means for improving the process of the generated data based on the optimal work patterns, means for recognizing user emotion data, means for collecting and storing the recognized emotion data in a database, and means for generating memos based on the collected and stored emotion data. This makes it possible to record, manage, and analyze the work history of the generative AI model and the user emotion data in detail, and to provide users with a transparent and reliable personalized service.

[1734] "Generated Data" refers to the responses or results generated by a generative AI model in response to a user's question or request.

[1735] "Monitoring means" refers to a system including hardware and software for monitoring the generated data in real time and understanding its contents.

[1736] "Collection means" refers to the set of processes or functions used to compile and record details of the data obtained through surveillance.

[1737] "Database" refers to a system of information collection that stores details of collected data in an organized manner so that they can be easily retrieved and used later.

[1738] "Means for generating notes" refers to algorithms or programs that create summary information based on the details and emotional data collected.

[1739] "User Interface" refers to the screens and applications that allow a user to access and manipulate generated notes and data details.

[1740] "Other artificial intelligence services" refers to artificial intelligence technologies and services that exist other than the system of the present invention, and refers to systems that can be linked with these.

[1741] "Means of integration" refers to the ability to bring together data and information obtained from other artificial intelligence services and manage them in a consistent manner.

[1742] "Optimal work patterns" refer to efficient and effective work procedures and processes discovered through detailed analysis of collected data.

[1743] "Measures for improvement" refers to methods and systems for improving the current data generation process based on the optimal work patterns identified.

[1744] "Emotion data" refers to data that indicates the emotional state recognized from the user's statements and actions.

[1745] "Means for recognizing emotions" refers to a system that includes hardware and software for analyzing data such as a user's facial expressions and voice and identifying the user's emotions.

[1746] "Collected and stored emotion data" refers to the state in which the recognized emotion data is stored in a database and can be used for later interpretation and analysis.

[1747] The AI ​​MemoSphere system of the present invention records, manages, and analyzes the work history of the generative AI model and the user's emotional data in detail, and provides users with a transparent, reliable, and personalized service based on the data. Specific embodiments of the system are described below.

[1748] Basic configuration

[1749] This system consists of a server, a terminal, and a user. The server mainly monitors, collects, stores, and analyzes data, while the terminal provides a user interface and collects emotion data.

[1750] Hardware and software used

[1751] Generative AI models: For example, using GPT-3 for natural language processing.

[1752] Server: A high-performance computer server that collects, stores, analyzes, and generates notes on data.

[1753] Database: For example, using a relational database management system (RDBMS).

[1754] Emotion Engine: Uses Microsoft Azure's Emotion API to recognize user emotions.

[1755] User interface: Provided as a web browser or desktop application.

[1756] Processing flow and specific examples

[1757] 1. Monitoring generated data:

[1758] The server monitors the process in real time of the generative AI model generating answers to user questions. For example, if a user asks, "What is the price of a new product?", the request is passed to the generative AI model (GPT-3).

[1759] 2. Collect task details:

[1760] The server collects detailed information about the algorithms used by the generative AI, input data, output results, and user emotional data, and stores it in a database. For example, it records the generative AI model used, the prompt (user question), the output result (product price), and the user's emotional data (dissatisfaction, satisfaction, etc.).

[1761] 3. User emotion recognition:

[1762] The device recognizes the user's facial expression and tone of voice when they input a question and uses an emotion engine to recognize their emotion. For example, if a user says "Is this really the right price?" in a dissatisfied tone, their facial expression and tone of voice are captured.

[1763] 4. Emotional Data Recording:

[1764] The server stores the recognized emotion data in a database. Specifically, the server receives emotion data (e.g., dissatisfaction) sent from the device and adds it to the database.

[1765] 5. Automatic note generation:

[1766] The server generates memos based on the collected work history and emotion data, including task summaries and key points. For example, it summarizes the tasks performed by the AI ​​over the past week and creates a summary memo that includes the user's emotions regarding each task.

[1767] 6. Accessing the User Interface:

[1768] Users can access their work history notes through a dedicated user interface, such as a dashboard in a web browser, to view details of past interactions and sentiment data.

[1769] 7. Data integration with other AI services:

[1770] The server also works with image recognition AI (e.g., an image analysis system) to integrate the entire work history and emotional data. For example, a user can upload a product image and record the answer generation process based on it.

[1771] 8. Extracting optimal work patterns:

[1772] The server then extracts the optimal work pattern based on the integrated work history and emotional data and applies it to the next task, for example by analyzing multiple response patterns and selecting the fastest and most effective algorithm to apply.

[1773] 9. Collaboration with Feedback:

[1774] The device collects user feedback and customizes the AI's behavior based on that feedback. For example, if the user provides feedback such as "Please provide more detailed explanation next time," that feedback is sent to the server and reflected in the next response.

[1775] 10. Performance Evaluation:

[1776] The server creates new evaluation indicators based on work history, emotional data, and user feedback to evaluate AI performance. For example, it creates an indicator that combines response speed, accuracy, and user satisfaction.

[1777] Examples and prompts

[1778] For example, a user inputs a prompt such as "Please tell me the price of the new product," and the answer generation process and emotional data based on that are processed. The AI's answer to the user's question (e.g., the price of the new product) and the user's emotion in response (e.g., dissatisfaction) are recorded.

[1779] In this way, the AI ​​MemoSphere system records and analyzes the work history of the generative AI model and the user's emotional data in detail, providing a transparent, reliable and personalized service.

[1780] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1781] Step 1:

[1782] Monitoring generated data

[1783] Input: A prompt from the user (e.g., "What is the price of the new product?")

[1784] Output: Details of the answer generation process by the generative AI model

[1785] Specific operation: The server monitors the prompt sentences entered by the user and tracks how the generative AI model (e.g., GPT-3) responds to the prompt. When a user enters "Please tell me the price of the new product," the server sends this prompt sentence to the generative AI model and records its response process.

[1786] Step 2:

[1787] Collecting task details

[1788] Input: Responses output by the generative AI model, prompts, algorithms, and user emotional data

[1789] Output: Collected task details data

[1790] What it does: The server collects the answer generated by the generative AI model (e.g., "The price of the new product is $100"), the algorithm used (e.g., the version and specific settings of GPT-3), the input prompt, and the user's emotional data. These details are then compiled and stored in a database.

[1791] Step 3:

[1792] Recognizing user emotions

[1793] Input: User's facial expression data, voice data

[1794] Output: Emotion recognition result (e.g., dissatisfied, satisfied)

[1795] Specific operation: The device captures the user's facial expressions and tone of voice when entering prompts using a camera and microphone, and analyzes them using an emotion engine (e.g., Microsoft Azure's Emotion API). The user may say, with a dissatisfied tone, "Is this really the right price?", and their facial and voice data is captured.

[1796] Step 4:

[1797] Emotional data recording

[1798] Input: Emotion recognition results

[1799] Output: Emotion data stored in a database

[1800] Specific operation: The server receives the emotion data sent from the device and stores it in a database. For example, it stores the emotion data "dissatisfied" in the database.

[1801] Step 5:

[1802] Auto-generate notes

[1803] Input: Collected task details data, emotion data

[1804] Output: Auto-generated notes

[1805] Specific operation: The server generates a memo based on the collected task details and emotion data. This includes a brief summary of the task and key points. For example, it may summarize all tasks performed by the AI ​​in the past week and create a summary memo including the user's associated emotion data.

[1806] Step 6:

[1807] Accessing the User Interface

[1808] Input: User's reference request

[1809] Output: Display of created note

[1810] Specific operation: The user refers to the work history memo through a dedicated user interface (e.g., a web browser). The user accesses the dashboard on the browser to view details of past interactions and emotion data.

[1811] Step 7:

[1812] Data integration with other AI services

[1813] Input: Data from other AI services (e.g., image recognition results)

[1814] Output: Integrated work history and emotion data

[1815] Specific operation: The server works with image recognition AI (e.g., image analysis system) in addition to the generation AI, and integrates their work history and emotional data. For example, a user uploads a product image and records the answer generation process based on it.

[1816] Step 8:

[1817] Extracting optimal work patterns

[1818] Input: Integrated work history, emotional data

[1819] Output: The optimal work pattern found

[1820] Specific operation: The server performs detailed analysis of the integrated work history and emotion data to extract the optimal work pattern. For example, it analyzes multiple response patterns to find the fastest and most effective algorithm and apply it to the next task.

[1821] Step 9:

[1822] Collaboration with Feedback

[1823] Input: User feedback

[1824] Output: Customized AI response

[1825] Specific operation: The device collects feedback from the user and sends it to the server. For example, if the user says, "I'd like more detailed explanation next time," the device customizes the AI's response based on this feedback and reflects it in the next response.

[1826] Step 10:

[1827] Performance Evaluation

[1828] Input: Work history, emotion data, feedback

[1829] Output: Generated performance metrics

[1830] Specific operation: The server creates new evaluation indicators based on work history, emotional data, and user feedback to evaluate the AI's performance. For example, it creates an indicator that combines response speed, accuracy of generated results, and user satisfaction, and evaluates the AI's performance based on that.

[1831] In this way, the AI ​​MemoSphere system records, manages, and analyzes the work history of the generative AI model and user emotional data in detail, providing transparent, reliable, and personalized services.

[1832] (Application example 2)

[1833] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1834] Modern customer service requires accurately understanding customer emotions and needs and providing appropriate responses in real time. However, traditional systems make it difficult to provide services that fully reflect customer emotions, and as a result, they are unable to provide responses that increase customer satisfaction. Furthermore, it is difficult to record detailed customer interaction history and refer to it later, resulting in a lack of data to continuously improve service quality.

[1835] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for monitoring generated data, means for collecting details, means for storing details in a database, means for generating memos, means for providing the memos through a user interface, means for integrating the collected details in cooperation with other AI services, means for analyzing the integrated details to find optimal work patterns, means for improving processes based on the optimal work patterns, means for capturing visual and audio data and recognizing user emotions, means for storing the emotion data in a database and incorporating it into memos, and means for real-time recording and emotion recognition using a smart device. This enables accurate understanding of customer emotions and appropriate responses in real time. Furthermore, by recording details of interactions with customers and referencing them later, service quality can be continuously improved and customer satisfaction can be increased.

[1836] "Generated data" is information that results from tasks performed by artificial intelligence.

[1837] "Details" are specific information including input data to the generated data, algorithms used, output results, and other relevant information.

[1838] A "database" is a system for storing collected details and emotional data for later reference and analysis.

[1839] A "memo" is a summary document generated based on collected details and emotion data, and is information provided to the user.

[1840] "User interface" refers to the interaction environment through which a user accesses the system and views and inputs notes and data.

[1841] An "artificial intelligence service" is a collection of AI systems designed to perform a specific task, such as text generation or image recognition.

[1842] An "optimal work pattern" refers to the most efficient and accurate algorithm or process for multiple tasks.

[1843] "Visual data" refers to image and video data captured by a camera or other imaging device.

[1844] "Audio data" refers to recorded audio data captured by an audio device such as a microphone.

[1845] "Emotion data" refers to data relating to the user's emotional state analyzed from facial expressions, tone of voice, and the like.

[1846] A "smart device" is an interactive device such as glasses equipped with a camera and microphone.

[1847] "Recording" is the act of saving interaction details and emotional data in a database.

[1848] "Emotion recognition" is the process of analyzing and identifying a user's emotions from visual and audio data.

[1849] The present invention is a system for capturing visual and audio data in real time, recognizing user emotions, and providing high-quality customer service based on the data. To implement the present invention, the following technical configuration and steps are required.

[1850] First, a smart device (e.g., smart glasses equipped with a camera and microphone) captures customer interactions in real time. The device then sends the captured video and audio data to the EmotionEngine (an emotion recognition library) to recognize the user's emotions. The recognized emotion data is then immediately sent to a server and stored in the MemoSphere system's database.

[1851] The server then monitors the data generated by the AI ​​model and collects details (such as input data, algorithms used, output results, and emotional data). The collected data is stored in a database for future reference. For example, if a customer asks about the price of a particular product, the customer's facial expression and tone of voice can be analyzed to record emotional data such as "interest" or "anxiety."

[1852] The server generates appropriate notes for users based on real-time interactions and emotional data. These notes reflect the customer's questions and emotional changes and are provided as specific, personalized information. The generated notes are provided to store clerks through the user interface, who then refer to them when assisting the customer.

[1853] In addition, the server will also work with other artificial intelligence services, integrating functions such as text generation and image recognition, enabling more advanced information provision. For example, when a customer presents a product image, the image can be recognized and relevant information can be instantly provided.

[1854] The system also analyzes optimal work patterns, identifying the most efficient algorithms and processes across multiple tasks and automatically applying them to the next task. Based on collected data and sentiment data, the system automatically improves processes and enhances customer service.

[1855] In this way, smart devices can be used to understand customer sentiment in real time and take appropriate action based on that sentiment. Furthermore, detailed records and analysis of the collected data can be used to continuously improve the quality of service.

[1856] For example, when a store clerk wearing smart glasses interacts with a customer, the customer asks about the price of a product. The smart glasses' camera captures the customer's facial expressions, and the Emotion Engine analyzes their expressions for interest and anxiety. The MemoSphere system then records the details of the interaction along with their emotions and generates a summary memo that can be referenced later.

[1857] An example prompt is:

[1858] "Analyze the emotions customers express when asking about product prices. Record the emotion data and details of the interaction."

[1859] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1860] Step 1:

[1861] The device captures visual and audio data. Specifically, it uses the camera and microphone of the smart glasses to record real-time interactions with customers. The input is the video and audio data obtained from the camera and microphone, and the output is the process of sending this data to the EmotionEngine.

[1862] Step 2:

[1863] The device uses the EmotionEngine to recognize customer emotions from captured visual and audio data. Specifically, it analyzes facial expressions from video and tone from audio. The input is visual and audio data, and the output is recognized emotional data.

[1864] Step 3:

[1865] The device sends the generated emotion data to the server. Specifically, the output data from the Emotion Engine is transferred to the server in real time. The input is the emotion data, and the output is the data reaching the server.

[1866] Step 4:

[1867] The server stores the received emotion data and interaction details in the MemoSphere system database. Specifically, it records detailed information such as the interaction content, the algorithm used, and the output results. The input is emotion data and detailed interaction information, and the output is storing this in the database.

[1868] Step 5:

[1869] The server generates memos based on the collected data. Specifically, it integrates the content of the interaction with emotional data to create memos containing key points and summaries. The input is the details and emotional data stored in the database, and the output is the generated memo.

[1870] Step 6:

[1871] The server provides the generated memo to the terminal through a user interface. Specifically, the memo can be viewed by the store clerk through smart glasses. The input is the generated memo, and the output is the memo as information provided to the user.

[1872] Step 7:

[1873] The server works with other AI services to integrate collected data and emotional data. Specifically, it integrates data from other AI services such as text generation and image recognition, enabling centralized information acquisition. The input is data from other AI services, and the output is the integrated detailed data.

[1874] Step 8:

[1875] The server analyzes the integrated detailed data and finds the optimal work pattern. Specifically, it identifies the most efficient algorithms and processes for multiple tasks. The input is the integrated detailed data, and the output is the optimal work pattern.

[1876] Step 9:

[1877] The server improves the process based on the optimal work pattern. Specifically, it automatically applies the newly identified work pattern to the next task, improving efficiency and quality. The input is the optimal work pattern, and the output is the improved process.

[1878] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1879] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1880] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1881] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1882] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1883] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1884] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1885] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1886] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1887] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1888] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1889] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1890] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1891] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1892] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1893] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1894] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1895] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1896] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1897] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1898] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1899] The following is further disclosed regarding the above embodiment.

[1900] (Claim 1)

[1901] a means for monitoring the generated data;

[1902] means for collecting details of said generated data;

[1903] means for storing said collected details in a database;

[1904] means for generating notes bas...

Claims

1. a means for monitoring the generated data; means for collecting details of said generated data; means for storing said collected details in a database; means for generating notes based on details stored in said database; means for providing the generated notes through a user interface; means for integrating said collected details in conjunction with other artificial intelligence services; and means for analyzing said integrated details to find optimal work patterns; means for improving the process of the generated data based on the optimal work pattern; A system including:

2. The system of claim 1 , further comprising means for combining the collected details with user feedback obtained through the user interface to evaluate performance of the generated data.

3. 10. The system of claim 1, further comprising means for learning user preferences and work styles from the collected details and customizing the processing of the generated data based on the learning.

Images