System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting business knowledge and know-how from a plurality of data sources within an enterprise; means for analyzing and visualizing the collected data using a generative AI model; and means for receiving a query from a user and providing the analyzed information to a user terminal.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Because business knowledge and know-how in a company is dispersed across various departments, the growth of new employees and general employees is slow, and managers' decision-making takes time and effort. Conventional systems make it difficult to quickly obtain the necessary information, preventing users from effectively carrying out their work. To improve this situation, a system that allows efficient and rapid access to business knowledge is needed. [Means for solving the problem]

[0005] This invention provides a means for collecting business knowledge and know-how from multiple data sources within a company. It then provides a means for analyzing and visualizing the collected data using a generative AI model. It also configures a system that includes a means for receiving user inquiries and providing analyzed information to the user's terminal. In particular, by providing a means for performing natural language processing using a generative AI model based on the user's inquiry and extracting relevant information, it is possible to understand the user's intent with high accuracy and quickly provide the necessary information. This system is compatible with a variety of devices, including smartphones, personal computers, and smart glasses, allowing users to quickly access information anytime, anywhere.

[0006] "Data source" refers to various information sources that are referenced to gather business knowledge and know-how within a company.

[0007] "Business knowledge" refers to the specific knowledge and know-how that a company needs to carry out its business efficiently.

[0008] "Know-how" refers to technical knowledge and skills based on experience required to carry out a specific task.

[0009] A "generative AI model" is an artificial intelligence technology that generates new information based on input data, such as GPT (Generative Pre-trained Transformer).

[0010] "Analysis" refers to the process of analyzing input data and information to extract useful insights and patterns.

[0011] "Visualization" refers to a method of making extracted data or information easier to understand by displaying it in the form of graphs, charts, etc.

[0012] A "user terminal" is a device used by a user to input and output information, and includes smartphones, personal computers, smart glasses, etc.

[0013] "Natural language processing" refers to technology for analyzing human language and understanding its meaning and intent.

[0014] A "question answering system" refers to a system that provides appropriate answers to questions from users. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] The present invention is a system for efficiently sharing business knowledge and know-how within a company and supporting rapid decision-making. Specific embodiments of this system will be described below.

[0037] 1. Data Collection

[0038] server

[0039] The server collects business knowledge and know-how from data sources within the company (document management systems, databases, internal SNS, etc.), obtains the content of each data source via API, and stores and manages it in an integrated data repository.

[0040] 2. Data analysis and visualization

[0041] server

[0042] The collected data is preprocessed and analyzed using a generative AI model, which uses, for example, GPT, to summarize the relevant information in an easy-to-understand format.

[0043] To visualize the generated information, visualization tools are used to convert the data into graphs, charts, flowcharts, and other formats, making it easier for users to understand and use the information.

[0044] 3. User Inquiry Processing

[0045] User

[0046] Users, such as new employees, general employees, and managers, use devices (smartphones, PCs, smart glasses, etc.) to inquire about the information they want to know. For example, they input specific questions such as, "Please tell me about the latest market trends."

[0047] Terminal

[0048] When the terminal receives an inquiry from a user, it analyzes the content using natural language processing and sends a request to the server.

[0049] 4. Server Response Generation

[0050] server

[0051] Based on the received inquiry, relevant data is extracted from the data repository, and information appropriate to the user's question is generated using a generative AI model.

[0052] A response is generated and sent to the terminal to provide the visualized information to the user.

[0053] 5. Providing Feedback

[0054] Terminal

[0055] The information received from the server is displayed to the user. For example, work procedures can be displayed in report format on a smartphone, or work procedures can be navigated in real time using smart glasses.

[0056] User

[0057] Users use the information provided to them to carry out their work, such as when a new employee learns and understands work procedures or when a manager makes decisions based on the latest market trends.

[0058] Specific examples

[0059] Example 1: A new employee wants to know the work procedure

[0060] User

[0061] A new employee types "Please tell me the work procedures" into their smartphone.

[0062] Terminal

[0063] This query is sent to the server.

[0064] server

[0065] Analyze inquiries and extract relevant data about business procedures.

[0066] We summarize the steps required in the GPT model and visualize them as a flowchart.

[0067] Send the flowchart to your smartphone.

[0068] Terminal

[0069] Display a flowchart on a smartphone to provide information to new employees.

[0070] User

[0071] New employees check the flowchart on their smartphones and understand the work procedures.

[0072] Example 2: Manager wants to know market trends

[0073] User

[0074] The manager types into his PC, "Tell me about the latest market trends."

[0075] Terminal

[0076] This query is sent to the server.

[0077] server

[0078] Analyze inquiries and extract relevant data on market trends.

[0079] The GPT model summarizes market trends and visualizes them as graphs and charts.

[0080] The visualized data is sent to a PC.

[0081] Terminal

[0082] Graphs and charts of market trends are displayed on a PC to provide information to managers.

[0083] User

[0084] Managers check market trends on their PCs and make appropriate decisions.

[0085] The processing flow will be explained below.

[0086] Step 1:

[0087] User

[0088] The user uses a device (e.g., a smartphone or PC) to input the information they want to know and query the system. For example, the user might input "Tell me the latest market trends" into their PC.

[0089] Step 2:

[0090] Terminal

[0091] The terminal receives the user's input and generates a request to send to the server. The request includes the user's question, the user's identification information, and the terminal information, and is sent to the server.

[0092] Step 3:

[0093] server

[0094] The server receives the request from the device and uses natural language processing (NLP) techniques to analyze the content of the received request. NLP techniques perform processes such as tokenization, stemming, and entity recognition.

[0095] Step 4:

[0096] server

[0097] Based on the query content analyzed by NLP technology, the server extracts relevant data from a data repository that stores business knowledge, know-how, the latest market information, and more.

[0098] Step 5:

[0099] server

[0100] The extracted data is fed into a generative AI model (e.g., GPT) to generate and summarize information, which is then organized into a format that can provide appropriate answers to user questions.

[0101] Step 6:

[0102] server

[0103] The generated information is converted into graphs, charts, etc. using visualization tools. The visualized information is organized so that it can be easily understood by users.

[0104] Step 7:

[0105] server

[0106] A response is generated to return the visualized information to the user terminal, the response including a display format of the visualized information and any necessary associated data.

[0107] Step 8:

[0108] Terminal

[0109] The terminal processes the response received from the server and displays it to the user, for example, by displaying graphs and charts of the latest market trends on a PC screen.

[0110] Step 9:

[0111] User

[0112] Users use the information displayed on the terminal to carry out necessary tasks, such as a manager making decisions based on the latest market information.

[0113] Example 1

[0114] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0115] In order to efficiently share business knowledge and know-how within a company and support rapid decision-making, a system is needed to centrally manage data from multiple sources and respond quickly and appropriately to user inquiries. However, current systems do not integrate the processes of data collection, analysis, visualization, and feedback, making it difficult to provide users with the information they need in a timely manner.

[0116] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0117] In this invention, the server includes a means for collecting business knowledge and know-how from multiple information sources within the company, a means for storing and managing the collected data in an integrated data repository, and a means for analyzing and summarizing the collected data using a generative AI model. This integrates a series of processes from data collection to analysis, visualization, and feedback, making it possible to respond quickly and appropriately to user inquiries.

[0118] "Multiple information sources within a company" refers to multiple data sources owned by a company, such as document management systems, databases, and internal social networking sites.

[0119] "Business knowledge and know-how" refers to information including knowledge and practical knowledge such as procedures, experience, rules, and technical information required for the operation of a company.

[0120] "Means of collection" refers to APIs and software tools that allow companies to acquire and integrate business knowledge and know-how from multiple sources within the company.

[0121] An "integrated data repository" refers to a database or storage system where collected data is centrally stored and managed.

[0122] A "generative AI model" refers to an artificial intelligence model that generates and summarizes useful information from large amounts of data, for example, using a generative AI model (such as GPT-4 (registered trademark)).

[0123] "Means of analysis" refers to the process of preprocessing collected data and using generative AI models to extract and summarize the necessary information.

[0124] "Visualization tools" refers to software tools (e.g., Tableau, Power BI) that transform analyzed data into visual formats such as graphs, charts, and flowcharts.

[0125] "Natural language processing" refers to the technology used to analyze user queries and extract key keywords and important information.

[0126] "User terminal" refers to a device used by a user to input and receive information, such as a PC, smartphone, or smart glasses.

[0127] "Server" refers to the central computer system used for data collection, analysis, visualization, and response to the user.

[0128] The present invention is a system for efficiently sharing business knowledge and know-how within a company and supporting rapid decision-making. Specific embodiments of this system will be described below.

[0129] System configuration and operation

[0130] server

[0131] Data collection: The server collects business knowledge and know-how from multiple information sources, such as the company's internal document management system, databases, and internal SNS. To do this, the server obtains data using the API of each data source (for example, a general-purpose API or database query). The collected data is stored in an integrated data repository. Specifically, it obtains PDF files from the document management system, table entries from the database, and message history from the internal SNS.

[0132] Data analysis and summarization: The collected data is preprocessed using Python's Pandas library. It is then analyzed and summarized using a generative AI model (e.g., GPT-4). An example prompt might be, "Please summarize the key business steps from the data below."

[0133] Visualization: Using visualization tools (e.g., Tableau, Power BI), the analyzed data is transformed into visual formats such as graphs, charts, and flowcharts, which helps users intuitively understand the information.

[0134] User

[0135] Inquiry: A user (new employee, general employee, manager, etc.) makes an inquiry about information using a device such as a smartphone, PC, or smart glasses. For example, they might type "Please tell me about the latest market trends" into their smartphone.

[0136] Terminal

[0137] Natural Language Processing: The terminal receives a user query and analyzes the content using a natural language processing library (e.g., spaCy, NLTK, or other major natural language processing tools), for example, tokenizing the input text and extracting key keywords.

[0138] Query sending: Send a request to the server based on the analysis results. This is done using the REST API.

[0139] Server Response

[0140] Information extraction: The server extracts relevant data from the integrated data repository based on the received query, for example, searching for the latest data on "market trends."

[0141] Response generation: Generative AI models (e.g., GPT-4) are used to generate information appropriate for the user's question. An example prompt might be, "Please summarize the latest market trends based on the market data below."

[0142] Visualization and Presentation: The generated information is visualized using a visualization tool and the results are sent to the terminal. For example, the latest market trends can be visualized as a bar graph and sent to a PC.

[0143] Providing Feedback

[0144] Display: The terminal displays the information received from the server to the user. Specifically, it displays a flowchart of business procedures on a smartphone, or a graph of market trends on a PC.

[0145] Work execution: Users perform their work based on the information provided. This can be a new employee learning how to do their job or a manager reviewing the latest market trends and making decisions.

[0146] Specific examples

[0147] When a new employee wants to know the work procedures

[0148] 1. User: A new employee types "Please tell me the work procedure" into their smartphone.

[0149] 2. Terminal: Analyzes the query using natural language processing and sends a request to the server.

[0150] 3. Server: Analyzes the query and extracts data about business procedures from the document management system. It uses GPT-4 to summarize the key steps and visualize them as a flowchart. It then sends the visualized flowchart to the smartphone.

[0151] 4. Terminal: Display the flowchart on a smartphone and provide information to new employees.

[0152] 5. User: New employees review the flowchart and understand the work procedures.

[0153] When managers want to know market trends

[0154] 1. User: The manager types into his PC, "Tell me the latest market trends."

[0155] 2. Terminal: Analyzes the query using natural language processing and sends a request to the server.

[0156] 3. Server: Analyzes the query and extracts data about market trends from the database. It uses GPT-4 to summarize the market trends and visualize them as graphs. It then sends the visualized data to the PC.

[0157] 4. Terminal: Displays graphs of market trends on a PC and provides information to managers.

[0158] 5. User: Managers review the graphs and make appropriate decisions.

[0159] As described above, this system collects data from multiple sources within a company and uses AI models and visualization tools to provide users with appropriate business knowledge and know-how.

[0160] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0161] Step 1:

[0162] Data collection

[0163] The server collects business knowledge and know-how from multiple sources within the company, specifically, using APIs to obtain data from document management systems, databases, internal social networking sites, etc.

[0164] Input: Data from document management systems, databases, and APIs of internal SNS.

[0165] Output: Data stored in a unified data repository.

[0166] Specific operations: For example, retrieve "Business Procedures.pdf" from the document management system, retrieve the latest entries in the "Market Trends" table from the database, and retrieve the message history of the "General" channel from the internal SNS, and store these in the integrated data repository.

[0167] Step 2:

[0168] Data Preprocessing

[0169] The server preprocesses the collected data using the Python Pandas library.

[0170] Input: Data from the Unified Data Repository.

[0171] Output: Cleaned and filtered data.

[0172] Specific Actions: For example, removing empty fields from database entries and fixing inconsistent formatting.

[0173] Step 3:

[0174] Data analysis and summary generation

[0175] The server analyzes the preprocessed data with a generative AI model (e.g., GPT-4) and generates a summary. The summary is generated using a prompt sentence.

[0176] Input: Cleaned and filtered data.

[0177] Output: The summary generated by GPT-4.

[0178] Specific operation: For example, input the prompt sentence "Please summarize the key business steps from the following data" into GPT-4 and obtain a summary.

[0179] Step 4:

[0180] Data Visualization

[0181] The server converts the generated summaries into a visual format using a visualization tool (e.g., Tableau, Power BI).

[0182] Input: The summary generated by GPT-4.

[0183] Output: Visual data such as graphs, charts, and flowcharts.

[0184] Specific actions: For example, visualizing business procedures as a flowchart and displaying Slack message history as a pie chart.

[0185] Step 5:

[0186] Receiving inquiries from users

[0187] Users can request information using devices such as smartphones, PCs, or smart glasses.

[0188] Input: User's query (e.g., "What are the latest market trends?").

[0189] Output: The parsing request sent to the terminal.

[0190] Specific action: For example, a user types "Please tell me the latest market trends" into their smartphone.

[0191] Step 6:

[0192] Natural Language Processing

[0193] The device analyzes user inquiries using natural language processing, specifically using natural language processing libraries (e.g., spaCy, NLTK).

[0194] Input: User's inquiry.

[0195] Output: Analysis results of the query.

[0196] Specific behavior: For example, tokenize the input text and extract key keywords (e.g., "market trends," "latest").

[0197] Step 7:

[0198] Sending a query to the server

[0199] The terminal transmits a request to the server based on the analysis result.

[0200] Input: Analysis results (e.g., "Market Trends", "Latest").

[0201] Output: REST API request to the server.

[0202] Specific operation: Send the analysis results to the server using the REST API.

[0203] Step 8:

[0204] Server information extraction and response generation

[0205] Based on the received inquiry, the server extracts relevant data from an integrated data repository and uses a generative AI model to generate information appropriate to the user's question.

[0206] Input: Data extraction request based on analysis results.

[0207] Output: Information suitable for the user's question.

[0208] Specific operation: For example, input the prompt sentence "Please summarize the latest market trends based on the market data below" into GPT-4 and obtain a summary.

[0209] Step 9:

[0210] Information Visualization

[0211] The server visualizes the generated information using a visualization tool and transmits the results to the terminal.

[0212] Input: Information generated by a generative AI model.

[0213] Output: Visualized data.

[0214] Specific operation: For example, visualize the latest market trends as a bar graph and send it to a PC.

[0215] Step 10:

[0216] Viewing information and providing feedback

[0217] The terminal displays the information received from the server to the user, who then performs the task based on the information provided.

[0218] Input: Visualization data sent from the server.

[0219] Output: Information displayed on the terminal.

[0220] Specific operations: For example, a graph of market trends is displayed on a PC, and the user checks the information and makes a decision.

[0221] (Application example 1)

[0222] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0223] Although logistics centers require the rapid acquisition and sharing of business knowledge and know-how, current systems lack the means to efficiently collect, analyze, visualize, and provide this information. In particular, it is difficult for staff to obtain optimal picking routes and inventory information in real time, which reduces work efficiency. There is a need for a system that can solve these issues and support the efficiency of work and rapid decision-making at logistics centers.

[0224] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0225] In this invention, the server includes means for collecting business knowledge and know-how from multiple data sources within the company, means for analyzing and visualizing the collected data using a generative AI model, means for receiving voice requests from a user terminal, analyzing the requests using natural language processing, and transmitting related information to the server, and means for visualizing the related information transmitted from the server and displaying it on the user terminal in real time using augmented reality technology. This enables logistics center staff to efficiently obtain optimal picking routes and inventory information in real time, enabling them to carry out their work promptly.

[0226] A "data source" is an information source used to collect business knowledge and know-how, such as a document management system, database, or in-house SNS within a company.

[0227] A "generative AI model" is an artificial intelligence model that analyzes collected data and summarizes it in an easy-to-understand format, such as GPT.

[0228] "Visualization" is the process of converting data analyzed by a generative AI model into visual formats such as graphs, charts, and flowcharts, making the information easier for users to understand.

[0229] A "user terminal" is a device used by a user to obtain information, and includes smartphones, PCs, smart glasses, head-mounted displays, etc.

[0230] "Natural language processing" is a technology for analyzing user inquiries and extracting relevant information, and is applied to voice requests and text data.

[0231] "Augmented reality technology" is a technology that displays visualized information overlaid on the real world in real time, and refers to the use of AR (augmented reality).

[0232] A "picking route" is the optimal route for staff to efficiently select products at a logistics center, and is generated based on inventory information.

[0233] "Inventory information" refers to information relating to the type, quantity, location, etc. of products currently held within the logistics center.

[0234] "Real-time" refers to immediate processing and response to user requests without delay.

[0235] This invention provides a system that supports operational efficiency and rapid decision-making in logistics centers. To achieve this, it is important to collect business knowledge and know-how from multiple data sources within a company, analyze and visualize it using a generative AI model, and provide information in real time based on user inquiries.

[0236] System hardware and software configuration

[0237] Server: The central system for data collection, analysis, and visualization. The server must be equipped with a high-performance processor, large memory capacity, and storage. The hardware used can be a general-purpose high-performance server or cloud service. The software used includes a generative AI model such as GPT, an NLP library (e.g., Spacy), a database (e.g., MySQL (registered trademark)), and a data visualization tool (e.g., D3.js).

[0238] User devices: Devices such as smartphones, smart glasses, and head-mounted displays that users use to obtain information. These devices must have voice input, display, and internet connectivity capabilities.

[0239] Data collection

[0240] The server collects business knowledge and know-how from multiple data sources within the company (document management systems, databases, internal SNS, etc.), obtains the content of each data source via API, and stores and manages it in an integrated data repository.

[0241] Parsing voice requests

[0242] The user device (such as smart glasses) receives the voice request and converts it into text data using a natural language processing library (such as spacy). The analyzed text data is then sent to the server via an API.

[0243] Data analysis and visualization

[0244] Based on the received voice request, the server extracts relevant data from a data repository, uses a generative AI model (e.g., GPT) to generate information relevant to the user's question, and then uses a data visualization tool (e.g., D3.js) to transform the data into visual formats such as graphs, charts, and flowcharts.

[0245] Augmented reality information provision

[0246] The visualized information is sent to the user's device and displayed in real time on the user's device using augmented reality technology (such as ARKit or ARCore). For example, by using smart glasses, logistics center staff can visually navigate and quickly determine the optimal picking route.

[0247] Specific examples

[0248] Example 1: Navigation with voice requests and AR

[0249] A staff member wears smart glasses and asks, "Tell me which shelf I should pick up the next item from." The smart glasses analyze the voice and send a request to the server. The server generates a picking route and sends the visualized information to the smart glasses. The smart glasses then use their AR functionality to navigate the staff member.

[0250] Example 2: Example of a prompt statement

[0251] "Can you tell me which shelf I should pick up next?"

[0252] As described above, using this system will significantly improve operational efficiency at logistics centers and enable faster decision-making.

[0253] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0254] Step 1:

[0255] The user enters a voice request

[0256] The user puts on the smart glasses and asks, "Tell me which shelf I should pick up the next item from." This voice data is the input.

[0257] Step 2:

[0258] Your device converts speech to text

[0259] The microphone in the smart glasses captures the voice data and converts it into text data using a natural language processing library (e.g., spacy). Here, the input is voice data and the output is text data.

[0260] Step 3:

[0261] The device sends text data to the server.

[0262] The terminal sends the parsed text data to the server via the API. The input is the text data, and the output is the request data sent to the server.

[0263] Step 4:

[0264] The server receives the request and parses the data

[0265] The server receives the request data and uses a generative AI model (e.g., GPT) to extract relevant data based on the request from a data repository, where the input is the request data and the output is the relevant data.

[0266] Step 5:

[0267] The server visualizes the relevant data

[0268] The server uses visualization tools (e.g., D3.js) to transform the relevant data into a visual format such as graphs, charts, flowcharts, etc. In this step, the input is the relevant data and the output is the visualized data.

[0269] Step 6:

[0270] The server sends the visualized data to the device.

[0271] The server sends the visualized data to the terminal through the API, where the input is the visualized data and the output is the response data to the terminal.

[0272] Step 7:

[0273] The device receives the visualized data and displays it using augmented reality technology.

[0274] The device displays the received visualization data in real time using augmented reality technology (e.g., ARKit or ARCore), where the input is the response data and the output is the AR display.

[0275] Step 8:

[0276] The user acts on the displayed information

[0277] The user picks up the specified item based on the visual guide displayed on the smart glasses. In this step, the input is the AR display, and the final task execution is the output.

[0278] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0279] This invention combines a system that efficiently shares business knowledge and know-how within a company and supports rapid decision-making with an emotion engine that recognizes user emotions. Specific embodiments of this system are described below.

[0280] 1. Data Collection

[0281] server

[0282] The server collects business knowledge and know-how from multiple data sources within the company, including document management systems, databases, and internal social networking sites, and retrieves content from each data source via API, storing and managing it in an integrated data repository.

[0283] 2. Data analysis and visualization

[0284] server

[0285] The server preprocesses the collected data and analyzes it using a generative AI model (e.g., GPT). The generative AI model summarizes relevant information from the collected data and organizes it into an easy-to-understand format. It then uses visualization tools to transform the data into graphs, charts, flowcharts, and other formats that users can easily understand.

[0286] 3. User Inquiry Processing

[0287] User

[0288] Users use devices (e.g., smartphones, PCs, smart glasses) to request information, for example, by inputting a specific question such as, "What are the latest market trends?"

[0289] Terminal

[0290] When the device receives user input, it analyzes it using natural language processing (NLP) technology and sends a request to the server.

[0291] 4. Server Response Generation

[0292] server

[0293] The server extracts relevant data from a data repository based on the received request, uses generative AI models to generate information relevant to the user's question, and organizes the information using visualization tools.

[0294] 5. Emotion engine integration

[0295] server

[0296] The server is integrated with an emotion engine that analyzes voice input and facial expression data to recognize the user's emotional state. The emotion engine optimizes the information provided based on the user's emotions and generates responses that correspond to the user's state of mind and situation.

[0297] 6. Providing Feedback

[0298] server

[0299] The server generates the optimized information as a response and sends it to the user terminal.

[0300] Terminal

[0301] The terminal processes the response received from the server and displays it to the user, for example by displaying graphs and charts of the latest market trends on a PC screen.

[0302] User

[0303] Users use the information displayed on the device to carry out the necessary tasks, such as when a manager makes decisions based on market information.

[0304] Specific examples

[0305] Example 1: A new employee wants to know the work procedure

[0306] User

[0307] A new employee types "Please tell me the work procedures" into their smartphone.

[0308] Terminal

[0309] The terminal sends this query to the server.

[0310] server

[0311] The server analyzes the query and extracts relevant data. A generative AI model summarizes the steps and visualizes them as a flowchart.

[0312] Emotion Engine

[0313] The server uses an emotion engine to analyze the user's voice and facial expression data while operating the device, and if the user appears anxious or confused, it responds by adding detailed instructions or supplementary information.

[0314] server

[0315] Finally, the server transmits the flowchart and supplementary information to the user terminal.

[0316] Terminal

[0317] Display flowcharts and supplemental information on smartphones to educate new employees.

[0318] User

[0319] New employees check the flowchart on their smartphones and understand the work procedures.

[0320] Example 2: Manager wants to know market trends

[0321] User

[0322] The manager types into his PC, "Tell me about the latest market trends."

[0323] Terminal

[0324] The terminal sends this query to the server.

[0325] server

[0326] The server analyzes the query and extracts data related to market trends. A generative AI model summarizes the market trends and visualizes them as graphs and charts.

[0327] Emotion Engine

[0328] The server analyzes the manager's emotions and optimizes the presentation of concise and intuitive information if the user is nervous.

[0329] server

[0330] Send visualized market trends graphs and charts to your device.

[0331] Terminal

[0332] Display graphs and charts on a PC to provide information to managers.

[0333] User

[0334] Managers check market trends on their PCs and make appropriate decisions.

[0335] The processing flow will be explained below.

[0336] Step 1:

[0337] User

[0338] The user uses a device (e.g., a smartphone or PC) to input the information they want to know and query the system. For example, the user might input "Tell me the latest market trends" into their PC.

[0339] Step 2:

[0340] Terminal

[0341] The terminal receives the user's input and generates a request to send to the server, which includes the user's question, the user's identification information, and information about the terminal.

[0342] Step 3:

[0343] server

[0344] The server analyzes the request received from the device using natural language processing (NLP) techniques, such as tokenization, stemming, and entity recognition.

[0345] Step 4:

[0346] server

[0347] The server searches and extracts relevant data from the data repository based on the query content analyzed by NLP.

[0348] Step 5:

[0349] server

[0350] The extracted data is fed into a generative AI model (e.g., GPT) to generate and summarize relevant information, which is organized into answers to user questions.

[0351] Step 6:

[0352] server

[0353] The generated information is converted into graphs, charts, and other formats using visualization tools, which makes it easier for users to understand the information.

[0354] Step 7:

[0355] server

[0356] The emotion engine analyzes the user's emotional state when making an inquiry. It uses voice input and a camera to collect facial expression data and analyzes the user's emotional state.

[0357] Step 8:

[0358] server

[0359] Based on the analysis results of the emotion engine, a generative AI model generates an optimized response, for example, if the user is nervous, it will provide a concise and reassuring answer.

[0360] Step 9:

[0361] server

[0362] The visualized information and the optimized response are generated as a response and transmitted to the user terminal.

[0363] Step 10:

[0364] Terminal

[0365] The terminal processes the response received from the server and displays it to the user, for example, by displaying graphs and charts of the latest market trends on a PC screen.

[0366] Step 11:

[0367] User

[0368] Users use the information displayed on the terminal to carry out necessary tasks, such as when a manager makes decisions based on market information.

[0369] Specific examples

[0370] Example 1: A new employee wants to know the work procedure

[0371] Step 1:

[0372] User

[0373] A new employee types "Please tell me the work procedures" into their smartphone.

[0374] Step 2:

[0375] Terminal

[0376] This query is sent to the server.

[0377] Step 3:

[0378] server

[0379] The received request is analyzed using NLP technology and data related to business procedures is extracted from the data repository.

[0380] Step 4:

[0381] server

[0382] The data is summarized using a generative AI model and visualized in flowchart format.

[0383] Step 5:

[0384] server

[0385] The emotion engine analyzes the voice and facial expression data of new employees, and if they appear anxious, detailed supplementary information is added.

[0386] Step 6:

[0387] server

[0388] Optimized information is sent to your smartphone.

[0389] Step 7:

[0390] Terminal

[0391] View the flowchart and supplemental information on your smartphone.

[0392] Step 8:

[0393] User

[0394] New employees review the information and understand the work procedures.

[0395] Example 2: Manager wants to know market trends

[0396] Step 1:

[0397] User

[0398] The manager types into his PC, "Tell me about the latest market trends."

[0399] Step 2:

[0400] Terminal

[0401] This query is sent to the server.

[0402] Step 3:

[0403] server

[0404] Analyze inquiries and extract data related to market trends.

[0405] Step 4:

[0406] server

[0407] Generative AI models summarize market trends and visualize them as graphs and charts.

[0408] Step 5:

[0409] server

[0410] The emotion engine analyzes the manager's emotions and provides concise information if they are nervous.

[0411] Step 6:

[0412] server

[0413] The visualized data is sent to a PC.

[0414] Step 7:

[0415] Terminal

[0416] Display graphs and charts on your PC.

[0417] Step 8:

[0418] User

[0419] The manager reviews the information and makes a decision.

[0420] Example 2

[0421] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0422] In conventional systems for sharing business knowledge and know-how within companies, the collection, analysis, and visualization of information are automated to a certain extent, but they are unable to provide information that takes into account the user's emotional state, which means that the specific information and supplementary explanations that users are looking for are insufficient, making it difficult to support effective decision-making.In particular, when users are feeling anxious or confused, it is necessary to provide appropriate information, but this has not been fully realized.

[0423] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0424] In this invention, the server includes means for collecting business knowledge and know-how from multiple data sources within the company, means for analyzing and visualizing the collected data using a generative AI model, means for receiving inquiries from users and providing analyzed information to the user terminal, means for integrating an emotion engine that analyzes voice input and facial expression data to recognize the emotional state of the user, and means for optimizing the information to be provided based on the emotional state of the user. This enables the provision of optimal information taking into account the user's emotions, thereby supporting effective knowledge sharing and rapid decision-making within the company.

[0425] "Multiple data sources within a company" refers to various information sources used within a company, such as document management systems, databases, and internal social networking sites.

[0426] "Business knowledge and know-how" refers to the specialized knowledge, experience, procedures and best practices a company needs to carry out its day-to-day business operations.

[0427] A "generative AI model" is an artificial intelligence model that has the ability to learn patterns from large amounts of data and generate new data, and specifically refers to natural language processing models such as GPT-4.

[0428] "Analysis and visualization" refers to a set of methods and techniques aimed at processing collected data and presenting the information in an understandable format, including, for example, generating graphs and charts.

[0429] "User inquiries" refer to questions or requests made to the system by users entering information or questions in natural language.

[0430] A "user terminal" is a hardware device that allows a user to use the system, and examples include smartphones, PCs, and smart glasses.

[0431] "Natural language processing" is a technology that enables computers to understand and generate human language, and refers to the process of analyzing text and audio data and extracting meaning.

[0432] "Emotion engine" refers to technology that recognizes a user's emotional state and generates an appropriate response based on that emotion.

[0433] "Voice input and facial expression data" refers to data used to analyze the user's voice and facial expressions, and is important input information used by the emotion engine.

[0434] "Optimization" refers to the process of adjusting the parameters and behavior of a system to achieve optimal results for a specific purpose.

[0435] This invention combines a system that efficiently shares business knowledge and know-how within a company and supports rapid decision-making with an emotion engine that recognizes user emotions.

[0436] System configuration

[0437] Data collection

[0438] server

[0439] The server collects business knowledge and know-how from multiple data sources within the company. These include document management systems, databases, and internal SNS. Data is obtained from each data source via API and stored and managed in an integrated data repository. Specifically, a general "document management system" is used as the document management system, a "database management system" as the database, and an "internal SNS system" as the internal SNS. A "database" is used as the integrated data repository.

[0440] Data analysis and visualization

[0441] server

[0442] The server preprocesses the collected data and analyzes it using a generative AI model (e.g., generative AI model). The generative AI model summarizes relevant information from the collected data and organizes it in an easy-to-understand format. It also uses visualization tools to convert the data into graphs, charts, flowcharts, and other formats. The visualization tools used are called "visualization tools."

[0443] Processing user inquiries

[0444] User

[0445] Users use devices (e.g., smartphones, PCs, smart glasses) to request information, for example, by inputting a specific question such as, "What are the latest market trends?"

[0446] Terminal

[0447] When the device receives user input, it analyzes the content using natural language processing technology and sends a request to the server. Natural language processing is performed using a "natural language processing system."

[0448] Server response generation

[0449] server

[0450] Based on the received request, the server extracts relevant data from a data repository, uses a generative AI model (e.g., a generative AI model) to generate information relevant to the user's question, and organizes the information using visualization tools.

[0451] Emotion engine integration

[0452] server

[0453] The server is integrated with an emotion engine that analyzes voice input and facial expression data to recognize the user's emotional state. The emotion engine uses an "emotion recognition system" to optimize the information provided according to the user's emotions.

[0454] Providing Feedback

[0455] server

[0456] The server generates the optimized information as a response and sends it to the user terminal.

[0457] Terminal

[0458] The terminal processes the response received from the server and displays it to the user, for example by displaying graphs and charts of the latest market trends on a PC screen.

[0459] User

[0460] Users use the information displayed on their devices to carry out the necessary tasks, such as when a manager makes decisions based on market information.

[0461] Specific examples

[0462] Example 1: A new employee wants to know the work procedure

[0463] User

[0464] A new employee types "Please tell me the work procedures" into their smartphone.

[0465] Terminal

[0466] The terminal sends this query to the server.

[0467] server

[0468] The server analyzes the query and extracts relevant data. A generative AI model summarizes the steps and visualizes them as a flowchart.

[0469] Emotion Engine

[0470] The server uses an emotion engine to analyze the user's voice and facial expression data, and if the user feels anxious or confused, it adds detailed instructions or supplementary information.

[0471] server

[0472] Finally, the server transmits the flowchart and supplementary information to the user terminal.

[0473] Terminal

[0474] Display flowcharts and supplemental information on smartphones to educate new employees.

[0475] User

[0476] New employees check the flowchart on their smartphones and understand the work procedures.

[0477] Example 2: Manager wants to know market trends

[0478] User

[0479] The manager types into his PC, "Tell me about the latest market trends."

[0480] Terminal

[0481] The terminal sends this query to the server.

[0482] server

[0483] The server analyzes the query and extracts data related to market trends. A generative AI model summarizes the market trends and visualizes them as graphs and charts.

[0484] Emotion Engine

[0485] The server analyzes the manager's emotions and optimizes to provide concise and intuitive information if the manager is nervous.

[0486] server

[0487] Send visualized market trends graphs and charts to your device.

[0488] Terminal

[0489] Display graphs and charts on a PC to provide information to managers.

[0490] User

[0491] Managers check market trends on their PCs and make appropriate decisions.

[0492] Prompt Sentence Examples

[0493] "Tell me the business procedures."

[0494] "What are the latest market trends?"

[0495] keyword

[0496] Generative AI model, prompt sentence

[0497] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0498] Step 1:

[0499] Data collection

[0500] server

[0501] Input: Multiple data sources within the company (e.g. document management systems, databases, internal SNS)

[0502] Output: Business knowledge and know-how stored in a unified data repository

[0503] The server collects business knowledge and know-how from multiple data sources within the company. Specifically, it periodically sends requests to the API endpoint of each data source, receives the retrieved data in JSON format, and temporarily stores it in memory. The data in memory is then written to the "database management system."

[0504] Step 2:

[0505] Data analysis and visualization

[0506] server

[0507] Input: Data stored in a unified data repository

[0508] Output: Summarized information and visualized graphs and charts

[0509] The server preprocesses the collected data and analyzes it using a generative AI model (e.g., a "generative AI model"). Preprocessing includes cleaning the data (removing unnecessary strings and standardizing formats). Next, the "generative AI model" is used to extract and summarize important information and save it as a new entry. Finally, a "visualization tool" is used to visualize the extracted data and generate graphs and charts.

[0510] Step 3:

[0511] Processing user inquiries

[0512] User

[0513] Input: A specific question (e.g., "What are the latest market trends?")

[0514] Output: None (user input is sent to the server)

[0515] A user uses a device (e.g., a smartphone, PC, or smart glasses) to request information. The user enters a question into an input field on the device. For example, the user enters a specific question such as, "Please tell me about the latest market trends."

[0516] Terminal

[0517] Input: User question

[0518] Output: Parsed user question sent to server

[0519] When the device receives user input, it analyzes the content using natural language processing technology. It uses a "natural language processing system" as a natural language processing tool, and sends the analyzed data to the server in the form of an HTTP request.

[0520] Step 4:

[0521] Server response generation

[0522] server

[0523] Input: The request sent from the terminal (the user's question)

[0524] Output: Generated response data and visualizations

[0525] The server extracts relevant data from a data repository based on the received request, uses a "generative AI model" to generate information appropriate for the user's question, and organizes and visualizes the information using a "visualization tool."

[0526] Step 5:

[0527] Emotion engine integration

[0528] server

[0529] Input: Voice input and facial expression data

[0530] Output: Optimized response based on the user's emotional state

[0531] The server is integrated with an emotion engine (e.g., an "emotion recognition system") that analyzes voice input and facial expression data to recognize the user's emotional state, and adjusts the information content and visualization format provided based on the analysis results.

[0532] Step 6:

[0533] Providing Feedback

[0534] server

[0535] Input: Optimized Information

[0536] Output: Information sent to the user's device

[0537] The server generates the optimized information as a response and sends it to the user terminal.

[0538] Terminal

[0539] Input: The response received from the server

[0540] Output: Information displayed to the user (e.g., graphs and charts)

[0541] The terminal processes the response received from the server and displays it to the user, for example, by displaying graphs and charts of the latest market trends on a PC screen.

[0542] User

[0543] Input: The information displayed on the terminal

[0544] Output: User decisions or actions

[0545] Users use the information displayed on their devices to carry out the necessary tasks, such as when a manager makes decisions based on market information.

[0546] (Application example 2)

[0547] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0548] In systems that efficiently share business knowledge and know-how within a company and support rapid decision-making, current technology faces the challenge of not being able to provide optimal information that takes user emotions into account. Furthermore, in situations where stress levels are particularly high, there is a risk of work efficiency declining, so there is a demand for technology that can analyze emotions in real time and optimize responses.

[0549] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting business knowledge and know-how from multiple data sources within the company, means for analyzing and visualizing the collected data using a generative AI model, means for receiving inquiries from users and providing analyzed information to the user terminal, means for recognizing the user's face and analyzing their emotions using a camera, and means for generating and providing optimal information based on the user's emotions. This makes it possible to provide quick and optimal information that takes the user's emotions into consideration.

[0550] "Data source" refers to various information sources used within a company to collect business knowledge and know-how.

[0551] A "generative AI model" is an artificial intelligence model used to analyze collected data and generate useful information.

[0552] "User terminal" refers to any device that a user uses to make inquiries and receive responses in order to use the system.

[0553] "Camera" refers to the imaging device used to recognize and analyze the User's face.

[0554] "Facial recognition" refers to the technology of detecting and identifying the face of a specific person from an image captured by a camera.

[0555] "Emotion analysis" refers to a technology that determines a user's emotional state based on facial and voice data.

[0556] "Visualization tools" refers to software or technology used to transform analyzed information into visual formats such as graphs and charts.

[0557] "Natural language processing" refers to the technology of analyzing human language and understanding its meaning using generative AI models.

[0558] "Optimization" refers to the process of optimally adjusting the information and responses provided based on the user's emotions and state.

[0559] The present invention combines a system that efficiently shares business knowledge and know-how within a company and supports rapid decision-making with an emotion engine that recognizes user emotions. Specific embodiments are described below.

[0560] First, the server collects business knowledge and know-how from multiple data sources within the company, including document management systems, databases, and internal social networking sites. The server retrieves content from each data source via API and stores and manages it in an integrated data repository.

[0561] The server then preprocesses the collected data and analyzes it using a generative AI model (e.g., GPT-3 (registered trademark)). The generative AI model summarizes relevant information from the collected data and organizes it into an easy-to-understand format. It then uses visualization tools to convert the data into formats such as graphs, charts, and flowcharts so that users can easily understand the information.

[0562] A user uses a device (such as smart glasses) to request information, for example, by entering a specific question such as "What are the latest market trends?" The device receives the user's input, analyzes it using natural language processing (NLP) techniques, and sends a request to the server.

[0563] Based on the received request, the server extracts relevant data from a data repository, uses a generative AI model to generate information appropriate to the user's question, and organizes and visualizes the information using visualization tools.

[0564] In addition, the server is integrated with an emotion engine, which performs facial recognition and emotion analysis through the smart glasses' camera to recognize the user's emotional state. The emotion engine analyzes voice input and facial expression data and optimizes the information provided based on the user's emotions. For example, if the user is feeling anxious or nervous, it will provide more detailed information or concise, intuitive information.

[0565] The optimized information is generated as a response and sent to the user's device. The device processes the response received from the server and displays it to the user. For example, the smart glasses display could show graphs and charts of the latest market trends. The user can then use the information displayed on the device to make appropriate decisions.

[0566] Examples:

[0567] As the security guard was making his rounds, the camera in his smart glasses began scanning his face and noticed he was nervous. As a result, the system immediately sent him a notification detailing a potential threat and a brief explanation of the steps he should take immediately. The prompt read, "The security guard is nervous, please provide a quick response."

[0568] As described above, by combining a server, user devices, emotion engines, generative AI models, visualization tools, etc., it becomes possible to provide fast and optimal information that takes into account the user's emotions.

[0569] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0570] Step 1:

[0571] The server collects business knowledge and know-how from multiple data sources within the company (document management systems, databases, internal SNS, etc.), obtains information from each data source via API, and stores and manages the obtained data in an integrated data repository.

[0572] Input: Raw data obtained from the API of each data source

[0573] Output: Business knowledge and know-how stored in a unified data repository

[0574] Step 2:

[0575] The server preprocesses the data stored in the integrated data repository and analyzes it using a generative AI model (such as GPT-3). The generative AI model summarizes relevant information from the collected data and organizes it into an easy-to-understand format. It then uses visualization tools to transform the data into graphs, charts, flowcharts, and other formats.

[0576] Input: Raw data from the integrated data repository

[0577] Output: Analyzed and visualized data (graphs, charts, etc.)

[0578] Step 3:

[0579] A user uses a device (such as smart glasses) to inquire about the information they want to know, for example, by inputting a specific question such as "Please tell me the latest market trends."

[0580] Input: User question

[0581] Output: The query is parsed and the request sent to the server.

[0582] Step 4:

[0583] The device analyzes the user's input using natural language processing (NLP) technology and sends the content to the server.

[0584] Input: User question

[0585] Output: Request sent to the server

[0586] Step 5:

[0587] Based on the received request, the server extracts relevant data from an integrated data repository, uses a generative AI model to generate information appropriate to the user's question, and organizes and visualizes the information using visualization tools.

[0588] Input: Natural language processed user question

[0589] Output: Specific responses to user questions (graphs, charts, etc.)

[0590] Step 6:

[0591] The server is integrated with an emotion engine that recognizes the user's face through the smart glasses' camera and performs real-time emotion analysis, determining the user's emotional state based on voice input and facial expression data.

[0592] Input: User's face data acquired by camera and voice input

[0593] Output: Parsed user's emotional state

[0594] Step 7:

[0595] The server optimizes the information it provides based on the user's emotional state analyzed by the emotion engine, generating detailed or concise information as needed.

[0596] Input: User's emotional state

[0597] Output: Specific responses optimized based on emotions

[0598] Step 8:

[0599] The server sends the optimized response to the user terminal.

[0600] Input: Emotion-based optimized response

[0601] Output: The optimized response sent to the user's device

[0602] Step 9:

[0603] The terminal processes the response received from the server and displays it to the user, for example by displaying graphs and charts of the latest market trends on the display of smart glasses.

[0604] Input: The response received from the server

[0605] Output: Analysis results displayed to the user (graphs, charts, etc.)

[0606] Step 10:

[0607] Users use the information displayed on their devices to perform their tasks and make appropriate decisions. For example, while a security guard was on patrol, the camera in his smart glasses began scanning his face and recognized that he was nervous. As a result, the system immediately sent a notification to him, notifying him of a possible threat and briefly explaining the steps that should be taken immediately. The prompt read, "The security guard is nervous, so please provide a quick response."

[0608] Input: Analysis results displayed to the user

[0609] Output: Business execution and decision-making based on analysis results

[0610] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0611] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0612] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0613] [Second embodiment]

[0614] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0615] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0616] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0617] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0618] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0619] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0620] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0621] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0622] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0623] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0624] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0625] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0626] The present invention is a system for efficiently sharing business knowledge and know-how within a company and supporting rapid decision-making. Specific embodiments of this system will be described below.

[0627] 1. Data Collection

[0628] server

[0629] The server collects business knowledge and know-how from data sources within the company (document management systems, databases, internal SNS, etc.), obtains the content of each data source via API, and stores and manages it in an integrated data repository.

[0630] 2. Data analysis and visualization

[0631] server

[0632] The collected data is preprocessed and analyzed using a generative AI model, which uses, for example, GPT, to summarize the relevant information in an easy-to-understand format.

[0633] To visualize the generated information, visualization tools are used to convert the data into graphs, charts, flowcharts, and other formats, making it easier for users to understand and use the information.

[0634] 3. User Inquiry Processing

[0635] User

[0636] Users, such as new employees, general employees, and managers, use devices (smartphones, PCs, smart glasses, etc.) to inquire about the information they want to know. For example, they input specific questions such as, "Please tell me about the latest market trends."

[0637] Terminal

[0638] When the terminal receives an inquiry from a user, it analyzes the content using natural language processing and sends a request to the server.

[0639] 4. Server Response Generation

[0640] server

[0641] Based on the received inquiry, relevant data is extracted from the data repository, and information appropriate to the user's question is generated using a generative AI model.

[0642] A response is generated and sent to the terminal to provide the visualized information to the user.

[0643] 5. Providing Feedback

[0644] Terminal

[0645] The information received from the server is displayed to the user. For example, work procedures can be displayed in report format on a smartphone, or work procedures can be navigated in real time using smart glasses.

[0646] User

[0647] Users use the information provided to them to carry out their work, such as when a new employee learns and understands work procedures or when a manager makes decisions based on the latest market trends.

[0648] Specific examples

[0649] Example 1: A new employee wants to know the work procedure

[0650] User

[0651] A new employee types "Please tell me the work procedures" into their smartphone.

[0652] Terminal

[0653] This query is sent to the server.

[0654] server

[0655] Analyze inquiries and extract relevant data about business procedures.

[0656] We summarize the steps required in the GPT model and visualize them as a flowchart.

[0657] Send the flowchart to your smartphone.

[0658] Terminal

[0659] Display a flowchart on a smartphone to provide information to new employees.

[0660] User

[0661] New employees check the flowchart on their smartphones and understand the work procedures.

[0662] Example 2: Manager wants to know market trends

[0663] User

[0664] The manager types into his PC, "Tell me about the latest market trends."

[0665] Terminal

[0666] This query is sent to the server.

[0667] server

[0668] Analyze inquiries and extract relevant data on market trends.

[0669] The GPT model summarizes market trends and visualizes them as graphs and charts.

[0670] The visualized data is sent to a PC.

[0671] Terminal

[0672] Graphs and charts of market trends are displayed on a PC to provide information to managers.

[0673] User

[0674] Managers check market trends on their PCs and make appropriate decisions.

[0675] The processing flow will be explained below.

[0676] Step 1:

[0677] User

[0678] The user uses a device (e.g., a smartphone or PC) to input the information they want to know and query the system. For example, the user might input "Tell me the latest market trends" into their PC.

[0679] Step 2:

[0680] Terminal

[0681] The terminal receives the user's input and generates a request to send to the server. The request includes the user's question, the user's identification information, and the terminal information, and is sent to the server.

[0682] Step 3:

[0683] server

[0684] The server receives the request from the device and uses natural language processing (NLP) techniques to analyze the content of the received request. NLP techniques perform processes such as tokenization, stemming, and entity recognition.

[0685] Step 4:

[0686] server

[0687] Based on the query content analyzed by NLP technology, the server extracts relevant data from a data repository that stores business knowledge, know-how, the latest market information, and more.

[0688] Step 5:

[0689] server

[0690] The extracted data is fed into a generative AI model (e.g., GPT) to generate and summarize information, which is then organized into a format that can provide appropriate answers to user questions.

[0691] Step 6:

[0692] server

[0693] The generated information is converted into graphs, charts, etc. using visualization tools. The visualized information is organized so that it can be easily understood by users.

[0694] Step 7:

[0695] server

[0696] A response is generated to return the visualized information to the user terminal, the response including a display format of the visualized information and any necessary associated data.

[0697] Step 8:

[0698] Terminal

[0699] The terminal processes the response received from the server and displays it to the user, for example, by displaying graphs and charts of the latest market trends on a PC screen.

[0700] Step 9:

[0701] User

[0702] Users use the information displayed on the terminal to carry out necessary tasks, such as a manager making decisions based on the latest market information.

[0703] Example 1

[0704] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0705] In order to efficiently share business knowledge and know-how within a company and support rapid decision-making, a system is needed to centrally manage data from multiple sources and respond quickly and appropriately to user inquiries. However, current systems do not integrate the processes of data collection, analysis, visualization, and feedback, making it difficult to provide users with the information they need in a timely manner.

[0706] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0707] In this invention, the server includes a means for collecting business knowledge and know-how from multiple information sources within the company, a means for storing and managing the collected data in an integrated data repository, and a means for analyzing and summarizing the collected data using a generative AI model. This integrates a series of processes from data collection to analysis, visualization, and feedback, making it possible to respond quickly and appropriately to user inquiries.

[0708] "Multiple information sources within a company" refers to multiple data sources owned by a company, such as document management systems, databases, and internal social networking sites.

[0709] "Business knowledge and know-how" refers to information including knowledge and practical knowledge such as procedures, experience, rules, and technical information required for the operation of a company.

[0710] "Means of collection" refers to APIs and software tools that allow companies to acquire and integrate business knowledge and know-how from multiple sources within the company.

[0711] An "integrated data repository" refers to a database or storage system where collected data is centrally stored and managed.

[0712] A "generative AI model" refers to an artificial intelligence model that generates and summarizes useful information from large amounts of data, for example, using a generative AI model (such as GPT-4).

[0713] "Means of analysis" refers to the process of preprocessing collected data and using generative AI models to extract and summarize the necessary information.

[0714] "Visualization tools" refers to software tools (e.g., Tableau, Power BI) that transform analyzed data into visual formats such as graphs, charts, and flowcharts.

[0715] "Natural language processing" refers to the technology used to analyze user queries and extract key keywords and important information.

[0716] "User terminal" refers to a device used by a user to input and receive information, such as a PC, smartphone, or smart glasses.

[0717] "Server" refers to the central computer system used for data collection, analysis, visualization, and response to the user.

[0718] The present invention is a system for efficiently sharing business knowledge and know-how within a company and supporting rapid decision-making. Specific embodiments of this system will be described below.

[0719] System configuration and operation

[0720] server

[0721] Data collection: The server collects business knowledge and know-how from multiple information sources, such as the company's internal document management system, databases, and internal SNS. To do this, the server obtains data using the API of each data source (for example, a general-purpose API or database query). The collected data is stored in an integrated data repository. Specifically, it obtains PDF files from the document management system, table entries from the database, and message history from the internal SNS.

[0722] Data analysis and summarization: The collected data is preprocessed using Python's Pandas library. It is then analyzed and summarized using a generative AI model (e.g., GPT-4). An example prompt might be, "Please summarize the key business steps from the data below."

[0723] Visualization: Using visualization tools (e.g., Tableau, Power BI), the analyzed data is transformed into visual formats such as graphs, charts, and flowcharts, which helps users intuitively understand the information.

[0724] User

[0725] Inquiry: A user (new employee, general employee, manager, etc.) makes an inquiry about information using a device such as a smartphone, PC, or smart glasses. For example, they might type "Please tell me about the latest market trends" into their smartphone.

[0726] Terminal

[0727] Natural Language Processing: The terminal receives a user query and analyzes the content using a natural language processing library (e.g., spaCy, NLTK, or other major natural language processing tools), for example, tokenizing the input text and extracting key keywords.

[0728] Query sending: Send a request to the server based on the analysis results. This is done using the REST API.

[0729] Server Response

[0730] Information extraction: The server extracts relevant data from the integrated data repository based on the received query, for example, searching for the latest data on "market trends."

[0731] Response generation: Generative AI models (e.g., GPT-4) are used to generate information appropriate for the user's question. An example prompt might be, "Please summarize the latest market trends based on the market data below."

[0732] Visualization and Presentation: The generated information is visualized using a visualization tool and the results are sent to the terminal. For example, the latest market trends can be visualized as a bar graph and sent to a PC.

[0733] Providing Feedback

[0734] Display: The terminal displays the information received from the server to the user. Specifically, it displays a flowchart of business procedures on a smartphone, or a graph of market trends on a PC.

[0735] Work execution: Users perform their work based on the information provided. This can be a new employee learning how to do their job or a manager reviewing the latest market trends and making decisions.

[0736] Specific examples

[0737] When a new employee wants to know the work procedures

[0738] 1. User: A new employee types "Please tell me the work procedure" into their smartphone.

[0739] 2. Terminal: Analyzes the query using natural language processing and sends a request to the server.

[0740] 3. Server: Analyzes the query and extracts data about business procedures from the document management system. It uses GPT-4 to summarize the key steps and visualize them as a flowchart. It then sends the visualized flowchart to the smartphone.

[0741] 4. Terminal: Display the flowchart on a smartphone and provide information to new employees.

[0742] 5. User: New employees review the flowchart and understand the work procedures.

[0743] When managers want to know market trends

[0744] 1. User: The manager types into his PC, "Tell me the latest market trends."

[0745] 2. Terminal: Analyzes the query using natural language processing and sends a request to the server.

[0746] 3. Server: Analyzes the query and extracts data about market trends from the database. It uses GPT-4 to summarize the market trends and visualize them as graphs. It then sends the visualized data to the PC.

[0747] 4. Terminal: Displays graphs of market trends on a PC and provides information to managers.

[0748] 5. User: Managers review the graphs and make appropriate decisions.

[0749] As described above, this system collects data from multiple sources within a company and uses AI models and visualization tools to provide users with appropriate business knowledge and know-how.

[0750] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0751] Step 1:

[0752] Data collection

[0753] The server collects business knowledge and know-how from multiple sources within the company, specifically, using APIs to obtain data from document management systems, databases, internal social networking sites, etc.

[0754] Input: Data from document management systems, databases, and APIs of internal SNS.

[0755] Output: Data stored in a unified data repository.

[0756] Specific operations: For example, retrieve "Business Procedures.pdf" from the document management system, retrieve the latest entries in the "Market Trends" table from the database, and retrieve the message history of the "General" channel from the internal SNS, and store these in the integrated data repository.

[0757] Step 2:

[0758] Data Preprocessing

[0759] The server preprocesses the collected data using the Python Pandas library.

[0760] Input: Data from the Unified Data Repository.

[0761] Output: Cleaned and filtered data.

[0762] Specific Actions: For example, removing empty fields from database entries and fixing inconsistent formatting.

[0763] Step 3:

[0764] Data analysis and summary generation

[0765] The server analyzes the preprocessed data with a generative AI model (e.g., GPT-4) and generates a summary. The summary is generated using a prompt sentence.

[0766] Input: Cleaned and filtered data.

[0767] Output: The summary generated by GPT-4.

[0768] Specific operation: For example, input the prompt sentence "Please summarize the key business steps from the following data" into GPT-4 and obtain a summary.

[0769] Step 4:

[0770] Data Visualization

[0771] The server converts the generated summaries into a visual format using a visualization tool (e.g., Tableau, Power BI).

[0772] Input: The summary generated by GPT-4.

[0773] Output: Visual data such as graphs, charts, and flowcharts.

[0774] Specific actions: For example, visualizing business procedures as a flowchart and displaying Slack message history as a pie chart.

[0775] Step 5:

[0776] Receiving inquiries from users

[0777] Users can request information using devices such as smartphones, PCs, or smart glasses.

[0778] Input: User's query (e.g., "What are the latest market trends?").

[0779] Output: The parsing request sent to the terminal.

[0780] Specific action: For example, a user types "Please tell me the latest market trends" into their smartphone.

[0781] Step 6:

[0782] Natural Language Processing

[0783] The device analyzes user inquiries using natural language processing, specifically using natural language processing libraries (e.g., spaCy, NLTK).

[0784] Input: User's inquiry.

[0785] Output: Analysis results of the query.

[0786] Specific behavior: For example, tokenize the input text and extract key keywords (e.g., "market trends," "latest").

[0787] Step 7:

[0788] Sending a query to the server

[0789] The terminal transmits a request to the server based on the analysis result.

[0790] Input: Analysis results (e.g., "Market Trends", "Latest").

[0791] Output: REST API request to the server.

[0792] Specific operation: Send the analysis results to the server using the REST API.

[0793] Step 8:

[0794] Server information extraction and response generation

[0795] Based on the received inquiry, the server extracts relevant data from an integrated data repository and uses a generative AI model to generate information appropriate to the user's question.

[0796] Input: Data extraction request based on analysis results.

[0797] Output: Information suitable for the user's question.

[0798] Specific operation: For example, input the prompt sentence "Please summarize the latest market trends based on the market data below" into GPT-4 and obtain a summary.

[0799] Step 9:

[0800] Information Visualization

[0801] The server visualizes the generated information using a visualization tool and transmits the results to the terminal.

[0802] Input: Information generated by a generative AI model.

[0803] Output: Visualized data.

[0804] Specific operation: For example, visualize the latest market trends as a bar graph and send it to a PC.

[0805] Step 10:

[0806] Viewing information and providing feedback

[0807] The terminal displays the information received from the server to the user, who then performs the task based on the information provided.

[0808] Input: Visualization data sent from the server.

[0809] Output: Information displayed on the terminal.

[0810] Specific operations: For example, a graph of market trends is displayed on a PC, and the user checks the information and makes a decision.

[0811] (Application example 1)

[0812] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0813] Although logistics centers require the rapid acquisition and sharing of business knowledge and know-how, current systems lack the means to efficiently collect, analyze, visualize, and provide this information. In particular, it is difficult for staff to obtain optimal picking routes and inventory information in real time, which reduces work efficiency. There is a need for a system that can solve these issues and support the efficiency of work and rapid decision-making at logistics centers.

[0814] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0815] In this invention, the server includes means for collecting business knowledge and know-how from multiple data sources within the company, means for analyzing and visualizing the collected data using a generative AI model, means for receiving voice requests from a user terminal, analyzing the requests using natural language processing, and transmitting related information to the server, and means for visualizing the related information transmitted from the server and displaying it on the user terminal in real time using augmented reality technology. This enables logistics center staff to efficiently obtain optimal picking routes and inventory information in real time, enabling them to carry out their work promptly.

[0816] A "data source" is an information source used to collect business knowledge and know-how, such as a document management system, database, or in-house SNS within a company.

[0817] A "generative AI model" is an artificial intelligence model that analyzes collected data and summarizes it in an easy-to-understand format, such as GPT.

[0818] "Visualization" is the process of converting data analyzed by a generative AI model into visual formats such as graphs, charts, and flowcharts, making the information easier for users to understand.

[0819] A "user terminal" is a device used by a user to obtain information, and includes smartphones, PCs, smart glasses, head-mounted displays, etc.

[0820] "Natural language processing" is a technology for analyzing user inquiries and extracting relevant information, and is applied to voice requests and text data.

[0821] "Augmented reality technology" is a technology that displays visualized information overlaid on the real world in real time, and refers to the use of AR (augmented reality).

[0822] A "picking route" is the optimal route for staff to efficiently select products at a logistics center, and is generated based on inventory information.

[0823] "Inventory information" refers to information relating to the type, quantity, location, etc. of products currently held within the logistics center.

[0824] "Real-time" refers to immediate processing and response to user requests without delay.

[0825] This invention provides a system that supports operational efficiency and rapid decision-making in logistics centers. To achieve this, it is important to collect business knowledge and know-how from multiple data sources within a company, analyze and visualize it using a generative AI model, and provide information in real time based on user inquiries.

[0826] System hardware and software configuration

[0827] Server: The central system for data collection, analysis, and visualization. The server must be equipped with a high-performance processor, large memory capacity, and storage. The hardware used can be a general-purpose high-performance server or cloud service. The software used includes a generative AI model such as GPT, an NLP library (e.g., spacy), a database (e.g., MySQL), and a data visualization tool (e.g., D3.js).

[0828] User devices: Devices such as smartphones, smart glasses, and head-mounted displays that users use to obtain information. These devices must have voice input, display, and internet connectivity capabilities.

[0829] Data collection

[0830] The server collects business knowledge and know-how from multiple data sources within the company (document management systems, databases, internal SNS, etc.), obtains the content of each data source via API, and stores and manages it in an integrated data repository.

[0831] Parsing voice requests

[0832] The user device (such as smart glasses) receives the voice request and converts it into text data using a natural language processing library (such as spacy). The analyzed text data is then sent to the server via an API.

[0833] Data analysis and visualization

[0834] Based on the received voice request, the server extracts relevant data from a data repository, uses a generative AI model (e.g., GPT) to generate information relevant to the user's question, and then uses a data visualization tool (e.g., D3.js) to transform the data into visual formats such as graphs, charts, and flowcharts.

[0835] Augmented reality information provision

[0836] The visualized information is sent to the user's device and displayed in real time on the user's device using augmented reality technology (such as ARKit or ARCore). For example, by using smart glasses, logistics center staff can visually navigate and quickly determine the optimal picking route.

[0837] Specific examples

[0838] Example 1: Navigation with voice requests and AR

[0839] A staff member wears smart glasses and asks, "Tell me which shelf I should pick up the next item from." The smart glasses analyze the voice and send a request to the server. The server generates a picking route and sends the visualized information to the smart glasses. The smart glasses then use their AR functionality to navigate the staff member.

[0840] Example 2: Example of a prompt statement

[0841] "Can you tell me which shelf I should pick up next?"

[0842] As described above, using this system will significantly improve operational efficiency at logistics centers and enable faster decision-making.

[0843] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0844] Step 1:

[0845] The user enters a voice request

[0846] The user puts on the smart glasses and asks, "Tell me which shelf I should pick up the next item from." This voice data is the input.

[0847] Step 2:

[0848] Your device converts speech to text

[0849] The microphone in the smart glasses captures the voice data and converts it into text data using a natural language processing library (e.g., spacy). Here, the input is voice data and the output is text data.

[0850] Step 3:

[0851] The device sends text data to the server.

[0852] The terminal sends the parsed text data to the server via the API. The input is the text data, and the output is the request data sent to the server.

[0853] Step 4:

[0854] The server receives the request and parses the data

[0855] The server receives the request data and uses a generative AI model (e.g., GPT) to extract relevant data based on the request from a data repository, where the input is the request data and the output is the relevant data.

[0856] Step 5:

[0857] The server visualizes the relevant data

[0858] The server uses visualization tools (e.g., D3.js) to transform the relevant data into a visual format such as graphs, charts, flowcharts, etc. In this step, the input is the relevant data and the output is the visualized data.

[0859] Step 6:

[0860] The server sends the visualized data to the device.

[0861] The server sends the visualized data to the terminal through the API, where the input is the visualized data and the output is the response data to the terminal.

[0862] Step 7:

[0863] The device receives the visualized data and displays it using augmented reality technology.

[0864] The device displays the received visualization data in real time using augmented reality technology (e.g., ARKit or ARCore), where the input is the response data and the output is the AR display.

[0865] Step 8:

[0866] The user acts on the displayed information

[0867] The user picks up the specified item based on the visual guide displayed on the smart glasses. In this step, the input is the AR display, and the final task execution is the output.

[0868] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0869] This invention combines a system that efficiently shares business knowledge and know-how within a company and supports rapid decision-making with an emotion engine that recognizes user emotions. Specific embodiments of this system are described below.

[0870] 1. Data Collection

[0871] server

[0872] The server collects business knowledge and know-how from multiple data sources within the company, including document management systems, databases, and internal social networking sites, and retrieves content from each data source via API, storing and managing it in an integrated data repository.

[0873] 2. Data analysis and visualization

[0874] server

[0875] The server preprocesses the collected data and analyzes it using a generative AI model (e.g., GPT). The generative AI model summarizes relevant information from the collected data and organizes it into an easy-to-understand format. It then uses visualization tools to transform the data into graphs, charts, flowcharts, and other formats that users can easily understand.

[0876] 3. User Inquiry Processing

[0877] User

[0878] Users use devices (e.g., smartphones, PCs, smart glasses) to request information, for example, by inputting a specific question such as, "What are the latest market trends?"

[0879] Terminal

[0880] When the device receives user input, it analyzes it using natural language processing (NLP) technology and sends a request to the server.

[0881] 4. Server Response Generation

[0882] server

[0883] The server extracts relevant data from a data repository based on the received request, uses generative AI models to generate information relevant to the user's question, and organizes the information using visualization tools.

[0884] 5. Emotion engine integration

[0885] server

[0886] The server is integrated with an emotion engine that analyzes voice input and facial expression data to recognize the user's emotional state. The emotion engine optimizes the information provided based on the user's emotions and generates responses that correspond to the user's state of mind and situation.

[0887] 6. Providing Feedback

[0888] server

[0889] The server generates the optimized information as a response and sends it to the user terminal.

[0890] Terminal

[0891] The terminal processes the response received from the server and displays it to the user, for example by displaying graphs and charts of the latest market trends on a PC screen.

[0892] User

[0893] Users use the information displayed on the device to carry out the necessary tasks, such as when a manager makes decisions based on market information.

[0894] Specific examples

[0895] Example 1: A new employee wants to know the work procedure

[0896] User

[0897] A new employee types "Please tell me the work procedures" into their smartphone.

[0898] Terminal

[0899] The terminal sends this query to the server.

[0900] server

[0901] The server analyzes the query and extracts relevant data. A generative AI model summarizes the steps and visualizes them as a flowchart.

[0902] Emotion Engine

[0903] The server uses an emotion engine to analyze the user's voice and facial expression data while operating the device, and if the user appears anxious or confused, it responds by adding detailed instructions or supplementary information.

[0904] server

[0905] Finally, the server transmits the flowchart and supplementary information to the user terminal.

[0906] Terminal

[0907] Display flowcharts and supplemental information on smartphones to educate new employees.

[0908] User

[0909] New employees check the flowchart on their smartphones and understand the work procedures.

[0910] Example 2: Manager wants to know market trends

[0911] User

[0912] The manager types into his PC, "Tell me about the latest market trends."

[0913] Terminal

[0914] The terminal sends this query to the server.

[0915] server

[0916] The server analyzes the query and extracts data related to market trends. A generative AI model summarizes the market trends and visualizes them as graphs and charts.

[0917] Emotion Engine

[0918] The server analyzes the manager's emotions and optimizes the presentation of concise and intuitive information if the user is nervous.

[0919] server

[0920] Send visualized market trends graphs and charts to your device.

[0921] Terminal

[0922] Display graphs and charts on a PC to provide information to managers.

[0923] User

[0924] Managers check market trends on their PCs and make appropriate decisions.

[0925] The processing flow will be explained below.

[0926] Step 1:

[0927] User

[0928] The user uses a device (e.g., a smartphone or PC) to input the information they want to know and query the system. For example, the user might input "Tell me the latest market trends" into their PC.

[0929] Step 2:

[0930] Terminal

[0931] The terminal receives the user's input and generates a request to send to the server, which includes the user's question, the user's identification information, and information about the terminal.

[0932] Step 3:

[0933] server

[0934] The server analyzes the request received from the device using natural language processing (NLP) techniques, such as tokenization, stemming, and entity recognition.

[0935] Step 4:

[0936] server

[0937] The server searches and extracts relevant data from the data repository based on the query content analyzed by NLP.

[0938] Step 5:

[0939] server

[0940] The extracted data is fed into a generative AI model (e.g., GPT) to generate and summarize relevant information, which is organized into answers to user questions.

[0941] Step 6:

[0942] server

[0943] The generated information is converted into graphs, charts, and other formats using visualization tools, which makes it easier for users to understand the information.

[0944] Step 7:

[0945] server

[0946] The emotion engine analyzes the user's emotional state when making an inquiry. It uses voice input and a camera to collect facial expression data and analyzes the user's emotional state.

[0947] Step 8:

[0948] server

[0949] Based on the analysis results of the emotion engine, a generative AI model generates an optimized response, for example, if the user is nervous, it will provide a concise and reassuring answer.

[0950] Step 9:

[0951] server

[0952] The visualized information and the optimized response are generated as a response and transmitted to the user terminal.

[0953] Step 10:

[0954] Terminal

[0955] The terminal processes the response received from the server and displays it to the user, for example, by displaying graphs and charts of the latest market trends on a PC screen.

[0956] Step 11:

[0957] User

[0958] Users use the information displayed on the terminal to carry out necessary tasks, such as when a manager makes decisions based on market information.

[0959] Specific examples

[0960] Example 1: A new employee wants to know the work procedure

[0961] Step 1:

[0962] User

[0963] A new employee types "Please tell me the work procedures" into their smartphone.

[0964] Step 2:

[0965] Terminal

[0966] This query is sent to the server.

[0967] Step 3:

[0968] server

[0969] The received request is analyzed using NLP technology and data related to business procedures is extracted from the data repository.

[0970] Step 4:

[0971] server

[0972] The data is summarized using a generative AI model and visualized in flowchart format.

[0973] Step 5:

[0974] server

[0975] The emotion engine analyzes the voice and facial expression data of new employees, and if they appear anxious, detailed supplementary information is added.

[0976] Step 6:

[0977] server

[0978] Optimized information is sent to your smartphone.

[0979] Step 7:

[0980] Terminal

[0981] View the flowchart and supplemental information on your smartphone.

[0982] Step 8:

[0983] User

[0984] New employees review the information and understand the work procedures.

[0985] Example 2: Manager wants to know market trends

[0986] Step 1:

[0987] User

[0988] The manager types into his PC, "Tell me about the latest market trends."

[0989] Step 2:

[0990] Terminal

[0991] This query is sent to the server.

[0992] Step 3:

[0993] server

[0994] Analyze inquiries and extract data related to market trends.

[0995] Step 4:

[0996] server

[0997] Generative AI models summarize market trends and visualize them as graphs and charts.

[0998] Step 5:

[0999] server

[1000] The emotion engine analyzes the manager's emotions and provides concise information if they are nervous.

[1001] Step 6:

[1002] server

[1003] The visualized data is sent to a PC.

[1004] Step 7:

[1005] Terminal

[1006] Display graphs and charts on your PC.

[1007] Step 8:

[1008] User

[1009] The manager reviews the information and makes a decision.

[1010] Example 2

[1011] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1012] In conventional systems for sharing business knowledge and know-how within companies, the collection, analysis, and visualization of information are automated to a certain extent, but they are unable to provide information that takes into account the user's emotional state, which means that the specific information and supplementary explanations that users are looking for are insufficient, making it difficult to support effective decision-making.In particular, when users are feeling anxious or confused, it is necessary to provide appropriate information, but this has not been fully realized.

[1013] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1014] In this invention, the server includes means for collecting business knowledge and know-how from multiple data sources within the company, means for analyzing and visualizing the collected data using a generative AI model, means for receiving inquiries from users and providing analyzed information to the user terminal, means for integrating an emotion engine that analyzes voice input and facial expression data to recognize the emotional state of the user, and means for optimizing the information to be provided based on the emotional state of the user. This enables the provision of optimal information taking into account the user's emotions, thereby supporting effective knowledge sharing and rapid decision-making within the company.

[1015] "Multiple data sources within a company" refers to various information sources used within a company, such as document management systems, databases, and internal social networking sites.

[1016] "Business knowledge and know-how" refers to the specialized knowledge, experience, procedures and best practices a company needs to carry out its day-to-day business operations.

[1017] A "generative AI model" is an artificial intelligence model that has the ability to learn patterns from large amounts of data and generate new data, and specifically refers to natural language processing models such as GPT-4.

[1018] "Analysis and visualization" refers to a set of methods and techniques aimed at processing collected data and presenting the information in an understandable format, including, for example, generating graphs and charts.

[1019] "User inquiries" refer to questions or requests made to the system by users entering information or questions in natural language.

[1020] A "user terminal" is a hardware device that allows a user to use the system, and examples include smartphones, PCs, and smart glasses.

[1021] "Natural language processing" is a technology that enables computers to understand and generate human language, and refers to the process of analyzing text and audio data and extracting meaning.

[1022] "Emotion engine" refers to technology that recognizes a user's emotional state and generates an appropriate response based on that emotion.

[1023] "Voice input and facial expression data" refers to data used to analyze the user's voice and facial expressions, and is important input information used by the emotion engine.

[1024] "Optimization" refers to the process of adjusting the parameters and behavior of a system to achieve optimal results for a specific purpose.

[1025] This invention combines a system that efficiently shares business knowledge and know-how within a company and supports rapid decision-making with an emotion engine that recognizes user emotions.

[1026] System configuration

[1027] Data collection

[1028] server

[1029] The server collects business knowledge and know-how from multiple data sources within the company. These include document management systems, databases, and internal SNS. Data is obtained from each data source via API and stored and managed in an integrated data repository. Specifically, a general "document management system" is used as the document management system, a "database management system" as the database, and an "internal SNS system" as the internal SNS. A "database" is used as the integrated data repository.

[1030] Data analysis and visualization

[1031] server

[1032] The server preprocesses the collected data and analyzes it using a generative AI model (e.g., generative AI model). The generative AI model summarizes relevant information from the collected data and organizes it in an easy-to-understand format. It also uses visualization tools to convert the data into graphs, charts, flowcharts, and other formats. The visualization tools used are called "visualization tools."

[1033] Processing user inquiries

[1034] User

[1035] Users use devices (e.g., smartphones, PCs, smart glasses) to request information, for example, by inputting a specific question such as, "What are the latest market trends?"

[1036] Terminal

[1037] When the device receives user input, it analyzes the content using natural language processing technology and sends a request to the server. Natural language processing is performed using a "natural language processing system."

[1038] Server response generation

[1039] server

[1040] Based on the received request, the server extracts relevant data from a data repository, uses a generative AI model (e.g., a generative AI model) to generate information relevant to the user's question, and organizes the information using visualization tools.

[1041] Emotion engine integration

[1042] server

[1043] The server is integrated with an emotion engine that analyzes voice input and facial expression data to recognize the user's emotional state. The emotion engine uses an "emotion recognition system" to optimize the information provided according to the user's emotions.

[1044] Providing Feedback

[1045] server

[1046] The server generates the optimized information as a response and sends it to the user terminal.

[1047] Terminal

[1048] The terminal processes the response received from the server and displays it to the user, for example by displaying graphs and charts of the latest market trends on a PC screen.

[1049] User

[1050] Users use the information displayed on their devices to carry out the necessary tasks, such as when a manager makes decisions based on market information.

[1051] Specific examples

[1052] Example 1: A new employee wants to know the work procedure

[1053] User

[1054] A new employee types "Please tell me the work procedures" into their smartphone.

[1055] Terminal

[1056] The terminal sends this query to the server.

[1057] server

[1058] The server analyzes the query and extracts relevant data. A generative AI model summarizes the steps and visualizes them as a flowchart.

[1059] Emotion Engine

[1060] The server uses an emotion engine to analyze the user's voice and facial expression data, and if the user feels anxious or confused, it adds detailed instructions or supplementary information.

[1061] server

[1062] Finally, the server transmits the flowchart and supplementary information to the user terminal.

[1063] Terminal

[1064] Display flowcharts and supplemental information on smartphones to educate new employees.

[1065] User

[1066] New employees check the flowchart on their smartphones and understand the work procedures.

[1067] Example 2: Manager wants to know market trends

[1068] User

[1069] The manager types into his PC, "Tell me about the latest market trends."

[1070] Terminal

[1071] The terminal sends this query to the server.

[1072] server

[1073] The server analyzes the query and extracts data related to market trends. A generative AI model summarizes the market trends and visualizes them as graphs and charts.

[1074] Emotion Engine

[1075] The server analyzes the manager's emotions and optimizes to provide concise and intuitive information if the manager is nervous.

[1076] server

[1077] Send visualized market trends graphs and charts to your device.

[1078] Terminal

[1079] Display graphs and charts on a PC to provide information to managers.

[1080] User

[1081] Managers check market trends on their PCs and make appropriate decisions.

[1082] Prompt Sentence Examples

[1083] "Tell me the business procedures."

[1084] "What are the latest market trends?"

[1085] keyword

[1086] Generative AI model, prompt sentence

[1087] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1088] Step 1:

[1089] Data collection

[1090] server

[1091] Input: Multiple data sources within the company (e.g. document management systems, databases, internal SNS)

[1092] Output: Business knowledge and know-how stored in a unified data repository

[1093] The server collects business knowledge and know-how from multiple data sources within the company. Specifically, it periodically sends requests to the API endpoint of each data source, receives the retrieved data in JSON format, and temporarily stores it in memory. The data in memory is then written to the "database management system."

[1094] Step 2:

[1095] Data analysis and visualization

[1096] server

[1097] Input: Data stored in a unified data repository

[1098] Output: Summarized information and visualized graphs and charts

[1099] The server preprocesses the collected data and analyzes it using a generative AI model (e.g., "generative AI model"). Preprocessing includes cleaning the data (removing unnecessary strings and standardizing formatting). Next, the "generative AI model" is used to extract and summarize important information and save it as a new entry. Finally, the extracted data is visualized using a "visualization tool" to generate graphs and charts.

[1100] Step 3:

[1101] Processing user inquiries

[1102] User

[1103] Input: A specific question (e.g., "What are the latest market trends?")

[1104] Output: None (user input is sent to the server)

[1105] A user uses a device (e.g., a smartphone, PC, or smart glasses) to request information. The user enters a question into an input field on the device. For example, the user enters a specific question such as, "Please tell me about the latest market trends."

[1106] Terminal

[1107] Input: User question

[1108] Output: Parsed user question sent to server

[1109] When the device receives user input, it analyzes the content using natural language processing technology. It uses a "natural language processing system" as a natural language processing tool, and sends the analyzed data to the server in the form of an HTTP request.

[1110] Step 4:

[1111] Server response generation

[1112] server

[1113] Input: The request sent from the terminal (the user's question)

[1114] Output: Generated response data and visualizations

[1115] The server extracts relevant data from a data repository based on the received request, uses a "generative AI model" to generate information appropriate for the user's question, and organizes and visualizes the information using a "visualization tool."

[1116] Step 5:

[1117] Emotion engine integration

[1118] server

[1119] Input: Voice input and facial expression data

[1120] Output: Optimized response based on the user's emotional state

[1121] The server is integrated with an emotion engine (e.g., an "emotion recognition system") that analyzes voice input and facial expression data to recognize the user's emotional state, and adjusts the information content and visualization format provided based on the analysis results.

[1122] Step 6:

[1123] Providing Feedback

[1124] server

[1125] Input: Optimized Information

[1126] Output: Information sent to the user's device

[1127] The server generates the optimized information as a response and sends it to the user terminal.

[1128] Terminal

[1129] Input: The response received from the server

[1130] Output: Information displayed to the user (e.g., graphs and charts)

[1131] The terminal processes the response received from the server and displays it to the user, for example, by displaying graphs and charts of the latest market trends on a PC screen.

[1132] User

[1133] Input: The information displayed on the terminal

[1134] Output: User decisions or actions

[1135] Users use the information displayed on their devices to carry out the necessary tasks, such as when a manager makes decisions based on market information.

[1136] (Application example 2)

[1137] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1138] In systems that efficiently share business knowledge and know-how within a company and support rapid decision-making, current technology faces the challenge of not being able to provide optimal information that takes user emotions into account. Furthermore, in situations where stress levels are particularly high, there is a risk of work efficiency declining, so there is a demand for technology that can analyze emotions in real time and optimize responses.

[1139] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting business knowledge and know-how from multiple data sources within the company, means for analyzing and visualizing the collected data using a generative AI model, means for receiving inquiries from users and providing analyzed information to the user terminal, means for recognizing the user's face and analyzing their emotions using a camera, and means for generating and providing optimal information based on the user's emotions. This makes it possible to provide quick and optimal information that takes the user's emotions into consideration.

[1140] "Data source" refers to various information sources used within a company to collect business knowledge and know-how.

[1141] A "generative AI model" is an artificial intelligence model used to analyze collected data and generate useful information.

[1142] "User terminal" refers to any device that a user uses to make inquiries and receive responses in order to use the system.

[1143] "Camera" refers to the imaging device used to recognize and analyze the User's face.

[1144] "Facial recognition" refers to the technology of detecting and identifying the face of a specific person from an image captured by a camera.

[1145] "Emotion analysis" refers to a technology that determines a user's emotional state based on facial and voice data.

[1146] "Visualization tools" refers to software or technology used to transform analyzed information into visual formats such as graphs and charts.

[1147] "Natural language processing" refers to the technology of analyzing human language and understanding its meaning using generative AI models.

[1148] "Optimization" refers to the process of optimally adjusting the information and responses provided based on the user's emotions and state.

[1149] The present invention combines a system that efficiently shares business knowledge and know-how within a company and supports rapid decision-making with an emotion engine that recognizes user emotions. Specific embodiments are described below.

[1150] First, the server collects business knowledge and know-how from multiple data sources within the company, including document management systems, databases, and internal social networking sites. The server retrieves content from each data source via API and stores and manages it in an integrated data repository.

[1151] The server then preprocesses the collected data and analyzes it using a generative AI model (e.g., GPT-3). The generative AI model summarizes relevant information from the collected data and organizes it into an easy-to-understand format. It then uses visualization tools to transform the data into graphs, charts, flowcharts, and other formats that users can easily understand.

[1152] A user uses a device (such as smart glasses) to request information, for example, by entering a specific question such as "What are the latest market trends?" The device receives the user's input, analyzes it using natural language processing (NLP) techniques, and sends a request to the server.

[1153] Based on the received request, the server extracts relevant data from a data repository, uses a generative AI model to generate information appropriate to the user's question, and organizes and visualizes the information using visualization tools.

[1154] In addition, the server is integrated with an emotion engine, which performs facial recognition and emotion analysis through the smart glasses' camera to recognize the user's emotional state. The emotion engine analyzes voice input and facial expression data and optimizes the information provided based on the user's emotions. For example, if the user is feeling anxious or nervous, it will provide more detailed information or concise, intuitive information.

[1155] The optimized information is generated as a response and sent to the user's device. The device processes the response received from the server and displays it to the user. For example, the smart glasses display could show graphs and charts of the latest market trends. The user can then use the information displayed on the device to make appropriate decisions.

[1156] Examples:

[1157] As the security guard was making his rounds, the camera in his smart glasses began scanning his face and noticed he was nervous. As a result, the system immediately sent him a notification detailing a potential threat and a brief explanation of the steps he should take immediately. The prompt read, "The security guard is nervous, please provide a quick response."

[1158] As described above, by combining a server, user devices, emotion engines, generative AI models, visualization tools, etc., it becomes possible to provide fast and optimal information that takes into account the user's emotions.

[1159] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1160] Step 1:

[1161] The server collects business knowledge and know-how from multiple data sources within the company (document management systems, databases, internal SNS, etc.), obtains information from each data source via API, and stores and manages the obtained data in an integrated data repository.

[1162] Input: Raw data obtained from the API of each data source

[1163] Output: Business knowledge and know-how stored in a unified data repository

[1164] Step 2:

[1165] The server preprocesses the data stored in the integrated data repository and analyzes it using a generative AI model (such as GPT-3). The generative AI model summarizes relevant information from the collected data and organizes it into an easy-to-understand format. It then uses visualization tools to transform the data into graphs, charts, flowcharts, and other formats.

[1166] Input: Raw data from the integrated data repository

[1167] Output: Analyzed and visualized data (graphs, charts, etc.)

[1168] Step 3:

[1169] A user uses a device (such as smart glasses) to inquire about the information they want to know, for example, by inputting a specific question such as "Please tell me the latest market trends."

[1170] Input: User question

[1171] Output: The query is parsed and the request sent to the server.

[1172] Step 4:

[1173] The device analyzes the user's input using natural language processing (NLP) technology and sends the content to the server.

[1174] Input: User question

[1175] Output: Request sent to the server

[1176] Step 5:

[1177] Based on the received request, the server extracts relevant data from an integrated data repository, uses a generative AI model to generate information appropriate to the user's question, and organizes and visualizes the information using visualization tools.

[1178] Input: Natural language processed user question

[1179] Output: Specific responses to user questions (graphs, charts, etc.)

[1180] Step 6:

[1181] The server is integrated with an emotion engine that recognizes the user's face through the smart glasses' camera and performs real-time emotion analysis, determining the user's emotional state based on voice input and facial expression data.

[1182] Input: User's face data acquired by camera and voice input

[1183] Output: Parsed user's emotional state

[1184] Step 7:

[1185] The server optimizes the information it provides based on the user's emotional state analyzed by the emotion engine, generating detailed or concise information as needed.

[1186] Input: User's emotional state

[1187] Output: Specific responses optimized based on emotions

[1188] Step 8:

[1189] The server sends the optimized response to the user terminal.

[1190] Input: Emotion-based optimized response

[1191] Output: The optimized response sent to the user's device

[1192] Step 9:

[1193] The terminal processes the response received from the server and displays it to the user, for example by displaying graphs and charts of the latest market trends on the display of smart glasses.

[1194] Input: The response received from the server

[1195] Output: Analysis results displayed to the user (graphs, charts, etc.)

[1196] Step 10:

[1197] Users use the information displayed on their devices to perform their tasks and make appropriate decisions. For example, while a security guard was on patrol, the camera in his smart glasses began scanning his face and recognized that he was nervous. As a result, the system immediately sent a notification to him, notifying him of a possible threat and briefly explaining the steps that should be taken immediately. The prompt read, "The security guard is nervous, so please provide a quick response."

[1198] Input: Analysis results displayed to the user

[1199] Output: Business execution and decision-making based on analysis results

[1200] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1201] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1202] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1203] [Third embodiment]

[1204] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1205] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1206] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1207] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1208] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1209] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1210] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1211] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1212] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1213] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1214] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1215] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1216] The present invention is a system for efficiently sharing business knowledge and know-how within a company and supporting rapid decision-making. Specific embodiments of this system will be described below.

[1217] 1. Data Collection

[1218] server

[1219] The server collects business knowledge and know-how from data sources within the company (document management systems, databases, internal SNS, etc.), obtains the content of each data source via API, and stores and manages it in an integrated data repository.

[1220] 2. Data analysis and visualization

[1221] server

[1222] The collected data is preprocessed and analyzed using a generative AI model, which uses, for example, GPT, to summarize the relevant information in an easy-to-understand format.

[1223] To visualize the generated information, visualization tools are used to convert the data into graphs, charts, flowcharts, and other formats, making it easier for users to understand and use the information.

[1224] 3. User Inquiry Processing

[1225] User

[1226] Users, such as new employees, general employees, and managers, use devices (smartphones, PCs, smart glasses, etc.) to inquire about the information they want to know. For example, they input specific questions such as, "Please tell me about the latest market trends."

[1227] Terminal

[1228] When the terminal receives an inquiry from a user, it analyzes the content using natural language processing and sends a request to the server.

[1229] 4. Server Response Generation

[1230] server

[1231] Based on the received inquiry, relevant data is extracted from the data repository, and information appropriate to the user's question is generated using a generative AI model.

[1232] A response is generated and sent to the terminal to provide the visualized information to the user.

[1233] 5. Providing Feedback

[1234] Terminal

[1235] The information received from the server is displayed to the user. For example, work procedures can be displayed in report format on a smartphone, or work procedures can be navigated in real time using smart glasses.

[1236] User

[1237] Users use the information provided to perform tasks, such as when a new employee learns and understands work procedures or when a manager makes decisions based on the latest market trends.

[1238] Specific examples

[1239] Example 1: A new employee wants to know the work procedure

[1240] User

[1241] A new employee types "Please tell me the work procedures" into their smartphone.

[1242] Terminal

[1243] This query is sent to the server.

[1244] server

[1245] Analyze inquiries and extract relevant data about business procedures.

[1246] We summarize the steps required in the GPT model and visualize them as a flowchart.

[1247] Send the flowchart to your smartphone.

[1248] Terminal

[1249] Display a flowchart on a smartphone to provide information to new employees.

[1250] User

[1251] New employees check the flowchart on their smartphones and understand the work procedures.

[1252] Example 2: Manager wants to know market trends

[1253] User

[1254] The manager types into his PC, "Tell me about the latest market trends."

[1255] Terminal

[1256] This query is sent to the server.

[1257] server

[1258] Analyze inquiries and extract relevant data on market trends.

[1259] The GPT model summarizes market trends and visualizes them as graphs and charts.

[1260] The visualized data is sent to a PC.

[1261] Terminal

[1262] Graphs and charts of market trends are displayed on a PC to provide information to managers.

[1263] User

[1264] Managers check market trends on their PCs and make appropriate decisions.

[1265] The processing flow will be explained below.

[1266] Step 1:

[1267] User

[1268] The user uses a device (e.g., a smartphone or PC) to input the information they want to know and query the system. For example, the user might input "Tell me the latest market trends" into their PC.

[1269] Step 2:

[1270] Terminal

[1271] The terminal receives the user's input and generates a request to send to the server. The request includes the user's question, the user's identification information, and the terminal information, and is sent to the server.

[1272] Step 3:

[1273] server

[1274] The server receives the request from the device and uses natural language processing (NLP) techniques to analyze the content of the received request. NLP techniques perform processes such as tokenization, stemming, and entity recognition.

[1275] Step 4:

[1276] server

[1277] Based on the query content analyzed by NLP technology, the server extracts relevant data from a data repository that stores business knowledge, know-how, the latest market information, and more.

[1278] Step 5:

[1279] server

[1280] The extracted data is fed into a generative AI model (e.g., GPT) to generate and summarize information, which is then organized into a format that can provide appropriate answers to user questions.

[1281] Step 6:

[1282] server

[1283] The generated information is converted into graphs, charts, etc. using visualization tools. The visualized information is organized so that it can be easily understood by users.

[1284] Step 7:

[1285] server

[1286] A response is generated to return the visualized information to the user terminal, the response including a display format of the visualized information and any necessary associated data.

[1287] Step 8:

[1288] Terminal

[1289] The terminal processes the response received from the server and displays it to the user, for example, by displaying graphs and charts of the latest market trends on a PC screen.

[1290] Step 9:

[1291] User

[1292] Users use the information displayed on the terminal to carry out necessary tasks, such as a manager making decisions based on the latest market information.

[1293] Example 1

[1294] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1295] In order to efficiently share business knowledge and know-how within a company and support rapid decision-making, a system is needed to centrally manage data from multiple sources and respond quickly and appropriately to user inquiries. However, current systems do not integrate the processes of data collection, analysis, visualization, and feedback, making it difficult to provide users with the information they need in a timely manner.

[1296] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1297] In this invention, the server includes a means for collecting business knowledge and know-how from multiple information sources within the company, a means for storing and managing the collected data in an integrated data repository, and a means for analyzing and summarizing the collected data using a generative AI model. This integrates a series of processes from data collection to analysis, visualization, and feedback, making it possible to respond quickly and appropriately to user inquiries.

[1298] "Multiple information sources within a company" refers to multiple data sources owned by a company, such as document management systems, databases, and internal social networking sites.

[1299] "Business knowledge and know-how" refers to information including knowledge and practical knowledge such as procedures, experience, rules, and technical information required for the operation of a company.

[1300] "Means of collection" refers to APIs and software tools that allow companies to acquire and integrate business knowledge and know-how from multiple sources within the company.

[1301] An "integrated data repository" refers to a database or storage system where collected data is centrally stored and managed.

[1302] A "generative AI model" refers to an artificial intelligence model that generates and summarizes useful information from large amounts of data, for example, using a generative AI model (such as GPT-4).

[1303] "Means of analysis" refers to the process of preprocessing collected data and using generative AI models to extract and summarize the necessary information.

[1304] "Visualization tools" refers to software tools (e.g., Tableau, Power BI) that transform analyzed data into visual formats such as graphs, charts, and flowcharts.

[1305] "Natural language processing" refers to the technology used to analyze user queries and extract key keywords and important information.

[1306] "User terminal" refers to a device used by a user to input and receive information, such as a PC, smartphone, or smart glasses.

[1307] "Server" refers to the central computer system used for data collection, analysis, visualization, and response to the user.

[1308] The present invention is a system for efficiently sharing business knowledge and know-how within a company and supporting rapid decision-making. Specific embodiments of this system will be described below.

[1309] System configuration and operation

[1310] server

[1311] Data collection: The server collects business knowledge and know-how from multiple information sources, such as the company's internal document management system, databases, and internal SNS. To do this, the server obtains data using the API of each data source (for example, a general-purpose API or database query). The collected data is stored in an integrated data repository. Specifically, it obtains PDF files from the document management system, table entries from the database, and message history from the internal SNS.

[1312] Data analysis and summarization: The collected data is preprocessed using Python's Pandas library. It is then analyzed and summarized using a generative AI model (e.g., GPT-4). An example prompt might be, "Please summarize the key business steps from the data below."

[1313] Visualization: Using visualization tools (e.g., Tableau, Power BI), the analyzed data is transformed into visual formats such as graphs, charts, and flowcharts, which helps users intuitively understand the information.

[1314] User

[1315] Inquiry: A user (new employee, general employee, manager, etc.) makes an inquiry about information using a device such as a smartphone, PC, or smart glasses. For example, they might type "Please tell me about the latest market trends" into their smartphone.

[1316] Terminal

[1317] Natural Language Processing: The terminal receives a user query and analyzes the content using a natural language processing library (e.g., spaCy, NLTK, or other major natural language processing tools), for example, tokenizing the input text and extracting key keywords.

[1318] Query sending: Send a request to the server based on the analysis results. This is done using the REST API.

[1319] Server Response

[1320] Information extraction: The server extracts relevant data from the integrated data repository based on the received query, for example, searching for the latest data on "market trends."

[1321] Response generation: Generative AI models (e.g., GPT-4) are used to generate information appropriate for the user's question. An example prompt might be, "Please summarize the latest market trends based on the market data below."

[1322] Visualization and Presentation: The generated information is visualized using a visualization tool and the results are sent to the terminal. For example, the latest market trends can be visualized as a bar graph and sent to a PC.

[1323] Providing Feedback

[1324] Display: The terminal displays the information received from the server to the user. Specifically, it displays a flowchart of business procedures on a smartphone, or a graph of market trends on a PC.

[1325] Work execution: Users perform their work based on the information provided. This can be a new employee learning how to do their job or a manager reviewing the latest market trends and making decisions.

[1326] Specific examples

[1327] When a new employee wants to know the work procedures

[1328] 1. User: A new employee types "Please tell me the work procedure" into their smartphone.

[1329] 2. Terminal: Analyzes the query using natural language processing and sends a request to the server.

[1330] 3. Server: Analyzes the query and extracts data about business procedures from the document management system. It uses GPT-4 to summarize the key steps and visualize them as a flowchart. It then sends the visualized flowchart to the smartphone.

[1331] 4. Terminal: Display the flowchart on a smartphone and provide information to new employees.

[1332] 5. User: New employees review the flowchart and understand the work procedures.

[1333] When managers want to know market trends

[1334] 1. User: The manager types into his PC, "Tell me the latest market trends."

[1335] 2. Terminal: Analyzes the query using natural language processing and sends a request to the server.

[1336] 3. Server: Analyzes the query and extracts data about market trends from the database. It uses GPT-4 to summarize the market trends and visualize them as graphs. It then sends the visualized data to the PC.

[1337] 4. Terminal: Displays graphs of market trends on a PC and provides information to managers.

[1338] 5. User: Managers review the graphs and make appropriate decisions.

[1339] As described above, this system collects data from multiple sources within a company and uses AI models and visualization tools to provide users with appropriate business knowledge and know-how.

[1340] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1341] Step 1:

[1342] Data collection

[1343] The server collects business knowledge and know-how from multiple sources within the company, specifically, using APIs to obtain data from document management systems, databases, internal social networking sites, etc.

[1344] Input: Data from document management systems, databases, and APIs of internal SNS.

[1345] Output: Data stored in a unified data repository.

[1346] Specific operations: For example, retrieve "Business Procedures.pdf" from the document management system, retrieve the latest entries in the "Market Trends" table from the database, and retrieve the message history of the "General" channel from the internal SNS, and store these in the integrated data repository.

[1347] Step 2:

[1348] Data Preprocessing

[1349] The server preprocesses the collected data using the Python Pandas library.

[1350] Input: Data from the Unified Data Repository.

[1351] Output: Cleaned and filtered data.

[1352] Specific Actions: For example, removing empty fields from database entries and fixing inconsistent formatting.

[1353] Step 3:

[1354] Data analysis and summary generation

[1355] The server analyzes the preprocessed data with a generative AI model (e.g., GPT-4) and generates a summary. The summary is generated using a prompt sentence.

[1356] Input: Cleaned and filtered data.

[1357] Output: The summary generated by GPT-4.

[1358] Specific operation: For example, input the prompt sentence "Please summarize the key business steps from the following data" into GPT-4 and obtain a summary.

[1359] Step 4:

[1360] Data Visualization

[1361] The server converts the generated summaries into a visual format using a visualization tool (e.g., Tableau, Power BI).

[1362] Input: The summary generated by GPT-4.

[1363] Output: Visual data such as graphs, charts, and flowcharts.

[1364] Specific actions: For example, visualizing business procedures as a flowchart and displaying Slack message history as a pie chart.

[1365] Step 5:

[1366] Receiving inquiries from users

[1367] Users can request information using devices such as smartphones, PCs, or smart glasses.

[1368] Input: User's query (e.g., "What are the latest market trends?").

[1369] Output: The parsing request sent to the terminal.

[1370] Specific action: For example, a user types "Please tell me the latest market trends" into their smartphone.

[1371] Step 6:

[1372] Natural Language Processing

[1373] The device analyzes user inquiries using natural language processing, specifically using natural language processing libraries (e.g., spaCy, NLTK).

[1374] Input: User's inquiry.

[1375] Output: Analysis results of the query.

[1376] Specific behavior: For example, tokenize the input text and extract key keywords (e.g., "market trends," "latest").

[1377] Step 7:

[1378] Sending a query to the server

[1379] The terminal transmits a request to the server based on the analysis result.

[1380] Input: Analysis results (e.g., "Market Trends", "Latest").

[1381] Output: REST API request to the server.

[1382] Specific operation: Send the analysis results to the server using the REST API.

[1383] Step 8:

[1384] Server information extraction and response generation

[1385] Based on the received inquiry, the server extracts relevant data from an integrated data repository and uses a generative AI model to generate information appropriate to the user's question.

[1386] Input: Data extraction request based on analysis results.

[1387] Output: Information suitable for the user's question.

[1388] Specific operation: For example, input the prompt sentence "Please summarize the latest market trends based on the market data below" into GPT-4 and obtain a summary.

[1389] Step 9:

[1390] Information Visualization

[1391] The server visualizes the generated information using a visualization tool and transmits the results to the terminal.

[1392] Input: Information generated by a generative AI model.

[1393] Output: Visualized data.

[1394] Specific operation: For example, visualize the latest market trends as a bar graph and send it to a PC.

[1395] Step 10:

[1396] Viewing information and providing feedback

[1397] The terminal displays the information received from the server to the user, who then performs the task based on the information provided.

[1398] Input: Visualization data sent from the server.

[1399] Output: Information displayed on the terminal.

[1400] Specific operations: For example, a graph of market trends is displayed on a PC, and the user checks the information and makes a decision.

[1401] (Application example 1)

[1402] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1403] Although logistics centers require the rapid acquisition and sharing of business knowledge and know-how, current systems lack the means to efficiently collect, analyze, visualize, and provide this information. In particular, it is difficult for staff to obtain optimal picking routes and inventory information in real time, which reduces work efficiency. There is a need for a system that can solve these issues and support the efficiency of work and rapid decision-making at logistics centers.

[1404] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1405] In this invention, the server includes means for collecting business knowledge and know-how from multiple data sources within the company, means for analyzing and visualizing the collected data using a generative AI model, means for receiving voice requests from a user terminal, analyzing the requests using natural language processing, and transmitting related information to the server, and means for visualizing the related information transmitted from the server and displaying it on the user terminal in real time using augmented reality technology. This enables logistics center staff to efficiently obtain optimal picking routes and inventory information in real time, enabling them to carry out their work promptly.

[1406] A "data source" is an information source used to collect business knowledge and know-how, such as a document management system, database, or in-house SNS within a company.

[1407] A "generative AI model" is an artificial intelligence model that analyzes collected data and summarizes it in an easy-to-understand format, such as GPT.

[1408] "Visualization" is the process of converting data analyzed by a generative AI model into visual formats such as graphs, charts, and flowcharts, making the information easier for users to understand.

[1409] A "user terminal" is a device used by a user to obtain information, and includes smartphones, PCs, smart glasses, head-mounted displays, etc.

[1410] "Natural language processing" is a technology for analyzing user inquiries and extracting relevant information, and is applied to voice requests and text data.

[1411] "Augmented reality technology" is a technology that displays visualized information overlaid on the real world in real time, and refers to the use of AR (augmented reality).

[1412] A "picking route" is the optimal route for staff to efficiently select products at a logistics center, and is generated based on inventory information.

[1413] "Inventory information" refers to information relating to the type, quantity, location, etc. of products currently held within the logistics center.

[1414] "Real-time" refers to immediate processing and response to user requests without delay.

[1415] This invention provides a system that supports operational efficiency and rapid decision-making in logistics centers. To achieve this, it is important to collect business knowledge and know-how from multiple data sources within a company, analyze and visualize it using a generative AI model, and provide information in real time based on user inquiries.

[1416] System hardware and software configuration

[1417] Server: The central system for data collection, analysis, and visualization. The server must be equipped with a high-performance processor, large memory capacity, and storage. The hardware used can be a general-purpose high-performance server or cloud service. The software used includes a generative AI model such as GPT, an NLP library (e.g., spacy), a database (e.g., MySQL), and a data visualization tool (e.g., D3.js).

[1418] User devices: Devices such as smartphones, smart glasses, and head-mounted displays that users use to obtain information. These devices must have voice input, display, and internet connectivity capabilities.

[1419] Data collection

[1420] The server collects business knowledge and know-how from multiple data sources within the company (document management systems, databases, internal SNS, etc.), obtains the content of each data source via API, and stores and manages it in an integrated data repository.

[1421] Parsing voice requests

[1422] The user device (such as smart glasses) receives the voice request and converts it into text data using a natural language processing library (such as spacy). The analyzed text data is then sent to the server via an API.

[1423] Data analysis and visualization

[1424] Based on the received voice request, the server extracts relevant data from a data repository, uses a generative AI model (e.g., GPT) to generate information relevant to the user's question, and then uses a data visualization tool (e.g., D3.js) to transform the data into visual formats such as graphs, charts, and flowcharts.

[1425] Augmented reality information provision

[1426] The visualized information is sent to the user's device and displayed in real time on the user's device using augmented reality technology (such as ARKit or ARCore). For example, by using smart glasses, logistics center staff can visually navigate and quickly determine the optimal picking route.

[1427] Specific examples

[1428] Example 1: Navigation with voice requests and AR

[1429] A staff member wears smart glasses and asks, "Tell me which shelf I should pick up the next item from." The smart glasses analyze the voice and send a request to the server. The server generates a picking route and sends the visualized information to the smart glasses. The smart glasses then use their AR functionality to navigate the staff member.

[1430] Example 2: Example of a prompt statement

[1431] "Can you tell me which shelf I should pick up next?"

[1432] As described above, using this system will significantly improve operational efficiency at logistics centers and enable faster decision-making.

[1433] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1434] Step 1:

[1435] The user enters a voice request

[1436] The user puts on the smart glasses and asks, "Tell me which shelf I should pick up the next item from." This voice data is the input.

[1437] Step 2:

[1438] Your device converts speech to text

[1439] The microphone in the smart glasses captures the voice data and converts it into text data using a natural language processing library (e.g., spacy). Here, the input is voice data and the output is text data.

[1440] Step 3:

[1441] The device sends text data to the server.

[1442] The terminal sends the parsed text data to the server via the API. The input is the text data, and the output is the request data sent to the server.

[1443] Step 4:

[1444] The server receives the request and parses the data

[1445] The server receives the request data and uses a generative AI model (e.g., GPT) to extract relevant data based on the request from a data repository, where the input is the request data and the output is the relevant data.

[1446] Step 5:

[1447] The server visualizes the relevant data

[1448] The server uses visualization tools (e.g., D3.js) to transform the relevant data into a visual format such as graphs, charts, flowcharts, etc. In this step, the input is the relevant data and the output is the visualized data.

[1449] Step 6:

[1450] The server sends the visualized data to the device.

[1451] The server sends the visualized data to the terminal through the API, where the input is the visualized data and the output is the response data to the terminal.

[1452] Step 7:

[1453] The device receives the visualized data and displays it using augmented reality technology.

[1454] The device displays the received visualization data in real time using augmented reality technology (e.g., ARKit or ARCore), where the input is the response data and the output is the AR display.

[1455] Step 8:

[1456] The user acts on the displayed information

[1457] The user picks up the specified item based on the visual guide displayed on the smart glasses. In this step, the input is the AR display, and the final task execution is the output.

[1458] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1459] This invention combines a system that efficiently shares business knowledge and know-how within a company and supports rapid decision-making with an emotion engine that recognizes user emotions. Specific embodiments of this system are described below.

[1460] 1. Data Collection

[1461] server

[1462] The server collects business knowledge and know-how from multiple data sources within the company, including document management systems, databases, and internal social networking sites, and retrieves content from each data source via API, storing and managing it in an integrated data repository.

[1463] 2. Data analysis and visualization

[1464] server

[1465] The server preprocesses the collected data and analyzes it using a generative AI model (e.g., GPT). The generative AI model summarizes relevant information from the collected data and organizes it into an easy-to-understand format. It then uses visualization tools to transform the data into graphs, charts, flowcharts, and other formats that users can easily understand.

[1466] 3. User Inquiry Processing

[1467] User

[1468] Users use devices (e.g., smartphones, PCs, smart glasses) to request information, for example, by inputting a specific question such as, "What are the latest market trends?"

[1469] Terminal

[1470] When the device receives user input, it analyzes it using natural language processing (NLP) technology and sends a request to the server.

[1471] 4. Server Response Generation

[1472] server

[1473] The server extracts relevant data from a data repository based on the received request, uses generative AI models to generate information relevant to the user's question, and organizes the information using visualization tools.

[1474] 5. Emotion engine integration

[1475] server

[1476] The server is integrated with an emotion engine that analyzes voice input and facial expression data to recognize the user's emotional state. The emotion engine optimizes the information provided based on the user's emotions and generates responses that correspond to the user's state of mind and situation.

[1477] 6. Providing Feedback

[1478] server

[1479] The server generates the optimized information as a response and sends it to the user terminal.

[1480] Terminal

[1481] The terminal processes the response received from the server and displays it to the user, for example by displaying graphs and charts of the latest market trends on a PC screen.

[1482] User

[1483] Users use the information displayed on the device to carry out the necessary tasks, such as when a manager makes decisions based on market information.

[1484] Specific examples

[1485] Example 1: A new employee wants to know the work procedure

[1486] User

[1487] A new employee types "Please tell me the work procedures" into their smartphone.

[1488] Terminal

[1489] The terminal sends this query to the server.

[1490] server

[1491] The server analyzes the query and extracts relevant data. A generative AI model summarizes the steps and visualizes them as a flowchart.

[1492] Emotion Engine

[1493] The server uses an emotion engine to analyze the user's voice and facial expression data while operating the device, and if the user appears anxious or confused, it responds by adding detailed instructions or supplementary information.

[1494] server

[1495] Finally, the server transmits the flowchart and supplementary information to the user terminal.

[1496] Terminal

[1497] Display flowcharts and supplemental information on smartphones to educate new employees.

[1498] User

[1499] New employees check the flowchart on their smartphones and understand the work procedures.

[1500] Example 2: Manager wants to know market trends

[1501] User

[1502] The manager types into his PC, "Tell me about the latest market trends."

[1503] Terminal

[1504] The terminal sends this query to the server.

[1505] server

[1506] The server analyzes the query and extracts data related to market trends. A generative AI model summarizes the market trends and visualizes them as graphs and charts.

[1507] Emotion Engine

[1508] The server analyzes the manager's emotions and optimizes the presentation of concise and intuitive information if the user is nervous.

[1509] server

[1510] Send visualized market trends graphs and charts to your device.

[1511] Terminal

[1512] Display graphs and charts on a PC to provide information to managers.

[1513] User

[1514] Managers check market trends on their PCs and make appropriate decisions.

[1515] The processing flow will be explained below.

[1516] Step 1:

[1517] User

[1518] The user uses a device (e.g., a smartphone or PC) to input the information they want to know and query the system. For example, the user might input "Tell me the latest market trends" into their PC.

[1519] Step 2:

[1520] Terminal

[1521] The terminal receives the user's input and generates a request to send to the server, which includes the user's question, the user's identification information, and information about the terminal.

[1522] Step 3:

[1523] server

[1524] The server analyzes the request received from the device using natural language processing (NLP) techniques, such as tokenization, stemming, and entity recognition.

[1525] Step 4:

[1526] server

[1527] The server searches and extracts relevant data from the data repository based on the query content analyzed by NLP.

[1528] Step 5:

[1529] server

[1530] The extracted data is fed into a generative AI model (e.g., GPT) to generate and summarize relevant information, which is organized into answers to user questions.

[1531] Step 6:

[1532] server

[1533] The generated information is converted into graphs, charts, and other formats using visualization tools, which makes it easier for users to understand the information.

[1534] Step 7:

[1535] server

[1536] The emotion engine analyzes the user's emotional state when making an inquiry. It uses voice input and a camera to collect facial expression data and analyzes the user's emotional state.

[1537] Step 8:

[1538] server

[1539] Based on the analysis results of the emotion engine, a generative AI model generates an optimized response, for example, if the user is nervous, it will provide a concise and reassuring answer.

[1540] Step 9:

[1541] server

[1542] The visualized information and the optimized response are generated as a response and transmitted to the user terminal.

[1543] Step 10:

[1544] Terminal

[1545] The terminal processes the response received from the server and displays it to the user, for example, by displaying graphs and charts of the latest market trends on a PC screen.

[1546] Step 11:

[1547] User

[1548] Users use the information displayed on the terminal to carry out necessary tasks, such as when a manager makes decisions based on market information.

[1549] Specific examples

[1550] Example 1: A new employee wants to know the work procedure

[1551] Step 1:

[1552] User

[1553] A new employee types "Please tell me the work procedures" into their smartphone.

[1554] Step 2:

[1555] Terminal

[1556] This query is sent to the server.

[1557] Step 3:

[1558] server

[1559] The received request is analyzed using NLP technology and data related to business procedures is extracted from the data repository.

[1560] Step 4:

[1561] server

[1562] The data is summarized using a generative AI model and visualized in flowchart format.

[1563] Step 5:

[1564] server

[1565] The emotion engine analyzes the voice and facial expression data of new employees, and if they appear anxious, detailed supplementary information is added.

[1566] Step 6:

[1567] server

[1568] Optimized information is sent to your smartphone.

[1569] Step 7:

[1570] Terminal

[1571] View the flowchart and supplemental information on your smartphone.

[1572] Step 8:

[1573] User

[1574] New employees review the information and understand the work procedures.

[1575] Example 2: Manager wants to know market trends

[1576] Step 1:

[1577] User

[1578] The manager types into his PC, "Tell me about the latest market trends."

[1579] Step 2:

[1580] Terminal

[1581] This query is sent to the server.

[1582] Step 3:

[1583] server

[1584] Analyze inquiries and extract data related to market trends.

[1585] Step 4:

[1586] server

[1587] Generative AI models summarize market trends and visualize them as graphs and charts.

[1588] Step 5:

[1589] server

[1590] The emotion engine analyzes the manager's emotions and provides concise information if they are nervous.

[1591] Step 6:

[1592] server

[1593] The visualized data is sent to a PC.

[1594] Step 7:

[1595] Terminal

[1596] Display graphs and charts on your PC.

[1597] Step 8:

[1598] User

[1599] The manager reviews the information and makes a decision.

[1600] Example 2

[1601] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1602] In conventional systems for sharing business knowledge and know-how within companies, the collection, analysis, and visualization of information are automated to a certain extent, but they are unable to provide information that takes into account the user's emotional state, which means that the specific information and supplementary explanations that users are looking for are insufficient, making it difficult to support effective decision-making.In particular, when users are feeling anxious or confused, it is necessary to provide appropriate information, but this has not been fully realized.

[1603] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1604] In this invention, the server includes means for collecting business knowledge and know-how from multiple data sources within the company, means for analyzing and visualizing the collected data using a generative AI model, means for receiving inquiries from users and providing analyzed information to the user terminal, means for integrating an emotion engine that analyzes voice input and facial expression data to recognize the emotional state of the user, and means for optimizing the information to be provided based on the emotional state of the user. This enables the provision of optimal information taking into account the user's emotions, thereby supporting effective knowledge sharing and rapid decision-making within the company.

[1605] "Multiple data sources within a company" refers to various information sources used within a company, such as document management systems, databases, and internal social networking sites.

[1606] "Business knowledge and know-how" refers to the specialized knowledge, experience, procedures and best practices a company needs to carry out its day-to-day business operations.

[1607] A "generative AI model" is an artificial intelligence model that has the ability to learn patterns from large amounts of data and generate new data, and specifically refers to natural language processing models such as GPT-4.

[1608] "Analysis and visualization" refers to a set of methods and techniques aimed at processing collected data and presenting the information in an understandable format, including, for example, generating graphs and charts.

[1609] "User inquiries" refer to questions or requests made to the system by users entering information or questions in natural language.

[1610] A "user terminal" is a hardware device that allows a user to use the system, and examples include smartphones, PCs, and smart glasses.

[1611] "Natural language processing" is a technology that enables computers to understand and generate human language, and refers to the process of analyzing text and audio data and extracting meaning.

[1612] "Emotion engine" refers to technology that recognizes a user's emotional state and generates an appropriate response based on that emotion.

[1613] "Voice input and facial expression data" refers to data used to analyze the user's voice and facial expressions, and is important input information used by the emotion engine.

[1614] "Optimization" refers to the process of adjusting the parameters and behavior of a system to achieve optimal results for a specific purpose.

[1615] This invention combines a system that efficiently shares business knowledge and know-how within a company and supports rapid decision-making with an emotion engine that recognizes user emotions.

[1616] System configuration

[1617] Data collection

[1618] server

[1619] The server collects business knowledge and know-how from multiple data sources within the company. These include document management systems, databases, and internal SNS. Data is obtained from each data source via API and stored and managed in an integrated data repository. Specifically, a general "document management system" is used as the document management system, a "database management system" as the database, and an "internal SNS system" as the internal SNS. A "database" is used as the integrated data repository.

[1620] Data analysis and visualization

[1621] server

[1622] The server preprocesses the collected data and analyzes it using a generative AI model (e.g., generative AI model). The generative AI model summarizes relevant information from the collected data and organizes it in an easy-to-understand format. It also uses visualization tools to convert the data into graphs, charts, flowcharts, and other formats. The visualization tools used are called "visualization tools."

[1623] Processing user inquiries

[1624] User

[1625] Users use devices (e.g., smartphones, PCs, smart glasses) to request information, for example, by inputting a specific question such as, "What are the latest market trends?"

[1626] Terminal

[1627] When the device receives user input, it analyzes the content using natural language processing technology and sends a request to the server. Natural language processing is performed using a "natural language processing system."

[1628] Server response generation

[1629] server

[1630] Based on the received request, the server extracts relevant data from a data repository, uses a generative AI model (e.g., a generative AI model) to generate information relevant to the user's question, and organizes the information using visualization tools.

[1631] Emotion engine integration

[1632] server

[1633] The server is integrated with an emotion engine that analyzes voice input and facial expression data to recognize the user's emotional state. The emotion engine uses an "emotion recognition system" to optimize the information provided according to the user's emotions.

[1634] Providing Feedback

[1635] server

[1636] The server generates the optimized information as a response and sends it to the user terminal.

[1637] Terminal

[1638] The terminal processes the response received from the server and displays it to the user, for example by displaying graphs and charts of the latest market trends on a PC screen.

[1639] User

[1640] Users use the information displayed on their devices to carry out the necessary tasks, such as when a manager makes decisions based on market information.

[1641] Specific examples

[1642] Example 1: A new employee wants to know the work procedure

[1643] User

[1644] A new employee types "Please tell me the work procedures" into their smartphone.

[1645] Terminal

[1646] The terminal sends this query to the server.

[1647] server

[1648] The server analyzes the query and extracts relevant data. A generative AI model summarizes the steps and visualizes them as a flowchart.

[1649] Emotion Engine

[1650] The server uses an emotion engine to analyze the user's voice and facial expression data, and if the user feels anxious or confused, it adds detailed instructions or supplementary information.

[1651] server

[1652] Finally, the server transmits the flowchart and supplementary information to the user terminal.

[1653] Terminal

[1654] Display flowcharts and supplemental information on smartphones to educate new employees.

[1655] User

[1656] New employees check the flowchart on their smartphones and understand the work procedures.

[1657] Example 2: Manager wants to know market trends

[1658] User

[1659] The manager types into his PC, "Tell me about the latest market trends."

[1660] Terminal

[1661] The terminal sends this query to the server.

[1662] server

[1663] The server analyzes the query and extracts data related to market trends. A generative AI model summarizes the market trends and visualizes them as graphs and charts.

[1664] Emotion Engine

[1665] The server analyzes the manager's emotions and optimizes to provide concise and intuitive information if the manager is nervous.

[1666] server

[1667] Send visualized market trends graphs and charts to your device.

[1668] Terminal

[1669] Display graphs and charts on a PC to provide information to managers.

[1670] User

[1671] Managers check market trends on their PCs and make appropriate decisions.

[1672] Prompt Sentence Examples

[1673] "Tell me the business procedures."

[1674] "What are the latest market trends?"

[1675] keyword

[1676] Generative AI model, prompt sentence

[1677] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1678] Step 1:

[1679] Data collection

[1680] server

[1681] Input: Multiple data sources within the company (e.g. document management systems, databases, internal SNS)

[1682] Output: Business knowledge and know-how stored in a unified data repository

[1683] The server collects business knowledge and know-how from multiple data sources within the company. Specifically, it periodically sends requests to the API endpoint of each data source, receives the retrieved data in JSON format, and temporarily stores it in memory. The data in memory is then written to the "database management system."

[1684] Step 2:

[1685] Data analysis and visualization

[1686] server

[1687] Input: Data stored in a unified data repository

[1688] Output: Summarized information and visualized graphs and charts

[1689] The server preprocesses the collected data and analyzes it using a generative AI model (e.g., "generative AI model"). Preprocessing includes cleaning the data (removing unnecessary strings and standardizing formatting). Next, the "generative AI model" is used to extract and summarize important information and save it as a new entry. Finally, the extracted data is visualized using a "visualization tool" to generate graphs and charts.

[1690] Step 3:

[1691] Processing user inquiries

[1692] User

[1693] Input: A specific question (e.g., "What are the latest market trends?")

[1694] Output: None (user input is sent to the server)

[1695] A user uses a device (e.g., a smartphone, PC, or smart glasses) to request information. The user enters a question into an input field on the device. For example, the user enters a specific question such as, "Please tell me about the latest market trends."

[1696] Terminal

[1697] Input: User question

[1698] Output: Parsed user question sent to server

[1699] When the device receives user input, it analyzes the content using natural language processing technology. It uses a "natural language processing system" as a natural language processing tool, and sends the analyzed data to the server in the form of an HTTP request.

[1700] Step 4:

[1701] Server response generation

[1702] server

[1703] Input: The request sent from the terminal (the user's question)

[1704] Output: Generated response data and visualizations

[1705] The server extracts relevant data from a data repository based on the received request, uses a "generative AI model" to generate information appropriate for the user's question, and organizes and visualizes the information using a "visualization tool."

[1706] Step 5:

[1707] Emotion engine integration

[1708] server

[1709] Input: Voice input and facial expression data

[1710] Output: Optimized response based on the user's emotional state

[1711] The server is integrated with an emotion engine (e.g., an "emotion recognition system") that analyzes voice input and facial expression data to recognize the user's emotional state, and adjusts the information content and visualization format provided based on the analysis results.

[1712] Step 6:

[1713] Providing Feedback

[1714] server

[1715] Input: Optimized Information

[1716] Output: Information sent to the user's device

[1717] The server generates the optimized information as a response and sends it to the user terminal.

[1718] Terminal

[1719] Input: The response received from the server

[1720] Output: Information displayed to the user (e.g., graphs and charts)

[1721] The terminal processes the response received from the server and displays it to the user, for example, by displaying graphs and charts of the latest market trends on a PC screen.

[1722] User

[1723] Input: The information displayed on the terminal

[1724] Output: User decisions or actions

[1725] Users use the information displayed on their devices to carry out the necessary tasks, such as when a manager makes decisions based on market information.

[1726] (Application example 2)

[1727] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1728] In systems that efficiently share business knowledge and know-how within a company and support rapid decision-making, current technology faces the challenge of not being able to provide optimal information that takes user emotions into account. Furthermore, in situations where stress levels are particularly high, there is a risk of work efficiency declining, so there is a demand for technology that can analyze emotions in real time and optimize responses.

[1729] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting business knowledge and know-how from multiple data sources within the company, means for analyzing and visualizing the collected data using a generative AI model, means for receiving inquiries from users and providing analyzed information to the user terminal, means for recognizing the user's face and analyzing their emotions using a camera, and means for generating and providing optimal information based on the user's emotions. This makes it possible to provide quick and optimal information that takes the user's emotions into consideration.

[1730] "Data source" refers to various information sources used within a company to collect business knowledge and know-how.

[1731] A "generative AI model" is an artificial intelligence model used to analyze collected data and generate useful information.

[1732] "User terminal" refers to any device that a user uses to make inquiries and receive responses in order to use the system.

[1733] "Camera" refers to the imaging device used to recognize and analyze the User's face.

[1734] "Facial recognition" refers to the technology of detecting and identifying the face of a specific person from an image captured by a camera.

[1735] "Emotion analysis" refers to a technology that determines a user's emotional state based on facial and voice data.

[1736] "Visualization tools" refers to software or technology used to transform analyzed information into visual formats such as graphs and charts.

[1737] "Natural language processing" refers to the technology of analyzing human language and understanding its meaning using generative AI models.

[1738] "Optimization" refers to the process of optimally adjusting the information and responses provided based on the user's emotions and state.

[1739] The present invention combines a system that efficiently shares business knowledge and know-how within a company and supports rapid decision-making with an emotion engine that recognizes user emotions. Specific embodiments are described below.

[1740] First, the server collects business knowledge and know-how from multiple data sources within the company, including document management systems, databases, and internal social networking sites. The content is retrieved from each data source via API and stored and managed in an integrated data repository.

[1741] The server then preprocesses the collected data and analyzes it using a generative AI model (e.g., GPT-3). The generative AI model summarizes relevant information from the collected data and organizes it into an easy-to-understand format. It then uses visualization tools to transform the data into graphs, charts, flowcharts, and other formats that users can easily understand.

[1742] A user uses a device (such as smart glasses) to request information, for example, by entering a specific question such as "What are the latest market trends?" The device receives the user's input, analyzes it using natural language processing (NLP) techniques, and sends a request to the server.

[1743] Based on the received request, the server extracts relevant data from a data repository, uses a generative AI model to generate information appropriate to the user's question, and organizes and visualizes the information using visualization tools.

[1744] In addition, the server is integrated with an emotion engine, which performs facial recognition and emotion analysis through the smart glasses' camera to recognize the user's emotional state. The emotion engine analyzes voice input and facial expression data and optimizes the information provided based on the user's emotions. For example, if the user is feeling anxious or nervous, it will provide more detailed information or concise, intuitive information.

[1745] The optimized information is generated as a response and sent to the user's device. The device processes the response received from the server and displays it to the user. For example, the smart glasses display could show graphs and charts of the latest market trends. The user can then use the information displayed on the device to make appropriate decisions.

[1746] Examples:

[1747] As the security guard was making his rounds, the camera in his smart glasses began scanning his face and noticed he was nervous. As a result, the system immediately sent him a notification detailing a potential threat and a brief explanation of the steps he should take immediately. The prompt read, "The security guard is nervous, please provide a quick response."

[1748] As described above, by combining a server, user devices, emotion engines, generative AI models, visualization tools, etc., it becomes possible to provide fast and optimal information that takes into account the user's emotions.

[1749] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1750] Step 1:

[1751] The server collects business knowledge and know-how from multiple data sources within the company (document management systems, databases, internal SNS, etc.), obtains information from each data source via API, and stores and manages the obtained data in an integrated data repository.

[1752] Input: Raw data obtained from the API of each data source

[1753] Output: Business knowledge and know-how stored in a unified data repository

[1754] Step 2:

[1755] The server preprocesses the data stored in the integrated data repository and analyzes it using a generative AI model (such as GPT-3). The generative AI model summarizes relevant information from the collected data and organizes it into an easy-to-understand format. It then uses visualization tools to transform the data into graphs, charts, flowcharts, and other formats.

[1756] Input: Raw data from the integrated data repository

[1757] Output: Analyzed and visualized data (graphs, charts, etc.)

[1758] Step 3:

[1759] A user uses a device (such as smart glasses) to inquire about the information they want to know, for example, by inputting a specific question such as "Please tell me the latest market trends."

[1760] Input: User question

[1761] Output: The query is parsed and the request sent to the server.

[1762] Step 4:

[1763] The device analyzes the user's input using natural language processing (NLP) technology and sends the content to the server.

[1764] Input: User question

[1765] Output: Request sent to the server

[1766] Step 5:

[1767] Based on the received request, the server extracts relevant data from an integrated data repository, uses a generative AI model to generate information appropriate to the user's question, and organizes and visualizes the information using visualization tools.

[1768] Input: Natural language processed user question

[1769] Output: Specific responses to user questions (graphs, charts, etc.)

[1770] Step 6:

[1771] The server is integrated with an emotion engine that recognizes the user's face through the smart glasses' camera and performs real-time emotion analysis, determining the user's emotional state based on voice input and facial expression data.

[1772] Input: User's face data acquired by camera and voice input

[1773] Output: Parsed user's emotional state

[1774] Step 7:

[1775] The server optimizes the information it provides based on the user's emotional state analyzed by the emotion engine, generating detailed or concise information as needed.

[1776] Input: User's emotional state

[1777] Output: Specific responses optimized based on emotions

[1778] Step 8:

[1779] The server sends the optimized response to the user terminal.

[1780] Input: Emotion-based optimized response

[1781] Output: The optimized response sent to the user's device

[1782] Step 9:

[1783] The terminal processes the response received from the server and displays it to the user, for example by displaying graphs and charts of the latest market trends on the display of smart glasses.

[1784] Input: The response received from the server

[1785] Output: Analysis results displayed to the user (graphs, charts, etc.)

[1786] Step 10:

[1787] Users use the information displayed on their devices to perform their tasks and make appropriate decisions. For example, while a security guard was on patrol, the camera in his smart glasses began scanning his face and recognized that he was nervous. As a result, the system immediately sent a notification to him, notifying him of a possible threat and briefly explaining the steps that should be taken immediately. The prompt read, "The security guard is nervous, so please provide a quick response."

[1788] Input: Analysis results displayed to the user

[1789] Output: Business execution and decision-making based on analysis results

[1790] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1791] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1792] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1793] [Fourth embodiment]

[1794] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1795] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1796] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1797] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1798] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1799] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1800] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1801] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1802] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1803] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1804] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1805] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1806] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1807] The present invention is a system for efficiently sharing business knowledge and know-how within a company and supporting rapid decision-making. Specific embodiments of this system will be described below.

[1808] 1. Data Collection

[1809] server

[1810] The server collects business knowledge and know-how from data sources within the company (document management systems, databases, internal SNS, etc.), obtains the content of each data source via API, and stores and manages it in an integrated data repository.

[1811] 2. Data analysis and visualization

[1812] server

[1813] The collected data is preprocessed and analyzed using a generative AI model, which uses, for example, GPT, to summarize the relevant information in an easy-to-understand format.

[1814] To visualize the generated information, visualization tools are used to convert the data into graphs, charts, flowcharts, and other formats, making it easier for users to understand and use the information.

[1815] 3. User Inquiry Processing

[1816] User

[1817] Users, such as new employees, general employees, and managers, use devices (smartphones, PCs, smart glasses, etc.) to inquire about the information they want to know. For example, they input specific questions such as, "Please tell me about the latest market trends."

[1818] Terminal

[1819] When the terminal receives an inquiry from a user, it analyzes the content using natural language processing and sends a request to the server.

[1820] 4. Server Response Generation

[1821] server

[1822] Based on the received inquiry, relevant data is extracted from the data repository, and information appropriate to the user's question is generated using a generative AI model.

[1823] A response is generated and sent to the terminal to provide the visualized information to the user.

[1824] 5. Providing Feedback

[1825] Terminal

[1826] The information received from the server is displayed to the user. For example, work procedures can be displayed in report format on a smartphone, or work procedures can be navigated in real time using smart glasses.

[1827] User

[1828] Users use the information provided to them to carry out their work, such as when a new employee learns and understands work procedures or when a manager makes decisions based on the latest market trends.

[1829] Specific examples

[1830] Example 1: A new employee wants to know the work procedure

[1831] User

[1832] A new employee types "Please tell me the work procedures" into their smartphone.

[1833] Terminal

[1834] This query is sent to the server.

[1835] server

[1836] Analyze inquiries and extract relevant data about business procedures.

[1837] We summarize the steps required in the GPT model and visualize them as a flowchart.

[1838] Send the flowchart to your smartphone.

[1839] Terminal

[1840] Display a flowchart on a smartphone to provide information to new employees.

[1841] User

[1842] New employees check the flowchart on their smartphones and understand the work procedures.

[1843] Example 2: Manager wants to know market trends

[1844] User

[1845] The manager types into his PC, "Tell me about the latest market trends."

[1846] Terminal

[1847] This query is sent to the server.

[1848] server

[1849] Analyze inquiries and extract relevant data on market trends.

[1850] The GPT model summarizes market trends and visualizes them as graphs and charts.

[1851] The visualized data is sent to a PC.

[1852] Terminal

[1853] Graphs and charts of market trends are displayed on a PC to provide information to managers.

[1854] User

[1855] Managers check market trends on their PCs and make appropriate decisions.

[1856] The processing flow will be explained below.

[1857] Step 1:

[1858] User

[1859] The user uses a device (e.g., a smartphone or PC) to input the information they want to know and query the system. For example, the user might input "Tell me the latest market trends" into their PC.

[1860] Step 2:

[1861] Terminal

[1862] The terminal receives the user's input and generates a request to send to the server. The request includes the user's question, the user's identification information, and the terminal information, and is sent to the server.

[1863] Step 3:

[1864] server

[1865] The server receives the request from the device and uses natural language processing (NLP) techniques to analyze the content of the received request. NLP techniques perform processes such as tokenization, stemming, and entity recognition.

[1866] Step 4:

[1867] server

[1868] Based on the query content analyzed by NLP technology, the server extracts relevant data from a data repository that stores business knowledge, know-how, the latest market information, and more.

[1869] Step 5:

[1870] server

[1871] The extracted data is fed into a generative AI model (e.g., GPT) to generate and summarize information, which is then organized into a format that can provide appropriate answers to user questions.

[1872] Step 6:

[1873] server

[1874] The generated information is converted into graphs, charts, etc. using visualization tools. The visualized information is organized so that it can be easily understood by users.

[1875] Step 7:

[1876] server

[1877] A response is generated to return the visualized information to the user terminal, the response including a display format of the visualized information and any necessary associated data.

[1878] Step 8:

[1879] Terminal

[1880] The terminal processes the response received from the server and displays it to the user, for example, by displaying graphs and charts of the latest market trends on a PC screen.

[1881] Step 9:

[1882] User

[1883] Users use the information displayed on the terminal to carry out necessary tasks, such as a manager making decisions based on the latest market information.

[1884] Example 1

[1885] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1886] In order to efficiently share business knowledge and know-how within a company and support rapid decision-making, a system is needed to centrally manage data from multiple sources and respond quickly and appropriately to user inquiries. However, current systems do not integrate the processes of data collection, analysis, visualization, and feedback, making it difficult to provide users with the information they need in a timely manner.

[1887] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1888] In this invention, the server includes a means for collecting business knowledge and know-how from multiple information sources within the company, a means for storing and managing the collected data in an integrated data repository, and a means for analyzing and summarizing the collected data using a generative AI model. This integrates a series of processes from data collection to analysis, visualization, and feedback, making it possible to respond quickly and appropriately to user inquiries.

[1889] "Multiple information sources within a company" refers to multiple data sources owned by a company, such as document management systems, databases, and internal social networking sites.

[1890] "Business knowledge and know-how" refers to information including knowledge and practical knowledge such as procedures, experience, rules, and technical information required for the operation of a company.

[1891] "Means of collection" refers to APIs and software tools that allow companies to acquire and integrate business knowledge and know-how from multiple sources within the company.

[1892] An "integrated data repository" refers to a database or storage system where collected data is centrally stored and managed.

[1893] A "generative AI model" refers to an artificial intelligence model that generates and summarizes useful information from large amounts of data, for example, using a generative AI model (such as GPT-4).

[1894] "Means of analysis" refers to the process of preprocessing collected data and using generative AI models to extract and summarize the necessary information.

[1895] "Visualization tools" refers to software tools (e.g., Tableau, Power BI) that transform analyzed data into visual formats such as graphs, charts, and flowcharts.

[1896] "Natural language processing" refers to the technology used to analyze user queries and extract key keywords and important information.

[1897] "User terminal" refers to a device used by a user to input and receive information, such as a PC, smartphone, or smart glasses.

[1898] "Server" refers to the central computer system used for data collection, analysis, visualization, and response to the user.

[1899] The present invention is a system for efficiently sharing business knowledge and know-how within a company and supporting rapid decision-making. Specific embodiments of this system will be described below.

[1900] System configuration and operation

[1901] server

[1902] Data collection: The server collects business knowledge and know-how from multiple information sources, such as the company's internal document management system, databases, and internal SNS. To do this, the server obtains data using the API of each data source (for example, a general-purpose API or database query). The collected data is stored in an integrated data repository. Specifically, it obtains PDF files from the document management system, table entries from the database, and message history from the internal SNS.

[1903] Data analysis and summarization: The collected data is preprocessed using Python's Pandas library. It is then analyzed and summarized using a generative AI model (e.g., GPT-4). An example prompt might be, "Please summarize the key business steps from the data below."

[1904] Visualization: Using visualization tools (e.g., Tableau, Power BI), the analyzed data is transformed into visual formats such as graphs, charts, and flowcharts, which helps users intuitively understand the information.

[1905] User

[1906] Inquiry: A user (new employee, general employee, manager, etc.) makes an inquiry about information using a device such as a smartphone, PC, or smart glasses. For example, they might type "Please tell me about the latest market trends" into their smartphone.

[1907] Terminal

[1908] Natural Language Processing: The terminal receives a user query and analyzes the content using a natural language processing library (e.g., spaCy, NLTK, or other major natural language processing tools), for example, tokenizing the input text and extracting key keywords.

[1909] Query sending: Send a request to the server based on the analysis results. This is done using the REST API.

[1910] Server Response

[1911] Information extraction: The server extracts relevant data from the integrated data repository based on the received query, for example, searching for the latest data on "market trends."

[1912] Response generation: Generative AI models (e.g., GPT-4) are used to generate information appropriate for the user's question. An example prompt might be, "Please summarize the latest market trends based on the market data below."

[1913] Visualization and Presentation: The generated information is visualized using a visualization tool and the results are sent to the terminal. For example, the latest market trends can be visualized as a bar graph and sent to a PC.

[1914] Providing Feedback

[1915] Display: The terminal displays the information received from the server to the user. Specifically, it displays a flowchart of business procedures on a smartphone, or a graph of market trends on a PC.

[1916] Work execution: Users perform their work based on the information provided. This can be a new employee learning how to do their job or a manager reviewing the latest market trends and making decisions.

[1917] Specific examples

[1918] When a new employee wants to know the work procedures

[1919] 1. User: A new employee types "Please tell me the work procedure" into their smartphone.

[1920] 2. Terminal: Analyzes the query using natural language processing and sends a request to the server.

[1921] 3. Server: Analyzes the query and extracts data about business procedures from the document management system. It uses GPT-4 to summarize the key steps and visualize them as a flowchart. It then sends the visualized flowchart to the smartphone.

[1922] 4. Terminal: Display the flowchart on a smartphone and provide information to new employees.

[1923] 5. User: New employees review the flowchart and understand the work procedures.

[1924] When managers want to know market trends

[1925] 1. User: The manager types into his PC, "Tell me the latest market trends."

[1926] 2. Terminal: Analyzes the query using natural language processing and sends a request to the server.

[1927] 3. Server: Analyzes the query and extracts data about market trends from the database. It uses GPT-4 to summarize the market trends and visualize them as graphs. It then sends the visualized data to the PC.

[1928] 4. Terminal: Displays graphs of market trends on a PC and provides information to managers.

[1929] 5. User: Managers review the graphs and make appropriate decisions.

[1930] As described above, this system collects data from multiple sources within a company and uses AI models and visualization tools to provide users with appropriate business knowledge and know-how.

[1931] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1932] Step 1:

[1933] Data collection

[1934] The server collects business knowledge and know-how from multiple sources within the company, specifically, using APIs to obtain data from document management systems, databases, internal social networking sites, etc.

[1935] Input: Data from document management systems, databases, and APIs of internal SNS.

[1936] Output: Data stored in a unified data repository.

[1937] Specific operations: For example, retrieve "Business Procedures.pdf" from the document management system, retrieve the latest entries in the "Market Trends" table from the database, and retrieve the message history of the "General" channel from the internal SNS, and store these in the integrated data repository.

[1938] Step 2:

[1939] Data Preprocessing

[1940] The server preprocesses the collected data using the Python Pandas library.

[1941] Input: Data from the Unified Data Repository.

[1942] Output: Cleaned and filtered data.

[1943] Specific Actions: For example, removing empty fields from database entries and fixing inconsistent formatting.

[1944] Step 3:

[1945] Data analysis and summary generation

[1946] The server analyzes the preprocessed data with a generative AI model (e.g., GPT-4) and generates a summary. The summary is generated using a prompt sentence.

[1947] Input: Cleaned and filtered data.

[1948] Output: The summary generated by GPT-4.

[1949] Specific operation: For example, input the prompt sentence "Please summarize the key business steps from the following data" into GPT-4 and obtain a summary.

[1950] Step 4:

[1951] Data Visualization

[1952] The server converts the generated summaries into a visual format using a visualization tool (e.g., Tableau, Power BI).

[1953] Input: The summary generated by GPT-4.

[1954] Output: Visual data such as graphs, charts, and flowcharts.

[1955] Specific actions: For example, visualizing business procedures as a flowchart and displaying Slack message history as a pie chart.

[1956] Step 5:

[1957] Receiving inquiries from users

[1958] Users can request information using devices such as smartphones, PCs, or smart glasses.

[1959] Input: User's query (e.g., "What are the latest market trends?").

[1960] Output: The parsing request sent to the terminal.

[1961] Specific action: For example, a user types "Please tell me the latest market trends" into their smartphone.

[1962] Step 6:

[1963] Natural Language Processing

[1964] The device analyzes user inquiries using natural language processing, specifically using natural language processing libraries (e.g., spaCy, NLTK).

[1965] Input: User's inquiry.

[1966] Output: Analysis results of the query.

[1967] Specific behavior: For example, tokenize the input text and extract key keywords (e.g., "market trends," "latest").

[1968] Step 7:

[1969] Sending a query to the server

[1970] The terminal transmits a request to the server based on the analysis result.

[1971] Input: Analysis results (e.g., "Market Trends", "Latest").

[1972] Output: REST API request to the server.

[1973] Specific operation: Send the analysis results to the server using the REST API.

[1974] Step 8:

[1975] Server information extraction and response generation

[1976] Based on the received inquiry, the server extracts relevant data from an integrated data repository and uses a generative AI model to generate information appropriate to the user's question.

[1977] Input: Data extraction request based on analysis results.

[1978] Output: Information suitable for the user's question.

[1979] Specific operation: For example, input the prompt sentence "Please summarize the latest market trends based on the market data below" into GPT-4 and obtain a summary.

[1980] Step 9:

[1981] Information Visualization

[1982] The server visualizes the generated information using a visualization tool and transmits the results to the terminal.

[1983] Input: Information generated by a generative AI model.

[1984] Output: Visualized data.

[1985] Specific operation: For example, visualize the latest market trends as a bar graph and send it to a PC.

[1986] Step 10:

[1987] Viewing information and providing feedback

[1988] The terminal displays the information received from the server to the user, who then performs the task based on the information provided.

[1989] Input: Visualization data sent from the server.

[1990] Output: Information displayed on the terminal.

[1991] Specific operations: For example, a graph of market trends is displayed on a PC, and the user checks the information and makes a decision.

[1992] (Application example 1)

[1993] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1994] Although logistics centers require the rapid acquisition and sharing of business knowledge and know-how, current systems lack the means to efficiently collect, analyze, visualize, and provide this information. In particular, it is difficult for staff to obtain optimal picking routes and inventory information in real time, which reduces work efficiency. There is a need for a system that can solve these issues and support the efficiency of work and rapid decision-making at logistics centers.

[1995] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1996] In this invention, the server includes means for collecting business knowledge and know-how from multiple data sources within the company, means for analyzing and visualizing the collected data using a generative AI model, means for receiving voice requests from a user terminal, analyzing the requests using natural language processing, and transmitting related information to the server, and means for visualizing the related information transmitted from the server and displaying it on the user terminal in real time using augmented reality technology. This enables logistics center staff to efficiently obtain optimal picking routes and inventory information in real time, enabling them to carry out their work promptly.

[1997] A "data source" is an information source used to collect business knowledge and know-how, such as a document management system, database, or in-house SNS within a company.

[1998] A "generative AI model" is an artificial intelligence model that analyzes collected data and summarizes it in an easy-to-understand format, such as GPT.

[1999] "Visualization" is the process of converting data analyzed by a generative AI model into visual formats such as graphs, charts, and flowcharts, making the information easier for users to understand.

[2000] A "user terminal" is a device used by a user to obtain information, and includes smartphones, PCs, smart glasses, head-mounted displays, etc.

[2001] "Natural language processing" is a technology for analyzing user inquiries and extracting relevant information, and is applied to voice requests and text data.

[2002] "Augmented reality technology" is a technology that displays visualized information overlaid on the real world in real time, and refers to the use of AR (augmented reality).

[2003] A "picking route" is the optimal route for staff to efficiently select products at a logistics center, and is generated based on inventory information.

[2004] "Inventory information" refers to information relating to the type, quantity, location, etc. of products currently held within the logistics center.

[2005] "Real-time" refers to immediate processing and response to user requests without delay.

[2006] This invention provides a system that supports operational efficiency and rapid decision-making in logistics centers. To achieve this, it is important to collect business knowledge and know-how from multiple data sources within a company, analyze and visualize it using a generative AI model, and provide information in real time based on user inquiries.

[2007] System hardware and software configuration

[2008] Server: The central system for data collection, analysis, and visualization. The server must be equipped with a high-performance processor, large memory capacity, and storage. The hardware used can be a general-purpose high-performance server or cloud service. The software used includes a generative AI model such as GPT, an NLP library (e.g., spacy), a database (e.g., MySQL), and a data visualization tool (e.g., D3.js).

[2009] User devices: Devices such as smartphones, smart glasses, and head-mounted displays that users use to obtain information. These devices must have voice input, display, and internet connectivity capabilities.

[2010] Data collection

[2011] The server collects business knowledge and know-how from multiple data sources within the company (document management systems, databases, internal SNS, etc.), obtains the content of each data source via API, and stores and manages it in an integrated data repository.

[2012] Parsing voice requests

[2013] The user device (such as smart glasses) receives the voice request and converts it into text data using a natural language processing library (such as spacy). The analyzed text data is then sent to the server via an API.

[2014] Data analysis and visualization

[2015] Based on the received voice request, the server extracts relevant data from a data repository, uses a generative AI model (e.g., GPT) to generate information relevant to the user's question, and then uses a data visualization tool (e.g., D3.js) to transform the data into visual formats such as graphs, charts, and flowcharts.

[2016] Augmented reality information provision

[2017] The visualized information is sent to the user's device and displayed in real time on the user's device using augmented reality technology (such as ARKit or ARCore). For example, by using smart glasses, logistics center staff can visually navigate and quickly determine the optimal picking route.

[2018] Specific examples

[2019] Example 1: Navigation with voice requests and AR

[2020] A staff member wears smart glasses and asks, "Tell me which shelf I should pick up the next item from." The smart glasses analyze the voice and send a request to the server. The server generates a picking route and sends the visualized information to the smart glasses. The smart glasses then use their AR functionality to navigate the staff member.

[2021] Example 2: Example of a prompt statement

[2022] "Can you tell me which shelf I should pick up next?"

[2023] As described above, using this system will significantly improve operational efficiency at logistics centers and enable faster decision-making.

[2024] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2025] Step 1:

[2026] The user enters a voice request

[2027] The user puts on the smart glasses and asks, "Tell me which shelf I should pick up the next item from." This voice data is the input.

[2028] Step 2:

[2029] Your device converts speech to text

[2030] The microphone in the smart glasses captures the voice data and converts it into text data using a natural language processing library (e.g., spacy). Here, the input is voice data and the output is text data.

[2031] Step 3:

[2032] The device sends text data to the server.

[2033] The terminal sends the parsed text data to the server via the API. The input is the text data, and the output is the request data sent to the server.

[2034] Step 4:

[2035] The server receives the request and parses the data

[2036] The server receives the request data and uses a generative AI model (e.g., GPT) to extract relevant data based on the request from a data repository, where the input is the request data and the output is the relevant data.

[2037] Step 5:

[2038] The server visualizes the relevant data

[2039] The server uses visualization tools (e.g., D3.js) to transform the relevant data into a visual format such as graphs, charts, flowcharts, etc. In this step, the input is the relevant data and the output is the visualized data.

[2040] Step 6:

[2041] The server sends the visualized data to the device.

[2042] The server sends the visualized data to the terminal through the API, where the input is the visualized data and the output is the response data to the terminal.

[2043] Step 7:

[2044] The device receives the visualized data and displays it using augmented reality technology.

[2045] The device displays the received visualization data in real time using augmented reality technology (e.g., ARKit or ARCore), where the input is the response data and the output is the AR display.

[2046] Step 8:

[2047] The user acts on the displayed information

[2048] The user picks up the specified item based on the visual guide displayed on the smart glasses. In this step, the input is the AR display, and the final task execution is the output.

[2049] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2050] This invention combines a system that efficiently shares business knowledge and know-how within a company and supports rapid decision-making with an emotion engine that recognizes user emotions. Specific embodiments of this system are described below.

[2051] 1. Data Collection

[2052] server

[2053] The server collects business knowledge and know-how from multiple data sources within the company, including document management systems, databases, and internal social networking sites, and retrieves content from each data source via API, storing and managing it in an integrated data repository.

[2054] 2. Data analysis and visualization

[2055] server

[2056] The server preprocesses the collected data and analyzes it using a generative AI model (e.g., GPT). The generative AI model summarizes relevant information from the collected data and organizes it into an easy-to-understand format. It then uses visualization tools to transform the data into graphs, charts, flowcharts, and other formats that users can easily understand.

[2057] 3. User Inquiry Processing

[2058] User

[2059] Users use devices (e.g., smartphones, PCs, smart glasses) to request information, for example, by inputting a specific question such as, "What are the latest market trends?"

[2060] Terminal

[2061] When the device receives user input, it analyzes it using natural language processing (NLP) technology and sends a request to the server.

[2062] 4. Server Response Generation

[2063] server

[2064] The server extracts relevant data from a data repository based on the received request, uses generative AI models to generate information relevant to the user's question, and organizes the information using visualization tools.

[2065] 5. Emotion engine integration

[2066] server

[2067] The server is integrated with an emotion engine that analyzes voice input and facial expression data to recognize the user's emotional state. The emotion engine optimizes the information provided based on the user's emotions and generates responses that correspond to the user's state of mind and situation.

[2068] 6. Providing Feedback

[2069] server

[2070] The server generates the optimized information as a response and sends it to the user terminal.

[2071] Terminal

[2072] The terminal processes the response received from the server and displays it to the user, for example by displaying graphs and charts of the latest market trends on a PC screen.

[2073] User

[2074] Users use the information displayed on the device to carry out the necessary tasks, such as when a manager makes decisions based on market information.

[2075] Specific examples

[2076] Example 1: A new employee wants to know the work procedure

[2077] User

[2078] A new employee types "Please tell me the work procedures" into their smartphone.

[2079] Terminal

[2080] The terminal sends this query to the server.

[2081] server

[2082] The server analyzes the query and extracts relevant data. A generative AI model summarizes the steps and visualizes them as a flowchart.

[2083] Emotion Engine

[2084] The server uses an emotion engine to analyze the user's voice and facial expression data while operating the device, and if the user appears anxious or confused, it responds by adding detailed instructions or supplementary information.

[2085] server

[2086] Finally, the server transmits the flowchart and supplementary information to the user terminal.

[2087] Terminal

[2088] Display flowcharts and supplemental information on smartphones to educate new employees.

[2089] User

[2090] New employees check the flowchart on their smartphones and understand the work procedures.

[2091] Example 2: Manager wants to know market trends

[2092] User

[2093] The manager types into his PC, "Tell me about the latest market trends."

[2094] Terminal

[2095] The terminal sends this query to the server.

[2096] server

[2097] The server analyzes the query and extracts data related to market trends. A generative AI model summarizes the market trends and visualizes them as graphs and charts.

[2098] Emotion Engine

[2099] The server analyzes the manager's emotions and optimizes the presentation of concise and intuitive information if the user is nervous.

[2100] server

[2101] Send visualized market trends graphs and charts to your device.

[2102] Terminal

[2103] Display graphs and charts on a PC to provide information to managers.

[2104] User

[2105] Managers check market trends on their PCs and make appropriate decisions.

[2106] The processing flow will be explained below.

[2107] Step 1:

[2108] User

[2109] The user uses a device (e.g., a smartphone or PC) to input the information they want to know and query the system. For example, the user might input "Tell me the latest market trends" into their PC.

[2110] Step 2:

[2111] Terminal

[2112] The terminal receives the user's input and generates a request to send to the server, which includes the user's question, the user's identification information, and information about the terminal.

[2113] Step 3:

[2114] server

[2115] The server analyzes the request received from the device using natural language processing (NLP) techniques, such as tokenization, stemming, and entity recognition.

[2116] Step 4:

[2117] server

[2118] The server searches and extracts relevant data from the data repository based on the query content analyzed by NLP.

[2119] Step 5:

[2120] server

[2121] The extracted data is fed into a generative AI model (e.g., GPT) to generate and summarize relevant information, which is organized into answers to user questions.

[2122] Step 6:

[2123] server

[2124] The generated information is converted into graphs, charts, and other formats using visualization tools, which makes it easier for users to understand the information.

[2125] Step 7:

[2126] server

[2127] The emotion engine analyzes the user's emotional state when making an inquiry. It uses voice input and a camera to collect facial expression data and analyzes the user's emotional state.

[2128] Step 8:

[2129] server

[2130] Based on the analysis results of the emotion engine, a generative AI model generates an optimized response, for example, if the user is nervous, it will provide a concise and reassuring answer.

[2131] Step 9:

[2132] server

[2133] The visualized information and the optimized response are generated as a response and transmitted to the user terminal.

[2134] Step 10:

[2135] Terminal

[2136] The terminal processes the response received from the server and displays it to the user, for example, by displaying graphs and charts of the latest market trends on a PC screen.

[2137] Step 11:

[2138] User

[2139] Users use the information displayed on the terminal to carry out necessary tasks, such as when a manager makes decisions based on market information.

[2140] Specific examples

[2141] Example 1: A new employee wants to know the work procedure

[2142] Step 1:

[2143] User

[2144] A new employee types "Please tell me the work procedures" into their smartphone.

[2145] Step 2:

[2146] Terminal

[2147] This query is sent to the server.

[2148] Step 3:

[2149] server

[2150] The received request is analyzed using NLP technology and data related to business procedures is extracted from the data repository.

[2151] Step 4:

[2152] server

[2153] The data is summarized using a generative AI model and visualized in flowchart format.

[2154] Step 5:

[2155] server

[2156] The emotion engine analyzes the voice and facial expression data of new employees, and if they appear anxious, detailed supplementary information is added.

[2157] Step 6:

[2158] server

[2159] Optimized information is sent to your smartphone.

[2160] Step 7:

[2161] Terminal

[2162] View the flowchart and supplemental information on your smartphone.

[2163] Step 8:

[2164] User

[2165] New employees review the information and understand the work procedures.

[2166] Example 2: Manager wants to know market trends

[2167] Step 1:

[2168] User

[2169] The manager types into his PC, "Tell me about the latest market trends."

[2170] Step 2:

[2171] Terminal

[2172] This query is sent to the server.

[2173] Step 3:

[2174] server

[2175] Analyze inquiries and extract data related to market trends.

[2176] Step 4:

[2177] server

[2178] Generative AI models summarize market trends and visualize them as graphs and charts.

[2179] Step 5:

[2180] server

[2181] The emotion engine analyzes the manager's emotions and provides concise information if they are nervous.

[2182] Step 6:

[2183] server

[2184] The visualized data is sent to a PC.

[2185] Step 7:

[2186] Terminal

[2187] Display graphs and charts on your PC.

[2188] Step 8:

[2189] User

[2190] The manager reviews the information and makes a decision.

[2191] Example 2

[2192] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2193] In conventional systems for sharing business knowledge and know-how within companies, the collection, analysis, and visualization of information are automated to a certain extent, but they are unable to provide information that takes into account the user's emotional state, which means that the specific information and supplementary explanations that users are looking for are insufficient, making it difficult to support effective decision-making.In particular, when users are feeling anxious or confused, it is necessary to provide appropriate information, but this has not been fully realized.

[2194] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2195] In this invention, the server includes means for collecting business knowledge and know-how from multiple data sources within the company, means for analyzing and visualizing the collected data using a generative AI model, means for receiving inquiries from users and providing analyzed information to the user terminal, means for integrating an emotion engine that analyzes voice input and facial expression data to recognize the emotional state of the user, and means for optimizing the information to be provided based on the emotional state of the user. This enables the provision of optimal information taking into account the user's emotions, thereby supporting effective knowledge sharing and rapid decision-making within the company.

[2196] "Multiple data sources within a company" refers to various information sources used within a company, such as document management systems, databases, and internal social networking sites.

[2197] "Business knowledge and know-how" refers to the specialized knowledge, experience, procedures and best practices a company needs to carry out its day-to-day business operations.

[2198] A "generative AI model" is an artificial intelligence model that has the ability to learn patterns from large amounts of data and generate new data, and specifically refers to natural language processing models such as GPT-4.

[2199] "Analysis and visualization" refers to a set of methods and techniques aimed at processing collected data and presenting the information in an understandable format, including, for example, generating graphs and charts.

[2200] "User inquiries" refer to questions or requests made to the system by users entering information or questions in natural language.

[2201] A "user terminal" is a hardware device that allows a user to use the system, and examples include smartphones, PCs, and smart glasses.

[2202] "Natural language processing" is a technology that enables computers to understand and generate human language, and refers to the process of analyzing text and audio data and extracting meaning.

[2203] "Emotion engine" refers to technology that recognizes a user's emotional state and generates an appropriate response based on that emotion.

[2204] "Voice input and facial expression data" refers to data used to analyze the user's voice and facial expressions, and is important input information used by the emotion engine.

[2205] "Optimization" refers to the process of adjusting the parameters and behavior of a system to achieve optimal results for a specific purpose.

[2206] This invention combines a system that efficiently shares business knowledge and know-how within a company and supports rapid decision-making with an emotion engine that recognizes user emotions.

[2207] System configuration

[2208] Data collection

[2209] server

[2210] The server collects business knowledge and know-how from multiple data sources within the company. These include document management systems, databases, and internal SNS. Data is obtained from each data source via API and stored and managed in an integrated data repository. Specifically, a general "document management system" is used as the document management system, a "database management system" as the database, and an "internal SNS system" as the internal SNS. A "database" is used as the integrated data repository.

[2211] Data analysis and visualization

[2212] server

[2213] The server preprocesses the collected data and analyzes it using a generative AI model (e.g., generative AI model). The generative AI model summarizes relevant information from the collected data and organizes it in an easy-to-understand format. It also uses visualization tools to convert the data into graphs, charts, flowcharts, and other formats. The visualization tools used are called "visualization tools."

[2214] Processing user inquiries

[2215] User

[2216] Users use devices (e.g., smartphones, PCs, smart glasses) to request information, for example, by inputting a specific question such as, "What are the latest market trends?"

[2217] Terminal

[2218] When the device receives user input, it analyzes the content using natural language processing technology and sends a request to the server. Natural language processing is performed using a "natural language processing system."

[2219] Server response generation

[2220] server

[2221] Based on the received request, the server extracts relevant data from a data repository, uses a generative AI model (e.g., a generative AI model) to generate information relevant to the user's question, and organizes the information using visualization tools.

[2222] Emotion engine integration

[2223] server

[2224] The server is integrated with an emotion engine that analyzes voice input and facial expression data to recognize the user's emotional state. The emotion engine uses an "emotion recognition system" to optimize the information provided according to the user's emotions.

[2225] Providing Feedback

[2226] server

[2227] The server generates the optimized information as a response and sends it to the user terminal.

[2228] Terminal

[2229] The terminal processes the response received from the server and displays it to the user, for example by displaying graphs and charts of the latest market trends on a PC screen.

[2230] User

[2231] Users use the information displayed on their devices to carry out the necessary tasks, such as when a manager makes decisions based on market information.

[2232] Specific examples

[2233] Example 1: A new employee wants to know the work procedure

[2234] User

[2235] A new employee types "Please tell me the work procedures" into their smartphone.

[2236] Terminal

[2237] The terminal sends this query to the server.

[2238] server

[2239] The server analyzes the query and extracts relevant data. A generative AI model summarizes the steps and visualizes them as a flowchart.

[2240] Emotion Engine

[2241] The server uses an emotion engine to analyze the user's voice and facial expression data, and if the user feels anxious or confused, it adds detailed instructions or supplementary information.

[2242] server

[2243] Finally, the server transmits the flowchart and supplementary information to the user terminal.

[2244] Terminal

[2245] Display flowcharts and supplemental information on smartphones to educate new employees.

[2246] User

[2247] New employees check the flowchart on their smartphones and understand the work procedures.

[2248] Example 2: Manager wants to know market trends

[2249] User

[2250] The manager types into his PC, "Tell me about the latest market trends."

[2251] Terminal

[2252] The terminal sends this query to the server.

[2253] server

[2254] The server analyzes the query and extracts data related to market trends. A generative AI model summarizes the market trends and visualizes them as graphs and charts.

[2255] Emotion Engine

[2256] The server analyzes the manager's emotions and optimizes to provide concise and intuitive information if the manager is nervous.

[2257] server

[2258] Send visualized market trends graphs and charts to your device.

[2259] Terminal

[2260] Display graphs and charts on a PC to provide information to managers.

[2261] User

[2262] Managers check market trends on their PCs and make appropriate decisions.

[2263] Prompt Sentence Examples

[2264] "Tell me the business procedures."

[2265] "What are the latest market trends?"

[2266] keyword

[2267] Generative AI model, prompt sentence

[2268] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2269] Step 1:

[2270] Data collection

[2271] server

[2272] Input: Multiple data sources within the company (e.g. document management systems, databases, internal SNS)

[2273] Output: Business knowledge and know-how stored in a unified data repository

[2274] The server collects business knowledge and know-how from multiple data sources within the company. Specifically, it periodically sends requests to the API endpoint of each data source, receives the retrieved data in JSON format, and temporarily stores it in memory. The data in memory is then written to the "database management system."

[2275] Step 2:

[2276] Data analysis and visualization

[2277] server

[2278] Input: Data stored in a unified data repository

[2279] Output: Summarized information and visualized graphs and charts

[2280] The server preprocesses the collected data and analyzes it using a generative AI model (e.g., "generative AI model"). Preprocessing includes cleaning the data (removing unnecessary strings and standardizing formatting). Next, the "generative AI model" is used to extract and summarize important information and save it as a new entry. Finally, the extracted data is visualized using a "visualization tool" to generate graphs and charts.

[2281] Step 3:

[2282] Processing user inquiries

[2283] User

[2284] Input: A specific question (e.g., "What are the latest market trends?")

[2285] Output: None (user input is sent to the server)

[2286] A user uses a device (e.g., a smartphone, PC, or smart glasses) to request information. The user enters a question into an input field on the device. For example, the user enters a specific question such as, "Please tell me about the latest market trends."

[2287] Terminal

[2288] Input: User question

[2289] Output: Parsed user question sent to server

[2290] When the device receives user input, it analyzes the content using natural language processing technology. It uses a "natural language processing system" as a natural language processing tool, and sends the analyzed data to the server in the form of an HTTP request.

[2291] Step 4:

[2292] Server response generation

[2293] server

[2294] Input: The request sent from the terminal (the user's question)

[2295] Output: Generated response data and visualizations

[2296] The server extracts relevant data from a data repository based on the received request, uses a "generative AI model" to generate information appropriate for the user's question, and organizes and visualizes the information using a "visualization tool."

[2297] Step 5:

[2298] Emotion engine integration

[2299] server

[2300] Input: Voice input and facial expression data

[2301] Output: Optimized response based on the user's emotional state

[2302] The server is integrated with an emotion engine (e.g., an "emotion recognition system") that analyzes voice input and facial expression data to recognize the user's emotional state, and adjusts the information content and visualization format provided based on the analysis results.

[2303] Step 6:

[2304] Providing Feedback

[2305] server

[2306] Input: Optimized Information

[2307] Output: Information sent to the user's device

[2308] The server generates the optimized information as a response and sends it to the user terminal.

[2309] Terminal

[2310] Input: The response received from the server

[2311] Output: Information displayed to the user (e.g., graphs and charts)

[2312] The terminal processes the response received from the server and displays it to the user, for example, by displaying graphs and charts of the latest market trends on a PC screen.

[2313] User

[2314] Input: The information displayed on the terminal

[2315] Output: User decisions or actions

[2316] Users use the information displayed on their devices to carry out the necessary tasks, such as when a manager makes decisions based on market information.

[2317] (Application example 2)

[2318] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2319] In systems that efficiently share business knowledge and know-how within a company and support rapid decision-making, current technology faces the challenge of not being able to provide optimal information that takes user emotions into account. Furthermore, in situations where stress levels are particularly high, there is a risk of work efficiency declining, so there is a demand for technology that can analyze emotions in real time and optimize responses.

[2320] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting business knowledge and know-how from multiple data sources within the company, means for analyzing and visualizing the collected data using a generative AI model, means for receiving inquiries from users and providing analyzed information to the user terminal, means for recognizing the user's face and analyzing their emotions using a camera, and means for generating and providing optimal information based on the user's emotions. This makes it possible to provide quick and optimal information that takes the user's emotions into consideration.

[2321] "Data source" refers to various information sources used within a company to collect business knowledge and know-how.

[2322] A "generative AI model" is an artificial intelligence model used to analyze collected data and generate useful information.

[2323] "User terminal" refers to any device that a user uses to make inquiries and receive responses in order to use the system.

[2324] "Camera" refers to the imaging device used to recognize and analyze the User's face.

[2325] "Facial recognition" refers to the technology of detecting and identifying the face of a specific person from an image captured by a camera.

[2326] "Emotion analysis" refers to a technology that determines a user's emotional state based on facial and voice data.

[2327] "Visualization tools" refers to software or technology used to transform analyzed information into visual formats such as graphs and charts.

[2328] "Natural language processing" refers to the technology of analyzing human language and understanding its meaning using generative AI models.

[2329] "Optimization" refers to the process of optimally adjusting the information and responses provided based on the user's emotions and state.

[2330] The present invention combines a system that efficiently shares business knowledge and know-how within a company and supports rapid decision-making with an emotion engine that recognizes user emotions. Specific embodiments are described below.

[2331] First, the server collects business knowledge and know-how from multiple data sources within the company, including document management systems, databases, and internal social networking sites. The server retrieves content from each data source via API and stores and manages it in an integrated data repository.

[2332] The server then preprocesses the collected data and analyzes it using a generative AI model (e.g., GPT-3). The generative AI model summarizes relevant information from the collected data and organizes it into an easy-to-understand format. It then uses visualization tools to transform the data into graphs, charts, flowcharts, and other formats that users can easily understand.

[2333] A user uses a device (such as smart glasses) to request information, for example, by entering a specific question such as "What are the latest market trends?" The device receives the user's input, analyzes it using natural language processing (NLP) techniques, and sends a request to the server.

[2334] Based on the received request, the server extracts relevant data from a data repository, uses a generative AI model to generate information appropriate to the user's question, and organizes and visualizes the information using visualization tools.

[2335] In addition, the server is integrated with an emotion engine, which performs facial recognition and emotion analysis through the smart glasses' camera to recognize the user's emotional state. The emotion engine analyzes voice input and facial expression data and optimizes the information provided based on the user's emotions. For example, if the user is feeling anxious or nervous, it will provide more detailed information or concise, intuitive information.

[2336] The optimized information is generated as a response and sent to the user's device. The device processes the response received from the server and displays it to the user. For example, the smart glasses display could show graphs and charts of the latest market trends. The user can then use the information displayed on the device to make appropriate decisions.

[2337] Examples:

[2338] As the security guard was making his rounds, the camera in his smart glasses began scanning his face and noticed he was nervous. As a result, the system immediately sent him a notification detailing a potential threat and a brief explanation of the steps he should take immediately. The prompt read, "The security guard is nervous, please provide a quick response."

[2339] As described above, by combining a server, user devices, emotion engines, generative AI models, visualization tools, etc., it becomes possible to provide fast and optimal information that takes into account the user's emotions.

[2340] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2341] Step 1:

[2342] The server collects business knowledge and know-how from multiple data sources within the company (document management systems, databases, internal SNS, etc.), obtains information from each data source via API, and stores and manages the obtained data in an integrated data repository.

[2343] Input: Raw data obtained from the API of each data source

[2344] Output: Business knowledge and know-how stored in a unified data repository

[2345] Step 2:

[2346] The server preprocesses the data stored in the integrated data repository and analyzes it using a generative AI model (such as GPT-3). The generative AI model summarizes relevant information from the collected data and organizes it into an easy-to-understand format. It then uses visualization tools to transform the data into graphs, charts, flowcharts, and other formats.

[2347] Input: Raw data from the integrated data repository

[2348] Output: Analyzed and visualized data (graphs, charts, etc.)

[2349] Step 3:

[2350] A user uses a device (such as smart glasses) to inquire about the information they want to know, for example, by inputting a specific question such as "Please tell me the latest market trends."

[2351] Input: User question

[2352] Output: The query is parsed and the request sent to the server.

[2353] Step 4:

[2354] The device analyzes the user's input using natural language processing (NLP) technology and sends the content to the server.

[2355] Input: User question

[2356] Output: Request sent to the server

[2357] Step 5:

[2358] Based on the received request, the server extracts relevant data from an integrated data repository, uses a generative AI model to generate information appropriate to the user's question, and organizes and visualizes the information using visualization tools.

[2359] Input: Natural language processed user question

[2360] Output: Specific responses to user questions (graphs, charts, etc.)

[2361] Step 6:

[2362] The server is integrated with an emotion engine that recognizes the user's face through the smart glasses' camera and performs real-time emotion analysis, determining the user's emotional state based on voice input and facial expression data.

[2363] Input: User's face data acquired by camera and voice input

[2364] Output: Parsed user's emotional state

[2365] Step 7:

[2366] The server optimizes the information it provides based on the user's emotional state analyzed by the emotion engine, generating detailed or concise information as needed.

[2367] Input: User's emotional state

[2368] Output: Specific responses optimized based on emotions

[2369] Step 8:

[2370] The server sends the optimized response to the user terminal.

[2371] Input: Emotion-based optimized response

[2372] Output: The optimized response sent to the user's device

[2373] Step 9:

[2374] The terminal processes the response received from the server and displays it to the user, for example by displaying graphs and charts of the latest market trends on the display of smart glasses.

[2375] Input: The response received from the server

[2376] Output: Analysis results displayed to the user (graphs, charts, etc.)

[2377] Step 10:

[2378] Users use the information displayed on their devices to perform their tasks and make appropriate decisions. For example, while a security guard was on patrol, the camera in his smart glasses began scanning his face and recognized that he was nervous. As a result, the system immediately sent a notification to him, notifying him of a possible threat and briefly explaining the steps that should be taken immediately. The prompt read, "The security guard is nervous, so please provide a quick response."

[2379] Input: Analysis results displayed to the user

[2380] Output: Business execution and decision-making based on analysis results

[2381] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2382] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2383] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2384] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2385] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2386] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2387] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2388] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2389] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2390] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2391] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2392] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2393] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2394] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2395] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2396] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2397] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2398] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2399] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2400] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2401] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2402] The following is further disclosed regarding the above embodiment.

[2403] (Claim 1)

[2404] A means of collecting business knowledge and know-how from multiple data sources within the enterprise;

[2405] A means of analyzing and visualizing the collected data using generative AI models; and

[2406] means for receiving an inquiry from a user and providing analyzed information to a user terminal;

[2407] A system including:

[2408] (Claim 2)

[2409] The system according to claim 1, further comprising means for performing natural language processing using a generative AI model based on the user's inquiry and extracting relevant information.

[2410] (Claim 3)

[2411] The system of claim 1, further comprising means for converting the analyzed information into a visual format such as a graph or chart using a visualization tool.

[2412] (Claim 4)

[2413] The system of claim 1, wherein the user terminal is compatible with a variety of devices such as smartphones, personal computers, and smart glasses.

[2414] (Claim 5)

[2415] The system of claim 1, wherein the generative AI model constitutes a question-answering system based on natural language processing and further includes means for understanding a user's intent with high accuracy.

[2416] "Example 1"

[2417] (Claim 1)

[2418] A means of gathering business knowledge and know-how from multiple sources within the enterprise;

[2419] a means of storing and managing the collected data in a unified data repository;

[2420] a means for analyzing and summarizing the collected data using a generative AI model; and

[2421] means for converting the analyzed data into a visual format using a visualization tool;

[2422] means for receiving an inquiry from a user and analyzing it using natural language processing;

[2423] means for transmitting the parsed information to a server as a request;

[2424] means for providing the analysis results to a user terminal;

[2425] A system including:

[2426] (Claim 2)

[2427] 10. The system of claim 1, further comprising means for extracting relevant information based on a user query using the generative AI model.

[2428] (Claim 3)

[2429] The system of claim 1, further comprising means for converting the analyzed information into a visual format such as a graph or chart using a visualization tool.

[2430] "Application Example 1"

[2431] (Claim 1)

[2432] A means of collecting business knowledge and know-how from multiple data sources within the enterprise;

[2433] A means of analyzing and visualizing the collected data using generative AI models; and

[2434] means for receiving an inquiry from a user and providing analyzed information to a user terminal;

[2435] means for receiving a voice request from a user terminal, analyzing the request using natural language processing, and transmitting related information to a server;

[2436] a means for visualizing the related information transmitted from the server and displaying it on the user terminal in real time using augmented reality technology;

[2437] A system including:

[2438] (Claim 2)

[2439] The system according to claim 1, further comprising means for performing natural language processing using a generative AI model based on the user's inquiry and extracting relevant information.

[2440] (Claim 3)

[2441] The system according to claim 1, further comprising means for converting the analyzed information into a visual format such as a graph or chart using a visualization tool and displaying it on a user terminal using augmented reality technology.

[2442] "Example 2: Combining Emotion Engines"

[2443] (Claim 1)

[2444] A means of collecting business knowledge and know-how from multiple data sources within the enterprise;

[2445] A means of analyzing and visualizing the collected data using generative AI models; and

[2446] means for receiving an inquiry from a user and providing analyzed information to a user terminal;

[2447] a means for integrating an emotion engine that analyzes voice input and facial expression data to recognize the user's emotional state;

[2448] means for optimizing the information provided based on the emotional state of the user;

[2449] A system including:

[2450] (Claim 2)

[2451] The system according to claim 1, further comprising means for performing natural language processing using a generative AI model based on the user's inquiry and extracting relevant information.

[2452] (Claim 3)

[2453] The system of claim 1, further comprising means for converting the analyzed information into a visual format such as a graph or chart using a visualization tool.

[2454] "Application example 2 when combining emotion engines"

[2455] (Claim 1)

[2456] A means of collecting business knowledge and know-how from multiple data sources within the enterprise;

[2457] A means of analyzing and visualizing the collected data using generative AI models; and

[2458] means for receiving an inquiry from a user and providing analyzed information to a user terminal;

[2459] means for recognizing a user's face and performing emotion analysis using a camera;

[2460] A means for generating and providing optimal information based on the user's emotions;

[2461] A system including:

[2462] (Claim 2)

[2463] The system according to claim 1, further comprising means for performing natural language processing using a generative AI model based on the user's inquiry and extracting relevant information.

[2464] (Claim 3)

[2465] The system of claim 1, further comprising means for converting the analyzed information into a visual format such as a graph or chart using a visualization tool. [Explanation of symbols]

[2466] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting business knowledge and know-how from multiple data sources within the enterprise; A means of analyzing and visualizing the collected data using generative AI models; and means for receiving an inquiry from a user and providing analyzed information to a user terminal; A system including:

2. The system according to claim 1, further comprising means for performing natural language processing using a generative AI model based on the user's inquiry and extracting relevant information.

3. The system of claim 1 , further comprising means for converting the analyzed information into a visual format such as a graph or chart using a visualization tool.

4. The system of claim 1 , wherein the user terminal supports a variety of devices such as smartphones, personal computers, and smart glasses.

5. The system according to claim 1 , wherein the generative AI model constitutes a question-answering system based on natural language processing, and further includes means for understanding a user's intention with high accuracy.

Images