system

Abstract

We provide the system. [Solution] A means for acquiring data including person information and business information, A means of analyzing acquired data and modeling the correlation between people and tasks, A means for generating a visual correlation diagram based on modeled correlations, A means for displaying the generated correlation diagram, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] With the merger and expansion of organizations, the number of new tasks and collaborative operations increases, and the efficiency of business operations tends to decline. In addition, with the generalization of remote work, there is a problem that it becomes more difficult to collaborate on business and to understand the relevant parties. In such an environment, a reliable tool for organizing and efficiently progressing business operations is required, but the current tools do not centralize data, and it is difficult to visually grasp the correlation between relevant parties and tasks.

Means for Solving the Problems

[0005] To solve the above problems, the present invention provides the following means: a system comprising means for acquiring data including person information and business information, means for analyzing the acquired data and modeling the correlation between people and business, means for generating a visual correlation diagram based on the modeled correlation, and means for displaying the generated correlation diagram. Furthermore, based on the analyzed data, the system includes means for displaying information about a node in a pop-up when the cursor is placed over that node on the correlation diagram, and means for receiving feedback from the user based on the generated correlation diagram, saving that feedback in a database, and considering it in the next data analysis. With such a system, users can visually grasp the coordination between stakeholders and business, and improve the efficiency of their work.

[0006] "Personal information" refers to information used to identify and specify a user, such as the user's name, job title, contact information, and profile picture.

[0007] "Business information" refers to information about the details and progress of a task, such as the task name, deadline, and progress status.

[0008] "Means of acquiring data" refers to the methods and processes by which a system collects necessary information from a specified data source.

[0009] "Means of data analysis" refer to methods and processes for identifying and modeling the relationship between people and tasks based on acquired data.

[0010] "Modeling correlations" means representing the relationship between people and tasks in a format such as a graph data structure.

[0011] A "correlation diagram" is a diagram that visually displays the correlation between people and tasks, and is a graph composed of nodes and edges.

[0012] A "node" is an element in a correlation diagram that represents a person or task.

[0013] An "edge" is a line in a correlation diagram that shows the relationship between nodes.

[0014] A "pop-up display" is a temporary display of additional detailed information that appears when a user performs a specific action (for example, hovering over an item with the cursor).

[0015] "Feedback" refers to the reactions and inputs that users provide to a system, such as opinions, evaluations, and suggestions for improvements.

[0016] A "database" is a system for efficiently storing collected and analyzed data, and for searching and retrieving it as needed.

[0017] A "server" is a computer system that collects, analyzes, generates correlation diagrams, and stores user feedback.

[0018] A "terminal" is a computer or device used by a user to view and manipulate correlation diagrams. [Brief explanation of the drawing]

[0019] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7]It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0020] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0021] First, the language used in the following description will be explained.

[0022] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0023] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0024] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0025] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0027] [First Embodiment]

[0028] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0029] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0032] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0035] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0039] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0040] To implement this invention, the system should be constructed and operated according to the following procedure.

[0041] 1. Data Collection Phase

[0042] server

[0043] The server makes API calls to various internal systems (e.g., email server, project management tool, calendar system) to retrieve personnel and business information.

[0044] The server authenticates with each system and secures the necessary access rights. The data retrieved by API calls includes user ID, task details, stakeholders, date and time, etc.

[0045] The acquired data is organized and stored in a central database. Each data entry undergoes validation to maintain data consistency and integrity.

[0046] 2. Correlation Diagram Generation Phase

[0047] server

[0048] The server periodically scans the database to retrieve the latest personnel and business information.

[0049] The analysis module analyzes the data and models the correlation between people and tasks as a graph structure (nodes and edges).

[0050] Based on the graph data, a visual correlation diagram is generated. This correlation diagram can be exported in JSON or XML format and used for display on the client side.

[0051] 3. UI / UX Phase

[0052] terminal

[0053] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server.

[0054] The device uses libraries such as D3.js and Cytoscape.js to display correlation diagrams. This allows users to view correlation diagrams that are easy to understand visually.

[0055] When you hover over a node, relevant information (such as the person's name, job title, and task details) pops up. If the user clicks on a node, even more detailed information is displayed, including past collaboration history and comments.

[0056] 4. Feedback Loop

[0057] User

[0058] Users provide feedback, evaluations, and suggestions for improvement to the system through a feedback form. This is treated as specific improvement requests to enhance operational efficiency.

[0059] The server stores the feedback in a database and considers it during subsequent data analysis and correlation diagram generation. This allows the system to continuously improve, increasing its accuracy and usefulness.

[0060] Specific example

[0061] New task and integration scenario

[0062] User A (Project Manager)

[0063] A new project is launched, and User A is appointed as the project manager. User A assigns Users B and C as team members.

[0064] User A assigns various tasks to each member through a project management tool.

[0065] Data collection

[0066] server

[0067] Retrieve the latest task information for User A and their team members from the mail server, project management tools, and calendar system.

[0068] The data is stored in a database.

[0069] Correlation diagram generation

[0070] server

[0071] The analysis module analyzes the acquired data and generates nodes (User A, User B, User C) and edges (task connections).

[0072] A visual correlation diagram is generated and exported in JSON format.

[0073] Displaying information

[0074] terminal

[0075] User A logs into the system and views the correlation diagram. In the correlation diagram, User A is displayed as the project manager, and Users B and C are displayed as members.

[0076] When user A hovers the cursor over a node, its task details and related information are displayed in a pop-up window.

[0077] feedback

[0078] User A

[0079] User A provides feedback on their experience with the system and suggestions for improvement through a feedback form. This allows the system to be improved to be more user-friendly.

[0080] By implementing this invention, coordination of operations and identification of stakeholders become easier, leading to a significant improvement in operational efficiency.

[0081] The following describes the processing flow.

[0082] Step 1:

[0083] server

[0084] The server makes API calls to internal systems (e.g., email server, project management tool, calendar system) to establish a connection. It then obtains access rights using authentication information (API key, OAuth token) for each system.

[0085] Step 2:

[0086] server

[0087] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time via API calls. To retrieve only the most recent data, it records the date and time of the previous data retrieval and collects differential data.

[0088] Step 3:

[0089] server

[0090] The acquired data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is removed.

[0091] Step 4:

[0092] server

[0093] The server periodically scans the database to retrieve the latest personnel and business information. This ensures that the data necessary for generating the correlation diagram is always up-to-date.

[0094] Step 5:

[0095] server

[0096] The analysis module is used to analyze the data and model the relationship between people and tasks in a graph structure. People and tasks are set as nodes, and their relationships are defined as edges.

[0097] Step 6:

[0098] server

[0099] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[0100] Step 7:

[0101] terminal

[0102] The terminal retrieves the latest correlation diagram data from the server when the user logs into the system. The data is received in JSON format and parsed (analyzed) on the client side.

[0103] Step 8:

[0104] terminal

[0105] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[0106] Step 9:

[0107] terminal

[0108] When a user hovers over a node, detailed information about that node (e.g., name, role, task details) pops up. Clicking on it displays even more detailed information (e.g., past collaboration history, comments).

[0109] Step 10:

[0110] User

[0111] Users view correlation diagrams and confirm the necessary information. Furthermore, they can provide feedback and suggestions for improvement to the system through a feedback form.

[0112] Step 11:

[0113] server

[0114] The server saves user feedback to a database. This feedback is then taken into consideration during the next data analysis, and improvements are implemented accordingly.

[0115] Step 12:

[0116] server

[0117] In the next data analysis and correlation diagram generation cycle, the saved feedback will be referenced, and necessary adjustments and improvements will be made. This allows the system to be continuously improved and optimized.

[0118] (Example 1)

[0119] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0120] Conventional systems made it difficult to grasp personnel and work information within companies, posing challenges to improving operational efficiency. Furthermore, data analysis for visually displaying the correlation between personnel and work, and interactive information display based on those results, were insufficient. As a result, it was difficult for users to intuitively grasp the status of work and make appropriate decisions.

[0121] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0122] In this invention, the server includes means for acquiring person information and business information via an API, means for authenticating the acquired data and securing access rights, and means for validating the acquired data and storing it in a central database. This makes it possible to accurately acquire necessary data and store and manage it safely and efficiently. It also includes means for an analysis module that periodically scans the database and analyzes the latest person information and business information, means for modeling the analyzed data as a graph structure, means for generating a visual correlation diagram based on the modeled correlations and exporting it in JSON or XML format, and means for acquiring the generated correlation diagram and displaying the visual correlation diagram. This makes it easier for users to intuitively understand the correlation between people and business, enabling them to make appropriate decisions quickly. Furthermore, it includes means for displaying information about a node in a pop-up when the cursor is placed over a node on the correlation diagram, and means for saving user feedback in the database and considering it during the next data analysis. This allows the system to be continuously improved and enables the provision of highly accurate information that meets the needs of users.

[0123] "Personal information" refers to identifiable information about a specific individual, including name, job title, contact information, and department.

[0124] "Business information" refers to information about business operations and tasks performed within a company, including task details, deadlines, assigned personnel, and related projects.

[0125] "API" stands for Application Programming Interface, and refers to a standardized interface that enables data exchange between different software systems.

[0126] Authentication is the process of verifying a user's identity when accessing an information system, and it primarily uses a username, password, token, etc.

[0127] "Access rights" refer to the operational permissions for data and resources within an information system, and include permissions such as read, write, and execute.

[0128] "Validation" is the process of verifying whether the acquired data conforms to predetermined standards and formats.

[0129] A "central database" is a database used to centrally store and manage all acquired data, and examples include SQL databases and NoSQL databases.

[0130] "Periodic scanning" is the process of checking, updating, and retrieving data within a database at regular time intervals.

[0131] An "analysis module" is a software component used to analyze collected data and extract specific patterns or relationships.

[0132] A "graph structure" is a data structure consisting of nodes (vertices) and edges, and is used to model the relationship between people and tasks.

[0133] A "visual correlation diagram" is a diagram that visually represents a graph structure, allowing for an intuitive understanding of the relationship between people and tasks.

[0134] "JSON" is an abbreviation for JavaScript (registered trademark) Object Notation, and is a lightweight data exchange format for structuring, storing, and exchanging data.

[0135] XML stands for eXtensible Markup Language, and it is a markup language used to structure, store, and exchange data.

[0136] A "node" refers to a vertex in a graph structure, and usually represents an entity such as a person or a task.

[0137] "Popup display" is a feature that displays a small window containing supplementary information in response to a specific action (for example, mouseover).

[0138] "Feedback" refers to opinions, suggestions for improvement, and evaluations provided by system users, which are used to improve the system.

[0139] "Data analysis" is the process of extracting patterns and relationships from collected data using statistical methods and algorithms to deepen our understanding of the data.

[0140] Modes for carrying out the invention

[0141] This invention is a system that acquires personnel and business information from various internal information systems, analyzes them, and generates and displays a correlation diagram that visually represents the relationships between them. The following describes a specific implementation of this system.

[0142] Data collection phase

[0143] server

[0144] The server retrieves data from the company's internal email system, project management system, and calendar system via APIs. This utilizes general-purpose APIs such as Google® Calendar API, Microsoft® Exchange API, and JIRA API.

[0145] Specifically, this involves sending API requests to retrieve user and task information. For example, you can access the API endpoint using the "curl" command and extract the necessary data.

[0146] The acquired data is authenticated using OAuth 2.0 or similar methods to ensure appropriate access rights. This authentication guarantees the security and reliability of the data.

[0147] The server validates the retrieved data and stores only the data that conforms to the schema in the central database. The database uses relational databases such as MySQL® or PostgreSQL.

[0148] Correlation diagram generation phase

[0149] server

[0150] The server periodically uses a job scheduler (e.g., a Cron job) to scan the database. This scan retrieves the latest personnel and business information, preparing it for analysis.

[0151] The server uses an analysis module (for example, the NetworkX library in Python) to analyze the person and task information in the database. As a result of the analysis, a graph structure is generated in which each person is a node and each task is an edge.

[0152] The server renders the generated graph data as a visual correlation diagram and exports it in JSON or XML format. This export is then used for display on the client side.

[0153] UI / UX Phase

[0154] terminal

[0155] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. AJAX requests are used to retrieve the data.

[0156] On the user's device, JavaScript libraries such as D3.js and Cytoscape.js are used to display correlation diagrams based on the acquired data. This allows users to intuitively understand the relationship between people and tasks.

[0157] When you hover your cursor over a node, related information will pop up, and if you need more detailed information, you can click on the node to display that information.

[0158] Feedback loop

[0159] User

[0160] Users provide feedback and suggestions for improvement to the system through a feedback form. For example, they can enter specific requests such as, "I'd like the UI layout to be a little easier to use."

[0161] The server stores the collected feedback in a database, which is then used for future data analysis and system updates.

[0162] Specific example

[0163] New task and integration scenario

[0164] User A (Project Manager)

[0165] User A launches a new project and assigns team members. User A assigns tasks to each member through a project management tool.

[0166] Data collection

[0167] server

[0168] The system retrieves the latest task information for User A and team members from email systems, project management systems, and calendar systems. The retrieved data is stored in a database.

[0169] Correlation diagram generation

[0170] server

[0171] The analysis module analyzes the data and generates nodes (User A, User B, User C) and edges (task connections). A visual correlation diagram is exported in JSON format.

[0172] Displaying information

[0173] terminal

[0174] User A logs into the system and views the correlation diagram. The diagram displays User A as the project manager, with Users B and C as members. Hovering the cursor over a node displays task details and collaboration information.

[0175] feedback

[0176] User A

[0177] User A enters their feedback on the system's usability and areas for improvement into a feedback form. This feedback will be used to improve the system.

[0178] Example of a prompt

[0179] This system collects internal company data and generates correlation diagrams of stakeholders. It collects data from the company's email system, project management system, and calendar system, and uses an analysis module to generate correlation diagrams. The generated correlation diagrams are viewable by users, and system improvements are made based on their feedback.

[0180] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0181] Step 1: Data Acquisition

[0182] server

[0183] The server retrieves data from the company's email system, project management system, and calendar system.

[0184] It sends requests to the APIs of various systems (such as Google Calendar API, Microsoft Exchange API, and JIRA API) to retrieve user and task information.

[0185] For example, execute a command like "curl -X GET 'https: / / api.calendar.google.com / calendar / v3 / calendars / primary / events' -H 'Authorization: Bearer [ACCESS_TOKEN]'".

[0186] Input: API endpoint, authentication token

[0187] Output: Data response in JSON format containing person information and job information.

[0188] Step 2: Authentication and securing access rights

[0189] server

[0190] The server authenticates the data it retrieves and secures the necessary access rights.

[0191] Obtain an access token using the OAuth 2.0 flow and include it in the API request header.

[0192] Input: User credentials, authentication server endpoint

[0193] Output: Access token

[0194] Step 3: Data validation and saving

[0195] server

[0196] The server validates the retrieved data and checks if it conforms to the schema.

[0197] Data that does not conform to the schema is removed, and only conforming data is stored in the central database.

[0198] Input: Acquired JSON data, schema definition

[0199] Output: Validated data stored in the central database

[0200] Step 4: Regular data scans

[0201] server

[0202] The server scans the data in the database using a periodic job scheduler (e.g., a Cron job).

[0203] Obtain the latest information from the database and prepare for analysis.

[0204] For example, the scan is performed daily in the format "0 0 / usr / bin / python3 / path / to / data_scan.py".

[0205] Input: Job scheduler settings, database connection information

[0206] Output: Latest dataset

[0207] Step 5: Data Analysis

[0208] server

[0209] The server analyzes the data using an analysis module (for example, the NetworkX library in Python).

[0210] A graph structure is generated using each person as a node and each task as an edge.

[0211] Input: Latest dataset

[0212] Output: Graph structure consisting of nodes and edges

[0213] Step 6: Generate and export the correlation diagram

[0214] server

[0215] The server renders a visual correlation diagram based on the generated graph structure.

[0216] The correlation diagram is exported in JSON or XML format and used for display on the client side.

[0217] Input: Graph structure

[0218] Output: Correlation diagram data in JSON or XML format

[0219] Step 7: Obtain and display correlation diagram data

[0220] terminal

[0221] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server.

[0222] This program uses AJAX requests to retrieve data in JSON format and displays correlation diagrams using D3.js or Cytoscape.js.

[0223] Input: User login information, server correlation diagram data

[0224] Output: Displayed interactive correlation diagram

[0225] Step 8: Interactive Information Display

[0226] terminal

[0227] When a user hovers their cursor over a node in the correlation diagram, related information pops up.

[0228] Clicking on a node will display more detailed information.

[0229] Input: User interaction, correlation diagram data

[0230] Output: Detailed information displayed in a pop-up window

[0231] Step 9: Gathering Feedback

[0232] User

[0233] Users can submit their opinions and suggestions for improvement regarding the system through a feedback form.

[0234] Input: Feedback content

[0235] Output: Feedback data sent to the server

[0236] Step 10: Saving and analyzing feedback

[0237] server

[0238] The server stores the collected feedback in a database and takes it into consideration during the next data analysis.

[0239] Input: Feedback data

[0240] Output: Feedback data stored in the database

[0241] (Application Example 1)

[0242] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0243] Traditional factories faced challenges in monitoring work and inventory information in real time and developing efficient work plans. Furthermore, it was difficult for factory robots and terminals to grasp work progress and inventory status and respond dynamically. This often hindered efficient work and reduced overall factory productivity.

[0244] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0245] In this invention, the server includes means for acquiring data including person information and work information; means for analyzing the acquired data and modeling the correlation between people and work; means for generating a visual correlation diagram based on the modeled correlation; means for robots and terminals to create efficient work plans based on the analysis results; means for displaying the generated correlation diagram; and means for robots and terminals to improve real-time work efficiency by referring to the displayed correlation diagram. This makes it possible to monitor work information and inventory information within the factory in real time and manage them efficiently.

[0246] "Personal information" refers to identifiable data about individual people, including their name, job title, and assigned tasks.

[0247] "Work information" refers to detailed data about a specific task, including the type of work, progress, person in charge, and deadline.

[0248] "Correlation" refers to the relationship between different data items, and specifically to the relationship between people and tasks.

[0249] "Modeling" refers to the process of transforming raw data into a mathematical or logical structure, making it easier to understand and analyze the relationships and patterns within the data.

[0250] A "visual correlation diagram" is a chart that visually represents the correlation between data, using nodes (data items) and edges (relationships) to show the structure.

[0251] The term "robot" refers to a machine or device that performs tasks automatically within a factory, and whose movements are controlled by work information and correlation diagrams.

[0252] "Terminal" refers to an electronic device used for displaying and manipulating data, and includes devices that display work information and correlation diagrams within a factory.

[0253] A "work plan" refers to a plan for carrying out work efficiently and effectively, and it manages the assignment and timing of each task.

[0254] "Real-time" refers to processing and reflecting data and information instantly with virtually no delay.

[0255] "Work efficiency" is a measure of how efficiently work is being performed, aiming to minimize wasted time and resources.

[0256] To implement this invention, the system is constructed and operated in the following steps: the server, robot, terminal, and user each play their respective roles.

[0257] 1. Data Collection Phase

[0258] The server makes API calls to various systems (e.g., inventory management system, work scheduling system, machine operation status monitoring system) to retrieve person and work information. The server authenticates with each system and secures the necessary access rights. The retrieved data includes user ID, work details, stakeholders, date and time, etc. The retrieved data is organized and stored in a central database.

[0259] 2. Correlation Diagram Generation Phase

[0260] The server periodically scans the database to retrieve the latest person and task information. An analysis module analyzes the data and models the correlation between people and tasks as a graph structure. Based on this graph data, a visual correlation diagram is generated. This correlation diagram is exported in JSON or XML format and used for display on the terminal.

[0261] 3. UI / UX Phase

[0262] The terminal and robot retrieve the latest correlation diagram data from the server and display the correlation diagram using libraries such as D3.js and Cytoscape.js. When a user views the correlation diagram and hovers over a node, relevant information (e.g., person's name, job title, and work details) pops up. If the user clicks on a node, more detailed information is displayed, including past collaboration history and comments.

[0263] 4. Feedback Loop

[0264] Users provide feedback, evaluations, and suggestions for improvement to the system through a feedback form. The server stores the feedback in a database and considers it during subsequent data analysis and correlation diagram generation. This allows the system to be continuously improved, increasing its accuracy and usefulness.

[0265] The specific software and hardware used in this system include the requests library for API calls, the networkx library and Flask framework for data analysis, and D3.js and Cytoscape.js for generating visual correlation diagrams.

[0266] Specific example

[0267] For example, suppose a robot in a factory is assembling parts. To enable this robot to understand its work progress and inventory status in real time and to carry out the work more efficiently, a system that monitors work progress and inventory status is used. This makes it possible to immediately notify the manager of instructions to replenish parts that are running low on stock.

[0268] Examples of prompts for generative AI models

[0269] "Please tell me how to improve work efficiency within the factory. Specifically, please tell me how to build a system that monitors work progress and inventory status in real time to improve efficiency."

[0270] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0271] Step 1:

[0272] The server makes API calls to various systems (inventory management system, work scheduling system, machine operation status monitoring system) to retrieve personnel and work information. It uses the API call URL and authentication information as input, and outputs JSON data of the retrieved personnel and work information. This data is temporarily stored in a central database.

[0273] Step 2:

[0274] The server periodically scans the database to retrieve the latest person and task information. The person and task information is input from the database scan results and analyzed to model the correlation between people and tasks. A data analysis library (e.g., networkx) is used for the analysis, and graph structure data is generated as output.

[0275] Step 3:

[0276] The server generates a visual correlation diagram based on the analyzed graph structure data. It receives the graph structure data as input and creates the correlation diagram using a visualization library such as D3.js or Cytoscape.js. As output, the correlation diagram is exported in JSON or XML format and used for subsequent display.

[0277] Step 4:

[0278] The terminal (or robot) retrieves the latest correlation diagram data from the server and visually displays the correlation diagram using a display library (e.g., D3.js, Cytoscape.js). The terminal treats the retrieved JSON or XML data as input and generates output (a visual correlation diagram) by rendering the correlation diagram.

[0279] Step 5:

[0280] The user views the correlation diagram generated using the terminal. When the user moves the cursor over a node on the correlation diagram, relevant information (person's name, position, work details) is displayed in a pop-up. This allows the user to view the detailed information of the corresponding node in real time.

[0281] Step 6:

[0282] The user provides opinions, evaluations, points for improvement, etc. to the system through a feedback form. The input feedback information is sent to the server, and the feedback content is saved in the database. This helps in the continuous improvement of the system as it is considered during the next data analysis and correlation diagram generation.

[0283] Step 7:

[0284] The server considers the saved feedback information during the next data analysis. It uses the past feedback information as input and affects the analysis results. As a result, analysis results with improvements based on the feedback are output.

[0285] Furthermore, an emotion engine for estimating the user's emotions may be combined. That is, the specific processing unit 290 may estimate the user's emotions using the emotion recognition model 59 and perform specific processing using the user's emotions.

[0286] To implement this invention, the system is constructed and operated according to the following procedures. Also, by combining an emotion engine for recognizing the user's emotions, a more detailed analysis of the correlation can be performed.

[0287] 1. Data collection phase

[0288] Server

[0289] The server makes an API call to the in-house system (e.g., mail server, project management tool, calendar system) to establish a connection. It acquires access rights using the authentication information (API key, OAuth token) of each system.

[0290] 0] The server acquires data such as user ID, task details (task name, deadline, progress status), related persons, date and time through the API call. Since only the latest data is acquired, the previous data acquisition date and time is recorded and differential data is collected.

[0291] The acquired data is stored in the central database. Validation (verification) is performed to maintain data consistency and integrity, and inconsistent data is excluded.

[0292] 2. Collection of emotion data

[0293] Server

[0294] The server uses the emotion engine to collect the user's emotion data. This is based on data obtained through the user's text messages, voice input, image recognition, etc.

[0295] The emotion engine analyzes the user's emotions (e.g., joy, anger, sadness, surprise, etc.) and stores the results in a database. This emotion data is linked to other business data and used to analyze correlations.

[0296] 3. Correlation Diagram Generation Phase

[0297] server

[0298] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[0299] Using an analysis module, we analyze this data and model the relationship between people and tasks in a graph structure. We set up people and tasks as nodes, define their relationships as edges, and simultaneously incorporate sentiment data into the analysis.

[0300] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[0301] 4. UI / UX Phase

[0302] terminal

[0303] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed (analyzed) on the client side.

[0304] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[0305] When the cursor is placed on a node, relevant information (e.g., name, position, task details) and the emotional status (e.g., the user's current emotional information) will be popped up. Furthermore, past interaction history and comments will be displayed by clicking.

[0306] 5. Feedback Loop

[0307] User

[0308] The user browses the correlation diagram and checks the necessary information. The user inputs opinions and improvement points for the system through the feedback form.

[0309] Server

[0310] The server saves the feedback from the user to the database. At the next data analysis, this feedback will be considered and reflected in the analysis model. As a result, the system will be continuously improved and optimized according to the user's needs.

[0311] Specific Example

[0312] Scenario of Linking with New Task

[0313] User A (Project Manager)

[0314] User A newly launches a project and assigns User B and User C as team members. User A assigns various tasks to each member through the project management tool.

[0315] User A grasps the progress of the project and checks the emotional data of the members.

[0316] Data Collection

[0317] Server

[0318] The system retrieves the latest task information and sentiment data for User A and their team members from the mail server, project management tools, calendar system, and sentiment engine.

[0319] The data is stored in a database.

[0320] Correlation diagram generation

[0321] server

[0322] The analysis module analyzes the acquired data and generates nodes (User A, User B, User C) and edges (task connections). Sentimental data is also incorporated, reflecting the user's current emotional status.

[0323] A visual correlation diagram is generated and exported in JSON format.

[0324] Displaying information

[0325] terminal

[0326] User A logs into the system and views the correlation diagram. In the correlation diagram, User A is displayed as the project manager, and Users B and C are displayed as members.

[0327] When user A hovers over a node, the task details, collaboration information, and the sentiment status of users B and C are displayed in a pop-up window.

[0328] feedback

[0329] User A

[0330] User A provides feedback on their experience with the system and suggestions for improvement through a feedback form. This allows the system to be improved to be more user-friendly.

[0331] By implementing this invention, collaboration in operations and identification of stakeholders become easier, and by considering emotional data, operational efficiency is further improved.

[0332] The following describes the processing flow.

[0333] Step 1:

[0334] server

[0335] The server establishes a connection by making API calls to internal systems (e.g., mail server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) for each system.

[0336] Step 2:

[0337] server

[0338] The server retrieves data from each system. This data includes user ID, task details (task name, deadline, progress), stakeholders, date, and time. API calls are used to collect the most up-to-date data.

[0339] Step 3:

[0340] server

[0341] The acquired data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is removed.

[0342] Step 4:

[0343] server

[0344] The system uses an emotion engine to acquire user emotion data. This is done by analyzing emotions through methods such as user text messages, voice input, and image recognition. The analyzed emotion data is stored in a database.

[0345] Step 5:

[0346] server

[0347] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[0348] Step 6:

[0349] server

[0350] The acquired data is analyzed, and the relationship between people and tasks is modeled using a graph structure (nodes and edges). People and tasks are set as nodes, their relationships are defined as edges, and sentiment data is also incorporated simultaneously.

[0351] Step 7:

[0352] server

[0353] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[0354] Step 8:

[0355] terminal

[0356] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed (analyzed) on the client side.

[0357] Step 9:

[0358] terminal

[0359] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[0360] Step 10:

[0361] terminal

[0362] When a user hovers over a node, a pop-up window displays detailed information about that node (e.g., name, role, task details) and its sentiment status (e.g., the user's current sentiment). Clicking on the node displays even more detailed information (e.g., past collaboration history, comments).

[0363] Step 11:

[0364] User

[0365] Users view correlation diagrams to check work-related information and emotional status. They also provide feedback and suggestions for improvement to the system through a feedback form.

[0366] Step 12:

[0367] server

[0368] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[0369] (Example 2)

[0370] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0371] Conventional data analysis systems had limitations in their ability to acquire person and business information and generate correlation diagrams. In particular, understanding business situations while considering users' emotional states was difficult, posing challenges to improving work efficiency and collaboration. Furthermore, there was insufficient mechanism for collecting user feedback on the generated correlation diagrams and incorporating it into subsequent data analyses.

[0372] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0373] In this invention, the server includes means for acquiring data containing person information and business information; means for analyzing the acquired data and modeling the correlation between people and business; means for collecting user emotion data and performing emotion analysis; means for generating a visual correlation diagram based on the modeled correlations and emotion data; and means for displaying the generated correlation diagram. This makes it easier to coordinate business operations and understand stakeholders, and by considering emotion data in particular, it becomes possible to improve business efficiency and facilitate smooth communication. Furthermore, by reflecting user feedback in the next data analysis, continuous improvement of the system can be achieved.

[0374] "Personal information" refers to data used to identify a specific individual within the system (e.g., user ID, name, job title).

[0375] "Business information" refers to data related to the progress and content of business operations (e.g., task name, deadline, progress status).

[0376] "Emotional data" refers to data that represents the user's emotional state (e.g., joy, anger, sadness, surprise).

[0377] "Sentiment analysis" is the process of extracting and analyzing emotional data from users' text messages, voice data, and other sources.

[0378] A "correlation diagram" is a graph-structured diagram that visually represents personal information, business information, and emotional data.

[0379] "Modeling" is the process of using data to represent the relationships between people and tasks as a graph structure.

[0380] A "means for generating visual correlation diagrams" refers to a system component that has the function of drawing correlation diagrams in a human-readable format using modeled data.

[0381] "Feedback" refers to information about opinions and suggestions for improvement provided by users who have used the system.

[0382] To implement this invention, the system should be constructed and operated according to the following procedure. Furthermore, by combining it with an emotion engine for recognizing user emotions, a more detailed correlation analysis can be performed.

[0383] 1. Data Collection Phase

[0384] server

[0385] The server establishes connections by making API calls to internal systems (e.g., mail server, project management tool, calendar system). It obtains access rights using authentication information (API keys or OAuth tokens) from each system. Specifically, we will use "JIRA" as the project management tool, "Google Calendar" as the calendar system, and "Microsoft Exchange" as the mail server.

[0386] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time via API calls. To retrieve only the most recent data, it records the date and time of the previous data retrieval and collects differential data. The retrieved data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is excluded.

[0387] 2. Collection of emotional data

[0388] server

[0389] The server uses an emotion engine to collect user emotion data. This data is based on information obtained through user text messages, voice input, image recognition, and other means. Specifically, IBM Watson® Natural Language Understanding is used for sentiment analysis of text messages, and Google Cloud Speech-to-Text is used for sentiment analysis of voice input.

[0390] The emotion engine analyzes the user's emotions (e.g., joy, anger, sadness, surprise, etc.) and stores the results in a database. This emotion data is linked to other business data and used to analyze correlations.

[0391] 3. Correlation Diagram Generation Phase

[0392] server

[0393] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[0394] Using an analysis module, we will analyze this data and model the relationship between people and tasks in a graph structure. We will set up people and tasks as nodes, define their relationships as edges, and simultaneously incorporate sentiment data into the analysis. As a concrete example, we will use the Python "NetworkX" library to generate the graph structure.

[0395] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[0396] 4. UI / UX Phase

[0397] terminal

[0398] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed on the client side. As a specific example, "React" is used as the frontend framework, and "D3.js" or "Cytoscape.js" is used as the correlation diagram library.

[0399] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on the analyzed data. Nodes and edges are visually arranged to make them easy for the user to understand. When the cursor hovers over a node, related information (e.g., name, job title, task details) and sentiment status (e.g., the user's current sentiment information) pop up. Furthermore, clicking on a node displays past interaction history and comments.

[0400] 5. Feedback Loop

[0401] User

[0402] Users view correlation diagrams and confirm necessary information. They also provide feedback and suggestions for improvement to the system through a feedback form.

[0403] server

[0404] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[0405] Example of a prompt

[0406] "Create a new task and generate a correlation diagram that includes an analysis of sentiment data."

[0407] This system makes it easier to coordinate tasks and identify stakeholders, and by considering emotional data, operational efficiency can be further improved.

[0408] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0409] Step 1:

[0410] The server makes an API call to the internal system and establishes a connection.

[0411] Input: Authentication information for internal systems (e.g., API key, OAuth token).

[0412] The server makes API calls to the mail server, project management tool, and calendar system to obtain access rights. Specifically, it adds the authentication information to the HTTP request header and establishes the connection.

[0413] Output: Handle to a successful API connection.

[0414] Step 2:

[0415] The server retrieves the necessary data from the internal system.

[0416] Input: API connection handle, last data retrieval date and time.

[0417] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time. For example, it sends a GET request to the JIRA API to retrieve the latest task information. It compares this to the previous data retrieval date and time and collects the difference data.

[0418] Output: A set of acquired data.

[0419] Step 3:

[0420] The server validates the data it retrieves and stores it in the central database.

[0421] Input: A set of retrieved data.

[0422] The server performs validation checks to ensure the data format is correct and all required fields are present. Only data that passes validation is inserted into the database. Specifically, the SQLAlchemy library in Python is used to insert data into the "tasks" table.

[0423] Output: Data stored in the database.

[0424] Step 4:

[0425] The server uses an emotion engine to collect user emotion data.

[0426] Input: User text messages, audio data.

[0427] The server sends text messages to sentiment analysis engines such as IBM Watson Natural Language Understanding for sentiment analysis. Similarly, sentiment analysis is performed on voice input using Google Cloud Speech-to-Text.

[0428] Output: Acquired sentiment data.

[0429] Step 5:

[0430] The server stores the emotion analysis results in a database.

[0431] Input: Sentiment analysis results.

[0432] The server inserts the analyzed emotion data into the "emotions" table in the database. Specifically, it stores the emotion status (e.g., joy, anger, sadness) and its associated task ID.

[0433] Output: Emotional data stored in the database.

[0434] Step 6:

[0435] The server scans the database to retrieve the latest information.

[0436] Input: Periodic scan timer.

[0437] The server scans the "Person Information," "Task Information," and "Emotional Data" tables to retrieve the latest data. This is done using SQL SELECT queries. For example, an SQL query can be used to retrieve updated task information.

[0438] Output: Set of the latest data.

[0439] Step 7:

[0440] The system analyzes the data acquired by the server and models its graph structure.

[0441] Input: Set of the latest data.

[0442] The server uses Python's NetworkX library to define people and tasks as nodes, and their relationships as edges. Sentiment data is also added to the nodes. For example, each user and task is added as a node, and their relationships are defined as edges.

[0443] Output: Modeled graph data.

[0444] Step 8:

[0445] The server converts the modeled graph data into a visual correlation diagram and sends it to the client in JSON format.

[0446] Input: Modeled graph data.

[0447] The server serializes the generated graph data into JSON format and sends it to the client. Specifically, it uses Flask's jsonify method to generate JSON data and returns it to the client.

[0448] Output: Data in JSON format.

[0449] Step 9:

[0450] The terminal retrieves the latest correlation diagram data from the server.

[0451] Input: Client request.

[0452] The device (the user's web browser) makes a GET request to a specified endpoint on the server and retrieves data in JSON format. Specifically, it uses the JavaScript fetch API to retrieve data from the / api / relation-graph endpoint.

[0453] Output: Retrieved JSON data.

[0454] Step 10:

[0455] The JSON data acquired by the device is analyzed, and a correlation diagram is drawn.

[0456] Input: Retrieved JSON data.

[0457] The device parses JSON data using D3.js or Cytoscape.js and visually arranges nodes and edges. For example, it adds nodes and edges using the cy.add method of Cytoscape.js and draws a correlation diagram.

[0458] Output: The generated correlation diagram.

[0459] Step 11:

[0460] When the device hovers over a node, it will display relevant information and sentiment status in a pop-up window.

[0461] Input: User hover event.

[0462] When a hover event occurs on a node, relevant information and sentiment status will be displayed as a tooltip. Specifically, the tooltip will be displayed using the Cytoscape.js method `cy.on('mouseover', 'node', function(evt) {...}`.

[0463] Output: The displayed tooltip.

[0464] Step 12:

[0465] Users submit their opinions and suggestions for improvement regarding the system through a feedback form.

[0466] Input: User feedback content.

[0467] Users enter their opinions and suggestions for improvement into a feedback form displayed on the UI and press the submit button. In other words, the form data is sent to the server.

[0468] Output: Sent feedback data.

[0469] Step 13:

[0470] The server saves user feedback to a database.

[0471] Input: Submitted feedback data.

[0472] The server inserts the feedback data received via POST request into the "feedback" table in the database. Specifically, it uses an SQL INSERT query to save the feedback content.

[0473] Output: Feedback stored in the database.

[0474] Step 14:

[0475] The server will take the feedback into consideration during the next data analysis.

[0476] Input: Feedback stored in the database.

[0477] The server will refer to the feedback during the next data analysis and incorporate it into the analysis model. Specifically, it will adjust the analysis algorithm and graph display based on the feedback.

[0478] Output: Analysis data reflecting the feedback.

[0479] This system will facilitate smoother collaboration between tasks and better identify stakeholders, and by considering emotional data, further improvements in operational efficiency can be expected.

[0480] (Application Example 2)

[0481] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0482] Traditional task management systems are limited to understanding correlations based on personal and task information, and lack consideration for user emotions, leading to problems such as decreased work efficiency and satisfaction. In particular, in workplaces where emotional changes directly impact work efficiency, the lack of collection and analysis of this information posed a significant challenge. There is a need to build a task management system that incorporates employee emotional data to ensure smooth workflow and improve employee satisfaction.

[0483] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0484] In this invention, the server includes means for acquiring data including person information, task information, and emotion data; means for analyzing the acquired data and modeling the correlation between people and tasks; and means for generating a visual correlation diagram based on the modeled correlations and emotion data. This makes it possible to perform effective task management while taking into account the user's emotions in real time.

[0485] "Personal information" refers to data related to an individual, such as the user's name, job title, and contact information.

[0486] "Business information" refers to data related to work or projects, such as task names, deadlines, and progress status.

[0487] "Emotional data" refers to data that indicates a user's emotional state, analyzed from sources such as text messages, voice input, and image recognition.

[0488] "Correlation" refers to the relationship or influence that exists between a person and a task.

[0489] "Modeling" refers to analyzing acquired data and representing the relationships between the data mathematically or graphically.

[0490] A "visual correlation diagram" is a graph that visually displays people and tasks as nodes and relationships as edges, based on analyzed data.

[0491] "Popup display" is a feature that displays related information in a small window format in response to a specific action (e.g., hovering the cursor over a node).

[0492] "Feedback" refers to the opinions, evaluations, and suggestions for improvement provided by system users.

[0493] A system for carrying out this invention includes the following elements.

[0494] Data collection phase

[0495] server

[0496] The server establishes a connection by making API calls to internal systems (e.g., email server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) for each system.

[0497] The server retrieves data such as user ID, task details, stakeholders, date, and time via API calls. To retrieve only the most recent data, it records the date and time of the previous data retrieval and collects differential data.

[0498] The acquired data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is excluded.

[0499] Collection of emotional data

[0500] server

[0501] The server uses an emotion engine to collect user emotion data. This data is based on information obtained through user text messages, voice input, image recognition, and other means.

[0502] The emotion engine analyzes the user's emotions (e.g., joy, anger, sadness, surprise, etc.) and stores the results in a database. This emotion data is linked to other business data and used to analyze correlations.

[0503] Correlation diagram generation phase

[0504] server

[0505] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[0506] Using an analysis module, we analyze this data and model the relationship between people and tasks in a graph structure. We set up people and tasks as nodes, define their relationships as edges, and simultaneously incorporate sentiment data into the analysis.

[0507] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[0508] UI / UX Phase

[0509] terminal

[0510] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed on the client side.

[0511] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[0512] When you hover your cursor over a node, relevant information (e.g., name, job title, task details) and sentiment status (e.g., the user's current sentiment information) will pop up. Furthermore, clicking on it will display past interaction history and comments.

[0513] Feedback loop

[0514] User

[0515] Users view correlation diagrams and confirm necessary information. They also provide feedback and suggestions for improvement to the system through a feedback form.

[0516] server

[0517] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[0518] Specific example

[0519] New task and integration scenario

[0520] Employee A (Manager)

[0521] Employee A launches a new project and assigns employees B and C as team members. Employee A uses a project management tool to assign various tasks to each member. Employee A monitors the project's progress and checks the members' emotional data.

[0522] Example of a prompt

[0523] Please tell me about the current progress of your task and how you feel about it.

[0524] Please share your thoughts on completing today's tasks.

[0525] Do you have any concerns about the next task?

[0526] Implementing this system makes it possible to manage tasks effectively while taking user emotions into consideration in real time.

[0527] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0528] Step 1:

[0529] The server establishes connections by making API calls to internal systems (e.g., mail server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) from each system. The server then retrieves the latest personnel and business information from each system. The input is the API endpoint and authentication information for each system, and the output is the retrieved personnel and business information. The data is stored in a central database.

[0530] Step 2:

[0531] The server retrieves data such as each user's ID, task details, stakeholders, date, and time via API calls. It records the date and time of the previous data retrieval and collects only differential data. The input is the date and time of the previous data retrieval and the result of the current API call, and the output is the new and updated data from the current call. This data is validated, and inconsistent data is excluded.

[0532] Step 3:

[0533] The server uses an emotion engine to collect user emotion data. The emotion engine analyzes emotions based on input data such as text messages, voice input, and image recognition, and stores the results in a database. The input is data related to the user's emotions, and the output is the analyzed emotion data.

[0534] Step 4:

[0535] The server periodically scans the database to retrieve the latest person information, task information, and sentiment data. Based on this data, the relationship between people and tasks is modeled in a graph structure. The input is the latest data retrieved from the database, and the output is the modeled graph data.

[0536] Step 5:

[0537] The server uses an analysis module to generate a visual correlation diagram based on modeled graph data. Sentiment data is also incorporated. The input is graph data and sentiment data, and the output is a visual correlation diagram. The generated correlation diagram can be exported in JSON or XML format.

[0538] Step 6:

[0539] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server and parses (analyzes) it on the client side. The input is correlation diagram data in JSON format provided by the server, and the output is the parsed data.

[0540] Step 7:

[0541] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand. The input is the parsed data, and the output is the displayed correlation diagram.

[0542] Step 8:

[0543] When a user hovers over a node, relevant information (e.g., name, role, task details) and sentiment status pop up. Clicking on the node displays past interaction history and comments. The input is the cursor's position, and the output is the detailed information displayed in the pop-up.

[0544] Step 9:

[0545] Users provide feedback on their experience with the system and suggestions for improvement through a feedback form. The input is the user's feedback content, and the output is feedback data stored in the database.

[0546] Step 10:

[0547] The server stores user feedback in a database and considers this feedback during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs. The input is the feedback data stored in the database, and the output is the improved analysis model that takes the feedback into account.

[0548] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0549] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0550] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0551] [Second Embodiment]

[0552] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0553] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0554] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0555] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0556] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0557] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0558] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0559] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0560] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0561] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0562] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0563] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0564] To implement this invention, the system should be constructed and operated according to the following procedure.

[0565] 1. Data Collection Phase

[0566] server

[0567] The server makes API calls to various internal systems (e.g., email server, project management tool, calendar system) to retrieve personnel and business information.

[0568] The server authenticates with each system and secures the necessary access rights. The data retrieved by API calls includes user ID, task details, stakeholders, date and time, etc.

[0569] The acquired data is organized and stored in a central database. Each data entry undergoes validation to maintain data consistency and integrity.

[0570] 2. Correlation Diagram Generation Phase

[0571] server

[0572] The server periodically scans the database to retrieve the latest personnel and business information.

[0573] The analysis module analyzes the data and models the correlation between people and tasks as a graph structure (nodes and edges).

[0574] Based on the graph data, a visual correlation diagram is generated. This correlation diagram can be exported in JSON or XML format and used for display on the client side.

[0575] 3. UI / UX Phase

[0576] terminal

[0577] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server.

[0578] The device uses libraries such as D3.js and Cytoscape.js to display correlation diagrams. This allows users to view correlation diagrams that are easy to understand visually.

[0579] When you hover over a node, relevant information (such as the person's name, job title, and task details) pops up. If the user clicks on a node, even more detailed information is displayed, including past collaboration history and comments.

[0580] 4. Feedback Loop

[0581] User

[0582] Users provide feedback, evaluations, and suggestions for improvement to the system through a feedback form. This is treated as specific improvement requests to enhance operational efficiency.

[0583] The server stores the feedback in a database and considers it during subsequent data analysis and correlation diagram generation. This allows the system to continuously improve, increasing its accuracy and usefulness.

[0584] Specific example

[0585] New task and integration scenario

[0586] User A (Project Manager)

[0587] A new project is launched, and User A is appointed as the project manager. User A assigns Users B and C as team members.

[0588] User A assigns various tasks to each member through a project management tool.

[0589] Data collection

[0590] server

[0591] Retrieve the latest task information for User A and their team members from the mail server, project management tools, and calendar system.

[0592] The data is stored in a database.

[0593] Correlation diagram generation

[0594] server

[0595] The analysis module analyzes the acquired data and generates nodes (User A, User B, User C) and edges (task connections).

[0596] A visual correlation diagram is generated and exported in JSON format.

[0597] Displaying information

[0598] terminal

[0599] User A logs into the system and views the correlation diagram. In the correlation diagram, User A is displayed as the project manager, and Users B and C are displayed as members.

[0600] When user A hovers the cursor over a node, its task details and related information are displayed in a pop-up window.

[0601] feedback

[0602] User A

[0603] User A provides feedback on their experience with the system and suggestions for improvement through a feedback form. This allows the system to be improved to be more user-friendly.

[0604] By implementing this invention, coordination of operations and identification of stakeholders become easier, leading to a significant improvement in operational efficiency.

[0605] The following describes the processing flow.

[0606] Step 1:

[0607] server

[0608] The server makes API calls to internal systems (e.g., email server, project management tool, calendar system) to establish a connection. It then obtains access rights using authentication information (API key, OAuth token) for each system.

[0609] Step 2:

[0610] server

[0611] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time via API calls. To retrieve only the most recent data, it records the date and time of the previous data retrieval and collects differential data.

[0612] Step 3:

[0613] server

[0614] The acquired data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is removed.

[0615] Step 4:

[0616] server

[0617] The server periodically scans the database to retrieve the latest personnel and business information. This ensures that the data necessary for generating the correlation diagram is always up-to-date.

[0618] Step 5:

[0619] server

[0620] The analysis module is used to analyze the data and model the relationship between people and tasks in a graph structure. People and tasks are set as nodes, and their relationships are defined as edges.

[0621] Step 6:

[0622] server

[0623] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[0624] Step 7:

[0625] terminal

[0626] The terminal retrieves the latest correlation diagram data from the server when the user logs into the system. The data is received in JSON format and parsed (analyzed) on the client side.

[0627] Step 8:

[0628] terminal

[0629] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[0630] Step 9:

[0631] terminal

[0632] When a user hovers over a node, detailed information about that node (e.g., name, role, task details) pops up. Clicking on it displays even more detailed information (e.g., past collaboration history, comments).

[0633] Step 10:

[0634] User

[0635] Users view correlation diagrams and confirm the necessary information. Furthermore, they can provide feedback and suggestions for improvement to the system through a feedback form.

[0636] Step 11:

[0637] server

[0638] The server saves user feedback to a database. This feedback is then taken into consideration during the next data analysis, and improvements are implemented accordingly.

[0639] Step 12:

[0640] server

[0641] In the next data analysis and correlation diagram generation cycle, the saved feedback will be referenced, and necessary adjustments and improvements will be made. This allows the system to be continuously improved and optimized.

[0642] (Example 1)

[0643] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0644] Conventional systems made it difficult to grasp personnel and work information within companies, posing challenges to improving operational efficiency. Furthermore, data analysis for visually displaying the correlation between personnel and work, and interactive information display based on those results, were insufficient. As a result, it was difficult for users to intuitively grasp the status of work and make appropriate decisions.

[0645] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0646] In this invention, the server includes means for acquiring person information and business information via an API, means for authenticating the acquired data and securing access rights, and means for validating the acquired data and storing it in a central database. This makes it possible to accurately acquire necessary data and store and manage it safely and efficiently. It also includes means for an analysis module that periodically scans the database and analyzes the latest person information and business information, means for modeling the analyzed data as a graph structure, means for generating a visual correlation diagram based on the modeled correlations and exporting it in JSON or XML format, and means for acquiring the generated correlation diagram and displaying the visual correlation diagram. This makes it easier for users to intuitively understand the correlation between people and business, enabling them to make appropriate decisions quickly. Furthermore, it includes means for displaying information about a node in a pop-up when the cursor is placed over a node on the correlation diagram, and means for saving user feedback in the database and considering it during the next data analysis. This allows the system to be continuously improved and enables the provision of highly accurate information that meets the needs of users.

[0647] "Personal information" refers to identifiable information about a specific individual, including name, job title, contact information, and department.

[0648] "Business information" refers to information about business operations and tasks performed within a company, including task details, deadlines, assigned personnel, and related projects.

[0649] "API" stands for Application Programming Interface, and refers to a standardized interface that enables data exchange between different software systems.

[0650] Authentication is the process of verifying a user's identity when accessing an information system, and it primarily uses a username, password, token, etc.

[0651] "Access rights" refer to the operational permissions for data and resources within an information system, and include permissions such as read, write, and execute.

[0652] "Validation" is the process of verifying whether the acquired data conforms to predetermined standards and formats.

[0653] A "central database" is a database used to centrally store and manage all acquired data, and examples include SQL databases and NoSQL databases.

[0654] "Periodic scanning" is the process of checking, updating, and retrieving data within a database at regular time intervals.

[0655] An "analysis module" is a software component used to analyze collected data and extract specific patterns or relationships.

[0656] A "graph structure" is a data structure consisting of nodes (vertices) and edges, and is used to model the relationship between people and tasks.

[0657] A "visual correlation diagram" is a diagram that visually represents a graph structure, allowing for an intuitive understanding of the relationship between people and tasks.

[0658] "JSON" stands for JavaScript Object Notation, and it is a lightweight data exchange format for structuring, storing, and exchanging data.

[0659] XML stands for eXtensible Markup Language, and it is a markup language used to structure, store, and exchange data.

[0660] A "node" refers to a vertex in a graph structure, and usually represents an entity such as a person or a task.

[0661] "Popup display" is a feature that displays a small window containing supplementary information in response to a specific action (for example, mouseover).

[0662] "Feedback" refers to opinions, suggestions for improvement, and evaluations provided by system users, which are used to improve the system.

[0663] "Data analysis" is the process of extracting patterns and relationships from collected data using statistical methods and algorithms to deepen our understanding of the data.

[0664] Modes for carrying out the invention

[0665] This invention is a system that acquires personnel and business information from various internal information systems, analyzes them, and generates and displays a correlation diagram that visually represents the relationships between them. The following describes a specific implementation of this system.

[0666] Data collection phase

[0667] server

[0668] The server retrieves data from the company's internal email system, project management system, and calendar system via APIs. This utilizes general-purpose APIs such as the Google Calendar API, Microsoft Exchange API, and JIRA API.

[0669] Specifically, this involves sending API requests to retrieve user and task information. For example, you can access the API endpoint using the "curl" command and extract the necessary data.

[0670] The acquired data is authenticated using OAuth 2.0 or similar methods to ensure appropriate access rights. This authentication guarantees the security and reliability of the data.

[0671] The server validates the retrieved data and stores only the data that conforms to the schema in the central database. The database used is a relational database such as MySQL or PostgreSQL.

[0672] Correlation diagram generation phase

[0673] server

[0674] The server periodically uses a job scheduler (e.g., a Cron job) to scan the database. This scan retrieves the latest personnel and business information, preparing it for analysis.

[0675] The server uses an analysis module (for example, the NetworkX library in Python) to analyze the person and task information in the database. As a result of the analysis, a graph structure is generated in which each person is a node and each task is an edge.

[0676] The server renders the generated graph data as a visual correlation diagram and exports it in JSON or XML format. This export is then used for display on the client side.

[0677] UI / UX Phase

[0678] terminal

[0679] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. AJAX requests are used to retrieve the data.

[0680] On the user's device, JavaScript libraries such as D3.js and Cytoscape.js are used to display correlation diagrams based on the acquired data. This allows users to intuitively understand the relationship between people and tasks.

[0681] When you hover your cursor over a node, related information will pop up, and if you need more detailed information, you can click on the node to display that information.

[0682] Feedback loop

[0683] User

[0684] Users provide feedback and suggestions for improvement to the system through a feedback form. For example, they can enter specific requests such as, "I'd like the UI layout to be a little easier to use."

[0685] The server stores the collected feedback in a database, which is then used for future data analysis and system updates.

[0686] Specific example

[0687] New task and integration scenario

[0688] User A (Project Manager)

[0689] User A launches a new project and assigns team members. User A assigns tasks to each member through a project management tool.

[0690] Data collection

[0691] server

[0692] The system retrieves the latest task information for User A and team members from email systems, project management systems, and calendar systems. The retrieved data is stored in a database.

[0693] Correlation diagram generation

[0694] server

[0695] The analysis module analyzes the data and generates nodes (User A, User B, User C) and edges (task connections). A visual correlation diagram is exported in JSON format.

[0696] Displaying information

[0697] terminal

[0698] User A logs into the system and views the correlation diagram. The diagram displays User A as the project manager, with Users B and C as members. Hovering the cursor over a node displays task details and collaboration information.

[0699] feedback

[0700] User A

[0701] User A enters their feedback on the system's usability and areas for improvement into a feedback form. This feedback will be used to improve the system.

[0702] Example of a prompt

[0703] This system collects internal company data and generates correlation diagrams of stakeholders. It collects data from the company's email system, project management system, and calendar system, and uses an analysis module to generate correlation diagrams. The generated correlation diagrams are viewable by users, and system improvements are made based on their feedback.

[0704] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0705] Step 1: Data Acquisition

[0706] server

[0707] The server retrieves data from the company's email system, project management system, and calendar system.

[0708] It sends requests to the APIs of various systems (such as Google Calendar API, Microsoft Exchange API, and JIRA API) to retrieve user and task information.

[0709] For example, execute a command like "curl -X GET 'https: / / api.calendar.google.com / calendar / v3 / calendars / primary / events' -H 'Authorization: Bearer [ACCESS_TOKEN]'".

[0710] Input: API endpoint, authentication token

[0711] Output: Data response in JSON format containing person information and job information.

[0712] Step 2: Authentication and securing access rights

[0713] server

[0714] The server authenticates the data it retrieves and secures the necessary access rights.

[0715] Obtain an access token using the OAuth 2.0 flow and include it in the API request header.

[0716] Input: User credentials, authentication server endpoint

[0717] Output: Access token

[0718] Step 3: Data validation and saving

[0719] server

[0720] The server validates the retrieved data and checks if it conforms to the schema.

[0721] Data that does not conform to the schema is removed, and only conforming data is stored in the central database.

[0722] Input: Acquired JSON data, schema definition

[0723] Output: Validated data stored in the central database

[0724] Step 4: Regular data scans

[0725] server

[0726] The server scans the data in the database using a periodic job scheduler (e.g., a Cron job).

[0727] Obtain the latest information from the database and prepare for analysis.

[0728] For example, the scan is performed daily in the format "0 0 / usr / bin / python3 / path / to / data_scan.py".

[0729] Input: Job scheduler settings, database connection information

[0730] Output: Latest dataset

[0731] Step 5: Data Analysis

[0732] server

[0733] The server analyzes the data using an analysis module (for example, the NetworkX library in Python).

[0734] A graph structure is generated using each person as a node and each task as an edge.

[0735] Input: Latest dataset

[0736] Output: Graph structure consisting of nodes and edges

[0737] Step 6: Generate and export the correlation diagram

[0738] server

[0739] The server renders a visual correlation diagram based on the generated graph structure.

[0740] The correlation diagram is exported in JSON or XML format and used for display on the client side.

[0741] Input: Graph structure

[0742] Output: Correlation diagram data in JSON or XML format

[0743] Step 7: Obtain and display correlation diagram data

[0744] terminal

[0745] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server.

[0746] This program uses AJAX requests to retrieve data in JSON format and displays correlation diagrams using D3.js or Cytoscape.js.

[0747] Input: User login information, server correlation diagram data

[0748] Output: Displayed interactive correlation diagram

[0749] Step 8: Interactive Information Display

[0750] terminal

[0751] When a user hovers their cursor over a node in the correlation diagram, related information pops up.

[0752] Clicking on a node will display more detailed information.

[0753] Input: User interaction, correlation diagram data

[0754] Output: Detailed information displayed in a pop-up window

[0755] Step 9: Gathering Feedback

[0756] User

[0757] Users can submit their opinions and suggestions for improvement regarding the system through a feedback form.

[0758] Input: Feedback content

[0759] Output: Feedback data sent to the server

[0760] Step 10: Saving and analyzing feedback

[0761] server

[0762] The server stores the collected feedback in a database and takes it into consideration during the next data analysis.

[0763] Input: Feedback data

[0764] Output: Feedback data stored in the database

[0765] (Application Example 1)

[0766] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0767] Traditional factories faced challenges in monitoring work and inventory information in real time and developing efficient work plans. Furthermore, it was difficult for factory robots and terminals to grasp work progress and inventory status and respond dynamically. This often hindered efficient work and reduced overall factory productivity.

[0768] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0769] In this invention, the server includes means for acquiring data including person information and work information; means for analyzing the acquired data and modeling the correlation between people and work; means for generating a visual correlation diagram based on the modeled correlation; means for robots and terminals to create efficient work plans based on the analysis results; means for displaying the generated correlation diagram; and means for robots and terminals to improve real-time work efficiency by referring to the displayed correlation diagram. This makes it possible to monitor work information and inventory information within the factory in real time and manage them efficiently.

[0770] "Personal information" refers to identifiable data about individual people, including their name, job title, and assigned tasks.

[0771] "Work information" refers to detailed data about a specific task, including the type of work, progress, person in charge, and deadline.

[0772] "Correlation" refers to the relationship between different data items, and specifically to the relationship between people and tasks.

[0773] "Modeling" refers to the process of transforming raw data into a mathematical or logical structure, making it easier to understand and analyze the relationships and patterns within the data.

[0774] A "visual correlation diagram" is a chart that visually represents the correlation between data, using nodes (data items) and edges (relationships) to show the structure.

[0775] The term "robot" refers to a machine or device that performs tasks automatically within a factory, and whose movements are controlled by work information and correlation diagrams.

[0776] "Terminal" refers to an electronic device used for displaying and manipulating data, and includes devices that display work information and correlation diagrams within a factory.

[0777] A "work plan" refers to a plan for carrying out work efficiently and effectively, and it manages the assignment and timing of each task.

[0778] "Real-time" refers to processing and reflecting data and information instantly with virtually no delay.

[0779] "Work efficiency" is a measure of how efficiently work is being performed, aiming to minimize wasted time and resources.

[0780] To implement this invention, the system is constructed and operated in the following steps: the server, robot, terminal, and user each play their respective roles.

[0781] 1. Data Collection Phase

[0782] The server makes API calls to various systems (e.g., inventory management system, work scheduling system, machine operation status monitoring system) to retrieve person and work information. The server authenticates with each system and secures the necessary access rights. The retrieved data includes user ID, work details, stakeholders, date and time, etc. The retrieved data is organized and stored in a central database.

[0783] 2. Correlation Diagram Generation Phase

[0784] The server periodically scans the database to retrieve the latest person and task information. An analysis module analyzes the data and models the correlation between people and tasks as a graph structure. Based on this graph data, a visual correlation diagram is generated. This correlation diagram is exported in JSON or XML format and used for display on the terminal.

[0785] 3. UI / UX Phase

[0786] The terminal and robot retrieve the latest correlation diagram data from the server and display the correlation diagram using libraries such as D3.js and Cytoscape.js. When a user views the correlation diagram and hovers over a node, relevant information (e.g., person's name, job title, and work details) pops up. If the user clicks on a node, more detailed information is displayed, including past collaboration history and comments.

[0787] 4. Feedback Loop

[0788] Users provide feedback, evaluations, and suggestions for improvement to the system through a feedback form. The server stores the feedback in a database and considers it during subsequent data analysis and correlation diagram generation. This allows the system to be continuously improved, increasing its accuracy and usefulness.

[0789] The specific software and hardware used in this system include the requests library for API calls, the networkx library and Flask framework for data analysis, and D3.js and Cytoscape.js for generating visual correlation diagrams.

[0790] Specific example

[0791] For example, suppose a robot in a factory is assembling parts. To enable this robot to understand its work progress and inventory status in real time and to carry out the work more efficiently, a system that monitors work progress and inventory status is used. This makes it possible to immediately notify the manager of instructions to replenish parts that are running low on stock.

[0792] Examples of prompts for generative AI models

[0793] "Please tell me how to improve work efficiency within the factory. Specifically, please tell me how to build a system that monitors work progress and inventory status in real time to improve efficiency."

[0794] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0795] Step 1:

[0796] The server makes API calls to various systems (inventory management system, work scheduling system, machine operation status monitoring system) to retrieve personnel and work information. It uses the API call URL and authentication information as input, and outputs JSON data of the retrieved personnel and work information. This data is temporarily stored in a central database.

[0797] Step 2:

[0798] The server periodically scans the database to retrieve the latest person and task information. The person and task information is input from the database scan results and analyzed to model the correlation between people and tasks. A data analysis library (e.g., networkx) is used for the analysis, and graph structure data is generated as output.

[0799] Step 3:

[0800] The server generates a visual correlation diagram based on the analyzed graph structure data. It accepts graph structure data as input and creates the correlation diagram using visualization libraries such as D3.js or Cytoscape.js. As output, the correlation diagram is exported in JSON or XML format for later display.

[0801] Step 4:

[0802] The terminal (or robot) retrieves the latest correlation diagram data from the server and uses a display library (e.g., D3.js, Cytoscape.js) to visually display the correlation diagram. The terminal takes the retrieved JSON or XML data as input and generates output (a visual correlation diagram) by rendering the correlation diagram.

[0803] Step 5:

[0804] Users view the generated correlation diagram using their device. When a user hovers their cursor over a node on the correlation diagram, relevant information (person's name, job title, and details of their work) pops up. This allows users to view detailed information about the corresponding node in real time.

[0805] Step 6:

[0806] Users provide feedback, evaluations, and suggestions for improvement to the system through a feedback form. The input feedback information is sent to the server and stored in a database. This feedback is then considered during subsequent data analysis and correlation diagram generation, contributing to the continuous improvement of the system.

[0807] Step 7:

[0808] The server considers the stored feedback information during the next data analysis. It uses past feedback information as input, influencing the analysis results. This results in an output analysis with improvements based on the feedback.

[0809] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0810] To implement this invention, the system should be constructed and operated according to the following procedure. Furthermore, by combining it with an emotion engine for recognizing user emotions, a more detailed correlation analysis can be performed.

[0811] 1. Data Collection Phase

[0812] server

[0813] The server establishes a connection by making API calls to internal systems (e.g., email server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) for each system.

[0814] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time via API calls. To retrieve only the most recent data, the server records the date and time of the previous data retrieval and collects differential data.

[0815] The acquired data is stored in a central database. To maintain data consistency and integrity, validation is performed to remove inconsistent data.

[0816] 2. Collection of emotional data

[0817] server

[0818] The server uses an emotion engine to collect user emotion data. This data is based on information obtained through user text messages, voice input, image recognition, and other means.

[0819] The emotion engine analyzes the user's emotions (e.g., joy, anger, sadness, surprise, etc.) and stores the results in a database. This emotion data is linked to other business data and used to analyze correlations.

[0820] 3. Correlation Diagram Generation Phase

[0821] server

[0822] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[0823] Using an analysis module, we analyze this data and model the relationship between people and tasks in a graph structure. We set up people and tasks as nodes, define their relationships as edges, and simultaneously incorporate sentiment data into the analysis.

[0824] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[0825] 4. UI / UX Phase

[0826] terminal

[0827] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed (analyzed) on the client side.

[0828] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[0829] When you hover your cursor over a node, relevant information (e.g., name, job title, task details) and sentiment status (e.g., the user's current sentiment information) will pop up. Furthermore, clicking on it will display past interaction history and comments.

[0830] 5. Feedback Loop

[0831] User

[0832] Users view correlation diagrams and confirm necessary information. They also provide feedback and suggestions for improvement to the system through a feedback form.

[0833] server

[0834] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[0835] Specific example

[0836] New task and integration scenario

[0837] User A (Project Manager)

[0838] User A starts a new project and assigns Users B and C as team members. User A then uses a project management tool to assign various tasks to each member.

[0839] User A monitors the project's progress and checks the members' emotional data.

[0840] Data collection

[0841] server

[0842] The system retrieves the latest task information and sentiment data for User A and their team members from the mail server, project management tools, calendar system, and sentiment engine.

[0843] The data is stored in a database.

[0844] Correlation diagram generation

[0845] server

[0846] The analysis module analyzes the acquired data and generates nodes (User A, User B, User C) and edges (task connections). Sentimental data is also incorporated, reflecting the user's current emotional status.

[0847] A visual correlation diagram is generated and exported in JSON format.

[0848] Displaying information

[0849] terminal

[0850] User A logs into the system and views the correlation diagram. In the correlation diagram, User A is displayed as the project manager, and Users B and C are displayed as members.

[0851] When user A hovers over a node, the task details, collaboration information, and the sentiment status of users B and C are displayed in a pop-up window.

[0852] feedback

[0853] User A

[0854] User A provides feedback on their experience with the system and suggestions for improvement through a feedback form. This allows the system to be improved to be more user-friendly.

[0855] By implementing this invention, collaboration in operations and identification of stakeholders become easier, and by considering emotional data, operational efficiency is further improved.

[0856] The following describes the processing flow.

[0857] Step 1:

[0858] server

[0859] The server establishes a connection by making API calls to internal systems (e.g., mail server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) for each system.

[0860] Step 2:

[0861] server

[0862] The server retrieves data from each system. This data includes user ID, task details (task name, deadline, progress), stakeholders, date, and time. API calls are used to collect the most up-to-date data.

[0863] Step 3:

[0864] server

[0865] The acquired data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is removed.

[0866] Step 4:

[0867] server

[0868] The system uses an emotion engine to acquire user emotion data. This is done by analyzing emotions through methods such as user text messages, voice input, and image recognition. The analyzed emotion data is stored in a database.

[0869] Step 5:

[0870] server

[0871] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[0872] Step 6:

[0873] server

[0874] The acquired data is analyzed, and the relationship between people and tasks is modeled using a graph structure (nodes and edges). People and tasks are set as nodes, their relationships are defined as edges, and sentiment data is also incorporated simultaneously.

[0875] Step 7:

[0876] server

[0877] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[0878] Step 8:

[0879] terminal

[0880] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed (analyzed) on the client side.

[0881] Step 9:

[0882] terminal

[0883] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[0884] Step 10:

[0885] terminal

[0886] When a user hovers over a node, a pop-up window displays detailed information about that node (e.g., name, role, task details) and its sentiment status (e.g., the user's current sentiment). Clicking on the node displays even more detailed information (e.g., past collaboration history, comments).

[0887] Step 11:

[0888] User

[0889] Users view correlation diagrams to check work-related information and emotional status. They also provide feedback and suggestions for improvement to the system through a feedback form.

[0890] Step 12:

[0891] server

[0892] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[0893] (Example 2)

[0894] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0895] Conventional data analysis systems had limitations in their ability to acquire person and business information and generate correlation diagrams. In particular, understanding business situations while considering users' emotional states was difficult, posing challenges to improving work efficiency and collaboration. Furthermore, there was insufficient mechanism for collecting user feedback on the generated correlation diagrams and incorporating it into subsequent data analyses.

[0896] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0897] In this invention, the server includes means for acquiring data containing person information and business information; means for analyzing the acquired data and modeling the correlation between people and business; means for collecting user emotion data and performing emotion analysis; means for generating a visual correlation diagram based on the modeled correlations and emotion data; and means for displaying the generated correlation diagram. This makes it easier to coordinate business operations and understand stakeholders, and by considering emotion data in particular, it becomes possible to improve business efficiency and facilitate smooth communication. Furthermore, by reflecting user feedback in the next data analysis, continuous improvement of the system can be achieved.

[0898] "Personal information" refers to data used to identify a specific individual within the system (e.g., user ID, name, job title).

[0899] "Business information" refers to data related to the progress and content of business operations (e.g., task name, deadline, progress status).

[0900] "Emotional data" refers to data that represents the user's emotional state (e.g., joy, anger, sadness, surprise).

[0901] "Sentiment analysis" is the process of extracting and analyzing emotional data from users' text messages, voice data, and other sources.

[0902] A "correlation diagram" is a graph-structured diagram that visually represents personal information, business information, and emotional data.

[0903] "Modeling" is the process of using data to represent the relationships between people and tasks as a graph structure.

[0904] A "means for generating visual correlation diagrams" refers to a system component that has the function of drawing correlation diagrams in a human-readable format using modeled data.

[0905] "Feedback" refers to information about opinions and suggestions for improvement provided by users who have used the system.

[0906] To implement this invention, the system should be constructed and operated according to the following procedure. Furthermore, by combining it with an emotion engine for recognizing user emotions, a more detailed correlation analysis can be performed.

[0907] 1. Data Collection Phase

[0908] server

[0909] The server establishes connections by making API calls to internal systems (e.g., mail server, project management tool, calendar system). It obtains access rights using authentication information (API keys or OAuth tokens) from each system. Specifically, we will use "JIRA" as the project management tool, "Google Calendar" as the calendar system, and "Microsoft Exchange" as the mail server.

[0910] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time via API calls. To retrieve only the most recent data, it records the date and time of the previous data retrieval and collects differential data. The retrieved data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is excluded.

[0911] 2. Collection of emotional data

[0912] server

[0913] The server uses an emotion engine to collect user emotion data. This data is based on information obtained through user text messages, voice input, image recognition, and other means. Specifically, IBM Watson Natural Language Understanding is used for emotion analysis of text messages, and Google Cloud Speech-to-Text is used for emotion analysis of voice input.

[0914] The emotion engine analyzes the user's emotions (e.g., joy, anger, sadness, surprise, etc.) and stores the results in a database. This emotion data is linked to other business data and used to analyze correlations.

[0915] 3. Correlation Diagram Generation Phase

[0916] server

[0917] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[0918] Using an analysis module, we will analyze this data and model the relationship between people and tasks in a graph structure. We will set up people and tasks as nodes, define their relationships as edges, and simultaneously incorporate sentiment data into the analysis. As a concrete example, we will use the Python "NetworkX" library to generate the graph structure.

[0919] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[0920] 4. UI / UX Phase

[0921] terminal

[0922] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed on the client side. As a specific example, "React" is used as the frontend framework, and "D3.js" or "Cytoscape.js" is used as the correlation diagram library.

[0923] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on the analyzed data. Nodes and edges are visually arranged to make them easy for the user to understand. When the cursor hovers over a node, related information (e.g., name, job title, task details) and sentiment status (e.g., the user's current sentiment information) pop up. Furthermore, clicking on a node displays past interaction history and comments.

[0924] 5. Feedback Loop

[0925] User

[0926] Users view correlation diagrams and confirm necessary information. They also provide feedback and suggestions for improvement to the system through a feedback form.

[0927] server

[0928] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[0929] Example of a prompt

[0930] "Create a new task and generate a correlation diagram that includes an analysis of sentiment data."

[0931] This system makes it easier to coordinate tasks and identify stakeholders, and by considering emotional data, operational efficiency can be further improved.

[0932] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0933] Step 1:

[0934] The server makes an API call to the internal system and establishes a connection.

[0935] Input: Authentication information for internal systems (e.g., API key, OAuth token).

[0936] The server makes API calls to the mail server, project management tool, and calendar system to obtain access rights. Specifically, it adds the authentication information to the HTTP request header and establishes the connection.

[0937] Output: Handle to a successful API connection.

[0938] Step 2:

[0939] The server retrieves the necessary data from the internal system.

[0940] Input: API connection handle, last data retrieval date and time.

[0941] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time. For example, it sends a GET request to the JIRA API to retrieve the latest task information. It compares this to the previous data retrieval date and time and collects the difference data.

[0942] Output: A set of acquired data.

[0943] Step 3:

[0944] The server validates the data it retrieves and stores it in the central database.

[0945] Input: A set of retrieved data.

[0946] The server performs validation checks to ensure the data format is correct and all required fields are present. Only data that passes validation is inserted into the database. Specifically, the SQLAlchemy library in Python is used to insert data into the "tasks" table.

[0947] Output: Data stored in the database.

[0948] Step 4:

[0949] The server uses an emotion engine to collect user emotion data.

[0950] Input: User text messages, audio data.

[0951] The server sends text messages to sentiment analysis engines such as IBM Watson Natural Language Understanding for sentiment analysis. Similarly, sentiment analysis is performed on voice input using Google Cloud Speech-to-Text.

[0952] Output: Acquired sentiment data.

[0953] Step 5:

[0954] The server stores the emotion analysis results in a database.

[0955] Input: Sentiment analysis results.

[0956] The server inserts the analyzed emotion data into the "emotions" table in the database. Specifically, it stores the emotion status (e.g., joy, anger, sadness) and its associated task ID.

[0957] Output: Emotional data stored in the database.

[0958] Step 6:

[0959] The server scans the database to retrieve the latest information.

[0960] Input: Periodic scan timer.

[0961] The server scans the "Person Information," "Task Information," and "Emotional Data" tables to retrieve the latest data. This is done using SQL SELECT queries. For example, an SQL query can be used to retrieve updated task information.

[0962] Output: Set of the latest data.

[0963] Step 7:

[0964] The system analyzes the data acquired by the server and models its graph structure.

[0965] Input: Set of the latest data.

[0966] The server uses Python's NetworkX library to define people and tasks as nodes, and their relationships as edges. Sentiment data is also added to the nodes. For example, each user and task is added as a node, and their relationships are defined as edges.

[0967] Output: Modeled graph data.

[0968] Step 8:

[0969] The server converts the modeled graph data into a visual correlation diagram and sends it to the client in JSON format.

[0970] Input: Modeled graph data.

[0971] The server serializes the generated graph data into JSON format and sends it to the client. Specifically, it uses Flask's jsonify method to generate JSON data and returns it to the client.

[0972] Output: Data in JSON format.

[0973] Step 9:

[0974] The terminal retrieves the latest correlation diagram data from the server.

[0975] Input: Client request.

[0976] The device (the user's web browser) makes a GET request to a specified endpoint on the server and retrieves data in JSON format. Specifically, it uses the JavaScript fetch API to retrieve data from the / api / relation-graph endpoint.

[0977] Output: Retrieved JSON data.

[0978] Step 10:

[0979] The JSON data acquired by the device is analyzed, and a correlation diagram is drawn.

[0980] Input: Retrieved JSON data.

[0981] The device parses JSON data using D3.js or Cytoscape.js and visually arranges nodes and edges. For example, it adds nodes and edges using the cy.add method of Cytoscape.js and draws a correlation diagram.

[0982] Output: The generated correlation diagram.

[0983] Step 11:

[0984] When the device hovers over a node, it will display relevant information and sentiment status in a pop-up window.

[0985] Input: User hover event.

[0986] When a hover event occurs on a node, relevant information and sentiment status will be displayed as a tooltip. Specifically, the tooltip will be displayed using the Cytoscape.js method `cy.on('mouseover', 'node', function(evt) {...}`.

[0987] Output: The displayed tooltip.

[0988] Step 12:

[0989] Users submit their opinions and suggestions for improvement regarding the system through a feedback form.

[0990] Input: User feedback content.

[0991] Users enter their opinions and suggestions for improvement into a feedback form displayed on the UI and press the submit button. In other words, the form data is sent to the server.

[0992] Output: Sent feedback data.

[0993] Step 13:

[0994] The server saves user feedback to a database.

[0995] Input: Submitted feedback data.

[0996] The server inserts the feedback data received via POST request into the "feedback" table in the database. Specifically, it uses an SQL INSERT query to save the feedback content.

[0997] Output: Feedback stored in the database.

[0998] Step 14:

[0999] The server will take the feedback into consideration during the next data analysis.

[1000] Input: Feedback stored in the database.

[1001] The server will refer to the feedback during the next data analysis and incorporate it into the analysis model. Specifically, it will adjust the analysis algorithm and graph display based on the feedback.

[1002] Output: Analysis data reflecting the feedback.

[1003] This system will facilitate smoother collaboration between tasks and better identify stakeholders, and by considering emotional data, further improvements in operational efficiency can be expected.

[1004] (Application Example 2)

[1005] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1006] Traditional task management systems are limited to understanding correlations based on personal and task information, and lack consideration for user emotions, leading to problems such as decreased work efficiency and satisfaction. In particular, in workplaces where emotional changes directly impact work efficiency, the lack of collection and analysis of this information posed a significant challenge. There is a need to build a task management system that incorporates employee emotional data to ensure smooth workflow and improve employee satisfaction.

[1007] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1008] In this invention, the server includes means for acquiring data including person information, task information, and emotion data; means for analyzing the acquired data and modeling the correlation between people and tasks; and means for generating a visual correlation diagram based on the modeled correlations and emotion data. This makes it possible to perform effective task management while taking into account the user's emotions in real time.

[1009] "Personal information" refers to data related to an individual, such as the user's name, job title, and contact information.

[1010] "Business information" refers to data related to work or projects, such as task names, deadlines, and progress status.

[1011] "Emotional data" refers to data that indicates a user's emotional state, analyzed from sources such as text messages, voice input, and image recognition.

[1012] "Correlation" refers to the relationship or influence that exists between a person and a task.

[1013] "Modeling" refers to analyzing acquired data and representing the relationships between the data mathematically or graphically.

[1014] A "visual correlation diagram" is a graph that visually displays people and tasks as nodes and relationships as edges, based on analyzed data.

[1015] "Popup display" is a feature that displays related information in a small window format in response to a specific action (e.g., hovering the cursor over a node).

[1016] "Feedback" refers to the opinions, evaluations, and suggestions for improvement provided by system users.

[1017] A system for carrying out this invention includes the following elements.

[1018] Data collection phase

[1019] server

[1020] The server establishes a connection by making API calls to internal systems (e.g., email server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) for each system.

[1021] The server retrieves data such as user ID, task details, stakeholders, date, and time via API calls. To retrieve only the most recent data, it records the date and time of the previous data retrieval and collects differential data.

[1022] The acquired data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is excluded.

[1023] Collection of emotional data

[1024] server

[1025] The server uses an emotion engine to collect user emotion data. This data is based on information obtained through user text messages, voice input, image recognition, and other means.

[1026] The emotion engine analyzes the user's emotions (e.g., joy, anger, sadness, surprise, etc.) and stores the results in a database. This emotion data is linked to other business data and used to analyze correlations.

[1027] Correlation diagram generation phase

[1028] server

[1029] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[1030] Using an analysis module, we analyze this data and model the relationship between people and tasks in a graph structure. We set up people and tasks as nodes, define their relationships as edges, and simultaneously incorporate sentiment data into the analysis.

[1031] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[1032] UI / UX Phase

[1033] terminal

[1034] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed on the client side.

[1035] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[1036] When you hover your cursor over a node, relevant information (e.g., name, job title, task details) and sentiment status (e.g., the user's current sentiment information) will pop up. Furthermore, clicking on it will display past interaction history and comments.

[1037] Feedback loop

[1038] User

[1039] Users view correlation diagrams and confirm necessary information. They also provide feedback and suggestions for improvement to the system through a feedback form.

[1040] server

[1041] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[1042] Specific example

[1043] New task and integration scenario

[1044] Employee A (Manager)

[1045] Employee A launches a new project and assigns employees B and C as team members. Employee A uses a project management tool to assign various tasks to each member. Employee A monitors the project's progress and checks the members' emotional data.

[1046] Example of a prompt

[1047] Please tell me about the current progress of your task and how you feel about it.

[1048] Please share your thoughts on completing today's tasks.

[1049] Do you have any concerns about the next task?

[1050] Implementing this system makes it possible to manage tasks effectively while taking user emotions into consideration in real time.

[1051] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1052] Step 1:

[1053] The server establishes connections by making API calls to internal systems (e.g., mail server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) from each system. The server then retrieves the latest personnel and business information from each system. The input is the API endpoint and authentication information for each system, and the output is the retrieved personnel and business information. The data is stored in a central database.

[1054] Step 2:

[1055] The server retrieves data such as each user's ID, task details, stakeholders, date, and time via API calls. It records the date and time of the previous data retrieval and collects only differential data. The input is the date and time of the previous data retrieval and the result of the current API call, and the output is the new and updated data from the current call. This data is validated, and inconsistent data is excluded.

[1056] Step 3:

[1057] The server uses an emotion engine to collect user emotion data. The emotion engine analyzes emotions based on input data such as text messages, voice input, and image recognition, and stores the results in a database. The input is data related to the user's emotions, and the output is the analyzed emotion data.

[1058] Step 4:

[1059] The server periodically scans the database to retrieve the latest person information, task information, and sentiment data. Based on this data, the relationship between people and tasks is modeled in a graph structure. The input is the latest data retrieved from the database, and the output is the modeled graph data.

[1060] Step 5:

[1061] The server uses an analysis module to generate a visual correlation diagram based on modeled graph data. Sentiment data is also incorporated. The input is graph data and sentiment data, and the output is a visual correlation diagram. The generated correlation diagram can be exported in JSON or XML format.

[1062] Step 6:

[1063] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server and parses (analyzes) it on the client side. The input is correlation diagram data in JSON format provided by the server, and the output is the parsed data.

[1064] Step 7:

[1065] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand. The input is the parsed data, and the output is the displayed correlation diagram.

[1066] Step 8:

[1067] When a user hovers over a node, relevant information (e.g., name, role, task details) and sentiment status pop up. Clicking on the node displays past interaction history and comments. The input is the cursor's position, and the output is the detailed information displayed in the pop-up.

[1068] Step 9:

[1069] Users provide feedback on their experience with the system and suggestions for improvement through a feedback form. The input is the user's feedback content, and the output is feedback data stored in the database.

[1070] Step 10:

[1071] The server stores user feedback in a database and considers this feedback during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs. The input is the feedback data stored in the database, and the output is the improved analysis model that takes the feedback into account.

[1072] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1073] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1074] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1075] [Third Embodiment]

[1076] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1077] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1078] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1079] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1080] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1081] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1082] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1083] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1084] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1085] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1086] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1087] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1088] To implement this invention, the system should be constructed and operated according to the following procedure.

[1089] 1. Data Collection Phase

[1090] server

[1091] The server makes API calls to various internal systems (e.g., email server, project management tool, calendar system) to retrieve personnel and business information.

[1092] The server authenticates with each system and secures the necessary access rights. The data retrieved by API calls includes user ID, task details, stakeholders, date and time, etc.

[1093] The acquired data is organized and stored in a central database. Each data entry undergoes validation to maintain data consistency and integrity.

[1094] 2. Correlation Diagram Generation Phase

[1095] server

[1096] The server periodically scans the database to retrieve the latest personnel and business information.

[1097] The analysis module analyzes the data and models the correlation between people and tasks as a graph structure (nodes and edges).

[1098] Based on the graph data, a visual correlation diagram is generated. This correlation diagram can be exported in JSON or XML format and used for display on the client side.

[1099] 3. UI / UX Phase

[1100] terminal

[1101] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server.

[1102] The device uses libraries such as D3.js and Cytoscape.js to display correlation diagrams. This allows users to view correlation diagrams that are easy to understand visually.

[1103] When you hover over a node, relevant information (such as the person's name, job title, and task details) pops up. If the user clicks on a node, even more detailed information is displayed, including past collaboration history and comments.

[1104] 4. Feedback Loop

[1105] User

[1106] Users provide feedback, evaluations, and suggestions for improvement to the system through a feedback form. This is treated as specific improvement requests to enhance operational efficiency.

[1107] The server stores the feedback in a database and considers it during subsequent data analysis and correlation diagram generation. This allows the system to continuously improve, increasing its accuracy and usefulness.

[1108] Specific example

[1109] New task and integration scenario

[1110] User A (Project Manager)

[1111] A new project is launched, and User A is appointed as the project manager. User A assigns Users B and C as team members.

[1112] User A assigns various tasks to each member through a project management tool.

[1113] Data collection

[1114] server

[1115] Retrieve the latest task information for User A and their team members from the mail server, project management tools, and calendar system.

[1116] The data is stored in a database.

[1117] Correlation diagram generation

[1118] server

[1119] The analysis module analyzes the acquired data and generates nodes (User A, User B, User C) and edges (task connections).

[1120] A visual correlation diagram is generated and exported in JSON format.

[1121] Displaying information

[1122] terminal

[1123] User A logs into the system and views the correlation diagram. In the correlation diagram, User A is displayed as the project manager, and Users B and C are displayed as members.

[1124] When user A hovers the cursor over a node, its task details and related information are displayed in a pop-up window.

[1125] feedback

[1126] User A

[1127] User A provides feedback on their experience with the system and suggestions for improvement through a feedback form. This allows the system to be improved to be more user-friendly.

[1128] By implementing this invention, coordination of operations and identification of stakeholders become easier, leading to a significant improvement in operational efficiency.

[1129] The following describes the processing flow.

[1130] Step 1:

[1131] server

[1132] The server makes API calls to internal systems (e.g., email server, project management tool, calendar system) to establish a connection. It then obtains access rights using authentication information (API key, OAuth token) for each system.

[1133] Step 2:

[1134] server

[1135] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time via API calls. To retrieve only the most recent data, it records the date and time of the previous data retrieval and collects differential data.

[1136] Step 3:

[1137] server

[1138] The acquired data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is removed.

[1139] Step 4:

[1140] server

[1141] The server periodically scans the database to retrieve the latest personnel and business information. This ensures that the data necessary for generating the correlation diagram is always up-to-date.

[1142] Step 5:

[1143] server

[1144] The analysis module is used to analyze the data and model the relationship between people and tasks in a graph structure. People and tasks are set as nodes, and their relationships are defined as edges.

[1145] Step 6:

[1146] server

[1147] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[1148] Step 7:

[1149] terminal

[1150] The terminal retrieves the latest correlation diagram data from the server when the user logs into the system. The data is received in JSON format and parsed (analyzed) on the client side.

[1151] Step 8:

[1152] terminal

[1153] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[1154] Step 9:

[1155] terminal

[1156] When a user hovers over a node, detailed information about that node (e.g., name, role, task details) pops up. Clicking on it displays even more detailed information (e.g., past collaboration history, comments).

[1157] Step 10:

[1158] User

[1159] Users view correlation diagrams and confirm the necessary information. Furthermore, they can provide feedback and suggestions for improvement to the system through a feedback form.

[1160] Step 11:

[1161] server

[1162] The server saves user feedback to a database. This feedback is then taken into consideration during the next data analysis, and improvements are implemented accordingly.

[1163] Step 12:

[1164] server

[1165] In the next data analysis and correlation diagram generation cycle, the saved feedback will be referenced, and necessary adjustments and improvements will be made. This allows the system to be continuously improved and optimized.

[1166] (Example 1)

[1167] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1168] Conventional systems made it difficult to grasp personnel and work information within companies, posing challenges to improving operational efficiency. Furthermore, data analysis for visually displaying the correlation between personnel and work, and interactive information display based on those results, were insufficient. As a result, it was difficult for users to intuitively grasp the status of work and make appropriate decisions.

[1169] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1170] In this invention, the server includes means for acquiring person information and business information via an API, means for authenticating the acquired data and securing access rights, and means for validating the acquired data and storing it in a central database. This makes it possible to accurately acquire necessary data and store and manage it safely and efficiently. It also includes means for an analysis module that periodically scans the database and analyzes the latest person information and business information, means for modeling the analyzed data as a graph structure, means for generating a visual correlation diagram based on the modeled correlations and exporting it in JSON or XML format, and means for acquiring the generated correlation diagram and displaying the visual correlation diagram. This makes it easier for users to intuitively understand the correlation between people and business, enabling them to make appropriate decisions quickly. Furthermore, it includes means for displaying information about a node in a pop-up when the cursor is placed over a node on the correlation diagram, and means for saving user feedback in the database and considering it during the next data analysis. This allows the system to be continuously improved and enables the provision of highly accurate information that meets the needs of users.

[1171] "Personal information" refers to identifiable information about a specific individual, including name, job title, contact information, and department.

[1172] "Business information" refers to information about business operations and tasks performed within a company, including task details, deadlines, assigned personnel, and related projects.

[1173] "API" stands for Application Programming Interface, and refers to a standardized interface that enables data exchange between different software systems.

[1174] Authentication is the process of verifying a user's identity when accessing an information system, and it primarily uses a username, password, token, etc.

[1175] "Access rights" refer to the operational permissions for data and resources within an information system, and include permissions such as read, write, and execute.

[1176] "Validation" is the process of verifying whether the acquired data conforms to predetermined standards and formats.

[1177] A "central database" is a database used to centrally store and manage all acquired data, and examples include SQL databases and NoSQL databases.

[1178] "Periodic scanning" is the process of checking, updating, and retrieving data within a database at regular time intervals.

[1179] An "analysis module" is a software component used to analyze collected data and extract specific patterns or relationships.

[1180] A "graph structure" is a data structure consisting of nodes (vertices) and edges, and is used to model the relationship between people and tasks.

[1181] A "visual correlation diagram" is a diagram that visually represents a graph structure, allowing for an intuitive understanding of the relationship between people and tasks.

[1182] "JSON" stands for JavaScript Object Notation, and it is a lightweight data exchange format for structuring, storing, and exchanging data.

[1183] XML stands for eXtensible Markup Language, and it is a markup language used to structure, store, and exchange data.

[1184] A "node" refers to a vertex in a graph structure, and usually represents an entity such as a person or a task.

[1185] "Popup display" is a feature that displays a small window containing supplementary information in response to a specific action (for example, mouseover).

[1186] "Feedback" refers to opinions, suggestions for improvement, and evaluations provided by system users, which are used to improve the system.

[1187] "Data analysis" is the process of extracting patterns and relationships from collected data using statistical methods and algorithms to deepen our understanding of the data.

[1188] Modes for carrying out the invention

[1189] This invention is a system that acquires personnel and business information from various internal information systems, analyzes them, and generates and displays a correlation diagram that visually represents the relationships between them. The following describes a specific implementation of this system.

[1190] Data collection phase

[1191] server

[1192] The server retrieves data from the company's internal email system, project management system, and calendar system via APIs. This utilizes general-purpose APIs such as the Google Calendar API, Microsoft Exchange API, and JIRA API.

[1193] Specifically, this involves sending API requests to retrieve user and task information. For example, you can access the API endpoint using the "curl" command and extract the necessary data.

[1194] The acquired data is authenticated using OAuth 2.0 or similar methods to ensure appropriate access rights. This authentication guarantees the security and reliability of the data.

[1195] The server validates the retrieved data and stores only the data that conforms to the schema in the central database. The database used is a relational database such as MySQL or PostgreSQL.

[1196] Correlation diagram generation phase

[1197] server

[1198] The server periodically uses a job scheduler (e.g., a Cron job) to scan the database. This scan retrieves the latest personnel and business information, preparing it for analysis.

[1199] The server uses an analysis module (for example, the NetworkX library in Python) to analyze the person and task information in the database. As a result of the analysis, a graph structure is generated in which each person is a node and each task is an edge.

[1200] The server renders the generated graph data as a visual correlation diagram and exports it in JSON or XML format. This export is then used for display on the client side.

[1201] UI / UX Phase

[1202] terminal

[1203] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. AJAX requests are used to retrieve the data.

[1204] On the user's device, JavaScript libraries such as D3.js and Cytoscape.js are used to display correlation diagrams based on the acquired data. This allows users to intuitively understand the relationship between people and tasks.

[1205] When you hover your cursor over a node, related information will pop up, and if you need more detailed information, you can click on the node to display that information.

[1206] Feedback loop

[1207] User

[1208] Users provide feedback and suggestions for improvement to the system through a feedback form. For example, they can enter specific requests such as, "I'd like the UI layout to be a little easier to use."

[1209] The server stores the collected feedback in a database, which is then used for future data analysis and system updates.

[1210] Specific example

[1211] New task and integration scenario

[1212] User A (Project Manager)

[1213] User A launches a new project and assigns team members. User A assigns tasks to each member through a project management tool.

[1214] Data collection

[1215] server

[1216] The system retrieves the latest task information for User A and team members from email systems, project management systems, and calendar systems. The retrieved data is stored in a database.

[1217] Correlation diagram generation

[1218] server

[1219] The analysis module analyzes the data and generates nodes (User A, User B, User C) and edges (task connections). A visual correlation diagram is exported in JSON format.

[1220] Displaying information

[1221] terminal

[1222] User A logs into the system and views the correlation diagram. The diagram displays User A as the project manager, with Users B and C as members. Hovering the cursor over a node displays task details and collaboration information.

[1223] feedback

[1224] User A

[1225] User A enters their feedback on the system's usability and areas for improvement into a feedback form. This feedback will be used to improve the system.

[1226] Example of a prompt

[1227] This system collects internal company data and generates correlation diagrams of stakeholders. It collects data from the company's email system, project management system, and calendar system, and uses an analysis module to generate correlation diagrams. The generated correlation diagrams are viewable by users, and system improvements are made based on their feedback.

[1228] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1229] Step 1: Data Acquisition

[1230] server

[1231] The server retrieves data from the company's email system, project management system, and calendar system.

[1232] It sends requests to the APIs of various systems (such as Google Calendar API, Microsoft Exchange API, and JIRA API) to retrieve user and task information.

[1233] For example, execute a command like "curl -X GET 'https: / / api.calendar.google.com / calendar / v3 / calendars / primary / events' -H 'Authorization: Bearer [ACCESS_TOKEN]'".

[1234] Input: API endpoint, authentication token

[1235] Output: Data response in JSON format containing person information and job information.

[1236] Step 2: Authentication and securing access rights

[1237] server

[1238] The server authenticates the data it retrieves and secures the necessary access rights.

[1239] Obtain an access token using the OAuth 2.0 flow and include it in the API request header.

[1240] Input: User credentials, authentication server endpoint

[1241] Output: Access token

[1242] Step 3: Data validation and saving

[1243] server

[1244] The server validates the retrieved data and checks if it conforms to the schema.

[1245] Data that does not conform to the schema is removed, and only conforming data is stored in the central database.

[1246] Input: Acquired JSON data, schema definition

[1247] Output: Validated data stored in the central database

[1248] Step 4: Regular data scans

[1249] server

[1250] The server scans the data in the database using a periodic job scheduler (e.g., a Cron job).

[1251] Obtain the latest information from the database and prepare for analysis.

[1252] For example, the scan is performed daily in the format "0 0 / usr / bin / python3 / path / to / data_scan.py".

[1253] Input: Job scheduler settings, database connection information

[1254] Output: Latest dataset

[1255] Step 5: Data Analysis

[1256] server

[1257] The server analyzes the data using an analysis module (for example, the NetworkX library in Python).

[1258] A graph structure is generated using each person as a node and each task as an edge.

[1259] Input: Latest dataset

[1260] Output: Graph structure consisting of nodes and edges

[1261] Step 6: Generate and export the correlation diagram

[1262] server

[1263] The server renders a visual correlation diagram based on the generated graph structure.

[1264] The correlation diagram is exported in JSON or XML format and used for display on the client side.

[1265] Input: Graph structure

[1266] Output: Correlation diagram data in JSON or XML format

[1267] Step 7: Obtain and display correlation diagram data

[1268] terminal

[1269] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server.

[1270] This program uses AJAX requests to retrieve data in JSON format and displays correlation diagrams using D3.js or Cytoscape.js.

[1271] Input: User login information, server correlation diagram data

[1272] Output: Displayed interactive correlation diagram

[1273] Step 8: Interactive Information Display

[1274] terminal

[1275] When a user hovers their cursor over a node in the correlation diagram, related information pops up.

[1276] Clicking on a node will display more detailed information.

[1277] Input: User interaction, correlation diagram data

[1278] Output: Detailed information displayed in a pop-up window

[1279] Step 9: Gathering Feedback

[1280] User

[1281] Users can submit their opinions and suggestions for improvement regarding the system through a feedback form.

[1282] Input: Feedback content

[1283] Output: Feedback data sent to the server

[1284] Step 10: Saving and analyzing feedback

[1285] server

[1286] The server stores the collected feedback in a database and takes it into consideration during the next data analysis.

[1287] Input: Feedback data

[1288] Output: Feedback data stored in the database

[1289] (Application Example 1)

[1290] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1291] Traditional factories faced challenges in monitoring work and inventory information in real time and developing efficient work plans. Furthermore, it was difficult for factory robots and terminals to grasp work progress and inventory status and respond dynamically. This often hindered efficient work and reduced overall factory productivity.

[1292] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1293] In this invention, the server includes means for acquiring data including person information and work information; means for analyzing the acquired data and modeling the correlation between people and work; means for generating a visual correlation diagram based on the modeled correlation; means for robots and terminals to create efficient work plans based on the analysis results; means for displaying the generated correlation diagram; and means for robots and terminals to improve real-time work efficiency by referring to the displayed correlation diagram. This makes it possible to monitor work information and inventory information within the factory in real time and manage them efficiently.

[1294] "Personal information" refers to identifiable data about individual people, including their name, job title, and assigned tasks.

[1295] "Work information" refers to detailed data about a specific task, including the type of work, progress, person in charge, and deadline.

[1296] "Correlation" refers to the relationship between different data items, and specifically to the relationship between people and tasks.

[1297] "Modeling" refers to the process of transforming raw data into a mathematical or logical structure, making it easier to understand and analyze the relationships and patterns within the data.

[1298] A "visual correlation diagram" is a chart that visually represents the correlation between data, using nodes (data items) and edges (relationships) to show the structure.

[1299] The term "robot" refers to a machine or device that performs tasks automatically within a factory, and whose movements are controlled by work information and correlation diagrams.

[1300] "Terminal" refers to an electronic device used for displaying and manipulating data, and includes devices that display work information and correlation diagrams within a factory.

[1301] A "work plan" refers to a plan for carrying out work efficiently and effectively, and it manages the assignment and timing of each task.

[1302] "Real-time" refers to processing and reflecting data and information instantly with virtually no delay.

[1303] "Work efficiency" is a measure of how efficiently work is being performed, aiming to minimize wasted time and resources.

[1304] To implement this invention, the system is constructed and operated in the following steps: the server, robot, terminal, and user each play their respective roles.

[1305] 1. Data Collection Phase

[1306] The server makes API calls to various systems (e.g., inventory management system, work scheduling system, machine operation status monitoring system) to retrieve person and work information. The server authenticates with each system and secures the necessary access rights. The retrieved data includes user ID, work details, stakeholders, date and time, etc. The retrieved data is organized and stored in a central database.

[1307] 2. Correlation Diagram Generation Phase

[1308] The server periodically scans the database to retrieve the latest person and task information. An analysis module analyzes the data and models the correlation between people and tasks as a graph structure. Based on this graph data, a visual correlation diagram is generated. This correlation diagram is exported in JSON or XML format and used for display on the terminal.

[1309] 3. UI / UX Phase

[1310] The terminal and robot retrieve the latest correlation diagram data from the server and display the correlation diagram using libraries such as D3.js and Cytoscape.js. When a user views the correlation diagram and hovers over a node, relevant information (e.g., person's name, job title, and work details) pops up. If the user clicks on a node, more detailed information is displayed, including past collaboration history and comments.

[1311] 4. Feedback Loop

[1312] Users provide feedback, evaluations, and suggestions for improvement to the system through a feedback form. The server stores the feedback in a database and considers it during subsequent data analysis and correlation diagram generation. This allows the system to be continuously improved, increasing its accuracy and usefulness.

[1313] The specific software and hardware used in this system include the requests library for API calls, the networkx library and Flask framework for data analysis, and D3.js and Cytoscape.js for generating visual correlation diagrams.

[1314] Specific example

[1315] For example, suppose a robot in a factory is assembling parts. To enable this robot to understand its work progress and inventory status in real time and to carry out the work more efficiently, a system that monitors work progress and inventory status is used. This makes it possible to immediately notify the manager of instructions to replenish parts that are running low on stock.

[1316] Examples of prompts for generative AI models

[1317] "Please tell me how to improve work efficiency within the factory. Specifically, please tell me how to build a system that monitors work progress and inventory status in real time to improve efficiency."

[1318] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1319] Step 1:

[1320] The server makes API calls to various systems (inventory management system, work scheduling system, machine operation status monitoring system) to retrieve personnel and work information. It uses the API call URL and authentication information as input, and outputs JSON data of the retrieved personnel and work information. This data is temporarily stored in a central database.

[1321] Step 2:

[1322] The server periodically scans the database to retrieve the latest person and task information. The person and task information is input from the database scan results and analyzed to model the correlation between people and tasks. A data analysis library (e.g., networkx) is used for the analysis, and graph structure data is generated as output.

[1323] Step 3:

[1324] The server generates a visual correlation diagram based on the analyzed graph structure data. It accepts graph structure data as input and creates the correlation diagram using visualization libraries such as D3.js or Cytoscape.js. As output, the correlation diagram is exported in JSON or XML format for later display.

[1325] Step 4:

[1326] The terminal (or robot) retrieves the latest correlation diagram data from the server and uses a display library (e.g., D3.js, Cytoscape.js) to visually display the correlation diagram. The terminal takes the retrieved JSON or XML data as input and generates output (a visual correlation diagram) by rendering the correlation diagram.

[1327] Step 5:

[1328] Users view the generated correlation diagram using their device. When a user hovers their cursor over a node on the correlation diagram, relevant information (person's name, job title, and details of their work) pops up. This allows users to view detailed information about the corresponding node in real time.

[1329] Step 6:

[1330] Users provide feedback, evaluations, and suggestions for improvement to the system through a feedback form. The input feedback information is sent to the server and stored in a database. This feedback is then considered during subsequent data analysis and correlation diagram generation, contributing to the continuous improvement of the system.

[1331] Step 7:

[1332] The server considers the stored feedback information during the next data analysis. It uses past feedback information as input, influencing the analysis results. This results in an output analysis with improvements based on the feedback.

[1333] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1334] To implement this invention, the system should be constructed and operated according to the following procedure. Furthermore, by combining it with an emotion engine for recognizing user emotions, a more detailed correlation analysis can be performed.

[1335] 1. Data Collection Phase

[1336] server

[1337] The server establishes a connection by making API calls to internal systems (e.g., email server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) for each system.

[1338] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time via API calls. To retrieve only the most recent data, the server records the date and time of the previous data retrieval and collects differential data.

[1339] The acquired data is stored in a central database. To maintain data consistency and integrity, validation is performed to remove inconsistent data.

[1340] 2. Collection of emotional data

[1341] server

[1342] The server uses an emotion engine to collect user emotion data. This data is based on information obtained through user text messages, voice input, image recognition, and other means.

[1343] The emotion engine analyzes the user's emotions (e.g., joy, anger, sadness, surprise, etc.) and stores the results in a database. This emotion data is linked to other business data and used to analyze correlations.

[1344] 3. Correlation Diagram Generation Phase

[1345] server

[1346] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[1347] Using an analysis module, we analyze this data and model the relationship between people and tasks in a graph structure. We set up people and tasks as nodes, define their relationships as edges, and simultaneously incorporate sentiment data into the analysis.

[1348] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[1349] 4. UI / UX Phase

[1350] terminal

[1351] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed (analyzed) on the client side.

[1352] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[1353] When you hover your cursor over a node, relevant information (e.g., name, job title, task details) and sentiment status (e.g., the user's current sentiment information) will pop up. Furthermore, clicking on it will display past interaction history and comments.

[1354] 5. Feedback Loop

[1355] User

[1356] Users view correlation diagrams and confirm necessary information. They also provide feedback and suggestions for improvement to the system through a feedback form.

[1357] server

[1358] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[1359] Specific example

[1360] New task and integration scenario

[1361] User A (Project Manager)

[1362] User A starts a new project and assigns Users B and C as team members. User A then uses a project management tool to assign various tasks to each member.

[1363] User A monitors the project's progress and checks the members' emotional data.

[1364] Data collection

[1365] server

[1366] The system retrieves the latest task information and sentiment data for User A and their team members from the mail server, project management tools, calendar system, and sentiment engine.

[1367] The data is stored in a database.

[1368] Correlation diagram generation

[1369] server

[1370] The analysis module analyzes the acquired data and generates nodes (User A, User B, User C) and edges (task connections). Sentimental data is also incorporated, reflecting the user's current emotional status.

[1371] A visual correlation diagram is generated and exported in JSON format.

[1372] Displaying information

[1373] terminal

[1374] User A logs into the system and views the correlation diagram. In the correlation diagram, User A is displayed as the project manager, and Users B and C are displayed as members.

[1375] When user A hovers over a node, the task details, collaboration information, and the sentiment status of users B and C are displayed in a pop-up window.

[1376] feedback

[1377] User A

[1378] User A provides feedback on their experience with the system and suggestions for improvement through a feedback form. This allows the system to be improved to be more user-friendly.

[1379] By implementing this invention, collaboration in operations and identification of stakeholders become easier, and by considering emotional data, operational efficiency is further improved.

[1380] The following describes the processing flow.

[1381] Step 1:

[1382] server

[1383] The server establishes a connection by making API calls to internal systems (e.g., mail server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) for each system.

[1384] Step 2:

[1385] server

[1386] The server retrieves data from each system. This data includes user ID, task details (task name, deadline, progress), stakeholders, date, and time. API calls are used to collect the most up-to-date data.

[1387] Step 3:

[1388] server

[1389] The acquired data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is removed.

[1390] Step 4:

[1391] server

[1392] The system uses an emotion engine to acquire user emotion data. This is done by analyzing emotions through methods such as user text messages, voice input, and image recognition. The analyzed emotion data is stored in a database.

[1393] Step 5:

[1394] server

[1395] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[1396] Step 6:

[1397] server

[1398] The acquired data is analyzed, and the relationship between people and tasks is modeled using a graph structure (nodes and edges). People and tasks are set as nodes, their relationships are defined as edges, and sentiment data is also incorporated simultaneously.

[1399] Step 7:

[1400] server

[1401] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[1402] Step 8:

[1403] terminal

[1404] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed (analyzed) on the client side.

[1405] Step 9:

[1406] terminal

[1407] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[1408] Step 10:

[1409] terminal

[1410] When a user hovers over a node, a pop-up window displays detailed information about that node (e.g., name, role, task details) and its sentiment status (e.g., the user's current sentiment). Clicking on the node displays even more detailed information (e.g., past collaboration history, comments).

[1411] Step 11:

[1412] User

[1413] Users view correlation diagrams to check work-related information and emotional status. They also provide feedback and suggestions for improvement to the system through a feedback form.

[1414] Step 12:

[1415] server

[1416] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[1417] (Example 2)

[1418] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1419] Conventional data analysis systems had limitations in their ability to acquire person and business information and generate correlation diagrams. In particular, understanding business situations while considering users' emotional states was difficult, posing challenges to improving work efficiency and collaboration. Furthermore, there was insufficient mechanism for collecting user feedback on the generated correlation diagrams and incorporating it into subsequent data analyses.

[1420] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1421] In this invention, the server includes means for acquiring data containing person information and business information; means for analyzing the acquired data and modeling the correlation between people and business; means for collecting user emotion data and performing emotion analysis; means for generating a visual correlation diagram based on the modeled correlations and emotion data; and means for displaying the generated correlation diagram. This makes it easier to coordinate business operations and understand stakeholders, and by considering emotion data in particular, it becomes possible to improve business efficiency and facilitate smooth communication. Furthermore, by reflecting user feedback in the next data analysis, continuous improvement of the system can be achieved.

[1422] "Personal information" refers to data used to identify a specific individual within the system (e.g., user ID, name, job title).

[1423] "Business information" refers to data related to the progress and content of business operations (e.g., task name, deadline, progress status).

[1424] "Emotional data" refers to data that represents the user's emotional state (e.g., joy, anger, sadness, surprise).

[1425] "Sentiment analysis" is the process of extracting and analyzing emotional data from users' text messages, voice data, and other sources.

[1426] A "correlation diagram" is a graph-structured diagram that visually represents personal information, business information, and emotional data.

[1427] "Modeling" is the process of using data to represent the relationships between people and tasks as a graph structure.

[1428] A "means for generating visual correlation diagrams" refers to a system component that has the function of drawing correlation diagrams in a human-readable format using modeled data.

[1429] "Feedback" refers to information about opinions and suggestions for improvement provided by users who have used the system.

[1430] To implement this invention, the system should be constructed and operated according to the following procedure. Furthermore, by combining it with an emotion engine for recognizing user emotions, a more detailed correlation analysis can be performed.

[1431] 1. Data Collection Phase

[1432] server

[1433] The server establishes connections by making API calls to internal systems (e.g., mail server, project management tool, calendar system). It obtains access rights using authentication information (API keys or OAuth tokens) from each system. Specifically, we will use "JIRA" as the project management tool, "Google Calendar" as the calendar system, and "Microsoft Exchange" as the mail server.

[1434] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time via API calls. To retrieve only the most recent data, it records the date and time of the previous data retrieval and collects differential data. The retrieved data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is excluded.

[1435] 2. Collection of emotional data

[1436] server

[1437] The server uses an emotion engine to collect user emotion data. This data is based on information obtained through user text messages, voice input, image recognition, and other means. Specifically, IBM Watson Natural Language Understanding is used for emotion analysis of text messages, and Google Cloud Speech-to-Text is used for emotion analysis of voice input.

[1438] The emotion engine analyzes the user's emotions (e.g., joy, anger, sadness, surprise, etc.) and stores the results in a database. This emotion data is linked to other business data and used to analyze correlations.

[1439] 3. Correlation Diagram Generation Phase

[1440] server

[1441] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[1442] Using an analysis module, we will analyze this data and model the relationship between people and tasks in a graph structure. We will set up people and tasks as nodes, define their relationships as edges, and simultaneously incorporate sentiment data into the analysis. As a concrete example, we will use the Python "NetworkX" library to generate the graph structure.

[1443] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[1444] 4. UI / UX Phase

[1445] terminal

[1446] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed on the client side. As a specific example, "React" is used as the frontend framework, and "D3.js" or "Cytoscape.js" is used as the correlation diagram library.

[1447] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on the analyzed data. Nodes and edges are visually arranged to make them easy for the user to understand. When the cursor hovers over a node, related information (e.g., name, job title, task details) and sentiment status (e.g., the user's current sentiment information) pop up. Furthermore, clicking on a node displays past interaction history and comments.

[1448] 5. Feedback Loop

[1449] User

[1450] Users view correlation diagrams and confirm necessary information. They also provide feedback and suggestions for improvement to the system through a feedback form.

[1451] server

[1452] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[1453] Example of a prompt

[1454] "Create a new task and generate a correlation diagram that includes an analysis of sentiment data."

[1455] This system makes it easier to coordinate tasks and identify stakeholders, and by considering emotional data, operational efficiency can be further improved.

[1456] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1457] Step 1:

[1458] The server makes an API call to the internal system and establishes a connection.

[1459] Input: Authentication information for internal systems (e.g., API key, OAuth token).

[1460] The server makes API calls to the mail server, project management tool, and calendar system to obtain access rights. Specifically, it adds the authentication information to the HTTP request header and establishes the connection.

[1461] Output: Handle to a successful API connection.

[1462] Step 2:

[1463] The server retrieves the necessary data from the internal system.

[1464] Input: API connection handle, last data retrieval date and time.

[1465] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time. For example, it sends a GET request to the JIRA API to retrieve the latest task information. It compares this to the previous data retrieval date and time and collects the difference data.

[1466] Output: A set of acquired data.

[1467] Step 3:

[1468] The server validates the data it retrieves and stores it in the central database.

[1469] Input: A set of retrieved data.

[1470] The server performs validation checks to ensure the data format is correct and all required fields are present. Only data that passes validation is inserted into the database. Specifically, the SQLAlchemy library in Python is used to insert data into the "tasks" table.

[1471] Output: Data stored in the database.

[1472] Step 4:

[1473] The server uses an emotion engine to collect user emotion data.

[1474] Input: User text messages, audio data.

[1475] The server sends text messages to sentiment analysis engines such as IBM Watson Natural Language Understanding for sentiment analysis. Similarly, sentiment analysis is performed on voice input using Google Cloud Speech-to-Text.

[1476] Output: Acquired sentiment data.

[1477] Step 5:

[1478] The server stores the emotion analysis results in a database.

[1479] Input: Sentiment analysis results.

[1480] The server inserts the analyzed emotion data into the "emotions" table in the database. Specifically, it stores the emotion status (e.g., joy, anger, sadness) and its associated task ID.

[1481] Output: Emotional data stored in the database.

[1482] Step 6:

[1483] The server scans the database to retrieve the latest information.

[1484] Input: Periodic scan timer.

[1485] The server scans the "Person Information," "Task Information," and "Emotional Data" tables to retrieve the latest data. This is done using SQL SELECT queries. For example, an SQL query can be used to retrieve updated task information.

[1486] Output: Set of the latest data.

[1487] Step 7:

[1488] The system analyzes the data acquired by the server and models its graph structure.

[1489] Input: Set of the latest data.

[1490] The server uses Python's NetworkX library to define people and tasks as nodes, and their relationships as edges. Sentiment data is also added to the nodes. For example, each user and task is added as a node, and their relationships are defined as edges.

[1491] Output: Modeled graph data.

[1492] Step 8:

[1493] The server converts the modeled graph data into a visual correlation diagram and sends it to the client in JSON format.

[1494] Input: Modeled graph data.

[1495] The server serializes the generated graph data into JSON format and sends it to the client. Specifically, it uses Flask's jsonify method to generate JSON data and returns it to the client.

[1496] Output: Data in JSON format.

[1497] Step 9:

[1498] The terminal retrieves the latest correlation diagram data from the server.

[1499] Input: Client request.

[1500] The device (the user's web browser) makes a GET request to a specified endpoint on the server and retrieves data in JSON format. Specifically, it uses the JavaScript fetch API to retrieve data from the / api / relation-graph endpoint.

[1501] Output: Retrieved JSON data.

[1502] Step 10:

[1503] The JSON data acquired by the device is analyzed, and a correlation diagram is drawn.

[1504] Input: Retrieved JSON data.

[1505] The device parses JSON data using D3.js or Cytoscape.js and visually arranges nodes and edges. For example, it adds nodes and edges using the cy.add method of Cytoscape.js and draws a correlation diagram.

[1506] Output: The generated correlation diagram.

[1507] Step 11:

[1508] When the device hovers over a node, it will display relevant information and sentiment status in a pop-up window.

[1509] Input: User hover event.

[1510] When a hover event occurs on a node, relevant information and sentiment status will be displayed as a tooltip. Specifically, the tooltip will be displayed using the Cytoscape.js method `cy.on('mouseover', 'node', function(evt) {...}`.

[1511] Output: The displayed tooltip.

[1512] Step 12:

[1513] Users submit their opinions and suggestions for improvement regarding the system through a feedback form.

[1514] Input: User feedback content.

[1515] Users enter their opinions and suggestions for improvement into a feedback form displayed on the UI and press the submit button. In other words, the form data is sent to the server.

[1516] Output: Sent feedback data.

[1517] Step 13:

[1518] The server saves user feedback to a database.

[1519] Input: Submitted feedback data.

[1520] The server inserts the feedback data received via POST request into the "feedback" table in the database. Specifically, it uses an SQL INSERT query to save the feedback content.

[1521] Output: Feedback stored in the database.

[1522] Step 14:

[1523] The server will take the feedback into consideration during the next data analysis.

[1524] Input: Feedback stored in the database.

[1525] The server will refer to the feedback during the next data analysis and incorporate it into the analysis model. Specifically, it will adjust the analysis algorithm and graph display based on the feedback.

[1526] Output: Analysis data reflecting the feedback.

[1527] This system will facilitate smoother collaboration between tasks and better identify stakeholders, and by considering emotional data, further improvements in operational efficiency can be expected.

[1528] (Application Example 2)

[1529] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1530] Traditional task management systems are limited to understanding correlations based on personal and task information, and lack consideration for user emotions, leading to problems such as decreased work efficiency and satisfaction. In particular, in workplaces where emotional changes directly impact work efficiency, the lack of collection and analysis of this information posed a significant challenge. There is a need to build a task management system that incorporates employee emotional data to ensure smooth workflow and improve employee satisfaction.

[1531] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1532] In this invention, the server includes means for acquiring data including person information, task information, and emotion data; means for analyzing the acquired data and modeling the correlation between people and tasks; and means for generating a visual correlation diagram based on the modeled correlations and emotion data. This makes it possible to perform effective task management while taking into account the user's emotions in real time.

[1533] "Personal information" refers to data related to an individual, such as the user's name, job title, and contact information.

[1534] "Business information" refers to data related to work or projects, such as task names, deadlines, and progress status.

[1535] "Emotional data" refers to data that indicates a user's emotional state, analyzed from sources such as text messages, voice input, and image recognition.

[1536] "Correlation" refers to the relationship or influence that exists between a person and a task.

[1537] "Modeling" refers to analyzing acquired data and representing the relationships between the data mathematically or graphically.

[1538] A "visual correlation diagram" is a graph that visually displays people and tasks as nodes and relationships as edges, based on analyzed data.

[1539] "Popup display" is a feature that displays related information in a small window format in response to a specific action (e.g., hovering the cursor over a node).

[1540] "Feedback" refers to the opinions, evaluations, and suggestions for improvement provided by system users.

[1541] A system for carrying out this invention includes the following elements.

[1542] Data collection phase

[1543] server

[1544] The server establishes a connection by making API calls to internal systems (e.g., email server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) for each system.

[1545] The server retrieves data such as user ID, task details, stakeholders, date, and time via API calls. To retrieve only the most recent data, it records the date and time of the previous data retrieval and collects differential data.

[1546] The acquired data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is excluded.

[1547] Collection of emotional data

[1548] server

[1549] The server uses an emotion engine to collect user emotion data. This data is based on information obtained through user text messages, voice input, image recognition, and other means.

[1550] The emotion engine analyzes the user's emotions (e.g., joy, anger, sadness, surprise, etc.) and stores the results in a database. This emotion data is linked to other business data and used to analyze correlations.

[1551] Correlation diagram generation phase

[1552] server

[1553] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[1554] Using an analysis module, we analyze this data and model the relationship between people and tasks in a graph structure. We set up people and tasks as nodes, define their relationships as edges, and simultaneously incorporate sentiment data into the analysis.

[1555] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[1556] UI / UX Phase

[1557] terminal

[1558] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed on the client side.

[1559] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[1560] When you hover your cursor over a node, relevant information (e.g., name, job title, task details) and sentiment status (e.g., the user's current sentiment information) will pop up. Furthermore, clicking on it will display past interaction history and comments.

[1561] Feedback loop

[1562] User

[1563] Users view correlation diagrams and confirm necessary information. They also provide feedback and suggestions for improvement to the system through a feedback form.

[1564] server

[1565] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[1566] Specific example

[1567] New task and integration scenario

[1568] Employee A (Manager)

[1569] Employee A launches a new project and assigns employees B and C as team members. Employee A uses a project management tool to assign various tasks to each member. Employee A monitors the project's progress and checks the members' emotional data.

[1570] Example of a prompt

[1571] Please tell me about the current progress of your task and how you feel about it.

[1572] Please share your thoughts on completing today's tasks.

[1573] Do you have any concerns about the next task?

[1574] Implementing this system makes it possible to manage tasks effectively while taking user emotions into consideration in real time.

[1575] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1576] Step 1:

[1577] The server establishes connections by making API calls to internal systems (e.g., mail server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) from each system. The server then retrieves the latest personnel and business information from each system. The input is the API endpoint and authentication information for each system, and the output is the retrieved personnel and business information. The data is stored in a central database.

[1578] Step 2:

[1579] The server retrieves data such as each user's ID, task details, stakeholders, date, and time via API calls. It records the date and time of the previous data retrieval and collects only differential data. The input is the date and time of the previous data retrieval and the result of the current API call, and the output is the new and updated data from the current call. This data is validated, and inconsistent data is excluded.

[1580] Step 3:

[1581] The server uses an emotion engine to collect user emotion data. The emotion engine analyzes emotions based on input data such as text messages, voice input, and image recognition, and stores the results in a database. The input is data related to the user's emotions, and the output is the analyzed emotion data.

[1582] Step 4:

[1583] The server periodically scans the database to retrieve the latest person information, task information, and sentiment data. Based on this data, the relationship between people and tasks is modeled in a graph structure. The input is the latest data retrieved from the database, and the output is the modeled graph data.

[1584] Step 5:

[1585] The server uses an analysis module to generate a visual correlation diagram based on modeled graph data. Sentiment data is also incorporated. The input is graph data and sentiment data, and the output is a visual correlation diagram. The generated correlation diagram can be exported in JSON or XML format.

[1586] Step 6:

[1587] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server and parses (analyzes) it on the client side. The input is correlation diagram data in JSON format provided by the server, and the output is the parsed data.

[1588] Step 7:

[1589] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand. The input is the parsed data, and the output is the displayed correlation diagram.

[1590] Step 8:

[1591] When a user hovers over a node, relevant information (e.g., name, role, task details) and sentiment status pop up. Clicking on the node displays past interaction history and comments. The input is the cursor's position, and the output is the detailed information displayed in the pop-up.

[1592] Step 9:

[1593] Users provide feedback on their experience with the system and suggestions for improvement through a feedback form. The input is the user's feedback content, and the output is feedback data stored in the database.

[1594] Step 10:

[1595] The server stores user feedback in a database and considers this feedback during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs. The input is the feedback data stored in the database, and the output is the improved analysis model that takes the feedback into account.

[1596] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1597] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1598] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1599] [Fourth Embodiment]

[1600] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1601] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1602] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1603] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1604] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1605] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1606] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1607] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1608] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1609] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1610] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1611] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1612] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1613] To implement this invention, the system should be constructed and operated according to the following procedure.

[1614] 1. Data Collection Phase

[1615] server

[1616] The server makes API calls to various internal systems (e.g., email server, project management tool, calendar system) to retrieve personnel and business information.

[1617] The server authenticates with each system and secures the necessary access rights. The data retrieved by API calls includes user ID, task details, stakeholders, date and time, etc.

[1618] The acquired data is organized and stored in a central database. Each data entry undergoes validation to maintain data consistency and integrity.

[1619] 2. Correlation Diagram Generation Phase

[1620] server

[1621] The server periodically scans the database to retrieve the latest personnel and business information.

[1622] The analysis module analyzes the data and models the correlation between people and tasks as a graph structure (nodes and edges).

[1623] Based on the graph data, a visual correlation diagram is generated. This correlation diagram can be exported in JSON or XML format and used for display on the client side.

[1624] 3. UI / UX Phase

[1625] terminal

[1626] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server.

[1627] The device uses libraries such as D3.js and Cytoscape.js to display correlation diagrams. This allows users to view correlation diagrams that are easy to understand visually.

[1628] When you hover over a node, relevant information (such as the person's name, job title, and task details) pops up. If the user clicks on a node, even more detailed information is displayed, including past collaboration history and comments.

[1629] 4. Feedback Loop

[1630] User

[1631] Users provide feedback, evaluations, and suggestions for improvement to the system through a feedback form. This is treated as specific improvement requests to enhance operational efficiency.

[1632] The server stores the feedback in a database and considers it during subsequent data analysis and correlation diagram generation. This allows the system to continuously improve, increasing its accuracy and usefulness.

[1633] Specific example

[1634] New task and integration scenario

[1635] User A (Project Manager)

[1636] A new project is launched, and User A is appointed as the project manager. User A assigns Users B and C as team members.

[1637] User A assigns various tasks to each member through a project management tool.

[1638] Data collection

[1639] server

[1640] Retrieve the latest task information for User A and their team members from the mail server, project management tools, and calendar system.

[1641] The data is stored in a database.

[1642] Correlation diagram generation

[1643] server

[1644] The analysis module analyzes the acquired data and generates nodes (User A, User B, User C) and edges (task connections).

[1645] A visual correlation diagram is generated and exported in JSON format.

[1646] Displaying information

[1647] terminal

[1648] User A logs into the system and views the correlation diagram. In the correlation diagram, User A is displayed as the project manager, and Users B and C are displayed as members.

[1649] When user A hovers the cursor over a node, its task details and related information are displayed in a pop-up window.

[1650] feedback

[1651] User A

[1652] User A provides feedback on their experience with the system and suggestions for improvement through a feedback form. This allows the system to be improved to be more user-friendly.

[1653] By implementing this invention, coordination of operations and identification of stakeholders become easier, leading to a significant improvement in operational efficiency.

[1654] The following describes the processing flow.

[1655] Step 1:

[1656] server

[1657] The server makes API calls to internal systems (e.g., email server, project management tool, calendar system) to establish a connection. It then obtains access rights using authentication information (API key, OAuth token) for each system.

[1658] Step 2:

[1659] server

[1660] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time via API calls. To retrieve only the most recent data, it records the date and time of the previous data retrieval and collects differential data.

[1661] Step 3:

[1662] server

[1663] The acquired data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is removed.

[1664] Step 4:

[1665] server

[1666] The server periodically scans the database to retrieve the latest personnel and business information. This ensures that the data necessary for generating the correlation diagram is always up-to-date.

[1667] Step 5:

[1668] server

[1669] The analysis module is used to analyze the data and model the relationship between people and tasks in a graph structure. People and tasks are set as nodes, and their relationships are defined as edges.

[1670] Step 6:

[1671] server

[1672] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[1673] Step 7:

[1674] terminal

[1675] The terminal retrieves the latest correlation diagram data from the server when the user logs into the system. The data is received in JSON format and parsed (analyzed) on the client side.

[1676] Step 8:

[1677] terminal

[1678] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[1679] Step 9:

[1680] terminal

[1681] When a user hovers over a node, detailed information about that node (e.g., name, role, task details) pops up. Clicking on it displays even more detailed information (e.g., past collaboration history, comments).

[1682] Step 10:

[1683] User

[1684] Users view correlation diagrams and confirm the necessary information. Furthermore, they can provide feedback and suggestions for improvement to the system through a feedback form.

[1685] Step 11:

[1686] server

[1687] The server saves user feedback to a database. This feedback is then taken into consideration during the next data analysis, and improvements are implemented accordingly.

[1688] Step 12:

[1689] server

[1690] In the next data analysis and correlation diagram generation cycle, the saved feedback will be referenced, and necessary adjustments and improvements will be made. This allows the system to be continuously improved and optimized.

[1691] (Example 1)

[1692] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1693] Conventional systems made it difficult to grasp personnel and work information within companies, posing challenges to improving operational efficiency. Furthermore, data analysis for visually displaying the correlation between personnel and work, and interactive information display based on those results, were insufficient. As a result, it was difficult for users to intuitively grasp the status of work and make appropriate decisions.

[1694] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1695] In this invention, the server includes means for acquiring person information and business information via an API, means for authenticating the acquired data and securing access rights, and means for validating the acquired data and storing it in a central database. This makes it possible to accurately acquire necessary data and store and manage it safely and efficiently. It also includes means for an analysis module that periodically scans the database and analyzes the latest person information and business information, means for modeling the analyzed data as a graph structure, means for generating a visual correlation diagram based on the modeled correlations and exporting it in JSON or XML format, and means for acquiring the generated correlation diagram and displaying the visual correlation diagram. This makes it easier for users to intuitively understand the correlation between people and business, enabling them to make appropriate decisions quickly. Furthermore, it includes means for displaying information about a node in a pop-up when the cursor is placed over a node on the correlation diagram, and means for saving user feedback in the database and considering it during the next data analysis. This allows the system to be continuously improved and enables the provision of highly accurate information that meets the needs of users.

[1696] "Personal information" refers to identifiable information about a specific individual, including name, job title, contact information, and department.

[1697] "Business information" refers to information about business operations and tasks performed within a company, including task details, deadlines, assigned personnel, and related projects.

[1698] "API" stands for Application Programming Interface, and refers to a standardized interface that enables data exchange between different software systems.

[1699] Authentication is the process of verifying a user's identity when accessing an information system, and it primarily uses a username, password, token, etc.

[1700] "Access rights" refer to the operational permissions for data and resources within an information system, and include permissions such as read, write, and execute.

[1701] "Validation" is the process of verifying whether the acquired data conforms to predetermined standards and formats.

[1702] A "central database" is a database used to centrally store and manage all acquired data, and examples include SQL databases and NoSQL databases.

[1703] "Periodic scanning" is the process of checking, updating, and retrieving data within a database at regular time intervals.

[1704] An "analysis module" is a software component used to analyze collected data and extract specific patterns or relationships.

[1705] A "graph structure" is a data structure consisting of nodes (vertices) and edges, and is used to model the relationship between people and tasks.

[1706] A "visual correlation diagram" is a diagram that visually represents a graph structure, allowing for an intuitive understanding of the relationship between people and tasks.

[1707] "JSON" stands for JavaScript Object Notation, and it is a lightweight data exchange format for structuring, storing, and exchanging data.

[1708] XML stands for eXtensible Markup Language, and it is a markup language used to structure, store, and exchange data.

[1709] A "node" refers to a vertex in a graph structure, and usually represents an entity such as a person or a task.

[1710] "Popup display" is a feature that displays a small window containing supplementary information in response to a specific action (for example, mouseover).

[1711] "Feedback" refers to opinions, suggestions for improvement, and evaluations provided by system users, which are used to improve the system.

[1712] "Data analysis" is the process of extracting patterns and relationships from collected data using statistical methods and algorithms to deepen our understanding of the data.

[1713] Modes for carrying out the invention

[1714] This invention is a system that acquires personnel and business information from various internal information systems, analyzes them, and generates and displays a correlation diagram that visually represents the relationships between them. The following describes a specific implementation of this system.

[1715] Data collection phase

[1716] server

[1717] The server retrieves data from the company's internal email system, project management system, and calendar system via APIs. This utilizes general-purpose APIs such as the Google Calendar API, Microsoft Exchange API, and JIRA API.

[1718] Specifically, this involves sending API requests to retrieve user and task information. For example, you can access the API endpoint using the "curl" command and extract the necessary data.

[1719] The acquired data is authenticated using OAuth 2.0 or similar methods to ensure appropriate access rights. This authentication guarantees the security and reliability of the data.

[1720] The server validates the retrieved data and stores only the data that conforms to the schema in the central database. The database used is a relational database such as MySQL or PostgreSQL.

[1721] Correlation diagram generation phase

[1722] server

[1723] The server periodically uses a job scheduler (e.g., a Cron job) to scan the database. This scan retrieves the latest personnel and business information, preparing it for analysis.

[1724] The server uses an analysis module (for example, the NetworkX library in Python) to analyze the person and task information in the database. As a result of the analysis, a graph structure is generated in which each person is a node and each task is an edge.

[1725] The server renders the generated graph data as a visual correlation diagram and exports it in JSON or XML format. This export is then used for display on the client side.

[1726] UI / UX Phase

[1727] terminal

[1728] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. AJAX requests are used to retrieve the data.

[1729] On the user's device, JavaScript libraries such as D3.js and Cytoscape.js are used to display correlation diagrams based on the acquired data. This allows users to intuitively understand the relationship between people and tasks.

[1730] When you hover your cursor over a node, related information will pop up, and if you need more detailed information, you can click on the node to display that information.

[1731] Feedback loop

[1732] User

[1733] Users provide feedback and suggestions for improvement to the system through a feedback form. For example, they can enter specific requests such as, "I'd like the UI layout to be a little easier to use."

[1734] The server stores the collected feedback in a database, which is then used for future data analysis and system updates.

[1735] Specific example

[1736] New task and integration scenario

[1737] User A (Project Manager)

[1738] User A launches a new project and assigns team members. User A assigns tasks to each member through a project management tool.

[1739] Data collection

[1740] server

[1741] The system retrieves the latest task information for User A and team members from email systems, project management systems, and calendar systems. The retrieved data is stored in a database.

[1742] Correlation diagram generation

[1743] server

[1744] The analysis module analyzes the data and generates nodes (User A, User B, User C) and edges (task connections). A visual correlation diagram is exported in JSON format.

[1745] Displaying information

[1746] terminal

[1747] User A logs into the system and views the correlation diagram. The diagram displays User A as the project manager, with Users B and C as members. Hovering the cursor over a node displays task details and collaboration information.

[1748] feedback

[1749] User A

[1750] User A enters their feedback on the system's usability and areas for improvement into a feedback form. This feedback will be used to improve the system.

[1751] Example of a prompt

[1752] This system collects internal company data and generates correlation diagrams of stakeholders. It collects data from the company's email system, project management system, and calendar system, and uses an analysis module to generate correlation diagrams. The generated correlation diagrams are viewable by users, and system improvements are made based on their feedback.

[1753] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1754] Step 1: Data Acquisition

[1755] server

[1756] The server retrieves data from the company's email system, project management system, and calendar system.

[1757] It sends requests to the APIs of various systems (such as Google Calendar API, Microsoft Exchange API, and JIRA API) to retrieve user and task information.

[1758] For example, execute a command like "curl -X GET 'https: / / api.calendar.google.com / calendar / v3 / calendars / primary / events' -H 'Authorization: Bearer [ACCESS_TOKEN]'".

[1759] Input: API endpoint, authentication token

[1760] Output: Data response in JSON format containing person information and job information.

[1761] Step 2: Authentication and securing access rights

[1762] server

[1763] The server authenticates the data it retrieves and secures the necessary access rights.

[1764] Obtain an access token using the OAuth 2.0 flow and include it in the API request header.

[1765] Input: User credentials, authentication server endpoint

[1766] Output: Access token

[1767] Step 3: Data validation and saving

[1768] server

[1769] The server validates the retrieved data and checks if it conforms to the schema.

[1770] Data that does not conform to the schema is removed, and only conforming data is stored in the central database.

[1771] Input: Acquired JSON data, schema definition

[1772] Output: Validated data stored in the central database

[1773] Step 4: Regular data scans

[1774] server

[1775] The server scans the data in the database using a periodic job scheduler (e.g., a Cron job).

[1776] Obtain the latest information from the database and prepare for analysis.

[1777] For example, the scan is performed daily in the format "0 0 / usr / bin / python3 / path / to / data_scan.py".

[1778] Input: Job scheduler settings, database connection information

[1779] Output: Latest dataset

[1780] Step 5: Data Analysis

[1781] server

[1782] The server analyzes the data using an analysis module (for example, the NetworkX library in Python).

[1783] A graph structure is generated using each person as a node and each task as an edge.

[1784] Input: Latest dataset

[1785] Output: Graph structure consisting of nodes and edges

[1786] Step 6: Generate and export the correlation diagram

[1787] server

[1788] The server renders a visual correlation diagram based on the generated graph structure.

[1789] The correlation diagram is exported in JSON or XML format and used for display on the client side.

[1790] Input: Graph structure

[1791] Output: Correlation diagram data in JSON or XML format

[1792] Step 7: Obtain and display correlation diagram data

[1793] terminal

[1794] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server.

[1795] This program uses AJAX requests to retrieve data in JSON format and displays correlation diagrams using D3.js or Cytoscape.js.

[1796] Input: User login information, server correlation diagram data

[1797] Output: Displayed interactive correlation diagram

[1798] Step 8: Interactive Information Display

[1799] terminal

[1800] When a user hovers their cursor over a node in the correlation diagram, related information pops up.

[1801] Clicking on a node will display more detailed information.

[1802] Input: User interaction, correlation diagram data

[1803] Output: Detailed information displayed in a pop-up window

[1804] Step 9: Gathering Feedback

[1805] User

[1806] Users can submit their opinions and suggestions for improvement regarding the system through a feedback form.

[1807] Input: Feedback content

[1808] Output: Feedback data sent to the server

[1809] Step 10: Saving and analyzing feedback

[1810] server

[1811] The server stores the collected feedback in a database and takes it into consideration during the next data analysis.

[1812] Input: Feedback data

[1813] Output: Feedback data stored in the database

[1814] (Application Example 1)

[1815] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1816] Traditional factories faced challenges in monitoring work and inventory information in real time and developing efficient work plans. Furthermore, it was difficult for factory robots and terminals to grasp work progress and inventory status and respond dynamically. This often hindered efficient work and reduced overall factory productivity.

[1817] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1818] In this invention, the server includes means for acquiring data including person information and work information; means for analyzing the acquired data and modeling the correlation between people and work; means for generating a visual correlation diagram based on the modeled correlation; means for robots and terminals to create efficient work plans based on the analysis results; means for displaying the generated correlation diagram; and means for robots and terminals to improve real-time work efficiency by referring to the displayed correlation diagram. This makes it possible to monitor work information and inventory information within the factory in real time and manage them efficiently.

[1819] "Personal information" refers to identifiable data about individual people, including their name, job title, and assigned tasks.

[1820] "Work information" refers to detailed data about a specific task, including the type of work, progress, person in charge, and deadline.

[1821] "Correlation" refers to the relationship between different data items, and specifically to the relationship between people and tasks.

[1822] "Modeling" refers to the process of transforming raw data into a mathematical or logical structure, making it easier to understand and analyze the relationships and patterns within the data.

[1823] A "visual correlation diagram" is a chart that visually represents the correlation between data, using nodes (data items) and edges (relationships) to show the structure.

[1824] The term "robot" refers to a machine or device that performs tasks automatically within a factory, and whose movements are controlled by work information and correlation diagrams.

[1825] "Terminal" refers to an electronic device used for displaying and manipulating data, and includes devices that display work information and correlation diagrams within a factory.

[1826] A "work plan" refers to a plan for carrying out work efficiently and effectively, and it manages the assignment and timing of each task.

[1827] "Real-time" refers to processing and reflecting data and information instantly with virtually no delay.

[1828] "Work efficiency" is a measure of how efficiently work is being performed, aiming to minimize wasted time and resources.

[1829] To implement this invention, the system is constructed and operated in the following steps: the server, robot, terminal, and user each play their respective roles.

[1830] 1. Data Collection Phase

[1831] The server makes API calls to various systems (e.g., inventory management system, work scheduling system, machine operation status monitoring system) to retrieve person and work information. The server authenticates with each system and secures the necessary access rights. The retrieved data includes user ID, work details, stakeholders, date and time, etc. The retrieved data is organized and stored in a central database.

[1832] 2. Correlation Diagram Generation Phase

[1833] The server periodically scans the database to retrieve the latest person and task information. An analysis module analyzes the data and models the correlation between people and tasks as a graph structure. Based on this graph data, a visual correlation diagram is generated. This correlation diagram is exported in JSON or XML format and used for display on the terminal.

[1834] 3. UI / UX Phase

[1835] The terminal and robot retrieve the latest correlation diagram data from the server and display the correlation diagram using libraries such as D3.js and Cytoscape.js. When a user views the correlation diagram and hovers over a node, relevant information (e.g., person's name, job title, and work details) pops up. If the user clicks on a node, more detailed information is displayed, including past collaboration history and comments.

[1836] 4. Feedback Loop

[1837] Users provide feedback, evaluations, and suggestions for improvement to the system through a feedback form. The server stores the feedback in a database and considers it during subsequent data analysis and correlation diagram generation. This allows the system to be continuously improved, increasing its accuracy and usefulness.

[1838] The specific software and hardware used in this system include the requests library for API calls, the networkx library and Flask framework for data analysis, and D3.js and Cytoscape.js for generating visual correlation diagrams.

[1839] Specific example

[1840] For example, suppose a robot in a factory is assembling parts. To enable this robot to understand its work progress and inventory status in real time and to carry out the work more efficiently, a system that monitors work progress and inventory status is used. This makes it possible to immediately notify the manager of instructions to replenish parts that are running low on stock.

[1841] Examples of prompts for generative AI models

[1842] "Please tell me how to improve work efficiency within the factory. Specifically, please tell me how to build a system that monitors work progress and inventory status in real time to improve efficiency."

[1843] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1844] Step 1:

[1845] The server makes API calls to various systems (inventory management system, work scheduling system, machine operation status monitoring system) to retrieve personnel and work information. It uses the API call URL and authentication information as input, and outputs JSON data of the retrieved personnel and work information. This data is temporarily stored in a central database.

[1846] Step 2:

[1847] The server periodically scans the database to retrieve the latest person and task information. The person and task information is input from the database scan results and analyzed to model the correlation between people and tasks. A data analysis library (e.g., networkx) is used for the analysis, and graph structure data is generated as output.

[1848] Step 3:

[1849] The server generates a visual correlation diagram based on the analyzed graph structure data. It accepts graph structure data as input and creates the correlation diagram using visualization libraries such as D3.js or Cytoscape.js. As output, the correlation diagram is exported in JSON or XML format for later display.

[1850] Step 4:

[1851] The terminal (or robot) retrieves the latest correlation diagram data from the server and uses a display library (e.g., D3.js, Cytoscape.js) to visually display the correlation diagram. The terminal takes the retrieved JSON or XML data as input and generates output (a visual correlation diagram) by rendering the correlation diagram.

[1852] Step 5:

[1853] Users view the generated correlation diagram using their device. When a user hovers their cursor over a node on the correlation diagram, relevant information (person's name, job title, and details of their work) pops up. This allows users to view detailed information about the corresponding node in real time.

[1854] Step 6:

[1855] Users provide feedback, evaluations, and suggestions for improvement to the system through a feedback form. The input feedback information is sent to the server and stored in a database. This feedback is then considered during subsequent data analysis and correlation diagram generation, contributing to the continuous improvement of the system.

[1856] Step 7:

[1857] The server considers the stored feedback information during the next data analysis. It uses past feedback information as input, influencing the analysis results. This results in an output analysis with improvements based on the feedback.

[1858] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1859] To implement this invention, the system should be constructed and operated according to the following procedure. Furthermore, by combining it with an emotion engine for recognizing user emotions, a more detailed correlation analysis can be performed.

[1860] 1. Data Collection Phase

[1861] server

[1862] The server establishes a connection by making API calls to internal systems (e.g., email server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) for each system.

[1863] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time via API calls. To retrieve only the most recent data, the server records the date and time of the previous data retrieval and collects differential data.

[1864] The acquired data is stored in a central database. To maintain data consistency and integrity, validation is performed to remove inconsistent data.

[1865] 2. Collection of emotional data

[1866] server

[1867] The server uses an emotion engine to collect user emotion data. This data is based on information obtained through user text messages, voice input, image recognition, and other means.

[1868] The emotion engine analyzes the user's emotions (e.g., joy, anger, sadness, surprise, etc.) and stores the results in a database. This emotion data is linked to other business data and used to analyze correlations.

[1869] 3. Correlation Diagram Generation Phase

[1870] server

[1871] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[1872] Using an analysis module, we analyze this data and model the relationship between people and tasks in a graph structure. We set up people and tasks as nodes, define their relationships as edges, and simultaneously incorporate sentiment data into the analysis.

[1873] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[1874] 4. UI / UX Phase

[1875] terminal

[1876] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed (analyzed) on the client side.

[1877] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[1878] When you hover your cursor over a node, relevant information (e.g., name, job title, task details) and sentiment status (e.g., the user's current sentiment information) will pop up. Furthermore, clicking on it will display past interaction history and comments.

[1879] 5. Feedback Loop

[1880] User

[1881] Users view correlation diagrams and confirm necessary information. They also provide feedback and suggestions for improvement to the system through a feedback form.

[1882] server

[1883] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[1884] Specific example

[1885] New task and integration scenario

[1886] User A (Project Manager)

[1887] User A starts a new project and assigns Users B and C as team members. User A then uses a project management tool to assign various tasks to each member.

[1888] User A monitors the project's progress and checks the members' emotional data.

[1889] Data collection

[1890] server

[1891] The system retrieves the latest task information and sentiment data for User A and their team members from the mail server, project management tools, calendar system, and sentiment engine.

[1892] The data is stored in a database.

[1893] Correlation diagram generation

[1894] server

[1895] The analysis module analyzes the acquired data and generates nodes (User A, User B, User C) and edges (task connections). Sentimental data is also incorporated, reflecting the user's current emotional status.

[1896] A visual correlation diagram is generated and exported in JSON format.

[1897] Displaying information

[1898] terminal

[1899] User A logs into the system and views the correlation diagram. In the correlation diagram, User A is displayed as the project manager, and Users B and C are displayed as members.

[1900] When user A hovers over a node, the task details, collaboration information, and the sentiment status of users B and C are displayed in a pop-up window.

[1901] feedback

[1902] User A

[1903] User A provides feedback on their experience with the system and suggestions for improvement through a feedback form. This allows the system to be improved to be more user-friendly.

[1904] By implementing this invention, collaboration in operations and identification of stakeholders become easier, and by considering emotional data, operational efficiency is further improved.

[1905] The following describes the processing flow.

[1906] Step 1:

[1907] server

[1908] The server establishes a connection by making API calls to internal systems (e.g., mail server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) for each system.

[1909] Step 2:

[1910] server

[1911] The server retrieves data from each system. This data includes user ID, task details (task name, deadline, progress), stakeholders, date, and time. API calls are used to collect the most up-to-date data.

[1912] Step 3:

[1913] server

[1914] The acquired data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is removed.

[1915] Step 4:

[1916] server

[1917] The system uses an emotion engine to acquire user emotion data. This is done by analyzing emotions through methods such as user text messages, voice input, and image recognition. The analyzed emotion data is stored in a database.

[1918] Step 5:

[1919] server

[1920] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[1921] Step 6:

[1922] server

[1923] The acquired data is analyzed, and the relationship between people and tasks is modeled using a graph structure (nodes and edges). People and tasks are set as nodes, their relationships are defined as edges, and sentiment data is also incorporated simultaneously.

[1924] Step 7:

[1925] server

[1926] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[1927] Step 8:

[1928] terminal

[1929] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed (analyzed) on the client side.

[1930] Step 9:

[1931] terminal

[1932] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[1933] Step 10:

[1934] terminal

[1935] When a user hovers over a node, a pop-up window displays detailed information about that node (e.g., name, role, task details) and its sentiment status (e.g., the user's current sentiment). Clicking on the node displays even more detailed information (e.g., past collaboration history, comments).

[1936] Step 11:

[1937] User

[1938] Users view correlation diagrams to check work-related information and emotional status. They also provide feedback and suggestions for improvement to the system through a feedback form.

[1939] Step 12:

[1940] server

[1941] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[1942] (Example 2)

[1943] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1944] Conventional data analysis systems had limitations in their ability to acquire person and business information and generate correlation diagrams. In particular, understanding business situations while considering users' emotional states was difficult, posing challenges to improving work efficiency and collaboration. Furthermore, there was insufficient mechanism for collecting user feedback on the generated correlation diagrams and incorporating it into subsequent data analyses.

[1945] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1946] In this invention, the server includes means for acquiring data containing person information and business information; means for analyzing the acquired data and modeling the correlation between people and business; means for collecting user emotion data and performing emotion analysis; means for generating a visual correlation diagram based on the modeled correlations and emotion data; and means for displaying the generated correlation diagram. This makes it easier to coordinate business operations and understand stakeholders, and by considering emotion data in particular, it becomes possible to improve business efficiency and facilitate smooth communication. Furthermore, by reflecting user feedback in the next data analysis, continuous improvement of the system can be achieved.

[1947] "Personal information" refers to data used to identify a specific individual within the system (e.g., user ID, name, job title).

[1948] "Business information" refers to data related to the progress and content of business operations (e.g., task name, deadline, progress status).

[1949] "Emotional data" refers to data that represents the user's emotional state (e.g., joy, anger, sadness, surprise).

[1950] "Sentiment analysis" is the process of extracting and analyzing emotional data from users' text messages, voice data, and other sources.

[1951] A "correlation diagram" is a graph-structured diagram that visually represents personal information, business information, and emotional data.

[1952] "Modeling" is the process of using data to represent the relationships between people and tasks as a graph structure.

[1953] A "means for generating visual correlation diagrams" refers to a system component that has the function of drawing correlation diagrams in a human-readable format using modeled data.

[1954] "Feedback" refers to information about opinions and suggestions for improvement provided by users who have used the system.

[1955] To implement this invention, the system should be constructed and operated according to the following procedure. Furthermore, by combining it with an emotion engine for recognizing user emotions, a more detailed correlation analysis can be performed.

[1956] 1. Data Collection Phase

[1957] server

[1958] The server establishes connections by making API calls to internal systems (e.g., mail server, project management tool, calendar system). It obtains access rights using authentication information (API keys or OAuth tokens) from each system. Specifically, we will use "JIRA" as the project management tool, "Google Calendar" as the calendar system, and "Microsoft Exchange" as the mail server.

[1959] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time via API calls. To retrieve only the most recent data, it records the date and time of the previous data retrieval and collects differential data. The retrieved data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is excluded.

[1960] 2. Collection of emotional data

[1961] server

[1962] The server uses an emotion engine to collect user emotion data. This data is based on information obtained through user text messages, voice input, image recognition, and other means. Specifically, IBM Watson Natural Language Understanding is used for emotion analysis of text messages, and Google Cloud Speech-to-Text is used for emotion analysis of voice input.

[1963] The emotion engine analyzes the user's emotions (e.g., joy, anger, sadness, surprise, etc.) and stores the results in a database. This emotion data is linked to other business data and used to analyze correlations.

[1964] 3. Correlation Diagram Generation Phase

[1965] server

[1966] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[1967] Using an analysis module, we will analyze this data and model the relationship between people and tasks in a graph structure. We will set up people and tasks as nodes, define their relationships as edges, and simultaneously incorporate sentiment data into the analysis. As a concrete example, we will use the Python "NetworkX" library to generate the graph structure.

[1968] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[1969] 4. UI / UX Phase

[1970] terminal

[1971] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed on the client side. As a specific example, "React" is used as the frontend framework, and "D3.js" or "Cytoscape.js" is used as the correlation diagram library.

[1972] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on the analyzed data. Nodes and edges are visually arranged to make them easy for the user to understand. When the cursor hovers over a node, related information (e.g., name, job title, task details) and sentiment status (e.g., the user's current sentiment information) pop up. Furthermore, clicking on a node displays past interaction history and comments.

[1973] 5. Feedback Loop

[1974] User

[1975] Users view correlation diagrams and confirm necessary information. They also provide feedback and suggestions for improvement to the system through a feedback form.

[1976] server

[1977] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[1978] Example of a prompt

[1979] "Create a new task and generate a correlation diagram that includes an analysis of sentiment data."

[1980] This system makes it easier to coordinate tasks and identify stakeholders, and by considering emotional data, operational efficiency can be further improved.

[1981] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1982] Step 1:

[1983] The server makes an API call to the internal system and establishes a connection.

[1984] Input: Authentication information for internal systems (e.g., API key, OAuth token).

[1985] The server makes API calls to the mail server, project management tool, and calendar system to obtain access rights. Specifically, it adds the authentication information to the HTTP request header and establishes the connection.

[1986] Output: Handle to a successful API connection.

[1987] Step 2:

[1988] The server retrieves the necessary data from the internal system.

[1989] Input: API connection handle, last data retrieval date and time.

[1990] The server retrieves data such as user ID, task details (task name, deadline, progress), stakeholders, date, and time. For example, it sends a GET request to the JIRA API to retrieve the latest task information. It compares this to the previous data retrieval date and time and collects the difference data.

[1991] Output: A set of acquired data.

[1992] Step 3:

[1993] The server validates the data it retrieves and stores it in the central database.

[1994] Input: A set of retrieved data.

[1995] The server performs validation checks to ensure the data format is correct and all required fields are present. Only data that passes validation is inserted into the database. Specifically, the SQLAlchemy library in Python is used to insert data into the "tasks" table.

[1996] Output: Data stored in the database.

[1997] Step 4:

[1998] The server uses an emotion engine to collect user emotion data.

[1999] Input: User text messages, audio data.

[2000] The server sends text messages to sentiment analysis engines such as IBM Watson Natural Language Understanding for sentiment analysis. Similarly, sentiment analysis is performed on voice input using Google Cloud Speech-to-Text.

[2001] Output: Acquired sentiment data.

[2002] Step 5:

[2003] The server stores the emotion analysis results in a database.

[2004] Input: Sentiment analysis results.

[2005] The server inserts the analyzed emotion data into the "emotions" table in the database. Specifically, it stores the emotion status (e.g., joy, anger, sadness) and its associated task ID.

[2006] Output: Emotional data stored in the database.

[2007] Step 6:

[2008] The server scans the database to retrieve the latest information.

[2009] Input: Periodic scan timer.

[2010] The server scans the "Person Information," "Task Information," and "Emotional Data" tables to retrieve the latest data. This is done using SQL SELECT queries. For example, an SQL query can be used to retrieve updated task information.

[2011] Output: Set of the latest data.

[2012] Step 7:

[2013] The system analyzes the data acquired by the server and models its graph structure.

[2014] Input: Set of the latest data.

[2015] The server uses Python's NetworkX library to define people and tasks as nodes, and their relationships as edges. Sentiment data is also added to the nodes. For example, each user and task is added as a node, and their relationships are defined as edges.

[2016] Output: Modeled graph data.

[2017] Step 8:

[2018] The server converts the modeled graph data into a visual correlation diagram and sends it to the client in JSON format.

[2019] Input: Modeled graph data.

[2020] The server serializes the generated graph data into JSON format and sends it to the client. Specifically, it uses Flask's jsonify method to generate JSON data and returns it to the client.

[2021] Output: Data in JSON format.

[2022] Step 9:

[2023] The terminal retrieves the latest correlation diagram data from the server.

[2024] Input: Client request.

[2025] The device (the user's web browser) makes a GET request to a specified endpoint on the server and retrieves data in JSON format. Specifically, it uses the JavaScript fetch API to retrieve data from the / api / relation-graph endpoint.

[2026] Output: Retrieved JSON data.

[2027] Step 10:

[2028] The JSON data acquired by the device is analyzed, and a correlation diagram is drawn.

[2029] Input: Retrieved JSON data.

[2030] The device parses JSON data using D3.js or Cytoscape.js and visually arranges nodes and edges. For example, it adds nodes and edges using the cy.add method of Cytoscape.js and draws a correlation diagram.

[2031] Output: The generated correlation diagram.

[2032] Step 11:

[2033] When the device hovers over a node, it will display relevant information and sentiment status in a pop-up window.

[2034] Input: User hover event.

[2035] When a hover event occurs on a node, relevant information and sentiment status will be displayed as a tooltip. Specifically, the tooltip will be displayed using the Cytoscape.js method `cy.on('mouseover', 'node', function(evt) {...}`.

[2036] Output: The displayed tooltip.

[2037] Step 12:

[2038] Users submit their opinions and suggestions for improvement regarding the system through a feedback form.

[2039] Input: User feedback content.

[2040] Users enter their opinions and suggestions for improvement into a feedback form displayed on the UI and press the submit button. In other words, the form data is sent to the server.

[2041] Output: Sent feedback data.

[2042] Step 13:

[2043] The server saves user feedback to a database.

[2044] Input: Submitted feedback data.

[2045] The server inserts the feedback data received via POST request into the "feedback" table in the database. Specifically, it uses an SQL INSERT query to save the feedback content.

[2046] Output: Feedback stored in the database.

[2047] Step 14:

[2048] The server will take the feedback into consideration during the next data analysis.

[2049] Input: Feedback stored in the database.

[2050] The server will refer to the feedback during the next data analysis and incorporate it into the analysis model. Specifically, it will adjust the analysis algorithm and graph display based on the feedback.

[2051] Output: Analysis data reflecting the feedback.

[2052] This system will facilitate smoother collaboration between tasks and better identify stakeholders, and by considering emotional data, further improvements in operational efficiency can be expected.

[2053] (Application Example 2)

[2054] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2055] Traditional task management systems are limited to understanding correlations based on personal and task information, and lack consideration for user emotions, leading to problems such as decreased work efficiency and satisfaction. In particular, in workplaces where emotional changes directly impact work efficiency, the lack of collection and analysis of this information posed a significant challenge. There is a need to build a task management system that incorporates employee emotional data to ensure smooth workflow and improve employee satisfaction.

[2056] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[2057] In this invention, the server includes means for acquiring data including person information, task information, and emotion data; means for analyzing the acquired data and modeling the correlation between people and tasks; and means for generating a visual correlation diagram based on the modeled correlations and emotion data. This makes it possible to perform effective task management while taking into account the user's emotions in real time.

[2058] "Personal information" refers to data related to an individual, such as the user's name, job title, and contact information.

[2059] "Business information" refers to data related to work or projects, such as task names, deadlines, and progress status.

[2060] "Emotional data" refers to data that indicates a user's emotional state, analyzed from sources such as text messages, voice input, and image recognition.

[2061] "Correlation" refers to the relationship or influence that exists between a person and a task.

[2062] "Modeling" refers to analyzing acquired data and representing the relationships between the data mathematically or graphically.

[2063] A "visual correlation diagram" is a graph that visually displays people and tasks as nodes and relationships as edges, based on analyzed data.

[2064] "Popup display" is a feature that displays related information in a small window format in response to a specific action (e.g., hovering the cursor over a node).

[2065] "Feedback" refers to the opinions, evaluations, and suggestions for improvement provided by system users.

[2066] A system for carrying out this invention includes the following elements.

[2067] Data collection phase

[2068] server

[2069] The server establishes a connection by making API calls to internal systems (e.g., email server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) for each system.

[2070] The server retrieves data such as user ID, task details, stakeholders, date, and time via API calls. To retrieve only the most recent data, it records the date and time of the previous data retrieval and collects differential data.

[2071] The acquired data is stored in a central database. Validation is performed to maintain data consistency and integrity, and inconsistent data is excluded.

[2072] Collection of emotional data

[2073] server

[2074] The server uses an emotion engine to collect user emotion data. This data is based on information obtained through user text messages, voice input, image recognition, and other means.

[2075] The emotion engine analyzes the user's emotions (e.g., joy, anger, sadness, surprise, etc.) and stores the results in a database. This emotion data is linked to other business data and used to analyze correlations.

[2076] Correlation diagram generation phase

[2077] server

[2078] The server periodically scans the database to retrieve the latest person information, business information, and sentiment data. This ensures that the data necessary for generating correlation diagrams is always up-to-date.

[2079] Using an analysis module, we analyze this data and model the relationship between people and tasks in a graph structure. We set up people and tasks as nodes, define their relationships as edges, and simultaneously incorporate sentiment data into the analysis.

[2080] A visual correlation diagram is generated based on the modeled graph data. The generated correlation diagram is exported in JSON or XML format and sent to the client.

[2081] UI / UX Phase

[2082] terminal

[2083] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server. The data is received in JSON format and parsed on the client side.

[2084] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand.

[2085] When you hover your cursor over a node, relevant information (e.g., name, job title, task details) and sentiment status (e.g., the user's current sentiment information) will pop up. Furthermore, clicking on it will display past interaction history and comments.

[2086] Feedback loop

[2087] User

[2088] Users view correlation diagrams and confirm necessary information. They also provide feedback and suggestions for improvement to the system through a feedback form.

[2089] server

[2090] The server stores user feedback in a database. This feedback is then considered and incorporated into the analysis model during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs.

[2091] Specific example

[2092] New task and integration scenario

[2093] Employee A (Manager)

[2094] Employee A launches a new project and assigns employees B and C as team members. Employee A uses a project management tool to assign various tasks to each member. Employee A monitors the project's progress and checks the members' emotional data.

[2095] Example of a prompt

[2096] Please tell me about the current progress of your task and how you feel about it.

[2097] Please share your thoughts on completing today's tasks.

[2098] Do you have any concerns about the next task?

[2099] Implementing this system makes it possible to manage tasks effectively while taking user emotions into consideration in real time.

[2100] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2101] Step 1:

[2102] The server establishes connections by making API calls to internal systems (e.g., mail server, project management tool, calendar system). It obtains access rights using authentication information (API key, OAuth token) from each system. The server then retrieves the latest personnel and business information from each system. The input is the API endpoint and authentication information for each system, and the output is the retrieved personnel and business information. The data is stored in a central database.

[2103] Step 2:

[2104] The server retrieves data such as each user's ID, task details, stakeholders, date, and time via API calls. It records the date and time of the previous data retrieval and collects only differential data. The input is the date and time of the previous data retrieval and the result of the current API call, and the output is the new and updated data from the current call. This data is validated, and inconsistent data is excluded.

[2105] Step 3:

[2106] The server uses an emotion engine to collect user emotion data. The emotion engine analyzes emotions based on input data such as text messages, voice input, and image recognition, and stores the results in a database. The input is data related to the user's emotions, and the output is the analyzed emotion data.

[2107] Step 4:

[2108] The server periodically scans the database to retrieve the latest person information, task information, and sentiment data. Based on this data, the relationship between people and tasks is modeled in a graph structure. The input is the latest data retrieved from the database, and the output is the modeled graph data.

[2109] Step 5:

[2110] The server uses an analysis module to generate a visual correlation diagram based on modeled graph data. Sentiment data is also incorporated. The input is graph data and sentiment data, and the output is a visual correlation diagram. The generated correlation diagram can be exported in JSON or XML format.

[2111] Step 6:

[2112] When a user logs into the system, the terminal retrieves the latest correlation diagram data from the server and parses (analyzes) it on the client side. The input is correlation diagram data in JSON format provided by the server, and the output is the parsed data.

[2113] Step 7:

[2114] The device uses libraries such as D3.js and Cytoscape.js to draw correlation diagrams based on parsed data. Nodes and edges are visually arranged to make them easy for the user to understand. The input is the parsed data, and the output is the displayed correlation diagram.

[2115] Step 8:

[2116] When a user hovers over a node, relevant information (e.g., name, role, task details) and sentiment status pop up. Clicking on the node displays past interaction history and comments. The input is the cursor's position, and the output is the detailed information displayed in the pop-up.

[2117] Step 9:

[2118] Users provide feedback on their experience with the system and suggestions for improvement through a feedback form. The input is the user's feedback content, and the output is feedback data stored in the database.

[2119] Step 10:

[2120] The server stores user feedback in a database and considers this feedback during subsequent data analysis. This allows the system to continuously improve and be optimized to meet user needs. The input is the feedback data stored in the database, and the output is the improved analysis model that takes the feedback into account.

[2121] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2122] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2123] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2124] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2125] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[2126] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[2127] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[2128] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[2129] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[2130] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[2131] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[2132] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[2133] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[2134] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2135] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[2136] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[2137] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[2138] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[2139] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[2140] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[2141] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[2142] The following is further disclosed regarding the embodiments described above.

[2143] (Claim 1)

[2144] A means of acquiring data including personal information and business information,

[2145] A means of analyzing acquired data and modeling the correlation between people and tasks,

[2146] A means for generating a visual correlation diagram based on modeled correlations,

[2147] A means for displaying the generated correlation diagram,

[2148] A system that includes this.

[2149] (Claim 2)

[2150] The system according to claim 1, further comprising means for displaying information about a node in a correlation diagram when the cursor is hovered over the node, based on the analyzed data.

[2151] (Claim 3)

[2152] The system according to claim 1, further comprising means for receiving user feedback based on the generated correlation diagram, storing that feedback in a database, and considering it during the next data analysis.

[2153] "Example 1"

[2154] (Claim 1)

[2155] A means of obtaining data including personal information and business information via an API,

[2156] A means of authenticating the acquired data and securing access rights,

[2157] A means of validating the acquired data and saving it to a central database,

[2158] A means including an analysis module that periodically scans stored data and analyzes the latest personnel and business information,

[2159] A means of modeling the analyzed data as a graph structure,

[2160] A method for generating a visual correlation diagram based on modeled correlations and exporting it in JSON or XML format,

[2161] A means for obtaining the generated correlation diagram and displaying the visual correlation diagram,

[2162] A system that includes this.

[2163] (Claim 2)

[2164] The system according to claim 1, further comprising means for displaying information about a node in a pop-up window when the cursor is placed over that node on the correlation diagram.

[2165] (Claim 3)

[2166] The system according to claim 1, further comprising means for receiving user feedback based on the generated correlation diagram, storing that feedback in a database, and considering it during the next data analysis.

[2167] "Application Example 1"

[2168] (Claim 1)

[2169] A means of acquiring data including person information and work information,

[2170] A means of analyzing acquired data and modeling the correlation between people and tasks,

[2171] A means for generating a visual correlation diagram based on modeled correlations,

[2172] A means for robots and terminals to create efficient work plans based on analysis results,

[2173] A means for displaying the...

Claims

1. A means of acquiring data including personal information and business information, A means of analyzing acquired data and modeling the correlation between people and tasks, A means for generating a visual correlation diagram based on modeled correlations, A means for displaying the generated correlation diagram, A system that includes this.

2. The system according to claim 1, further comprising means for displaying information about a node in a correlation diagram when the cursor is hovered over the node, based on the analyzed data.

3. The system according to claim 1, further comprising means for receiving user feedback based on the generated correlation diagram, storing that feedback in a database, and considering it during the next data analysis.

Images