system

Abstract

We provide the system. [Solution] A means of automatically collecting relevant information from sources, A means of analyzing collected information and integrating related information, A means of evaluating progress and generating a report based on integrated information, A means of detecting and pointing out inconsistencies between multiple pieces of information, A system that includes this.

Description

Technical Field

[0005]

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In project management, since information is distributed among a large number of information sources (such as emails, chats, meetings, etc.), it is difficult for stakeholders to grasp the latest progress. Also, due to information inconsistencies between different departments and task delays, a lot of wasted time is often spent in management operations. Such problems prevent the efficient execution of projects.

Means for Solving the Problems

[0005] This invention provides means for automatically collecting relevant information from information sources, and means for analyzing the collected information and integrating related information. It also provides means for evaluating progress based on the integrated information and generating reports. This makes it possible to detect and point out inconsistencies in information, manage the progress of tasks, and notify the user if there are delays. Furthermore, by providing means for checking the consistency of annotations and suggesting corrections as necessary, it reduces the effort required in project management and improves efficiency.

[0006] "Information sources" refers to any medium or platform in which project-related information, such as emails, chats, and meeting minutes, is generated and stored.

[0007] "Means of collection" refers to the technologies and processes for automatically obtaining relevant information from information sources.

[0008] "Analysis means" refers to techniques and methods for analyzing collected information and identifying and integrating relevant data.

[0009] "Integration methods" refer to methods and processes for organizing data collected from different sources and building a unified information base.

[0010] "Progress evaluation means" refers to the techniques and methods for determining the progress of project activities based on integrated information and organizing the results.

[0011] "Report generation means" refers to the technology and processes for automatically creating reports for managers based on evaluated progress information.

[0012] "Inconsistency detection means" refers to techniques and methods for identifying contradictions and discrepancies between multiple sources or pieces of information, and for issuing warnings to users as necessary.

[0013] "Task progress management means" refers to the techniques and methods for monitoring the current status of each task within a project and identifying any delays.

[0014] "Annotation verification methods" refer to techniques and methods for checking the consistency and coherence of annotations attached to documents, advertisements, and other materials. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Modes for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), etc.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disk (e.g., hard disk), or magnetic tape, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention is a system aimed at centralizing information and efficiently managing progress in project management. It grasps and manages the progress of a project through a process of collecting relevant information from information sources, analyzing, integrating, and evaluating it.

[0037] Specifically, the system includes the following operations: First, the user enters project-related keywords through the interface to identify the information to be collected. This prepares the system to accurately retrieve information related to those keywords.

[0038] Next, the server accesses the information sources specified by the user, such as email servers, chat applications, and online meeting systems, and collects data that matches the keywords. The information sources accessed are designed to be handled securely in accordance with security protocols.

[0039] The collected information is analyzed within the server. Natural language processing technology is used to identify related information and integrate information obtained from multiple sources. This integration allows users to view dispersed information in a single, centralized manner.

[0040] Furthermore, the server generates progress reports based on the integrated information. These reports are provided to project managers and supervisors, allowing them to gain a detailed understanding of the project's current status. The reports include an overview of progress, challenges, and potential risks.

[0041] In addition, the server automatically detects inconsistencies in the collected information and task delays, and notifies the user. For example, if there are conflicting decisions between different departments, it will point them out and propose corrective measures to support smooth project progress.

[0042] Finally, using the integrated data, the AI ​​module checks for consistency in annotations accompanying advertisements and documents and suggests corrections as needed. This ensures consistent communication within the project.

[0043] As a concrete example, suppose a user is managing a new product development project. In this case, by integrating all project-related information into the system and quickly identifying inconsistencies between different teams, it is possible to achieve smoother collaboration between teams and improve the efficiency of project execution.

[0044] The following describes the processing flow.

[0045] Step 1:

[0046] Users log in to the project management system and enter keywords related to the projects they are managing through the interface. These keywords act as filters for information gathering and are used to narrow down specific information.

[0047] Step 2:

[0048] The server securely accesses user-specified information sources (e.g., email servers, chat applications, meeting platforms) based on configured keywords. During this process, it automatically collects information related to the keywords from each information source.

[0049] Step 3:

[0050] The server stores the collected information in a temporary database and analyzes the data using natural language processing techniques. Here, relevant information is identified, and preparations are made to group similar or related information together.

[0051] Step 4:

[0052] The server integrates the analyzed information to create a centralized information platform. This platform functions as a project-specific information base, accessible to users for review.

[0053] Step 5:

[0054] The server evaluates project progress based on integrated information. An AI algorithm analyzes the progress and generates daily progress reports. These reports are automatically delivered according to user settings.

[0055] Step 6:

[0056] The server initiates a process to detect inconsistencies in information, identifying conflicting decisions and information between different teams. The findings are compiled into a report, and alerts are sent to users as needed.

[0057] Step 7:

[0058] The server performs task progress management and identifies tasks that are behind schedule. Based on this information, users receive real-time notifications.

[0059] Step 8:

[0060] The AI ​​module initiates the annotation verification process, checking for consistency in annotations within documents and advertisements. If inconsistencies are found, suggested corrections are generated and the user is notified.

[0061] Step 9:

[0062] The terminal ultimately provides users with progress reports, task notifications, and inconsistency detection reports from the server, allowing users to take necessary actions and make corrections based on this information, thereby improving the efficiency of project management.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] In project management, the widespread and dispersed nature of information makes it difficult to accurately grasp progress, often leading to task delays and inconsistencies. Furthermore, inconsistent information across different teams can reduce the efficiency of project execution.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server includes means for automatically collecting relevant information based on keywords from information sources, means for analyzing the collected information using natural language processing technology and integrating related information, and means for evaluating the progress of the project based on the integrated information and generating a report. This enables centralized management of information within a project and smooth collaboration between different teams.

[0068] "Information sources" refer to the media or systems from which data is collected, specifically including email, chat applications, and online meeting records.

[0069] A "keyword" is a term that a user identifies as the focus of information gathering during project management, and related information is searched based on this keyword.

[0070] "Means of automatic data collection" refers to system functions that obtain necessary data from information sources without user intervention.

[0071] "Natural language processing technology" is a technology that enables computers to process and understand human language, and is used to analyze collected data.

[0072] A "means for integrating related information" refers to a system that has the function of identifying relevant information from collected data and compiling it in a consistent format.

[0073] "Means for evaluating progress and generating reports" refers to the function of a system that analyzes the current status of a project and compiles the results into a report.

[0074] "Means for detecting and pointing out inconsistencies and task delays" refers to a system that has the functionality to discover inconsistencies and delays in data and communicate them to the user.

[0075] A "means for checking the consistency of annotations and proposing corrections" refers to a system that has the function of checking whether the annotations in documents used within a project are consistent and, if necessary, making suggestions to maintain consistency.

[0076] This invention provides a system that enables centralized information management and efficient progress management in project management, and grasps the project status through information collection, analysis, integration, and evaluation. Specific embodiments are shown below.

[0077] First, the user enters project-related keywords through an interface. This interface is a software module that runs on the terminal and identifies the information to be collected based on the user's input of specific keywords.

[0078] Next, the server accesses information sources specified based on keywords, such as email servers, chat applications, and online meeting systems, and collects relevant information. The server ensures the security of the information by accessing it in accordance with robust security protocols. The technologies used include modern cloud computing platforms and APIs for server access.

[0079] The collected information is analyzed on the server using natural language processing technology. This identifies and integrates related data obtained from different sources. Natural language processing libraries in Python and R are used in this analysis and integration process.

[0080] Furthermore, based on the integrated data, the server generates a project progress report. This report is provided to project managers and supervisors, allowing them to gain a detailed understanding of the project's current status. The report includes an overview of the project's progress, issues, and a presentation of potential risks.

[0081] Furthermore, the server analyzes the integrated data and automatically detects inconsistencies between information and task delays. This allows it to identify conflicts between different departments and propose appropriate corrections.

[0082] Finally, the AI ​​module uses the integrated data to check the consistency of annotations accompanying advertisements, documents, and other materials, and suggests corrections as needed. This helps maintain consistency in communication within the project.

[0083] As a concrete example, suppose a user is managing a "new product development" project. In this case, all relevant information is integrated into the system, and inconsistencies between different teams can be quickly identified, leading to smoother team collaboration and improved project efficiency.

[0084] An example of a prompt for a generative AI model is, "Please summarize the progress and challenges of the new product development project in a report."

[0085] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0086] Step 1:

[0087] The user enters project-related keywords into the terminal interface. This input instructs the system on what information to collect. For example, if the user enters keywords such as "new product development" or "market research," the collection of relevant data will begin. This clarifies the target of information collection.

[0088] Step 2:

[0089] The server accesses specified information sources based on keywords and collects relevant information. Using the entered keywords, it accesses mail servers, chat applications, and online meeting systems to retrieve relevant messages, documents, etc. Here, it filters the data matching the keywords from the accessed information sources, formats it, and collects it to form a consistent dataset.

[0090] Step 3:

[0091] The server collects information and analyzes it using natural language processing techniques. The input is the raw data collected in step 2. By processing this data, relevant content is extracted and irrelevant information is eliminated. Here, natural language processing libraries in Python and R are used to analyze the text content and integrate important information relevant to the project.

[0092] Step 4:

[0093] The server generates a progress report based on the analyzed and integrated information. The input for this step is integrated data, which is used to create an output report summarizing the project's progress, challenges, and risks. Here, the information is organized according to the report format and output in an easily understandable manner. The report is generated in text format and provided to the project manager.

[0094] Step 5:

[0095] The server analyzes the integrated data, detects inconsistencies between pieces of information and task delays, and notifies the user. The input here is the dataset generated in steps 3 and 4. By checking the consistency between the data, it identifies inconsistencies and delays and generates a notification as output, including suggestions.

[0096] Step 6:

[0097] The AI ​​module uses integrated data to check the consistency of annotations attached to documents and advertisements within the project and suggests corrections as needed. The input is the integrated information from step 3, and the output provides the user with suggested corrections and suggestions to maintain annotation consistency. This ensures consistent communication throughout the project.

[0098] (Application Example 1)

[0099] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0100] In modern manufacturing environments, it is crucial for engineers to efficiently grasp project progress and respond quickly. However, because information is scattered across various locations, integrating it and providing it to on-site workers in a consistent format is not easy. As a result, delays and inconsistencies can occur in project progress, negatively impacting productivity. Therefore, there is a need to solve these problems and achieve centralized and efficient use of information.

[0101] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0102] In this invention, the server includes means for automatically collecting relevant information from information sources, means for analyzing the collected information and integrating related information, and means for visualizing the data collected from various devices and presenting the information to the user. This enables workers to grasp the progress of projects in real time and take effective action.

[0103] "Information sources" refer to various devices and platforms accessed to acquire data, such as email servers and online meeting systems.

[0104] "Collection means" refers to the part that has the function of automatically acquiring relevant information, and is responsible for extracting data from information sources and sending it to the server.

[0105] "Analysis means" refers to the part that has the function of analyzing acquired information and evaluating its relationships, and performs the process of extracting information that can be integrated.

[0106] The term "integration means" refers to the part that has the function of aggregating and centrally managing the analyzed information, and making related data available for display.

[0107] "Visualization means" refers to the part that has the function of presenting information to the worker in an intuitively easy-to-understand form, and displays it in real time through a head-mounted display or monitor.

[0108] "Inconsistency detection means" refers to a component that has the function of identifying contradictions and inconsistencies between pieces of information, and is necessary to improve the accuracy of progress management.

[0109] "Notification means" refers to the part of the system that has the function of conveying important information to the user, such as informing them of task delays or the need for new instructions.

[0110] The system implementing this invention primarily has functions for collecting, analyzing, and integrating data, and for visually presenting it. The entire system consists of a server, user terminals, and a group of information source devices.

[0111] The server automatically collects data from information sources. These sources include mail servers, chat applications, and online meeting systems, which are accessed securely through security protocols. This collected data is analyzed within the server using natural language processing libraries such as NLTK and spaCy to identify relevant information and integrate it centrally.

[0112] The integrated information is displayed on the user's head-mounted display (HMD). Unity is used to visualize the information in a way that is intuitively understandable to the user. Furthermore, users can issue voice commands through the HMD's built-in voice recognition function, allowing them to obtain detailed information about specific projects or instruct changes to be made.

[0113] The AI ​​module installed in the server automatically detects inconsistencies and task delays and sends real-time notifications to the user. These notifications are delivered using the HMD's visual display and audio output, prompting immediate action at the work site.

[0114] For example, if the system detects a delay in the production of product A at a factory, the server analyzes the reason and proposes a quick solution. In this case, the worker receives new instructions via HMD and can efficiently readjust their tasks. An example of a prompt message for the generated AI model is: "Production of product A is delayed at the factory. Collect the latest data from the manufacturing department, identify the cause of the delay, and propose a solution to the manager."

[0115] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0116] Step 1:

[0117] The server connects to the information source and collects data. The input is a keyword specified by the user, and the output is raw data obtained from the information source. At this stage, the server accesses the information source's API and securely extracts the data using security protocols.

[0118] Step 2:

[0119] The server analyzes the collected raw data. The input is the raw data obtained in step 1, and the output is a snapshot of the analyzed data. Here, natural language processing is performed using NLTK or spaCy to clarify the structure of the information. For example, relevant information is tagged based on keywords in the manufacturing process.

[0120] Step 3:

[0121] The server integrates and centralizes the analysis results. The input is a data snapshot obtained from step 2, and the output is a dataset of the integrated information. This process combines relevant information from different sources and transforms it into a consistent format, allowing for a comprehensive view of the project.

[0122] Step 4:

[0123] The server visualizes integrated information and sends it to the terminal. The input is an integrated dataset, and the output is a visualized user interface. The server uses an interface built with Unity to allow workers to visually check the information in real time through an HMD (Head-Mounted Display). Here, graphs and dashboards are displayed on the user interface, making the progress intuitively understandable.

[0124] Step 5:

[0125] The server uses an AI module to detect inconsistencies and delays and notify the user. Inputs are an integrated dataset and current progress information, and output is an alert message. The AI ​​uses pattern recognition and predictive algorithms to identify problems and generates visual and audible warnings for the worker. For example, it might send a notification such as, "There is a delay on the production line for product A. We are investigating the cause."

[0126] Step 6:

[0127] The user provides voice instructions to the terminal as needed. The input is the user's voice instructions, and the output is interactive feedback actions. Voice recognition software analyzes the instructions, and the server performs the next corresponding action based on the results. Specifically, when an instruction such as "Plan additional production of product A" is input, the project plan is automatically adjusted based on that information.

[0128] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0129] This invention combines a project management system with an emotion engine, aiming to provide a more personalized experience by recognizing the user's emotions within the project management process and adjusting the presentation of information and notifications based on those emotions.

[0130] Specifically, when a user uses the system, the emotion engine recognizes the user's emotions in real time. This emotional state is analyzed from text entered by the user and non-verbal cues (such as facial expressions and tone of voice) acquired through the device's camera and microphone.

[0131] The server analyzes project-related data collected from information sources and presents relevant information to the user. During this process, the presentation of the information is adjusted according to the user's emotions, as recognized by the emotion engine. For example, if the user is stressed, the system reduces the user's burden by presenting information more concisely and focusing on high-priority tasks.

[0132] In addition, the server dynamically adjusts task priorities based on the user's emotional state. When the user is calm, it optimizes project progress by generating more detailed reports and notifying users of complex tasks.

[0133] This system allows users to receive project information and manage tasks in a way that is optimally tailored to their emotional state. For example, if a user is feeling nervous right before an important meeting, the emotion engine recognizes this state, and the server creates a concise list of key action items for the day, providing it on the user's device to reduce their burden. In this way, the present invention enables more efficient project execution through information management that takes emotions into account.

[0134] The following describes the processing flow.

[0135] Step 1:

[0136] The user logs into the project management system and sets project-related keywords on the interface. These keywords determine the target of information collection.

[0137] Step 2:

[0138] The server accesses user-defined information sources and automatically collects relevant information that matches specified keywords. This information can range from email servers and chat applications to meeting records.

[0139] Step 3:

[0140] An emotion engine within the server operates, analyzing text messages entered by the user, as well as facial expressions and voice recordings collected by the device's microphone and camera, to identify the user's emotional state in real time.

[0141] Step 4:

[0142] The server analyzes the collected information, integrating and centralizing relevant data. During this process, it refers to the output of the emotion engine to determine how to present information according to the user's emotional state.

[0143] Step 5:

[0144] Based on the integrated information, the server evaluates project progress and generates a report with appropriate content and format, taking into account the sentiment index. For example, if a user is experiencing stress, the information is summarized concisely, and the report prioritizes including only the most important information.

[0145] Step 6:

[0146] The server adjusts task priorities based on the user's emotional state. If the user appears anxious, it prioritizes presenting achievable short-term goals to prevent them from falling behind.

[0147] Step 7:

[0148] The terminal provides users with generated reports and alert notifications. Users use these to check project progress and take necessary management and action.

[0149] Step 8:

[0150] The terminal continuously updates the emotion engine's results based on newly entered information and operation history from the user, and feeds this information back to the server. This enables project management that is always based on the latest user status.

[0151] (Example 2)

[0152] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal".

[0153] In project management, users are often overburdened with information because their emotional state is not taken into consideration. As a result, users may lose sight of the priorities of important tasks or experience unnecessary stress. Furthermore, the failure to present information in a way that is sensitive to the user's emotions can lead to decreased work efficiency and negatively impact the overall progress of the project.

[0154] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0155] In this invention, the server includes means for automatically collecting relevant information from information sources, means for analyzing the collected information and integrating related information, and means for analyzing the user's emotional state and dynamically adjusting the information presentation method based on the emotional state. This makes it possible to optimize information presentation and task prioritization according to the user's emotional state.

[0156] "Information source" refers to the source or medium from which a system obtains data, and this includes document databases, online resources, or other digital recording media.

[0157] "Means of collection" refers to the techniques and methods used to obtain necessary data from information sources, and includes database queries and web scraping techniques.

[0158] "Means of analysis" refers to methods and techniques for analyzing collected data and understanding its structure and meaning, and includes data mining and machine learning algorithms.

[0159] "Means of integration" refer to techniques and methods for combining analyzed information to achieve a holistic understanding and evaluation, and these include data normalization and merging operations.

[0160] "Means for analyzing emotional states" refers to methods and technologies for understanding a user's emotions, and includes natural language processing, speech recognition, and facial expression recognition technologies.

[0161] "Means of dynamically adjusting the method of information presentation" refers to technologies that change how information is displayed according to the user's emotional state, and this includes changing the display format and optimizing the UI.

[0162] "Means of dynamically adjusting priorities" refers to technologies that flexibly change task priorities while taking into account the user's emotional state, and this includes algorithmic automatic adjustment functions.

[0163] This invention provides comprehensive emotion recognition functionality for project management systems. The core of the invention involves implementing an emotion engine and data collection, analysis, and integration functions on both the server and the terminal. This allows users to receive optimal support tailored to their emotional state.

[0164] This system utilizes the following hardware and software: The server automatically collects relevant information from information sources using a database management system (e.g., MySQL®). It also uses generative AI models for natural language processing and speech analysis to analyze the user's emotional state in real time. The terminal functions as hardware that collects nonverbal information from the user (facial expressions, tone of voice, etc.) using a camera and microphone. In addition, the end user is provided with a user interface that is dynamically adjusted based on their emotional state.

[0165] The server uses an emotion engine to detect the user's emotional state and adjusts the information display method and task priorities accordingly. This enables the presentation of information tailored to the user's situation and emotions, resulting in reduced stress and improved work efficiency.

[0166] For example, if tension is detected in a user preparing an important presentation, the server will provide a list of materials that should be displayed in summary format and focus on key action items to help the user prepare more easily. Furthermore, the operation of this mechanism can be verified by entering an example prompt message into the server, such as, "If sentiment analysis indicates the user is experiencing stress, please help design a program that adjusts task priorities and presents information more concisely."

[0167] Thus, the present invention is a system that provides a new method for recognizing a user's emotional state with high accuracy and realizing project management tailored to individual circumstances.

[0168] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0169] Step 1:

[0170] The user logs into the project management system. The device collects nonverbal data such as the user's facial expressions and tone of voice through its camera and microphone. Text entered by the user is also collected as input data. This data becomes input to the emotion engine.

[0171] Step 2:

[0172] The server inputs nonverbal and text data collected from the terminal into a generative AI model. The generative AI model uses natural language processing and speech analysis algorithms to analyze the user's emotional state. As a result of the analysis, the user's emotional state is classified into categories such as "tension," "relaxation," and "stress," and this is returned to the server as output.

[0173] Step 3:

[0174] The server collects project-related information from the database within the project management system. This information includes task progress, deadlines, and assigned personnel schedules. The server retrieves this information using database queries and integrates relevant data. This integration process prepares the project information to be presented to the user.

[0175] Step 4:

[0176] The server dynamically adjusts how information is presented based on the analyzed emotional state of the user. For example, if the user is feeling stressed, the server generates a concise list and sends it to the terminal to present the information in a simple and easy-to-understand format. The adjusted information is then displayed on the user's terminal as output.

[0177] Step 5:

[0178] The server adjusts task priorities based on the emotion analysis results. For users in a state of tension, it clearly identifies high-priority tasks, while for calm users, it presents more detailed task information. The adjusted task priorities are output and notified to the user via the terminal.

[0179] (Application Example 2)

[0180] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0181] In the work environment, the emotional state of employees and operators can affect work efficiency. However, conventional task management systems do not take users' emotions into consideration, and work can be hindered by stress and fatigue. Furthermore, the information presented is not adjusted to the user's situation, often increasing the user's burden. A new system is needed to solve these problems.

[0182] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0183] In this invention, the server includes means for automatically collecting relevant information from an information medium, means for analyzing the collected information and integrating related information, and means for analyzing the user's emotions and dynamically adjusting the method of presenting information based on that analysis. This enables efficient task management and information presentation in accordance with the user's emotional state.

[0184] "Information media" refers to media used to acquire and transmit information, such as computer networks and databases.

[0185] "Automatic collection of relevant information" is a process that autonomously acquires useful data based on specific conditions or keywords.

[0186] "Information integration" is the process of organizing collected information and linking related data to create a unified whole.

[0187] "User emotion analysis" is the process of analyzing nonverbal information such as a user's facial expressions and voice to determine their emotional state.

[0188] "Dynamic adjustment" means automatically changing the operation or settings in real time in response to changes in circumstances or conditions.

[0189] "Evaluating the progress of a task" involves measuring and analyzing the level of completion and progress of the current task.

[0190] "Report generation" is the process of compiling information in document format based on analysis results and progress.

[0191] The system for implementing this invention consists of a server for data processing, a cloud service for sentiment analysis, and a terminal used by the user. The details of each element are as follows.

[0192] The server has the functionality to automatically collect relevant data from information media and analyze and integrate that information. Specifically, it collects and stores information from databases using the Python language and MongoDB, and performs user sentiment analysis through the OpenCV library and Google Cloud Sentiment Analysis API. It also dynamically delivers information to the user's device using the Flask framework.

[0193] The user's device refers to a smartphone or tablet, and uses its camera and microphone to capture nonverbal cues (facial expressions and voice). This data is transmitted to the server in real time and used to analyze the user's emotional state. Based on the analysis results, the server dynamically adjusts how the information is presented.

[0194] For example, when a smartphone application is used on a factory floor, the priority of tasks is automatically adjusted according to the user's level of concentration and stress, enabling efficient task management.

[0195] As a concrete example, when supervising work on-site, it is possible to send a prompt message to the system such as, "Analyze the user's current emotional state from their facial expressions and tone of voice, and generate a priority task list." This will provide the optimal work procedure according to the emotional state.

[0196] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0197] Step 1:

[0198] The device captures the user's facial expressions and voice through its camera and microphone. The input consists of real-time video and audio data, which the device then transmits to the server.

[0199] Step 2:

[0200] The server analyzes the received video and audio data. It uses the OpenCV library to analyze faces and expressions, and the Google Cloud Sentiment Analysis API to extract emotional states from the audio data. This results in an output that determines the user's current emotional state.

[0201] Step 3:

[0202] The server dynamically adjusts the content and order of information presented to the user based on the analyzed emotional state. The input is the result of the emotional analysis, which is used to select high-priority information and tasks.

[0203] Step 4:

[0204] The server generates a task list and information tailored to the user's needs. Using the Flask framework, the generated information is sent to the user's terminal.

[0205] Step 5:

[0206] The user reviews information and tasks sent from the server through an application on their device and enters prompt messages as needed. These prompt messages are used to provide additional instructions to the system.

[0207] Step 6:

[0208] The server re-evaluates the system status based on the received prompt message and updates the information content and task management methods as needed. It then sends the newly updated information back to the user as output.

[0209] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0210] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0211] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0212] [Second Embodiment]

[0213] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0214] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0215] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0216] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0217] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0218] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0219] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0220] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0221] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0222] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0223] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0224] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0225] This invention is a system aimed at centralizing information and efficiently managing progress in project management. It grasps and manages the progress of a project through a process of collecting relevant information from information sources, analyzing, integrating, and evaluating it.

[0226] Specifically, the system includes the following operations: First, the user enters project-related keywords through the interface to identify the information to be collected. This prepares the system to accurately retrieve information related to those keywords.

[0227] Next, the server accesses the information sources specified by the user, such as email servers, chat applications, and online meeting systems, and collects data that matches the keywords. The information sources accessed are designed to be handled securely in accordance with security protocols.

[0228] The collected information is analyzed within the server. Natural language processing technology is used to identify related information and integrate information obtained from multiple sources. This integration allows users to view dispersed information in a single, centralized manner.

[0229] Furthermore, the server generates progress reports based on the integrated information. These reports are provided to project managers and supervisors, allowing them to gain a detailed understanding of the project's current status. The reports include an overview of progress, challenges, and potential risks.

[0230] In addition, the server automatically detects inconsistencies in the collected information and task delays, and notifies the user. For example, if there are conflicting decisions between different departments, it will point them out and propose corrective measures to support smooth project progress.

[0231] Finally, using the integrated data, the AI ​​module checks for consistency in annotations accompanying advertisements and documents and suggests corrections as needed. This ensures consistent communication within the project.

[0232] As a concrete example, suppose a user is managing a new product development project. In this case, by integrating all project-related information into the system and quickly identifying inconsistencies between different teams, it is possible to achieve smoother collaboration between teams and improve the efficiency of project execution.

[0233] The following describes the processing flow.

[0234] Step 1:

[0235] Users log in to the project management system and enter keywords related to the projects they are managing through the interface. These keywords act as filters for information gathering and are used to narrow down specific information.

[0236] Step 2:

[0237] The server securely accesses user-specified information sources (e.g., email servers, chat applications, meeting platforms) based on configured keywords. During this process, it automatically collects information related to the keywords from each information source.

[0238] Step 3:

[0239] The server stores the collected information in a temporary database and analyzes the data using natural language processing techniques. Here, relevant information is identified, and preparations are made to group similar or related information together.

[0240] Step 4:

[0241] The server integrates the analyzed information to create a centralized information platform. This platform functions as a project-specific information base, accessible to users for review.

[0242] Step 5:

[0243] The server evaluates project progress based on integrated information. An AI algorithm analyzes the progress and generates daily progress reports. These reports are automatically delivered according to user settings.

[0244] Step 6:

[0245] The server initiates a process to detect inconsistencies in information, identifying conflicting decisions and information between different teams. The findings are compiled into a report, and alerts are sent to users as needed.

[0246] Step 7:

[0247] The server performs task progress management and identifies tasks that are behind schedule. Based on this information, users receive real-time notifications.

[0248] Step 8:

[0249] The AI ​​module initiates the annotation verification process, checking for consistency in annotations within documents and advertisements. If inconsistencies are found, suggested corrections are generated and the user is notified.

[0250] Step 9:

[0251] The terminal ultimately provides users with progress reports, task notifications, and inconsistency detection reports from the server, allowing users to take necessary actions and make corrections based on this information, thereby improving the efficiency of project management.

[0252] (Example 1)

[0253] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0254] In project management, the widespread and dispersed nature of information makes it difficult to accurately grasp progress, often leading to task delays and inconsistencies. Furthermore, inconsistent information across different teams can reduce the efficiency of project execution.

[0255] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0256] In this invention, the server includes means for automatically collecting relevant information based on keywords from information sources, means for analyzing the collected information using natural language processing technology and integrating related information, and means for evaluating the progress of the project based on the integrated information and generating a report. This enables centralized management of information within a project and smooth collaboration between different teams.

[0257] "Information sources" refer to the media or systems from which data is collected, specifically including email, chat applications, and online meeting records.

[0258] A "keyword" is a term that a user identifies as the focus of information gathering during project management, and related information is searched based on this keyword.

[0259] "Means of automatic data collection" refers to system functions that obtain necessary data from information sources without user intervention.

[0260] "Natural language processing technology" is a technology that enables computers to process and understand human language, and is used to analyze collected data.

[0261] A "means for integrating related information" refers to a system that has the function of identifying relevant information from collected data and compiling it in a consistent format.

[0262] "Means for evaluating progress and generating reports" refers to the function of a system that analyzes the current status of a project and compiles the results into a report.

[0263] "Means for detecting and pointing out inconsistencies and task delays" refers to a system that has the functionality to discover inconsistencies and delays in data and communicate them to the user.

[0264] A "means for checking the consistency of annotations and proposing corrections" refers to a system that has the function of checking whether the annotations in documents used within a project are consistent and, if necessary, making suggestions to maintain consistency.

[0265] This invention provides a system that enables centralized information management and efficient progress management in project management, and grasps the project status through information collection, analysis, integration, and evaluation. Specific embodiments are shown below.

[0266] First, the user enters project-related keywords through an interface. This interface is a software module that runs on the terminal and identifies the information to be collected based on the user's input of specific keywords.

[0267] Next, the server accesses information sources specified based on keywords, such as email servers, chat applications, and online meeting systems, and collects relevant information. The server ensures the security of the information by accessing it in accordance with robust security protocols. The technologies used include modern cloud computing platforms and APIs for server access.

[0268] The collected information is analyzed on the server using natural language processing technology. This identifies and integrates related data obtained from different sources. Natural language processing libraries in Python and R are used in this analysis and integration process.

[0269] Furthermore, based on the integrated data, the server generates a project progress report. This report is provided to project managers and supervisors, allowing them to gain a detailed understanding of the project's current status. The report includes an overview of the project's progress, issues, and a presentation of potential risks.

[0270] Furthermore, the server analyzes the integrated data and automatically detects inconsistencies between information and task delays. This allows it to identify conflicts between different departments and propose appropriate corrections.

[0271] Finally, the AI ​​module uses the integrated data to check the consistency of annotations accompanying advertisements, documents, and other materials, and suggests corrections as needed. This helps maintain consistency in communication within the project.

[0272] As a concrete example, suppose a user is managing a "new product development" project. In this case, all relevant information is integrated into the system, and inconsistencies between different teams can be quickly identified, leading to smoother team collaboration and improved project efficiency.

[0273] An example of a prompt for a generative AI model is, "Please summarize the progress and challenges of the new product development project in a report."

[0274] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0275] Step 1:

[0276] The user enters project-related keywords into the terminal interface. This input instructs the system on what information to collect. For example, if the user enters keywords such as "new product development" or "market research," the collection of relevant data will begin. This clarifies the target of information collection.

[0277] Step 2:

[0278] The server accesses the specified information sources based on the keywords and collects relevant information. Using the input keywords, it accesses the mail server, chat application, and online meeting system to obtain relevant messages, documents, etc. Here, by filtering the data that matches the keywords from the accessed information sources, formatting it, and collecting it, a consistent dataset is formed.

[0279] Step 3:

[0280] The server analyzes the information collected using natural language processing techniques. The input is the raw data collected in Step 2. By processing this data, relevant content is extracted and irrelevant information is excluded. Here, natural language processing libraries such as Python and R are used to analyze the text content and integrate important information related to the project.

[0281] Step 4:

[0282] The server generates a progress report based on the analyzed and integrated information. The input for this step is the integrated data, and using it, a report summarizing the progress, issues, and risks of the project is created as the output. Here, the information is organized according to the report format and output in an easily understandable form. The report is generated in text format and provided to the project manager.

[0283] Step 5:

[0284] The server analyzes the integrated data, detects inconsistencies and task delays between the information, and notifies the user. The input here is the dataset generated in Step 3 and Step 4. By checking the consistency between the data, contradictions and delays are identified, and a notification including suggestions is generated as the output.

[0285] Step 6:

[0286] Based on the integrated data, the AI module checks the consistency of the annotations attached to the documents and advertisements within the project and proposes corrections if necessary. The input is the integrated information from Step 3, and the output is to provide the user with amendments and proposals to maintain the consistency of the annotations. This ensures that the overall communication is coherent.

[0287] (Application Example 1)

[0288] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0289] In modern production sites, it is important for engineers to efficiently grasp the progress of projects and respond quickly. However, since information is scattered everywhere, it is not easy to integrate them and provide them to on-site workers in a consistent manner. As a result, delays and inconsistencies may occur in the progress of projects, which may have an adverse impact on productivity. Therefore, it is required to solve these problems and achieve the integration and efficient utilization of information.

[0290] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0291] In this invention, the server includes means for automatically collecting relevant information from information sources, means for analyzing the collected information and integrating relevant information, and means for visualizing using the data collected from various devices and presenting the information to users. This enables workers to grasp the progress of projects in real time and respond effectively.

[0292] The "information source" refers to various devices and platforms accessed for data acquisition, such as email servers and online meeting systems.

[0293] "Collection means" refers to the part that has the function of automatically acquiring relevant information, and is responsible for extracting data from information sources and sending it to the server.

[0294] "Analysis means" refers to the part that has the function of analyzing acquired information and evaluating its relationships, and performs the process of extracting information that can be integrated.

[0295] The term "integration means" refers to the part that has the function of aggregating and centrally managing the analyzed information, and making related data available for display.

[0296] "Visualization means" refers to the part that has the function of presenting information to the worker in an intuitively easy-to-understand form, and displays it in real time through a head-mounted display or monitor.

[0297] "Inconsistency detection means" refers to a component that has the function of identifying contradictions and inconsistencies between pieces of information, and is necessary to improve the accuracy of progress management.

[0298] "Notification means" refers to the part of the system that has the function of conveying important information to the user, such as informing them of task delays or the need for new instructions.

[0299] The system implementing this invention primarily has functions for collecting, analyzing, and integrating data, and for visually presenting it. The entire system consists of a server, user terminals, and a group of information source devices.

[0300] The server automatically collects data from information sources. These sources include mail servers, chat applications, and online meeting systems, which are accessed securely through security protocols. This collected data is analyzed within the server using natural language processing libraries such as NLTK and spaCy to identify relevant information and integrate it centrally.

[0301] The integrated information is displayed on the head-mounted display (HMD), which is the user terminal. Here, Unity is utilized for visualization to enable intuitive understanding by the operator. Additionally, the user can issue voice commands through the voice recognition function incorporated in the HMD, thereby enabling detailed acquisition of specific project information and giving instructions for changes.

[0302] The AI module installed on the server automatically detects inconsistencies and task delays and sends real-time notifications to the user. These notifications are carried out using the visual display and voice output of the HMD, prompting immediate response at the work site.

[0303] For example, when the system detects that the production of Product A in the factory is delayed, the server analyzes the reason and proposes a quick solution. At this time, the operator can receive new instructions via the HMD and efficiently readjust the task. As an example of a prompt sentence for the generative AI model, "The production of Product A in the factory is currently delayed. Collect the latest data from the manufacturing department, identify the cause of the delay, and propose a solution to the manager" can be cited.

[0304] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0305] Step 1:

[0306] The server connects to the information source to collect data. The input is the keyword specified by the user, and the output is the raw data obtained from the information source. At this stage, the server accesses the API of the information source and securely extracts data using a security protocol.

[0307] Step 2:

[0308] The server analyzes the collected raw data. The input is the raw data obtained in step 1, and the output is a snapshot of the analyzed data. Here, natural language processing is performed using NLTK or spaCy to clarify the structure of the information. For example, relevant information is tagged based on keywords in the manufacturing process.

[0309] Step 3:

[0310] The server integrates and centralizes the analysis results. The input is a data snapshot obtained from step 2, and the output is a dataset of the integrated information. This process combines relevant information from different sources and transforms it into a consistent format, allowing for a comprehensive view of the project.

[0311] Step 4:

[0312] The server visualizes integrated information and sends it to the terminal. The input is an integrated dataset, and the output is a visualized user interface. The server uses an interface built with Unity to allow workers to visually check the information in real time through an HMD (Head-Mounted Display). Here, graphs and dashboards are displayed on the user interface, making the progress intuitively understandable.

[0313] Step 5:

[0314] The server uses an AI module to detect inconsistencies and delays and notify the user. Inputs are an integrated dataset and current progress information, and output is an alert message. The AI ​​uses pattern recognition and predictive algorithms to identify problems and generates visual and audible warnings for the worker. For example, it might send a notification such as, "There is a delay on the production line for product A. We are investigating the cause."

[0315] Step 6:

[0316] The user provides voice instructions to the terminal as needed. The input is the user's voice instructions, and the output is interactive feedback actions. Voice recognition software analyzes the instructions, and the server performs the next corresponding action based on the results. Specifically, when an instruction such as "Plan additional production of product A" is input, the project plan is automatically adjusted based on that information.

[0317] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0318] This invention combines a project management system with an emotion engine, aiming to provide a more personalized experience by recognizing the user's emotions within the project management process and adjusting the presentation of information and notifications based on those emotions.

[0319] Specifically, when a user uses the system, the emotion engine recognizes the user's emotions in real time. This emotional state is analyzed from text entered by the user and non-verbal cues (such as facial expressions and tone of voice) acquired through the device's camera and microphone.

[0320] The server analyzes project-related data collected from information sources and presents relevant information to the user. During this process, the presentation of the information is adjusted according to the user's emotions, as recognized by the emotion engine. For example, if the user is stressed, the system reduces the user's burden by presenting information more concisely and focusing on high-priority tasks.

[0321] In addition, the server dynamically adjusts task priorities based on the user's emotional state. When the user is calm, it optimizes project progress by generating more detailed reports and notifying users of complex tasks.

[0322] This system allows users to receive project information and manage tasks in a way that is optimally tailored to their emotional state. For example, if a user is feeling nervous right before an important meeting, the emotion engine recognizes this state, and the server creates a concise list of key action items for the day, providing it on the user's device to reduce their burden. In this way, the present invention enables more efficient project execution through information management that takes emotions into account.

[0323] The following describes the processing flow.

[0324] Step 1:

[0325] The user logs into the project management system and sets project-related keywords on the interface. These keywords determine the target of information collection.

[0326] Step 2:

[0327] The server accesses user-defined information sources and automatically collects relevant information that matches specified keywords. This information can range from email servers and chat applications to meeting records.

[0328] Step 3:

[0329] An emotion engine within the server operates, analyzing text messages entered by the user, as well as facial expressions and voice recordings collected by the device's microphone and camera, to identify the user's emotional state in real time.

[0330] Step 4:

[0331] The server analyzes the collected information, integrating and centralizing relevant data. During this process, it refers to the output of the emotion engine to determine how to present information according to the user's emotional state.

[0332] Step 5:

[0333] Based on the integrated information, the server evaluates project progress and generates a report with appropriate content and format, taking into account the sentiment index. For example, if a user is experiencing stress, the information is summarized concisely, and the report prioritizes including only the most important information.

[0334] Step 6:

[0335] The server adjusts task priorities based on the user's emotional state. If the user appears anxious, it prioritizes presenting achievable short-term goals to prevent them from falling behind.

[0336] Step 7:

[0337] The terminal provides users with generated reports and alert notifications. Users use these to check project progress and take necessary management and action.

[0338] Step 8:

[0339] The terminal continuously updates the emotion engine's results based on newly entered information and operation history from the user, and feeds this information back to the server. This enables project management that is always based on the latest user status.

[0340] (Example 2)

[0341] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0342] In project management, users are often overburdened with information because their emotional state is not taken into consideration. As a result, users may lose sight of the priorities of important tasks or experience unnecessary stress. Furthermore, the failure to present information in a way that is sensitive to the user's emotions can lead to decreased work efficiency and negatively impact the overall progress of the project.

[0343] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0344] In this invention, the server includes means for automatically collecting relevant information from information sources, means for analyzing the collected information and integrating related information, and means for analyzing the user's emotional state and dynamically adjusting the information presentation method based on the emotional state. This makes it possible to optimize information presentation and task prioritization according to the user's emotional state.

[0345] "Information source" refers to the source or medium from which a system obtains data, and this includes document databases, online resources, or other digital recording media.

[0346] "Means of collection" refers to the techniques and methods used to obtain necessary data from information sources, and includes database queries and web scraping techniques.

[0347] "Means of analysis" refers to methods and techniques for analyzing collected data and understanding its structure and meaning, and includes data mining and machine learning algorithms.

[0348] "Means of integration" refer to techniques and methods for combining analyzed information to achieve a holistic understanding and evaluation, and these include data normalization and merging operations.

[0349] "Means for analyzing emotional states" refers to methods and technologies for understanding a user's emotions, and includes natural language processing, speech recognition, and facial expression recognition technologies.

[0350] "Means of dynamically adjusting the method of information presentation" refers to technologies that change how information is displayed according to the user's emotional state, and this includes changing the display format and optimizing the UI.

[0351] "Means of dynamically adjusting priorities" refers to technologies that flexibly change task priorities while taking into account the user's emotional state, and this includes algorithmic automatic adjustment functions.

[0352] This invention provides comprehensive emotion recognition functionality for project management systems. The core of the invention involves implementing an emotion engine and data collection, analysis, and integration functions on both the server and the terminal. This allows users to receive optimal support tailored to their emotional state.

[0353] This system utilizes the following hardware and software: The server automatically collects relevant information from sources using a database management system (e.g., MySQL). It also uses generative AI models for natural language processing and speech analysis to analyze the user's emotional state in real time. The terminal functions as hardware that collects nonverbal information from the user (facial expressions, tone of voice, etc.) using a camera and microphone. In addition, the end user is provided with a user interface that is dynamically adjusted based on their emotional state.

[0354] The server uses an emotion engine to detect the user's emotional state and adjusts the information display method and task priorities accordingly. This enables the presentation of information tailored to the user's situation and emotions, resulting in reduced stress and improved work efficiency.

[0355] For example, if tension is detected in a user preparing an important presentation, the server will provide a list of materials that should be displayed in summary format and focus on key action items to help the user prepare more easily. Furthermore, the operation of this mechanism can be verified by entering an example prompt message into the server, such as, "If sentiment analysis indicates the user is experiencing stress, please help design a program that adjusts task priorities and presents information more concisely."

[0356] Thus, the present invention is a system that provides a new method for recognizing a user's emotional state with high accuracy and realizing project management tailored to individual circumstances.

[0357] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0358] Step 1:

[0359] The user logs into the project management system. The device collects nonverbal data such as the user's facial expressions and tone of voice through its camera and microphone. Text entered by the user is also collected as input data. This data becomes input to the emotion engine.

[0360] Step 2:

[0361] The server inputs nonverbal and text data collected from the terminal into a generative AI model. The generative AI model uses natural language processing and speech analysis algorithms to analyze the user's emotional state. As a result of the analysis, the user's emotional state is classified into categories such as "tension," "relaxation," and "stress," and this is returned to the server as output.

[0362] Step 3:

[0363] The server collects project-related information from the database within the project management system. This information includes task progress, deadlines, and assigned personnel schedules. The server retrieves this information using database queries and integrates relevant data. This integration process prepares the project information to be presented to the user.

[0364] Step 4:

[0365] The server dynamically adjusts how information is presented based on the analyzed emotional state of the user. For example, if the user is feeling stressed, the server generates a concise list and sends it to the terminal to present the information in a simple and easy-to-understand format. The adjusted information is then displayed on the user's terminal as output.

[0366] Step 5:

[0367] The server adjusts task priorities based on the emotion analysis results. For users in a state of tension, it clearly identifies high-priority tasks, while for calm users, it presents more detailed task information. The adjusted task priorities are output and notified to the user via the terminal.

[0368] (Application Example 2)

[0369] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0370] In the work environment, the emotional state of employees and operators can affect work efficiency. However, conventional task management systems do not take users' emotions into consideration, and work can be hindered by stress and fatigue. Furthermore, the information presented is not adjusted to the user's situation, often increasing the user's burden. A new system is needed to solve these problems.

[0371] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0372] In this invention, the server includes means for automatically collecting relevant information from an information medium, means for analyzing the collected information and integrating related information, and means for analyzing the user's emotions and dynamically adjusting the method of presenting information based on that analysis. This enables efficient task management and information presentation in accordance with the user's emotional state.

[0373] "Information media" refers to media used to acquire and transmit information, such as computer networks and databases.

[0374] "Automatic collection of relevant information" is a process that autonomously acquires useful data based on specific conditions or keywords.

[0375] "Information integration" is the process of organizing collected information and linking related data to create a unified whole.

[0376] "User emotion analysis" is the process of analyzing nonverbal information such as a user's facial expressions and voice to determine their emotional state.

[0377] "Dynamic adjustment" means automatically changing the operation or settings in real time in response to changes in circumstances or conditions.

[0378] "Evaluating the progress of a task" involves measuring and analyzing the level of completion and progress of the current task.

[0379] "Report generation" is the process of compiling information in document format based on analysis results and progress.

[0380] The system for implementing this invention consists of a server for data processing, a cloud service for sentiment analysis, and a terminal used by the user. The details of each element are as follows.

[0381] The server has the functionality to automatically collect relevant data from information media and analyze and integrate that information. Specifically, it collects and stores information from databases using the Python language and MongoDB, and performs user sentiment analysis through the OpenCV library and the Google Cloud Sentiment Analysis API. It also dynamically delivers information to the user's device using the Flask framework.

[0382] The user's device refers to a smartphone or tablet, and uses its camera and microphone to capture nonverbal cues (facial expressions and voice). This data is transmitted to the server in real time and used to analyze the user's emotional state. Based on the analysis results, the server dynamically adjusts how the information is presented.

[0383] For example, when a smartphone application is used on a factory floor, the priority of tasks is automatically adjusted according to the user's level of concentration and stress, enabling efficient task management.

[0384] As a concrete example, when supervising work on-site, it is possible to send a prompt message to the system such as, "Analyze the user's current emotional state from their facial expressions and tone of voice, and generate a priority task list." This will provide the optimal work procedure according to the emotional state.

[0385] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0386] Step 1:

[0387] The device captures the user's facial expressions and voice through its camera and microphone. The input consists of real-time video and audio data, which the device then transmits to the server.

[0388] Step 2:

[0389] The server analyzes the received video and audio data. It uses the OpenCV library to analyze faces and expressions, and the Google Cloud Sentiment Analysis API to extract emotional states from the audio data. This results in an output that determines the user's current emotional state.

[0390] Step 3:

[0391] The server dynamically adjusts the content and order of information presented to the user based on the analyzed emotional state. The input is the result of the emotional analysis, which is used to select high-priority information and tasks.

[0392] Step 4:

[0393] The server generates a task list and information tailored to the user's needs. Using the Flask framework, the generated information is sent to the user's terminal.

[0394] Step 5:

[0395] The user reviews information and tasks sent from the server through an application on their device and enters prompt messages as needed. These prompt messages are used to provide additional instructions to the system.

[0396] Step 6:

[0397] The server re-evaluates the system status based on the received prompt message and updates the information content and task management methods as needed. It then sends the newly updated information back to the user as output.

[0398] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0399] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0400] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0401] [Third Embodiment]

[0402] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0403] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0404] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0405] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0406] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0407] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0408] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0409] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0410] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0411] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0412] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0413] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0414] This invention is a system aimed at centralizing information and efficiently managing progress in project management. It grasps and manages the progress of a project through a process of collecting relevant information from information sources, analyzing, integrating, and evaluating it.

[0415] Specifically, the system includes the following operations: First, the user enters project-related keywords through the interface to identify the information to be collected. This prepares the system to accurately retrieve information related to those keywords.

[0416] Next, the server accesses the information sources specified by the user, such as email servers, chat applications, and online meeting systems, and collects data that matches the keywords. The information sources accessed are designed to be handled securely in accordance with security protocols.

[0417] The collected information is analyzed within the server. Natural language processing technology is used to identify related information and integrate information obtained from multiple sources. This integration allows users to view dispersed information in a single, centralized manner.

[0418] Furthermore, the server generates progress reports based on the integrated information. These reports are provided to project managers and supervisors, allowing them to gain a detailed understanding of the project's current status. The reports include an overview of progress, challenges, and potential risks.

[0419] In addition, the server automatically detects inconsistencies in the collected information and task delays, and notifies the user. For example, if there are conflicting decisions between different departments, it will point them out and propose corrective measures to support smooth project progress.

[0420] Finally, using the integrated data, the AI ​​module checks for consistency in annotations accompanying advertisements and documents and suggests corrections as needed. This ensures consistent communication within the project.

[0421] As a concrete example, suppose a user is managing a new product development project. In this case, by integrating all project-related information into the system and quickly identifying inconsistencies between different teams, it is possible to achieve smoother collaboration between teams and improve the efficiency of project execution.

[0422] The following describes the processing flow.

[0423] Step 1:

[0424] Users log in to the project management system and enter keywords related to the projects they are managing through the interface. These keywords act as filters for information gathering and are used to narrow down specific information.

[0425] Step 2:

[0426] The server securely accesses user-specified information sources (e.g., email servers, chat applications, meeting platforms) based on configured keywords. During this process, it automatically collects information related to the keywords from each information source.

[0427] Step 3:

[0428] The server stores the collected information in a temporary database and analyzes the data using natural language processing techniques. Here, relevant information is identified, and preparations are made to group similar or related information together.

[0429] Step 4:

[0430] The server integrates the analyzed information to create a centralized information platform. This platform functions as a project-specific information base, accessible to users for review.

[0431] Step 5:

[0432] The server evaluates project progress based on integrated information. An AI algorithm analyzes the progress and generates daily progress reports. These reports are automatically delivered according to user settings.

[0433] Step 6:

[0434] The server initiates a process to detect inconsistencies in information, identifying conflicting decisions and information between different teams. The findings are compiled into a report, and alerts are sent to users as needed.

[0435] Step 7:

[0436] The server performs task progress management and identifies tasks that are behind schedule. Based on this information, users receive real-time notifications.

[0437] Step 8:

[0438] The AI ​​module initiates the annotation verification process, checking for consistency in annotations within documents and advertisements. If inconsistencies are found, suggested corrections are generated and the user is notified.

[0439] Step 9:

[0440] The terminal ultimately provides users with progress reports, task notifications, and inconsistency detection reports from the server, allowing users to take necessary actions and make corrections based on this information, thereby improving the efficiency of project management.

[0441] (Example 1)

[0442] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0443] In project management, the widespread and dispersed nature of information makes it difficult to accurately grasp progress, often leading to task delays and inconsistencies. Furthermore, inconsistent information across different teams can reduce the efficiency of project execution.

[0444] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0445] In this invention, the server includes means for automatically collecting relevant information based on keywords from information sources, means for analyzing the collected information using natural language processing technology and integrating related information, and means for evaluating the progress of the project based on the integrated information and generating a report. This enables centralized management of information within a project and smooth collaboration between different teams.

[0446] "Information sources" refer to the media or systems from which data is collected, specifically including email, chat applications, and online meeting records.

[0447] A "keyword" is a term that a user identifies as the focus of information gathering during project management, and related information is searched based on this keyword.

[0448] "Means of automatic data collection" refers to system functions that obtain necessary data from information sources without user intervention.

[0449] "Natural language processing technology" is a technology that enables computers to process and understand human language, and is used to analyze collected data.

[0450] A "means for integrating related information" refers to a system that has the function of identifying relevant information from collected data and compiling it in a consistent format.

[0451] "Means for evaluating progress and generating reports" refers to the function of a system that analyzes the current status of a project and compiles the results into a report.

[0452] "Means for detecting and pointing out inconsistencies and task delays" refers to a system that has the functionality to discover inconsistencies and delays in data and communicate them to the user.

[0453] A "means for checking the consistency of annotations and proposing corrections" refers to a system that has the function of checking whether the annotations in documents used within a project are consistent and, if necessary, making suggestions to maintain consistency.

[0454] This invention provides a system that enables centralized information management and efficient progress management in project management, and grasps the project status through information collection, analysis, integration, and evaluation. Specific embodiments are shown below.

[0455] First, the user enters project-related keywords through an interface. This interface is a software module that runs on the terminal and identifies the information to be collected based on the user's input of specific keywords.

[0456] Next, the server accesses information sources specified based on keywords, such as email servers, chat applications, and online meeting systems, and collects relevant information. The server ensures the security of the information by accessing it in accordance with robust security protocols. The technologies used include modern cloud computing platforms and APIs for server access.

[0457] The collected information is analyzed on the server using natural language processing technology. This identifies and integrates related data obtained from different sources. Natural language processing libraries in Python and R are used in this analysis and integration process.

[0458] Furthermore, based on the integrated data, the server generates a project progress report. This report is provided to project managers and supervisors, allowing them to gain a detailed understanding of the project's current status. The report includes an overview of the project's progress, issues, and a presentation of potential risks.

[0459] Furthermore, the server analyzes the integrated data and automatically detects inconsistencies between information and task delays. This allows it to identify conflicts between different departments and propose appropriate corrections.

[0460] Finally, the AI ​​module uses the integrated data to check the consistency of annotations accompanying advertisements, documents, and other materials, and suggests corrections as needed. This helps maintain consistency in communication within the project.

[0461] As a concrete example, suppose a user is managing a "new product development" project. In this case, all relevant information is integrated into the system, and inconsistencies between different teams can be quickly identified, leading to smoother team collaboration and improved project efficiency.

[0462] An example of a prompt for a generative AI model is, "Please summarize the progress and challenges of the new product development project in a report."

[0463] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0464] Step 1:

[0465] The user enters project-related keywords into the terminal interface. This input instructs the system on what information to collect. For example, if the user enters keywords such as "new product development" or "market research," the collection of relevant data will begin. This clarifies the target of information collection.

[0466] Step 2:

[0467] The server accesses specified information sources based on keywords and collects relevant information. Using the entered keywords, it accesses mail servers, chat applications, and online meeting systems to retrieve relevant messages, documents, etc. Here, it filters the data matching the keywords from the accessed information sources, formats it, and collects it to form a consistent dataset.

[0468] Step 3:

[0469] The server collects information and analyzes it using natural language processing techniques. The input is the raw data collected in step 2. By processing this data, relevant content is extracted and irrelevant information is eliminated. Here, natural language processing libraries in Python and R are used to analyze the text content and integrate important information relevant to the project.

[0470] Step 4:

[0471] The server generates a progress report based on the analyzed and integrated information. The input for this step is integrated data, which is used to create an output report summarizing the project's progress, challenges, and risks. Here, the information is organized according to the report format and output in an easily understandable manner. The report is generated in text format and provided to the project manager.

[0472] Step 5:

[0473] The server analyzes the integrated data, detects inconsistencies between pieces of information and task delays, and notifies the user. The input here is the dataset generated in steps 3 and 4. By checking the consistency between the data, it identifies inconsistencies and delays and generates a notification as output, including suggestions.

[0474] Step 6:

[0475] The AI ​​module uses integrated data to check the consistency of annotations attached to documents and advertisements within the project and suggests corrections as needed. The input is the integrated information from step 3, and the output provides the user with suggested corrections and suggestions to maintain annotation consistency. This ensures consistent communication throughout the project.

[0476] (Application Example 1)

[0477] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0478] In modern manufacturing environments, it is crucial for engineers to efficiently grasp project progress and respond quickly. However, because information is scattered across various locations, integrating it and providing it to on-site workers in a consistent format is not easy. As a result, delays and inconsistencies can occur in project progress, negatively impacting productivity. Therefore, there is a need to solve these problems and achieve centralized and efficient use of information.

[0479] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0480] In this invention, the server includes means for automatically collecting relevant information from information sources, means for analyzing the collected information and integrating related information, and means for visualizing the data collected from various devices and presenting the information to the user. This enables workers to grasp the progress of projects in real time and take effective action.

[0481] "Information sources" refer to various devices and platforms accessed to acquire data, such as email servers and online meeting systems.

[0482] "Collection means" refers to the part that has the function of automatically acquiring relevant information, and is responsible for extracting data from information sources and sending it to the server.

[0483] "Analysis means" refers to the part that has the function of analyzing acquired information and evaluating its relationships, and performs the process of extracting information that can be integrated.

[0484] The term "integration means" refers to the part that has the function of aggregating and centrally managing the analyzed information, and making related data available for display.

[0485] "Visualization means" refers to the part that has the function of presenting information to the worker in an intuitively easy-to-understand form, and displays it in real time through a head-mounted display or monitor.

[0486] "Inconsistency detection means" refers to a component that has the function of identifying contradictions and inconsistencies between pieces of information, and is necessary to improve the accuracy of progress management.

[0487] "Notification means" refers to the part of the system that has the function of conveying important information to the user, such as informing them of task delays or the need for new instructions.

[0488] The system implementing this invention primarily has functions for collecting, analyzing, and integrating data, and for visually presenting it. The entire system consists of a server, user terminals, and a group of information source devices.

[0489] The server automatically collects data from information sources. These sources include mail servers, chat applications, and online meeting systems, which are accessed securely through security protocols. This collected data is analyzed within the server using natural language processing libraries such as NLTK and spaCy to identify relevant information and integrate it centrally.

[0490] The integrated information is displayed on the user's head-mounted display (HMD). Unity is used to visualize the information in a way that is intuitively understandable to the user. Furthermore, users can issue voice commands through the HMD's built-in voice recognition function, allowing them to obtain detailed information about specific projects or instruct changes to be made.

[0491] The AI ​​module installed in the server automatically detects inconsistencies and task delays and sends real-time notifications to the user. These notifications are delivered using the HMD's visual display and audio output, prompting immediate action at the work site.

[0492] For example, if the system detects a delay in the production of product A at a factory, the server analyzes the reason and proposes a quick solution. In this case, the worker receives new instructions via HMD and can efficiently readjust their tasks. An example of a prompt message for the generated AI model is: "Production of product A is delayed at the factory. Collect the latest data from the manufacturing department, identify the cause of the delay, and propose a solution to the manager."

[0493] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0494] Step 1:

[0495] The server connects to the information source and collects data. The input is a keyword specified by the user, and the output is raw data obtained from the information source. At this stage, the server accesses the information source's API and securely extracts the data using security protocols.

[0496] Step 2:

[0497] The server analyzes the collected raw data. The input is the raw data obtained in step 1, and the output is a snapshot of the analyzed data. Here, natural language processing is performed using NLTK or spaCy to clarify the structure of the information. For example, relevant information is tagged based on keywords in the manufacturing process.

[0498] Step 3:

[0499] The server integrates and centralizes the analysis results. The input is a data snapshot obtained from step 2, and the output is a dataset of the integrated information. This process combines relevant information from different sources and transforms it into a consistent format, allowing for a comprehensive view of the project.

[0500] Step 4:

[0501] The server visualizes integrated information and sends it to the terminal. The input is an integrated dataset, and the output is a visualized user interface. The server uses an interface built with Unity to allow workers to visually check the information in real time through an HMD (Head-Mounted Display). Here, graphs and dashboards are displayed on the user interface, making the progress intuitively understandable.

[0502] Step 5:

[0503] The server uses an AI module to detect inconsistencies and delays and notify the user. Inputs are an integrated dataset and current progress information, and output is an alert message. The AI ​​uses pattern recognition and predictive algorithms to identify problems and generates visual and audible warnings for the worker. For example, it might send a notification such as, "There is a delay on the production line for product A. We are investigating the cause."

[0504] Step 6:

[0505] The user provides voice instructions to the terminal as needed. The input is the user's voice instructions, and the output is interactive feedback actions. Voice recognition software analyzes the instructions, and the server performs the next corresponding action based on the results. Specifically, when an instruction such as "Plan additional production of product A" is input, the project plan is automatically adjusted based on that information.

[0506] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0507] This invention combines a project management system with an emotion engine, aiming to provide a more personalized experience by recognizing the user's emotions within the project management process and adjusting the presentation of information and notifications based on those emotions.

[0508] Specifically, when a user uses the system, the emotion engine recognizes the user's emotions in real time. This emotional state is analyzed from text entered by the user and non-verbal cues (such as facial expressions and tone of voice) acquired through the device's camera and microphone.

[0509] The server analyzes project-related data collected from information sources and presents relevant information to the user. During this process, the presentation of the information is adjusted according to the user's emotions, as recognized by the emotion engine. For example, if the user is stressed, the system reduces the user's burden by presenting information more concisely and focusing on high-priority tasks.

[0510] In addition, the server dynamically adjusts task priorities based on the user's emotional state. When the user is calm, it optimizes project progress by generating more detailed reports and notifying users of complex tasks.

[0511] This system allows users to receive project information and manage tasks in a way that is optimally tailored to their emotional state. For example, if a user is feeling nervous right before an important meeting, the emotion engine recognizes this state, and the server creates a concise list of key action items for the day, providing it on the user's device to reduce their burden. In this way, the present invention enables more efficient project execution through information management that takes emotions into account.

[0512] The following describes the processing flow.

[0513] Step 1:

[0514] The user logs into the project management system and sets project-related keywords on the interface. These keywords determine the target of information collection.

[0515] Step 2:

[0516] The server accesses user-defined information sources and automatically collects relevant information that matches specified keywords. This information can range from email servers and chat applications to meeting records.

[0517] Step 3:

[0518] An emotion engine within the server operates, analyzing text messages entered by the user, as well as facial expressions and voice recordings collected by the device's microphone and camera, to identify the user's emotional state in real time.

[0519] Step 4:

[0520] The server analyzes the collected information, integrating and centralizing relevant data. During this process, it refers to the output of the emotion engine to determine how to present information according to the user's emotional state.

[0521] Step 5:

[0522] Based on the integrated information, the server evaluates project progress and generates a report with appropriate content and format, taking into account the sentiment index. For example, if a user is experiencing stress, the information is summarized concisely, and the report prioritizes including only the most important information.

[0523] Step 6:

[0524] The server adjusts task priorities based on the user's emotional state. If the user appears anxious, it prioritizes presenting achievable short-term goals to prevent them from falling behind.

[0525] Step 7:

[0526] The terminal provides users with generated reports and alert notifications. Users use these to check project progress and take necessary management and action.

[0527] Step 8:

[0528] The terminal continuously updates the emotion engine's results based on newly entered information and operation history from the user, and feeds this information back to the server. This enables project management that is always based on the latest user status.

[0529] (Example 2)

[0530] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0531] In project management, users are often overburdened with information because their emotional state is not taken into consideration. As a result, users may lose sight of the priorities of important tasks or experience unnecessary stress. Furthermore, the failure to present information in a way that is sensitive to the user's emotions can lead to decreased work efficiency and negatively impact the overall progress of the project.

[0532] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0533] In this invention, the server includes means for automatically collecting relevant information from information sources, means for analyzing the collected information and integrating related information, and means for analyzing the user's emotional state and dynamically adjusting the information presentation method based on the emotional state. This makes it possible to optimize information presentation and task prioritization according to the user's emotional state.

[0534] "Information source" refers to the source or medium from which a system obtains data, and this includes document databases, online resources, or other digital recording media.

[0535] "Means of collection" refers to the techniques and methods used to obtain necessary data from information sources, and includes database queries and web scraping techniques.

[0536] "Means of analysis" refers to methods and techniques for analyzing collected data and understanding its structure and meaning, and includes data mining and machine learning algorithms.

[0537] "Means of integration" refer to techniques and methods for combining analyzed information to achieve a holistic understanding and evaluation, and these include data normalization and merging operations.

[0538] "Means for analyzing emotional states" refers to methods and technologies for understanding a user's emotions, and includes natural language processing, speech recognition, and facial expression recognition technologies.

[0539] "Means of dynamically adjusting the method of information presentation" refers to technologies that change how information is displayed according to the user's emotional state, and this includes changing the display format and optimizing the UI.

[0540] "Means of dynamically adjusting priorities" refers to technologies that flexibly change task priorities while taking into account the user's emotional state, and this includes algorithmic automatic adjustment functions.

[0541] This invention provides comprehensive emotion recognition functionality for project management systems. The core of the invention involves implementing an emotion engine and data collection, analysis, and integration functions on both the server and the terminal. This allows users to receive optimal support tailored to their emotional state.

[0542] This system utilizes the following hardware and software: The server automatically collects relevant information from sources using a database management system (e.g., MySQL). It also uses generative AI models for natural language processing and speech analysis to analyze the user's emotional state in real time. The terminal functions as hardware that collects nonverbal information from the user (facial expressions, tone of voice, etc.) using a camera and microphone. In addition, the end user is provided with a user interface that is dynamically adjusted based on their emotional state.

[0543] The server uses an emotion engine to detect the user's emotional state and adjusts the information display method and task priorities accordingly. This enables the presentation of information tailored to the user's situation and emotions, resulting in reduced stress and improved work efficiency.

[0544] For example, if tension is detected in a user preparing an important presentation, the server will provide a list of materials that should be displayed in summary format and focus on key action items to help the user prepare more easily. Furthermore, the operation of this mechanism can be verified by entering an example prompt message into the server, such as, "If sentiment analysis indicates the user is experiencing stress, please help design a program that adjusts task priorities and presents information more concisely."

[0545] Thus, the present invention is a system that provides a new method for recognizing a user's emotional state with high accuracy and realizing project management tailored to individual circumstances.

[0546] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0547] Step 1:

[0548] The user logs into the project management system. The device collects nonverbal data such as the user's facial expressions and tone of voice through its camera and microphone. Text entered by the user is also collected as input data. This data becomes input to the emotion engine.

[0549] Step 2:

[0550] The server inputs nonverbal and text data collected from the terminal into a generative AI model. The generative AI model uses natural language processing and speech analysis algorithms to analyze the user's emotional state. As a result of the analysis, the user's emotional state is classified into categories such as "tension," "relaxation," and "stress," and this is returned to the server as output.

[0551] Step 3:

[0552] The server collects project-related information from the database within the project management system. This information includes task progress, deadlines, and assigned personnel schedules. The server retrieves this information using database queries and integrates relevant data. This integration process prepares the project information to be presented to the user.

[0553] Step 4:

[0554] The server dynamically adjusts how information is presented based on the analyzed emotional state of the user. For example, if the user is feeling stressed, the server generates a concise list and sends it to the terminal to present the information in a simple and easy-to-understand format. The adjusted information is then displayed on the user's terminal as output.

[0555] Step 5:

[0556] The server adjusts task priorities based on the emotion analysis results. For users in a state of tension, it clearly identifies high-priority tasks, while for calm users, it presents more detailed task information. The adjusted task priorities are output and notified to the user via the terminal.

[0557] (Application Example 2)

[0558] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0559] In the work environment, the emotional state of employees and operators can affect work efficiency. However, conventional task management systems do not take users' emotions into consideration, and work can be hindered by stress and fatigue. Furthermore, the information presented is not adjusted to the user's situation, often increasing the user's burden. A new system is needed to solve these problems.

[0560] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0561] In this invention, the server includes means for automatically collecting relevant information from an information medium, means for analyzing the collected information and integrating related information, and means for analyzing the user's emotions and dynamically adjusting the method of presenting information based on that analysis. This enables efficient task management and information presentation in accordance with the user's emotional state.

[0562] "Information media" refers to media used to acquire and transmit information, such as computer networks and databases.

[0563] "Automatic collection of relevant information" is a process that autonomously acquires useful data based on specific conditions or keywords.

[0564] "Information integration" is the process of organizing collected information and linking related data to create a unified whole.

[0565] "User emotion analysis" is the process of analyzing nonverbal information such as a user's facial expressions and voice to determine their emotional state.

[0566] "Dynamic adjustment" means automatically changing the operation or settings in real time in response to changes in circumstances or conditions.

[0567] "Evaluating the progress of a task" involves measuring and analyzing the level of completion and progress of the current task.

[0568] "Report generation" is the process of compiling information in document format based on analysis results and progress.

[0569] The system for implementing this invention consists of a server for data processing, a cloud service for sentiment analysis, and a terminal used by the user. The details of each element are as follows.

[0570] The server has the functionality to automatically collect relevant data from information media and analyze and integrate that information. Specifically, it collects and stores information from databases using the Python language and MongoDB, and performs user sentiment analysis through the OpenCV library and the Google Cloud Sentiment Analysis API. It also dynamically delivers information to the user's device using the Flask framework.

[0571] The user's device refers to a smartphone or tablet, and uses its camera and microphone to capture nonverbal cues (facial expressions and voice). This data is transmitted to the server in real time and used to analyze the user's emotional state. Based on the analysis results, the server dynamically adjusts how the information is presented.

[0572] For example, when a smartphone application is used on a factory floor, the priority of tasks is automatically adjusted according to the user's level of concentration and stress, enabling efficient task management.

[0573] As a concrete example, when supervising work on-site, it is possible to send a prompt message to the system such as, "Analyze the user's current emotional state from their facial expressions and tone of voice, and generate a priority task list." This will provide the optimal work procedure according to the emotional state.

[0574] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0575] Step 1:

[0576] The device captures the user's facial expressions and voice through its camera and microphone. The input consists of real-time video and audio data, which the device then transmits to the server.

[0577] Step 2:

[0578] The server analyzes the received video and audio data. It uses the OpenCV library to analyze faces and expressions, and the Google Cloud Sentiment Analysis API to extract emotional states from the audio data. This results in an output that determines the user's current emotional state.

[0579] Step 3:

[0580] The server dynamically adjusts the content and order of information presented to the user based on the analyzed emotional state. The input is the result of the emotional analysis, which is used to select high-priority information and tasks.

[0581] Step 4:

[0582] The server generates a task list and information tailored to the user's needs. Using the Flask framework, the generated information is sent to the user's terminal.

[0583] Step 5:

[0584] The user reviews information and tasks sent from the server through an application on their device and enters prompt messages as needed. These prompt messages are used to provide additional instructions to the system.

[0585] Step 6:

[0586] The server re-evaluates the system status based on the received prompt message and updates the information content and task management methods as needed. It then sends the newly updated information back to the user as output.

[0587] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0588] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0589] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0590] [Fourth Embodiment]

[0591] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0592] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0593] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0594] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0595] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0596] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0597] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0598] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0599] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0600] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0601] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0602] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0603] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0604] This invention is a system aimed at centralizing information and efficiently managing progress in project management. It grasps and manages the progress of a project through a process of collecting relevant information from information sources, analyzing, integrating, and evaluating it.

[0605] Specifically, the system includes the following operations: First, the user enters project-related keywords through the interface to identify the information to be collected. This prepares the system to accurately retrieve information related to those keywords.

[0606] Next, the server accesses the information sources specified by the user, such as email servers, chat applications, and online meeting systems, and collects data that matches the keywords. The information sources accessed are designed to be handled securely in accordance with security protocols.

[0607] The collected information is analyzed within the server. Natural language processing technology is used to identify related information and integrate information obtained from multiple sources. This integration allows users to view dispersed information in a single, centralized manner.

[0608] Furthermore, the server generates progress reports based on the integrated information. These reports are provided to project managers and supervisors, allowing them to gain a detailed understanding of the project's current status. The reports include an overview of progress, challenges, and potential risks.

[0609] In addition, the server automatically detects inconsistencies in the collected information and task delays, and notifies the user. For example, if there are conflicting decisions between different departments, it will point them out and propose corrective measures to support smooth project progress.

[0610] Finally, using the integrated data, the AI ​​module checks for consistency in annotations accompanying advertisements and documents and suggests corrections as needed. This ensures consistent communication within the project.

[0611] As a concrete example, suppose a user is managing a new product development project. In this case, by integrating all project-related information into the system and quickly identifying inconsistencies between different teams, it is possible to achieve smoother collaboration between teams and improve the efficiency of project execution.

[0612] The following describes the processing flow.

[0613] Step 1:

[0614] Users log in to the project management system and enter keywords related to the projects they are managing through the interface. These keywords act as filters for information gathering and are used to narrow down specific information.

[0615] Step 2:

[0616] The server securely accesses user-specified information sources (e.g., email servers, chat applications, meeting platforms) based on configured keywords. During this process, it automatically collects information related to the keywords from each information source.

[0617] Step 3:

[0618] The server stores the collected information in a temporary database and analyzes the data using natural language processing techniques. Here, relevant information is identified, and preparations are made to group similar or related information together.

[0619] Step 4:

[0620] The server integrates the analyzed information to create a centralized information platform. This platform functions as a project-specific information base, accessible to users for review.

[0621] Step 5:

[0622] The server evaluates project progress based on integrated information. An AI algorithm analyzes the progress and generates daily progress reports. These reports are automatically delivered according to user settings.

[0623] Step 6:

[0624] The server initiates a process to detect inconsistencies in information, identifying conflicting decisions and information between different teams. The findings are compiled into a report, and alerts are sent to users as needed.

[0625] Step 7:

[0626] The server performs task progress management and identifies tasks that are behind schedule. Based on this information, users receive real-time notifications.

[0627] Step 8:

[0628] The AI ​​module initiates the annotation verification process, checking for consistency in annotations within documents and advertisements. If inconsistencies are found, suggested corrections are generated and the user is notified.

[0629] Step 9:

[0630] The terminal ultimately provides users with progress reports, task notifications, and inconsistency detection reports from the server, allowing users to take necessary actions and make corrections based on this information, thereby improving the efficiency of project management.

[0631] (Example 1)

[0632] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0633] In project management, the widespread and dispersed nature of information makes it difficult to accurately grasp progress, often leading to task delays and inconsistencies. Furthermore, inconsistent information across different teams can reduce the efficiency of project execution.

[0634] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0635] In this invention, the server includes means for automatically collecting relevant information based on keywords from information sources, means for analyzing the collected information using natural language processing technology and integrating related information, and means for evaluating the progress of the project based on the integrated information and generating a report. This enables centralized management of information within a project and smooth collaboration between different teams.

[0636] "Information sources" refer to the media or systems from which data is collected, specifically including email, chat applications, and online meeting records.

[0637] A "keyword" is a term that a user identifies as the focus of information gathering during project management, and related information is searched based on this keyword.

[0638] "Means of automatic data collection" refers to system functions that obtain necessary data from information sources without user intervention.

[0639] "Natural language processing technology" is a technology that enables computers to process and understand human language, and is used to analyze collected data.

[0640] A "means for integrating related information" refers to a system that has the function of identifying relevant information from collected data and compiling it in a consistent format.

[0641] "Means for evaluating progress and generating reports" refers to the function of a system that analyzes the current status of a project and compiles the results into a report.

[0642] "Means for detecting and pointing out inconsistencies and task delays" refers to a system that has the functionality to discover inconsistencies and delays in data and communicate them to the user.

[0643] A "means for checking the consistency of annotations and proposing corrections" refers to a system that has the function of checking whether the annotations in documents used within a project are consistent and, if necessary, making suggestions to maintain consistency.

[0644] This invention provides a system that enables centralized information management and efficient progress management in project management, and grasps the project status through information collection, analysis, integration, and evaluation. Specific embodiments are shown below.

[0645] First, the user enters project-related keywords through an interface. This interface is a software module that runs on the terminal and identifies the information to be collected based on the user's input of specific keywords.

[0646] Next, the server accesses information sources specified based on keywords, such as email servers, chat applications, and online meeting systems, and collects relevant information. The server ensures the security of the information by accessing it in accordance with robust security protocols. The technologies used include modern cloud computing platforms and APIs for server access.

[0647] The collected information is analyzed on the server using natural language processing technology. This identifies and integrates related data obtained from different sources. Natural language processing libraries in Python and R are used in this analysis and integration process.

[0648] Furthermore, based on the integrated data, the server generates a project progress report. This report is provided to project managers and supervisors, allowing them to gain a detailed understanding of the project's current status. The report includes an overview of the project's progress, issues, and a presentation of potential risks.

[0649] Furthermore, the server analyzes the integrated data and automatically detects inconsistencies between information and task delays. This allows it to identify conflicts between different departments and propose appropriate corrections.

[0650] Finally, the AI ​​module uses the integrated data to check the consistency of annotations accompanying advertisements, documents, and other materials, and suggests corrections as needed. This helps maintain consistency in communication within the project.

[0651] As a concrete example, suppose a user is managing a "new product development" project. In this case, all relevant information is integrated into the system, and inconsistencies between different teams can be quickly identified, leading to smoother team collaboration and improved project efficiency.

[0652] An example of a prompt for a generative AI model is, "Please summarize the progress and challenges of the new product development project in a report."

[0653] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0654] Step 1:

[0655] The user enters project-related keywords into the terminal interface. This input instructs the system on what information to collect. For example, if the user enters keywords such as "new product development" or "market research," the collection of relevant data will begin. This clarifies the target of information collection.

[0656] Step 2:

[0657] The server accesses specified information sources based on keywords and collects relevant information. Using the entered keywords, it accesses mail servers, chat applications, and online meeting systems to retrieve relevant messages, documents, etc. Here, it filters the data matching the keywords from the accessed information sources, formats it, and collects it to form a consistent dataset.

[0658] Step 3:

[0659] The server collects information and analyzes it using natural language processing techniques. The input is the raw data collected in step 2. By processing this data, relevant content is extracted and irrelevant information is eliminated. Here, natural language processing libraries in Python and R are used to analyze the text content and integrate important information relevant to the project.

[0660] Step 4:

[0661] The server generates a progress report based on the analyzed and integrated information. The input for this step is integrated data, which is used to create an output report summarizing the project's progress, challenges, and risks. Here, the information is organized according to the report format and output in an easily understandable manner. The report is generated in text format and provided to the project manager.

[0662] Step 5:

[0663] The server analyzes the integrated data, detects inconsistencies between pieces of information and task delays, and notifies the user. The input here is the dataset generated in steps 3 and 4. By checking the consistency between the data, it identifies inconsistencies and delays and generates a notification as output, including suggestions.

[0664] Step 6:

[0665] The AI ​​module uses integrated data to check the consistency of annotations attached to documents and advertisements within the project and suggests corrections as needed. The input is the integrated information from step 3, and the output provides the user with suggested corrections and suggestions to maintain annotation consistency. This ensures consistent communication throughout the project.

[0666] (Application Example 1)

[0667] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0668] In modern manufacturing environments, it is crucial for engineers to efficiently grasp project progress and respond quickly. However, because information is scattered across various locations, integrating it and providing it to on-site workers in a consistent format is not easy. As a result, delays and inconsistencies can occur in project progress, negatively impacting productivity. Therefore, there is a need to solve these problems and achieve centralized and efficient use of information.

[0669] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0670] In this invention, the server includes means for automatically collecting relevant information from information sources, means for analyzing the collected information and integrating related information, and means for visualizing the data collected from various devices and presenting the information to the user. This enables workers to grasp the progress of projects in real time and take effective action.

[0671] "Information sources" refer to various devices and platforms accessed to acquire data, such as email servers and online meeting systems.

[0672] "Collection means" refers to the part that has the function of automatically acquiring relevant information, and is responsible for extracting data from information sources and sending it to the server.

[0673] "Analysis means" refers to the part that has the function of analyzing acquired information and evaluating its relationships, and performs the process of extracting information that can be integrated.

[0674] The term "integration means" refers to the part that has the function of aggregating and centrally managing the analyzed information, and making related data available for display.

[0675] "Visualization means" refers to the part that has the function of presenting information to the worker in an intuitively easy-to-understand form, and displays it in real time through a head-mounted display or monitor.

[0676] "Inconsistency detection means" refers to a component that has the function of identifying contradictions and inconsistencies between pieces of information, and is necessary to improve the accuracy of progress management.

[0677] "Notification means" refers to the part of the system that has the function of conveying important information to the user, such as informing them of task delays or the need for new instructions.

[0678] The system implementing this invention primarily has functions for collecting, analyzing, and integrating data, and for visually presenting it. The entire system consists of a server, user terminals, and a group of information source devices.

[0679] The server automatically collects data from information sources. These sources include mail servers, chat applications, and online meeting systems, which are accessed securely through security protocols. This collected data is analyzed within the server using natural language processing libraries such as NLTK and spaCy to identify relevant information and integrate it centrally.

[0680] The integrated information is displayed on the user's head-mounted display (HMD). Unity is used to visualize the information in a way that is intuitively understandable to the user. Furthermore, users can issue voice commands through the HMD's built-in voice recognition function, allowing them to obtain detailed information about specific projects or instruct changes to be made.

[0681] The AI ​​module installed in the server automatically detects inconsistencies and task delays and sends real-time notifications to the user. These notifications are delivered using the HMD's visual display and audio output, prompting immediate action at the work site.

[0682] For example, if the system detects a delay in the production of product A at a factory, the server analyzes the reason and proposes a quick solution. In this case, the worker receives new instructions via HMD and can efficiently readjust their tasks. An example of a prompt message for the generated AI model is: "Production of product A is delayed at the factory. Collect the latest data from the manufacturing department, identify the cause of the delay, and propose a solution to the manager."

[0683] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0684] Step 1:

[0685] The server connects to the information source and collects data. The input is a keyword specified by the user, and the output is raw data obtained from the information source. At this stage, the server accesses the information source's API and securely extracts the data using security protocols.

[0686] Step 2:

[0687] The server analyzes the collected raw data. The input is the raw data obtained in step 1, and the output is a snapshot of the analyzed data. Here, natural language processing is performed using NLTK or spaCy to clarify the structure of the information. For example, relevant information is tagged based on keywords in the manufacturing process.

[0688] Step 3:

[0689] The server integrates and centralizes the analysis results. The input is a data snapshot obtained from step 2, and the output is a dataset of the integrated information. This process combines relevant information from different sources and transforms it into a consistent format, allowing for a comprehensive view of the project.

[0690] Step 4:

[0691] The server visualizes integrated information and sends it to the terminal. The input is an integrated dataset, and the output is a visualized user interface. The server uses an interface built with Unity to allow workers to visually check the information in real time through an HMD (Head-Mounted Display). Here, graphs and dashboards are displayed on the user interface, making the progress intuitively understandable.

[0692] Step 5:

[0693] The server uses an AI module to detect inconsistencies and delays and notify the user. Inputs are an integrated dataset and current progress information, and output is an alert message. The AI ​​uses pattern recognition and predictive algorithms to identify problems and generates visual and audible warnings for the worker. For example, it might send a notification such as, "There is a delay on the production line for product A. We are investigating the cause."

[0694] Step 6:

[0695] The user provides voice instructions to the terminal as needed. The input is the user's voice instructions, and the output is interactive feedback actions. Voice recognition software analyzes the instructions, and the server performs the next corresponding action based on the results. Specifically, when an instruction such as "Plan additional production of product A" is input, the project plan is automatically adjusted based on that information.

[0696] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0697] This invention combines a project management system with an emotion engine, aiming to provide a more personalized experience by recognizing the user's emotions within the project management process and adjusting the presentation of information and notifications based on those emotions.

[0698] Specifically, when a user uses the system, the emotion engine recognizes the user's emotions in real time. This emotional state is analyzed from text entered by the user and non-verbal cues (such as facial expressions and tone of voice) acquired through the device's camera and microphone.

[0699] The server analyzes project-related data collected from information sources and presents relevant information to the user. During this process, the presentation of the information is adjusted according to the user's emotions, as recognized by the emotion engine. For example, if the user is stressed, the system reduces the user's burden by presenting information more concisely and focusing on high-priority tasks.

[0700] In addition, the server dynamically adjusts task priorities based on the user's emotional state. When the user is calm, it optimizes project progress by generating more detailed reports and notifying users of complex tasks.

[0701] This system allows users to receive project information and manage tasks in a way that is optimally tailored to their emotional state. For example, if a user is feeling nervous right before an important meeting, the emotion engine recognizes this state, and the server creates a concise list of key action items for the day, providing it on the user's device to reduce their burden. In this way, the present invention enables more efficient project execution through information management that takes emotions into account.

[0702] The following describes the processing flow.

[0703] Step 1:

[0704] The user logs into the project management system and sets project-related keywords on the interface. These keywords determine the target of information collection.

[0705] Step 2:

[0706] The server accesses user-defined information sources and automatically collects relevant information that matches specified keywords. This information can range from email servers and chat applications to meeting records.

[0707] Step 3:

[0708] An emotion engine within the server operates, analyzing text messages entered by the user, as well as facial expressions and voice recordings collected by the device's microphone and camera, to identify the user's emotional state in real time.

[0709] Step 4:

[0710] The server analyzes the collected information, integrating and centralizing relevant data. During this process, it refers to the output of the emotion engine to determine how to present information according to the user's emotional state.

[0711] Step 5:

[0712] Based on the integrated information, the server evaluates project progress and generates a report with appropriate content and format, taking into account the sentiment index. For example, if a user is experiencing stress, the information is summarized concisely, and the report prioritizes including only the most important information.

[0713] Step 6:

[0714] The server adjusts task priorities based on the user's emotional state. If the user appears anxious, it prioritizes presenting achievable short-term goals to prevent them from falling behind.

[0715] Step 7:

[0716] The terminal provides users with generated reports and alert notifications. Users use these to check project progress and take necessary management and action.

[0717] Step 8:

[0718] The terminal continuously updates the emotion engine's results based on newly entered information and operation history from the user, and feeds this information back to the server. This enables project management that is always based on the latest user status.

[0719] (Example 2)

[0720] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0721] In project management, users are often overburdened with information because their emotional state is not taken into consideration. As a result, users may lose sight of the priorities of important tasks or experience unnecessary stress. Furthermore, the failure to present information in a way that is sensitive to the user's emotions can lead to decreased work efficiency and negatively impact the overall progress of the project.

[0722] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0723] In this invention, the server includes means for automatically collecting relevant information from information sources, means for analyzing the collected information and integrating related information, and means for analyzing the user's emotional state and dynamically adjusting the information presentation method based on the emotional state. This makes it possible to optimize information presentation and task prioritization according to the user's emotional state.

[0724] "Information source" refers to the source or medium from which a system obtains data, and this includes document databases, online resources, or other digital recording media.

[0725] "Means of collection" refers to the techniques and methods used to obtain necessary data from information sources, and includes database queries and web scraping techniques.

[0726] "Means of analysis" refers to methods and techniques for analyzing collected data and understanding its structure and meaning, and includes data mining and machine learning algorithms.

[0727] "Means of integration" refer to techniques and methods for combining analyzed information to achieve a holistic understanding and evaluation, and these include data normalization and merging operations.

[0728] "Means for analyzing emotional states" refers to methods and technologies for understanding a user's emotions, and includes natural language processing, speech recognition, and facial expression recognition technologies.

[0729] "Means of dynamically adjusting the method of information presentation" refers to technologies that change how information is displayed according to the user's emotional state, and this includes changing the display format and optimizing the UI.

[0730] "Means of dynamically adjusting priorities" refers to technologies that flexibly change task priorities while taking into account the user's emotional state, and this includes algorithmic automatic adjustment functions.

[0731] This invention provides comprehensive emotion recognition functionality for project management systems. The core of the invention involves implementing an emotion engine and data collection, analysis, and integration functions on both the server and the terminal. This allows users to receive optimal support tailored to their emotional state.

[0732] This system utilizes the following hardware and software: The server automatically collects relevant information from sources using a database management system (e.g., MySQL). It also uses generative AI models for natural language processing and speech analysis to analyze the user's emotional state in real time. The terminal functions as hardware that collects nonverbal information from the user (facial expressions, tone of voice, etc.) using a camera and microphone. In addition, the end user is provided with a user interface that is dynamically adjusted based on their emotional state.

[0733] The server uses an emotion engine to detect the user's emotional state and adjusts the information display method and task priorities accordingly. This enables the presentation of information tailored to the user's situation and emotions, resulting in reduced stress and improved work efficiency.

[0734] For example, if tension is detected in a user preparing an important presentation, the server will provide a list of materials that should be displayed in summary format and focus on key action items to help the user prepare more easily. Furthermore, the operation of this mechanism can be verified by entering an example prompt message into the server, such as, "If sentiment analysis indicates the user is experiencing stress, please help design a program that adjusts task priorities and presents information more concisely."

[0735] Thus, the present invention is a system that provides a new method for recognizing a user's emotional state with high accuracy and realizing project management tailored to individual circumstances.

[0736] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0737] Step 1:

[0738] The user logs into the project management system. The device collects nonverbal data such as the user's facial expressions and tone of voice through its camera and microphone. Text entered by the user is also collected as input data. This data becomes input to the emotion engine.

[0739] Step 2:

[0740] The server inputs nonverbal and text data collected from the terminal into a generative AI model. The generative AI model uses natural language processing and speech analysis algorithms to analyze the user's emotional state. As a result of the analysis, the user's emotional state is classified into categories such as "tension," "relaxation," and "stress," and this is returned to the server as output.

[0741] Step 3:

[0742] The server collects project-related information from the database within the project management system. This information includes task progress, deadlines, and assigned personnel schedules. The server retrieves this information using database queries and integrates relevant data. This integration process prepares the project information to be presented to the user.

[0743] Step 4:

[0744] The server dynamically adjusts how information is presented based on the analyzed emotional state of the user. For example, if the user is feeling stressed, the server generates a concise list and sends it to the terminal to present the information in a simple and easy-to-understand format. The adjusted information is then displayed on the user's terminal as output.

[0745] Step 5:

[0746] The server adjusts task priorities based on the emotion analysis results. For users in a state of tension, it clearly identifies high-priority tasks, while for calm users, it presents more detailed task information. The adjusted task priorities are output and notified to the user via the terminal.

[0747] (Application Example 2)

[0748] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0749] In the work environment, the emotional state of employees and operators can affect work efficiency. However, conventional task management systems do not take users' emotions into consideration, and work can be hindered by stress and fatigue. Furthermore, the information presented is not adjusted to the user's situation, often increasing the user's burden. A new system is needed to solve these problems.

[0750] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0751] In this invention, the server includes means for automatically collecting relevant information from an information medium, means for analyzing the collected information and integrating related information, and means for analyzing the user's emotions and dynamically adjusting the method of presenting information based on that analysis. This enables efficient task management and information presentation in accordance with the user's emotional state.

[0752] "Information media" refers to media used to acquire and transmit information, such as computer networks and databases.

[0753] "Automatic collection of relevant information" is a process that autonomously acquires useful data based on specific conditions or keywords.

[0754] "Information integration" is the process of organizing collected information and linking related data to create a unified whole.

[0755] "User emotion analysis" is the process of analyzing nonverbal information such as a user's facial expressions and voice to determine their emotional state.

[0756] "Dynamic adjustment" means automatically changing the operation or settings in real time in response to changes in circumstances or conditions.

[0757] "Evaluating the progress of a task" involves measuring and analyzing the level of completion and progress of the current task.

[0758] "Report generation" is the process of compiling information in document format based on analysis results and progress.

[0759] The system for implementing this invention consists of a server for data processing, a cloud service for sentiment analysis, and a terminal used by the user. The details of each element are as follows.

[0760] The server has the functionality to automatically collect relevant data from information media and analyze and integrate that information. Specifically, it collects and stores information from databases using the Python language and MongoDB, and performs user sentiment analysis through the OpenCV library and the Google Cloud Sentiment Analysis API. It also dynamically delivers information to the user's device using the Flask framework.

[0761] The user's device refers to a smartphone or tablet, and uses its camera and microphone to capture nonverbal cues (facial expressions and voice). This data is transmitted to the server in real time and used to analyze the user's emotional state. Based on the analysis results, the server dynamically adjusts how the information is presented.

[0762] For example, when a smartphone application is used on a factory floor, the priority of tasks is automatically adjusted according to the user's level of concentration and stress, enabling efficient task management.

[0763] As a concrete example, when supervising work on-site, it is possible to send a prompt message to the system such as, "Analyze the user's current emotional state from their facial expressions and tone of voice, and generate a priority task list." This will provide the optimal work procedure according to the emotional state.

[0764] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0765] Step 1:

[0766] The device captures the user's facial expressions and voice through its camera and microphone. The input consists of real-time video and audio data, which the device then transmits to the server.

[0767] Step 2:

[0768] The server analyzes the received video and audio data. It uses the OpenCV library to analyze faces and expressions, and the Google Cloud Sentiment Analysis API to extract emotional states from the audio data. This results in an output that determines the user's current emotional state.

[0769] Step 3:

[0770] The server dynamically adjusts the content and order of information presented to the user based on the analyzed emotional state. The input is the result of the emotional analysis, which is used to select high-priority information and tasks.

[0771] Step 4:

[0772] The server generates a task list and information tailored to the user's needs. Using the Flask framework, the generated information is sent to the user's terminal.

[0773] Step 5:

[0774] The user reviews information and tasks sent from the server through an application on their device and enters prompt messages as needed. These prompt messages are used to provide additional instructions to the system.

[0775] Step 6:

[0776] The server re-evaluates the system status based on the received prompt message and updates the information content and task management methods as needed. It then sends the newly updated information back to the user as output.

[0777] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0778] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0779] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0780] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0781] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0782] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0783] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0784] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0785] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0786] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0787] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0788] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0789] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0790] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0791] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0792] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0793] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0794] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0795] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0796] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0797] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0798] The following is further disclosed regarding the embodiments described above.

[0799] (Claim 1)

[0800] A means of automatically collecting relevant information from sources,

[0801] A means of analyzing collected information and integrating related information,

[0802] A means of evaluating progress and generating a report based on integrated information,

[0803] A means of detecting and pointing out inconsistencies between multiple pieces of information,

[0804] A system that includes this.

[0805] (Claim 2)

[0806] The system according to claim 1, comprising means for managing the progress of a task and notifying the user if there is a delay from the schedule.

[0807] (Claim 3)

[0808] The system according to claim 1, which has means for checking the consistency of annotations and proposing corrections if necessary.

[0809] "Example 1"

[0810] (Claim 1)

[0811] A means of automatically collecting relevant information based on keywords from information sources,

[0812] A means of analyzing collected information using natural language processing technology and integrating related information,

[0813] A means of evaluating project progress and generating reports based on integrated information,

[0814] A means of detecting and pointing out inconsistencies in the data and task delays,

[0815] A means to check the consistency of annotations in a document and propose corrections if necessary,

[0816] A system that includes this.

[0817] (Claim 2)

[0818] The system according to claim 1, which has means to centralize information in project management and quickly identify inconsistencies in information between different teams, thereby improving smooth collaboration between teams and the efficiency of project execution.

[0819] (Claim 3)

[0820] The system according to claim 1, comprising means for providing the generated report to the user in real time and for facilitating efficient understanding of the project's progress.

[0821] "Application Example 1"

[0822] (Claim 1)

[0823] A means of automatically collecting relevant information from sources,

[0824] A means of analyzing collected information and integrating related information,

[0825] A means of evaluating progress and generating a report based on integrated information,

[0826] A means of detecting and pointing out inconsistencies between multiple pieces of information,

[0827] A means of visualizing data collected from various devices and presenting that information to users,

[0828] A system that includes this.

[0829] (Claim 2)

[0830] The system according to claim 1, having means for managing the progress of a task, notifying the user if there is a delay from the schedule, and enabling the worker to provide instructions.

[0831] (Claim 3)

[0832] The system according to claim 1, comprising means for verifying the consistency of annotations and proposing corrections if necessary, and means for detecting and notifying inconsistencies or risks using an AI module.

[0833] "Example 2 of combining an emotion engine"

[0834] (Claim 1)

[0835] A means of automatically collecting relevant information from sources,

[0836] A means of analyzing collected information and integrating related information,

[0837] A means of evaluating progress and generating a report based on integrated information,

[0838] A means of detecting and pointing out inconsistencies between multiple pieces of information,

[0839] A means for analyzing the user's emotional state and dynamically adjusting the information presentation method based on that emotional state,

[0840] A means of dynamically adjusting the priority of tasks based on the user's emotional state,

[0841] A system that includes this.

[0842] (Claim 2)

[0843] The system according to claim 1, comprising means for managing the progress of a task and notifying the user if there is a delay from the schedule.

[0844] (Claim 3)

[0845] The system according to claim 1, which has means for checking the consistency of annotations and proposing corrections if necessary.

[0846] "Application example 2 of combining emotional engines"

[0847] (Claim 1)

[0848] A means of automatically collecting relevant information from information media,

[0849] A means of analyzing collected information and integrating related information,

[0850] A means of analyzing user emotions and dynamically adjusting the way information is presented based on those emotions,

[0851] A means of evaluating the progress of work and generating a report based on integrated information,

[0852] A means of detecting and pointing out inconsistencies between multiple pieces of information,

[0853] A system that includes this.

[0854] (Claim 2)

[0855] The system according to claim 1, further comprising means for adjusting the priority of tasks according to the user's emotional state and notifying the user if there is a delay in the schedule.

[0856] (Claim 3)

[0857] The system according to claim 1, which has means for checking the consistency of annotations and proposing corrections if necessary. [Explanation of Symbols]

[0858] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of automatically collecting relevant information from sources, A means of analyzing collected information and integrating related information, A means of evaluating progress and generating a report based on integrated information, A means of detecting and pointing out inconsistencies between multiple pieces of information, A system that includes this.

2. The system according to claim 1, comprising means for managing the progress of a task and notifying the user if there is a delay from the schedule.

3. The system according to claim 1, which has means for checking the consistency of annotations and proposing corrections if necessary.

Images