system

Abstract

This system significantly improves the efficiency of project management and prevents project delays and quality degradation. [Solution] A system comprising: means for recording meeting audio of stakeholders during the progress of a project and converting it into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for sharing the project plan with stakeholders; means for providing an interface for inputting daily progress and issues; means for transmitting the progress and issue data to a server; means for analyzing the progress and issue data, updating the progress, and generating an issue list and proposed solutions; means for notifying stakeholders of the issue list and proposed solutions; and means for automatically generating a project progress report and transmitting it to a higher level.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In conventional project management systems, tasks such as creating project plans, progress reports, and issue management are often performed manually, which is a problem because it requires a lot of labor and time. In particular, in large-scale projects, delays and quality degradation of the project may occur due to insufficient communication and delayed information sharing among stakeholders. Also, if the daily progress status and issue reports are delayed, upper management cannot accurately grasp the current situation of the project, making it difficult to make appropriate decisions. Furthermore, there is a lack of a mechanism to respond quickly and effectively when an urgent issue occurs. To solve these problems, an automated system for creating and managing project plans is required.

Means for Solving the Problems

[0005] This invention provides a means for recording meeting audio among stakeholders during a project, converting it into text data, and further analyzing that text data using natural language processing technology to automatically generate a project plan. It also includes means for distributing this automatically generated project plan to stakeholders in a format easily shareable. Furthermore, it includes an interface for inputting daily progress and issues, and means for transmitting the progress and issue data to a server. The system also includes means for analyzing the data transmitted to the server, automatically updating the project progress, and generating a list of issues and proposed solutions. This ensures that stakeholders are notified of the issue list and proposed solutions, facilitating a rapid response. Additionally, it provides means for automatically generating and transmitting project progress reports to higher levels, enabling an accurate understanding of the project's current status. Moreover, in the event of an urgent issue, it includes means for immediately notifying stakeholders of the issue and presenting solutions, supporting a swift and effective response. These means significantly improve the efficiency of project management and provide a system that prevents project delays and quality degradation.

[0006] A "project plan" is a document that outlines the project's objectives, overview, schedule, key milestones, resource requirements, etc., and is used to manage the progress of the project.

[0007] "Stakeholders" refers to individuals and organizations participating in the project who hold various roles and responsibilities, including project managers, team members, and senior management (project sponsors and executives).

[0008] A "server" is a computer system that stores, processes, and distributes data, and is a central device that provides the core functions of a project management system.

[0009] A "terminal" is a device, such as a computer or mobile device, that a user uses to access a system and transmit input information.

[0010] A "user" refers to a person who uses a project management system to manage project progress and input data.

[0011] "Natural language processing technology" is a technology that uses computers to analyze, understand, and process the language that humans normally use.

[0012] "Progress status" refers to information indicating the extent to which each task in a project has been completed.

[0013] A "challenge" refers to a problem or obstacle that arises during the progress of a project and requires resolution.

[0014] A "proposed solution" is a set of measures that outline how to address a problem that has arisen.

[0015] "Interface" refers to the screen or operating environment that a user uses to access and operate a system.

[0016] "Upper levels" refers to individuals and organizations such as management and project sponsors who make important decisions within a project. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6]It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0019] First, the language used in the following description will be explained.

[0020] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0021] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] Modes for carrying out the invention

[0039] This invention relates to a system for efficiently managing projects to build new systems, improving the efficiency of project progress by automating the content of meetings among stakeholders, daily progress reports, and issue management.

[0040] System Components and Their Functions

[0041] Recording and transcribing meeting audio.

[0042] The user begins project planning meetings and progress meetings.

[0043] The terminals (microphones and recording devices in the conference room) acquire the conference audio and convert it into text data in real time.

[0044] All statements made during the meeting are converted into text data and used for subsequent processing. At this stage, speech recognition technology is applied to perform highly accurate speech-to-text conversion.

[0045] Text data analysis and project plan generation

[0046] The terminal sends the generated text data to the server.

[0047] The server analyzes text data using natural language processing (NLP) techniques. It automatically extracts the project's objectives, overview, schedule, milestones, and required resources, and generates a project plan.

[0048] The generated project plan is automatically shared with stakeholders, allowing each project member to review it.

[0049] Daily progress reporting and task management

[0050] Users input daily work reports, progress updates, and any issues that arise as daily reports.

[0051] The terminal sends this input data to the server.

[0052] When the server receives the daily report data, it immediately begins analysis. The analysis results are saved in the database, and the project progress and issue list are updated accordingly.

[0053] Generation and notification of proposed solutions to the problem.

[0054] The server lists newly arising issues and generates proposed solutions. It automatically suggests the optimal solution based on historical data and a knowledge base.

[0055] The generated response plan is notified to the user via their device, prompting them to take the necessary action.

[0056] Progress reports and sharing with upper management

[0057] The server periodically retrieves the latest progress information from the database and updates the project plan. The updated plan is automatically sent as a report to higher levels.

[0058] The server provides this progress report to higher levels in the form of email or dashboards, allowing them to accurately understand the project's current status.

[0059] Addressing urgent issues

[0060] If the server detects an urgent issue, it will immediately notify the relevant parties. The notification will also include specific solutions.

[0061] This notification will be communicated to relevant parties in real time to facilitate a swift response.

[0062] Specific example

[0063] Examples of meeting recording and transcription

[0064] User: "The goal of this project is to improve operational efficiency by revamping our customer management system."

[0065] Device: Recorded the meeting audio and transcribed it as, "The objective of this project is to revamp the customer management system to improve operational efficiency."

[0066] Server: Receives text data, analyzes it, and adds it to the project plan as "Purpose: Renewal of customer management system" and "Goal: Improve operational efficiency."

[0067] Examples of progress reporting and issue management

[0068] User: "Today's progress: 40% of customer data migration completed. There are issues with data format mismatches."

[0069] Terminal: Submit data via input form.

[0070] Server: Updated progress and added "Issue: Data format mismatch" to the list. Proposed "Create a data conversion script" as a solution and notified the user.

[0071] In this way, a system is provided that significantly improves the efficiency of project management by automatically analyzing and managing the statements and daily reports of stakeholders.

[0072] The following describes the processing flow.

[0073] Processing steps

[0074] 1. Initial setup and project plan generation

[0075] Step 1:

[0076] The user initiates a meeting and shares the objectives, overview, and schedule of the new project with stakeholders.

[0077] Step 2:

[0078] The device records meeting audio in real time and converts that audio data into text data.

[0079] Specific example: A microphone or recording device in a conference room acquires audio data, and speech recognition software converts it into text.

[0080] Step 3:

[0081] The terminal sends the converted text data to the server.

[0082] Specific example: A text-based data file is uploaded to a server via the internet.

[0083] Step 4:

[0084] The server analyzes the received text data using natural language processing (NLP) techniques to extract project objectives, overview, schedule, milestones, resource requirements, and other relevant information from the conversation.

[0085] Specific example: An NLP engine analyzes text data to identify important keywords and phrases.

[0086] Step 5:

[0087] The server generates an initial project plan based on the analysis results and automatically shares that plan with the relevant parties.

[0088] Specific example: A project plan is formatted and sent to stakeholders via an email program.

[0089] 2. Daily progress and task management

[0090] Step 6:

[0091] Users input their daily progress and challenges as daily reports and submit them to the system.

[0092] Specific example: Enter progress information and any problems encountered into a dedicated input form.

[0093] Step 7:

[0094] The terminal sends the daily report data to the server.

[0095] Specific example: When you press the submit button on the form, the entered data is uploaded to the server.

[0096] Step 8:

[0097] The server receives the daily report data and performs analysis. Based on the analysis results, it updates the project progress and lists any newly arising issues.

[0098] Specific example: The analysis engine analyzes daily report data and saves progress and issues to a database.

[0099] Step 9:

[0100] The server generates proposed solutions for the issues it has listed and notifies the user of these solutions.

[0101] Specific example: An AI algorithm generates the optimal solution and proposes it to the user via email notification.

[0102] 3. Progress reports and sharing of issues

[0103] Step 10:

[0104] The server periodically retrieves the latest project progress information from the database and updates the project plan.

[0105] Specific example: The clone job is executed according to schedule, and the latest data is used.

[0106] Step 11:

[0107] The server automatically sends the updated project plan as a report to the higher layer.

[0108] Specific example: A program sends an email containing a report summarizing the latest progress.

[0109] Step 12:

[0110] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes a solution.

[0111] Specific example: When an urgent issue arises, the notification system sends real-time alerts to relevant parties and proposes solutions.

[0112] summary

[0113] Through these steps, the project management system operates efficiently through the coordination of users, terminals, and servers, effectively supporting project planning, progress management, and issue resolution. Including specific program processing flows clarifies how the system will be implemented.

[0114] (Example 1)

[0115] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0116] In project management, it is crucial for stakeholders to communicate efficiently, share progress, and resolve issues quickly. However, manually recording project meeting audio, converting it to text, and incorporating it into project plans is a laborious process. Furthermore, delays in reporting progress and issues can impact the overall project schedule. Moreover, the inability to quickly address urgent issues increases project risk. Thus, streamlining the collection, sharing, and updating of information is a major challenge in project management.

[0117] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0118] This invention includes a server that transcribes meeting audio into text in real time and transmits it to the server immediately, a server that notifies relevant parties in real time when it detects an urgent issue and presents specific solutions, and a server that periodically updates the project plan and automatically generates the latest progress status and issue list. This enables rapid transcription and analysis of meeting content, realizing a project management system that can respond quickly to urgent issues.

[0119] "Meeting audio" refers to the audio generated when stakeholders discuss and exchange opinions about a project during its progress.

[0120] "Text data" refers to data obtained by converting recorded meeting audio into text information using speech recognition technology.

[0121] "Natural language processing technology" is a general term for technologies that enable computers to understand, interpret, and generate human language, and is particularly used for analyzing text data.

[0122] A "project plan" is a document that includes information such as the project's objectives, overview, schedule, milestones, and required resources.

[0123] An "interface" is an input method used by users to enter data such as daily work reports, progress status, and issues that have arisen.

[0124] A "server" is a central computer system that receives text data and daily report data sent from each terminal and performs processing such as analysis, storage, and notification.

[0125] "Progress status" refers to information that indicates the current stage of a project's work and how far along it is.

[0126] "Issue data" refers to information that records problems and obstacles that occur during the progress of a project.

[0127] A "list of issues" refers to a compilation of all issues that arose during the project's progress.

[0128] A "proposed solution" refers to the proposed solutions or action plans for the listed issues.

[0129] "Upper management" refers to senior personnel such as managers and executives who receive project progress reports.

[0130] "Real-time" refers to a timeframe in which processing and notifications are carried out immediately the moment an event occurs.

[0131] This invention relates to a system for efficiently managing projects to build new systems. It improves the efficiency of project progress by automating the content of meetings among stakeholders, daily progress reports, and issue management.

[0132] First, users begin project planning and progress meetings. Terminals (such as microphones or recording devices in the meeting room) capture the meeting audio and convert it into text data in real time. At this stage, speech recognition technology is applied, specifically using speech recognition services such as "Google® Speech-to-Text API" or "IBM Watson® Speech to Text".

[0133] The acquired text data is sent from the terminal to the server. The server analyzes this text data using natural language processing techniques (such as Python's NLTK library or Google Cloud Natural Language API) and automatically extracts the project's objectives, overview, schedule, milestones, and required resources. Based on the extracted information, the server generates a project plan and automatically shares it with stakeholders.

[0134] Users also input daily work reports, progress updates, and issues encountered as daily reports. This daily report data is sent from the terminal to the server. Upon receiving the daily report data, the server immediately begins analysis and saves the analysis results to a database, thereby updating the project progress and issue list.

[0135] The server lists newly arising issues and generates optimal solutions. This process utilizes "ElasticSearch®" and the Python machine learning module (scikit-learn) to generate the best solutions from historical data and a knowledge base. The generated solutions are then communicated to the user via their terminal, and they are required to take action.

[0136] Furthermore, the server periodically retrieves the latest progress information from the database and updates the project plan. This updated plan is automatically sent to higher levels. Reports to higher levels are provided via email or dashboard tools such as Tableau and Power BI.

[0137] If the server detects an urgent issue, it will immediately notify relevant parties in real time and propose specific solutions. This notification will be sent in real time using the Slack API or Twilio API.

[0138] Specific examples include the following:

[0139] User: "The goal of this project is to improve operational efficiency by revamping our customer management system."

[0140] Terminal: Recorded the meeting audio and transcribed it as, "The objective of this project is to revamp the customer management system to improve operational efficiency."

[0141] Server: Receives text data, analyzes it, and adds it to the project plan as "Purpose: Renewal of the customer management system" and "Goal: Improvement of operational efficiency."

[0142] Furthermore, the following specific examples are given for progress reporting and issue management.

[0143] User: "Today's progress: 40% of customer data migration completed. There are issues with data format mismatches."

[0144] Terminal: Submit data from the input form.

[0145] Server: Updated progress and added "Issue: Data format mismatch" to the list. Proposed "Create a data conversion script" as a solution and notified the user.

[0146] Examples of prompt statements to input into the generative AI model are as follows:

[0147] "Automatically generate a project plan that includes the project objectives, overview, schedule, milestones, and required resources."

[0148] "Based on past data, please propose the best solution to the data format mismatch issue."

[0149] This system makes it possible to efficiently analyze and manage conversations and daily reports from stakeholders, significantly improving the efficiency of project management.

[0150] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0151] Step 1:

[0152] The user initiates a project planning or progress meeting. The input is the meeting audio, which serves as the starting point for processing.

[0153] The device activates the microphones and recording devices in the conference room and begins capturing conference audio in real time. Specifically, the device starts recording as soon as it recognizes the voice command "start".

[0154] Step 2:

[0155] The terminal sends the acquired meeting audio data to a speech recognition API (e.g., Google Speech-to-Text API). The input is audio data, and the output returned by the API is text data.

[0156] The terminal receives text data and saves the meeting's statements as text information. Specifically, the audio data is converted to text, such as "The objective of this project is to improve operational efficiency by revamping the customer management system."

[0157] Step 3:

[0158] The terminal sends the generated text data to the server. The input is text data, which is sent to the server for further analysis.

[0159] The server receives and stores the text data. This prepares it for creating the project plan.

[0160] Step 4:

[0161] The server analyzes the received text data using natural language processing techniques (e.g., Python's NLTK library). The input is text data, and the output is extracted information such as the project's objectives, overview, schedule, milestones, and required resources.

[0162] The server automatically generates a project plan based on these analysis results. Specifically, details such as "Objective: Renewal of the customer management system" and "Goal: Improvement of operational efficiency" are added to the plan.

[0163] Step 5:

[0164] The server automatically shares the generated project plan with stakeholders. The input is the automatically generated plan, and the output is the information shared with stakeholders. Sharing is done via email, cloud storage, and project management tools.

[0165] Step 6:

[0166] Users input daily work reports, progress updates, and any issues encountered as daily reports. These inputs consist of user work reports and progress data, which are then recorded on the terminal as daily reports.

[0167] The terminal sends this daily report data to the server. The input here is the daily report data, and the output is the data sent to the server.

[0168] Step 7:

[0169] The server immediately begins analyzing the daily report data upon receiving it and saves the analysis results to the database. The input is the daily report data, and the output is the analysis results and updated database information.

[0170] The server updates the project progress and issue list. Specifically, it updates information such as "Progress: 40% of customer data migration complete" and "Issues: Data format mismatch."

[0171] Step 8:

[0172] The server lists newly arising issues and generates optimal solutions. The input is issue data, and the output is the proposed solutions. Elasticsearch and Python's machine learning module (scikit-learn) are used here.

[0173] The server sends the generated solution to the terminal and notifies the user. Specifically, the user receives a notification stating "Problem: Data format mismatch" and "Suggested solution: Create a data conversion script."

[0174] Step 9:

[0175] The server periodically retrieves the latest progress information from the database and updates the project plan. The input is the latest progress information, and the output is the updated project plan.

[0176] The server automatically sends updated plans to higher levels. Specifically, it provides information to higher levels via email or dashboard tools (Tableau, Power BI).

[0177] Step 10:

[0178] When the server detects an urgent issue, it immediately notifies relevant parties in real time and provides concrete solutions. The input is data on the urgent issue, and the output is the notification and solution.

[0179] Notifications are sent to all relevant parties via the terminal. Specifically, if a "system failure" occurs as an urgent issue, relevant parties will be immediately notified of a solution such as "system restart."

[0180] (Application Example 1)

[0181] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0182] In conventional project management systems, managing meeting content, progress reports, and issues among stakeholders is done manually, requiring a significant amount of time and effort, often resulting in project delays. Furthermore, even in factory settings, manual data entry and management are necessary, hindering efficient project progress. A system is needed to address these challenges and improve the efficiency of project management.

[0183] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0184] In this invention, the server includes means for recording meeting audio conducted by stakeholders during the progress of a project and converting it into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for sharing the project plan with stakeholders; means for providing an interface for inputting daily progress and issues; means for transmitting the progress and issue data to the server; means for analyzing the progress and issue data, updating the progress, and generating an issue list and proposed solutions; means for notifying stakeholders of the issue list and proposed solutions; means for automatically generating a project progress report and transmitting it to a higher level; means for recording meeting audio in real time via a factory robot and converting it into text data; means for transmitting the text data to the server, analyzing it using natural language processing technology, and automatically generating a project plan; and means for automatically inputting the progress and issue data via a factory robot and transmitting it to the server. This makes it possible to automate the work of stakeholders and improve the efficiency of project management.

[0185] "Meeting audio" refers to audio recordings made during meetings held by stakeholders in the course of a project.

[0186] "Text data" refers to data obtained by converting recorded meeting audio into text information.

[0187] "Natural language processing technology" is a technology that interprets and analyzes human language.

[0188] A "project plan" is a document that outlines the project's objectives, overview, schedule, milestones, and required resources.

[0189] "Stakeholders" refers to all members, managers, and other interested parties involved in the project.

[0190] An "interface" refers to an input device or software that users use to record their daily progress and tasks.

[0191] A "server" is a computing system used for analyzing, storing, and transmitting data.

[0192] A "problem list" is a list of issues that arose during the project's progress.

[0193] A "solution plan" is a proposed solution to a problem that has arisen.

[0194] A "progress report" is a report that describes the progress of a project.

[0195] "Upper management" refers to senior managers who monitor project progress and make decisions.

[0196] A "factory robot" is a machine used in a factory to perform automated tasks.

[0197] "Real-time" refers to performing actions or processing in accordance with real-world time.

[0198] "Analysis" is the process of examining data in detail to clarify its structure and meaning.

[0199] "Automatic generation" refers to the process where a system automatically creates documents or data without human intervention.

[0200] This invention relates to a system for streamlining project management in factories, enabling rapid and accurate project progress by automating meeting audio recording, transcription, progress reporting, and issue management.

[0201] System Configuration

[0202] This system consists of the following main elements:

[0203] Meeting audio and text

[0204] The server records meeting audio using microphones and recording devices mounted on factory robots and converts it into text data in real time. It uses speech recognition technology (for example, Google Cloud Speech-to-Text or Amazon Transcribe).

[0205] Text data analysis and project plan generation

[0206] The server analyzes text data using natural language processing (NLP) technology and automatically extracts the project's objectives, overview, schedule, milestones, and required resources to generate a project plan.

[0207] Progress reporting and issue management

[0208] This system provides an interface for inputting daily progress and challenges via factory robots. The data entered by users is sent to a server, where it is automatically analyzed.

[0209] Generating a list of issues and proposed solutions

[0210] Upon receiving a progress report, the server immediately begins analysis, updates the issue list, and generates proposed solutions. These proposed solutions are then notified to the user.

[0211] Project progress report

[0212] The server periodically retrieves the latest progress information from the database, updates the project plan, and reports the latest progress to higher levels.

[0213] Hardware and software

[0214] Hardware:

[0215] Microphones and recording devices mounted on factory robots

[0216] Server (used as a computing resource)

[0217] software:

[0218] Speech recognition technology (Google Cloud Speech-to-Text, Amazon Transcribe, etc.)

[0219] Natural language processing techniques (such as Python's NLTK library)

[0220] Specific example

[0221] For example, suppose a user makes the following statement in a meeting.

[0222] Example of a prompt:

[0223] "The objective of this project is to improve production efficiency by introducing a new manufacturing line."

[0224] The server records the meeting audio and converts it into text data. Then, it analyzes the text data to automatically extract the following information.

[0225] Objective: To introduce a new manufacturing line.

[0226] Objective: Improve production efficiency

[0227] As a concrete example of progress reporting and issue management, the user will input the following information.

[0228] Example of a prompt:

[0229] "Today's progress: 50% of the equipment installation is complete. A new challenge has arisen: a power supply issue."

[0230] The server updates the progress, adds "Issue: Power supply issues" to the list, generates "Inspect and repair the power supply system" as a suggested solution, and notifies the user.

[0231] Thus, the system of the present invention can automate and efficiently and effectively perform project management in a factory.

[0232] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0233] Step 1:

[0234] A microphone and recording device mounted on the factory robot acquire the conference audio. The microphone records all sounds emitted during the conference and converts them to a digital format in real time. This results in the conference audio being acquired as digital audio data.

[0235] Input: Audio from a meeting

[0236] Output: Digital audio data

[0237] Step 2:

[0238] The acquired digital audio data is sent to the server. The server uses speech recognition technology to convert the digital audio data into text data. Google Cloud Speech-to-Text and Amazon Transcribe are used as speech recognition technologies.

[0239] Input: Digital audio data

[0240] Output: Text data

[0241] Step 3:

[0242] Text data is analyzed by the server using natural language processing (NLP) techniques. This analysis automatically extracts information such as the project's objectives, overview, schedule, and required resources. This then generates an initial project plan.

[0243] Input: Text data

[0244] Output: Project plan

[0245] Step 4:

[0246] The generated project plan is shared with stakeholders via the server. Sharing methods include email, display on a dashboard, and other collaboration tools.

[0247] Input: Project plan

[0248] Output: Sharing with stakeholders

[0249] Step 5:

[0250] Users input daily progress and challenges through factory robots. The interface provides progress input forms and challenge reporting forms, and this data is sent to the server in real time.

[0251] Input: Progress status, issue data

[0252] Output: Progress status and issue data sent to the server

[0253] Step 6:

[0254] The server analyzes the received progress and issue data, updates the progress status, and generates an issue list and proposed solutions. Historical data and knowledge bases are used for data analysis to propose the optimal solution.

[0255] Input: Progress status, issue data

[0256] Output: Updated progress, issue list, proposed solutions

[0257] Step 7:

[0258] The server notifies stakeholders of the generated list of issues and proposed solutions. Notification methods include alerts displayed via factory robots, email, and notifications on the dashboard.

[0259] Input: List of issues, proposed solutions

[0260] Output: Notification to relevant parties

[0261] Step 8:

[0262] The server periodically retrieves the latest progress information from the database and updates the project plan. This ensures that stakeholders are always aware of the latest project status.

[0263] Input: Progress information

[0264] Output: Updated project plan

[0265] Step 9:

[0266] The updated project plan is sent to higher levels as an automatically generated progress report by the server. Methods of transmission include periodic emails, dashboard displays, and automated report generation.

[0267] Input: Updated project plan

[0268] Output: Progress report to higher layers

[0269] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0270] Modes for carrying out the invention

[0271] This invention relates to a system that combines an emotion engine with a project management system to more effectively manage project progress and issues. In particular, it aims to increase the success rate of projects by analyzing the emotions of stakeholders and reflecting them in plans and progress reports.

[0272] System Configuration and Functions

[0273] Recording and transcribing meeting audio.

[0274] The user initiates project planning meetings and progress meetings.

[0275] The terminal (the microphone or recording device in the conference room) records the conference audio and converts that audio data into text data.

[0276] All speech during the meeting is converted into text data in real time and used for subsequent processing. Speech recognition technology is used to perform highly accurate speech-to-text conversion.

[0277] Text data analysis and project plan generation

[0278] The terminal sends the generated text data to the server.

[0279] The server analyzes text data using natural language processing (NLP) technology, automatically extracting project objectives, overview, schedule, milestones, resource requirements, and other information from the conversation to generate a project plan.

[0280] The generated project plan is automatically shared with stakeholders and becomes accessible to everyone.

[0281] Daily progress reporting and task management

[0282] The user inputs the daily progress and issues into the system as a daily report.

[0283] The terminal sends the daily report data to the server.

[0284] The server analyzes the received daily report data and automatically updates the progress of the project and the issues that have occurred.

[0285] Sentiment analysis by the sentiment engine

[0286] A sentiment engine is incorporated into the server to analyze the user's sentiment from meeting audio and daily report data.

[0287] The server reflects the analysis results in the project plan document and generates countermeasures for the progress and issues taking into account the sentiment information.

[0288] The sentiment engine analyzes the sentiment trend from the user's speech and input content, and can extract information such as "stress is increasing" and "satisfaction is high". Thereby, the risks of the project can be detected early, and specific countermeasures for problem-solving can be proposed.

[0289] Generation and notification of countermeasures for issues

[0290] The server generates countermeasures for the listed issues, reflects the sentiment information, and notifies the user.

[0291] The countermeasures include optimal solutions based on past data and knowledge bases. Solutions that also take into account sentiment information are more likely to be accepted by the user.

[0292] Progress report and sharing with upper layers

[0293] The server regularly retrieves the latest progress information from the database and updates the project plan document.

[0294] Updated project plans and progress reports are automatically sent to higher levels of the system.

[0295] This allows upper management to always have an accurate grasp of the latest project status.

[0296] Addressing urgent issues

[0297] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[0298] Real-time notifications allow for a quick response to urgent issues.

[0299] Specific example

[0300] Examples of meeting recording and transcription

[0301] User: "The goal of this project is to improve operational efficiency by revamping our customer management system."

[0302] Device: Recorded the meeting audio and transcribed it as, "The objective of this project is to revamp the customer management system to improve operational efficiency."

[0303] Server: Receives text data, analyzes it, and adds it to the project plan as "Purpose: Renewal of customer management system" and "Goal: Improve operational efficiency."

[0304] Examples of progress reporting and issue management

[0305] User: "Today we started migrating customer data, but we encountered problems due to data format mismatches. It's very stressful."

[0306] Terminal: Submit data via input form.

[0307] Server: Update the progress status and add it to the list as "Issue: Data format mismatch". The emotion engine detects that "stress" is increasing, proposes "Create a data conversion script" as a solution, and notifies the user.

[0308] By using this system, project management can be carried out more efficiently and effectively, improving the satisfaction of stakeholders and the success rate of projects.

[0309] The following describes the process flow.

[0310] Processing steps

[0311] 1. Initial setup and generation of project plan

[0312] Step 1:

[0313] The user starts a meeting and shares the purpose, overview, schedule, etc. of the new project with the stakeholders.

[0314] Step 2:

[0315] The terminal records the meeting audio in real-time and converts the audio data into text data.

[0316] Specific example: The microphone or recording device in the meeting room acquires the audio data, and the speech recognition software converts it into text.

[0317] Step 3:

[0318] The terminal sends the converted text data to the server.

[0319] Specific example: The texturized data file is uploaded to the server via the Internet.

[0320] Step 4:

[0321] The server analyzes the received text data using natural language processing (NLP) techniques to extract project objectives, overview, schedule, milestones, resource requirements, and other relevant information from the conversation.

[0322] Specific example: An NLP engine analyzes text data to identify important keywords and phrases.

[0323] Step 5:

[0324] The server generates an initial project plan based on the analysis results and automatically shares that plan with the relevant parties.

[0325] Specific example: A project plan is formatted and sent to stakeholders via an email program.

[0326] 2. Daily progress and task management

[0327] Step 6:

[0328] Users input their daily progress and challenges as daily reports and submit them to the system.

[0329] Specific example: Enter progress information and any problems encountered into a dedicated input form.

[0330] Step 7:

[0331] The terminal sends the daily report data to the server.

[0332] Specific example: When you press the submit button on the form, the entered data is uploaded to the server.

[0333] Step 8:

[0334] The server receives the daily report data and performs analysis. Based on the analysis results, it updates the project progress and lists any newly arising issues.

[0335] Specific example: The analysis engine analyzes daily report data and saves progress and issues to a database.

[0336] Step 9:

[0337] The server generates proposed solutions for the issues it has listed and notifies the user of these solutions.

[0338] Specific example: An AI algorithm generates the optimal solution and proposes it to the user via email notification.

[0339] 3. Emotional analysis through the introduction of an emotion engine

[0340] Step 10:

[0341] The server has an emotion engine built in that analyzes user emotions from meeting audio and daily report data.

[0342] Specific example: Extract emotional indicators such as "stress" and "satisfaction level" from meeting audio and text-based daily reports.

[0343] Step 11:

[0344] The server incorporates the analysis results into the project plan and generates progress updates and proposed solutions to challenges, taking emotional information into account.

[0345] Specific example: For users experiencing high stress levels, we suggest solutions such as "recommending rest" or "enhancing support."

[0346] Step 12:

[0347] The server notifies relevant parties of emotional information, facilitating smooth communication among project members.

[0348] Specific example: Display emotional information on a dashboard so that everyone can share the situation in real time.

[0349] 4. Progress reports and sharing with upper management

[0350] Step 13:

[0351] The server periodically retrieves the latest project progress information from the database and updates the project plan.

[0352] Specific example: Cloning jobs are executed regularly according to schedule, and the latest data is reflected in the project plan.

[0353] Step 14:

[0354] The server automatically sends updated project plans and progress reports to higher levels.

[0355] Specific example: A program sends an email containing a report summarizing the latest progress.

[0356] 5. Addressing urgent issues

[0357] Step 15:

[0358] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[0359] Specific example: When an urgent issue arises, the notification system sends real-time alerts to relevant parties and proposes solutions that take emotional information into account.

[0360] summary

[0361] Through these steps, the project management system operates efficiently through the coordination of users, terminals, and servers, effectively supporting project planning, progress management, and issue resolution. By introducing an emotion engine, emotional information is reflected in project plans and solutions, further improving the project's success rate.

[0362] (Example 2)

[0363] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0364] Traditional project management systems lacked sufficient automation in areas such as recording and transcribing meeting audio, automatically generating project plans, and managing progress and issues. Furthermore, they struggled to provide solutions that considered the feelings of stakeholders. As a result, accurately understanding project progress and efficiently resolving issues was difficult. Moreover, the inability to respond quickly to urgent issues contributed to a lower project success rate.

[0365] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0366] In this invention, the server includes means for recording the voices of stakeholders during a meeting and converting them into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for sharing the project plan with stakeholders; means for providing an interface for inputting daily progress and issues; means for transmitting the progress and issue data to a central processing unit; means for analyzing the progress and issue data, updating the progress, and generating an issue list and proposed solutions; means for notifying stakeholders of the issue list and proposed solutions; means for automatically generating a project progress report and transmitting it to a higher level; means for analyzing emotions from the voice and text data; and means for reflecting the emotion analysis results in the project plan and generating countermeasures that take emotional information into account. This makes project management more efficient and effective, and improves stakeholder satisfaction and the success rate of the project.

[0367] A "meeting" is a place where stakeholders gather to discuss the project's plan, progress, and challenges.

[0368] "Audio" refers to data that includes the words and statements made by participants during a meeting.

[0369] "Text data" refers to information obtained by converting speech into written text.

[0370] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate human language.

[0371] A "project plan" is a document that outlines the project's objectives, overview, schedule, milestones, resource requirements, and other relevant information.

[0372] "Stakeholders" refers to all people involved in the project.

[0373] An "interface" refers to the means by which users input their daily progress and tasks into a system.

[0374] A "central processing unit" refers to a device that processes data for the entire system.

[0375] A "task list" is a document that lists all unresolved issues and tasks within a project.

[0376] A "proposed solution" is a proposal that outlines solutions to a problem.

[0377] A "progress report" is a document that reports on the current progress of a project.

[0378] "Upper management" refers to the people or organizations in a position to oversee and manage the entire project.

[0379] "Emotions" refer to information about the psychological state and feelings of those involved.

[0380] "Emotional analysis" refers to the process of identifying a user's emotions by analyzing text data and audio data.

[0381] "System" refers to a set of computing resources and software that possess the aforementioned functions.

[0382] "Updating the progress status" refers to the act of keeping the project's progress information up to date.

[0383] "To notify" means to convey information to the relevant parties.

[0384] This invention is a system that combines an emotion engine with a project management system. The system includes recording and transcribing meeting audio, analyzing text data, automatically generating and sharing project plans, managing daily progress and issues, sentiment analysis, and generating and notifying solutions to issues.

[0385] Hardware and software usage

[0386] The user initiates project planning meetings and progress meetings.

[0387] The terminal records meeting audio using the conference room's microphone or recording device, and converts the audio data into text using the Google Cloud Speech-to-Text API. It also utilizes an internet connection for data transmission. The recorded audio is transcribed in real time. All statements made during the meeting are recorded as text and used for subsequent processing.

[0388] Processing flow

[0389] The terminal sends the generated text data to the server.

[0390] The server receives text data and performs analysis using natural language processing (NLP) techniques. Here, the OpenAI® GPT-4® model is used for text analysis to automatically extract project objectives, overview, schedule, milestones, resource requirements, etc., and generate a project plan. The generated project plan is automatically shared with stakeholders and made accessible to everyone.

[0391] Progress management and issue management

[0392] Users use an interface to input daily progress and issues into the system as daily reports. The terminal sends the entered daily report data to the server. The server receives the daily report data, analyzes it, and automatically updates the project progress and any issues that have arisen. Programming languages ​​such as Python are used for this analysis.

[0393] Emotion analysis and response plan generation

[0394] The emotion engine built into the server analyzes user emotions from meeting audio and daily report data. For example, it uses the Sentiment Analysis API to detect emotions within text. The server then incorporates these emotion analysis results into project plans, generating progress reports and countermeasures for issues that take emotional information into account.

[0395] The emotion engine can extract information such as "stress is increasing" or "satisfaction is high" from user statements and input. This allows for early detection of project risks and the concrete proposal of appropriate countermeasures for problem solving.

[0396] Notification of proposed response and emergency response

[0397] The server notifies stakeholders of the generated list of issues and proposed solutions. These solutions include optimal solutions based on historical data and a knowledge base. Sentimental information is also taken into account, resulting in solutions that are more likely to be accepted by users.

[0398] The server periodically retrieves project progress reports from the database and updates the project plan. The updated plan and progress reports are automatically sent to higher levels, allowing them to always stay informed about the latest project status.

[0399] In the event of an urgent issue, the server immediately notifies relevant parties and proposes solutions that take emotional information into consideration. This real-time notification enables a rapid response.

[0400] Examples of specific actions

[0401] User: "The goal of this project is to revamp our customer management system to improve operational efficiency."

[0402] Terminal: Records meeting audio, transcribes it into text, and translates it as, "The objective of this project is to improve operational efficiency by revamping the customer management system."

[0403] Server: Receives and analyzes text data, and adds it to the project plan with the objectives "Renewal of customer management system" and the goal "Improvement of operational efficiency."

[0404] Example of a prompt

[0405] "Automatically generate a project plan from the meeting audio. The following is a transcript of the meeting: 'The objective of this project is to improve operational efficiency by revamping the customer management system.'"

[0406] By using this system, project management is expected to be more efficient and effective, leading to increased stakeholder satisfaction and a higher project success rate.

[0407] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0408] Step 1:

[0409] The user initiates project planning meetings and progress meetings.

[0410] Input: Conference audio

[0411] Output: Recorded audio data

[0412] Specific action: The user starts a meeting and makes a statement. That statement is recorded as audio data by the recording device.

[0413] Step 2:

[0414] The device converts recorded meeting audio into text data in real time using the Google Cloud Speech-to-Text API.

[0415] Input: Recorded audio data

[0416] Output: Text converted to character data

[0417] Specific operation: The recording device sends the recorded audio data to the Google Cloud Speech-to-Text API, and the data, converted to text with high accuracy, is returned to the device.

[0418] Step 3:

[0419] The terminal sends text data to the server.

[0420] Input: Text data

[0421] Output: Text data sent to the server

[0422] Specific operation: Text data is sent from the terminal to the server. The data is transmitted in real time using a network connection and received by the server.

[0423] Step 4:

[0424] The server analyzes the received text data using Natural Language Processing (NLP) technology and automatically generates a project plan.

[0425] Input: Text data sent to the server

[0426] Output: Project plan generated based on the analyzed information

[0427] Specific operation: The server analyzes text data, uses the OpenAI GPT-4 model to extract project objectives, overview, schedule, resource requirements, etc., and automatically generates a project plan.

[0428] Step 5:

[0429] The server automatically shares the generated project plan with relevant parties.

[0430] Input: Generated project plan

[0431] Output: Project plan shared with stakeholders

[0432] Specific operation: The server sends the generated project plan to relevant parties' email addresses or cloud systems, making it accessible to everyone.

[0433] Step 6:

[0434] Users input their daily progress and challenges into the system as daily reports.

[0435] Input: Progress status and details of the issues

[0436] Output: Daily report data

[0437] Specific operation: Users input their daily progress and challenges using input forms provided in the system, and this data is saved in the system as daily report data.

[0438] Step 7:

[0439] The terminal sends the daily report data to the server.

[0440] Input: Daily report data

[0441] Output: Daily report data sent to the server

[0442] Specific operation: Daily report data is sent from the terminal to the server. Daily report data is sent over the network and received by the server.

[0443] Step 8:

[0444] The server analyzes the daily report data and automatically updates the project progress and issue list.

[0445] Input: Daily report data sent to the server

[0446] Output: Updated progress and issue list

[0447] Specific operation: The server analyzes the daily report data and updates the progress status and task list using Python or similar tools.

[0448] Step 9:

[0449] The emotion engine built into the server analyzes the user's emotions from daily report data and meeting audio.

[0450] Input: Daily report data and meeting audio data

[0451] Output: User sentiment analysis results

[0452] Specific operation: The emotion engine on the server analyzes the data using the Sentiment Analysis API and extracts emotional information (e.g., "stress is increasing," "satisfaction is high").

[0453] Step 10:

[0454] The server incorporates the sentiment analysis results into the project plan and generates response proposals that take sentiment information into account.

[0455] Input: Sentiment analysis results

[0456] Output: Project plan and response plan reflecting emotional information

[0457] Specific operation: The server updates the project plan based on the sentiment analysis results and generates countermeasures that take sentiment information into account.

[0458] Step 11:

[0459] The server will notify relevant parties of the listed issues and proposed solutions.

[0460] Input: List of issues and proposed solutions

[0461] Output: List of issues and proposed solutions notified to stakeholders.

[0462] Specific operation: The server notifies relevant parties of the issue list and proposed solutions, and sends them via email or messaging system.

[0463] Step 12:

[0464] The server periodically retrieves the latest progress information from the database and updates the project plan.

[0465] Input: Progress information in the database

[0466] Output: Updated project plan

[0467] Specific operation: The server queries the database to retrieve the latest progress information and update the project plan.

[0468] Step 13:

[0469] The server automatically sends updated project plans and progress reports to higher levels.

[0470] Input: Updated project plan and progress report

[0471] Output: Progress report sent to the upper layer

[0472] Specific operation: The server sends updated project plans and progress reports to higher levels via email or cloud systems.

[0473] Step 14:

[0474] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[0475] Input: Urgent issue information and sentiment analysis results

[0476] Output: Urgent issues and solutions notified to stakeholders

[0477] Specific operation: The server detects urgent issues, considers solutions that take emotional information into account, and notifies relevant parties in real time.

[0478] (Application Example 2)

[0479] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0480] Traditional project management systems often manage progress and issues without considering the feelings of stakeholders, leading to decreased stakeholder satisfaction and a lower project success rate. Furthermore, the lack of real-time notifications for urgent issues and analysis results made rapid response difficult.

[0481] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0482] In this invention, the server includes means for recording meeting audio of stakeholders during the project and converting it into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for analyzing the user's emotions using an emotion engine and reflecting the analysis results in the project plan and progress reports; and means for notifying the user of the progress status, issues, and emotion analysis results in real time using a smart device. This enables effective project management that takes into account the emotions of stakeholders, thereby improving the project success rate and stakeholder satisfaction.

[0483] A "project plan" is a document that outlines the project's objectives, overview, schedule, milestones, resource requirements, and other relevant information.

[0484] "Text data" refers to a data format in which audio data is converted into written text.

[0485] "Natural language processing technology" refers to technologies that enable computers to understand, analyze, and generate human language.

[0486] An "emotion engine" is a system that analyzes a user's emotions from text and audio data.

[0487] "Progress status" refers to information indicating the degree of progress of a project.

[0488] "Issue data" refers to information about problems that arise during the progress of a project and matters that require attention.

[0489] A "server" is a computer system that performs specific services or data processing over a network.

[0490] A "smart device" is a portable electronic device with internet connectivity and typically possesses advanced computing capabilities.

[0491] An "interface" is the window or means through which a user interacts with a system.

[0492] "Notification" refers to a means or action used to inform a user of specific information.

[0493] "Real-time" means that information is generated and processed almost instantly.

[0494] "Upper management" refers to managers and leaders who are responsible for overseeing the progress and results of a project.

[0495] An "urgent issue" is a serious problem or obstacle that requires immediate attention.

[0496] A "solution plan" refers to a proposed solution or proposal for a problem that has arisen.

[0497] This invention relates to a system that combines an emotion engine with a project management system to more effectively manage project progress and issues. Specifically, it aims to increase the success rate of projects by analyzing the emotions of stakeholders and reflecting them in project plans and progress reports.

[0498] System Configuration

[0499] The system consists of the following main components. These components are implemented through a combination of hardware and software.

[0500] Recording and transcribing meeting audio.

[0501] A device (such as a smart device or a dedicated recording device) records project planning meetings and progress meetings. The recorded audio data is converted into text data using speech recognition technology. This text data is then used for subsequent processing.

[0502] Text data analysis and project plan generation

[0503] The server receives text data sent from the terminal. This text data is analyzed using natural language processing (NLP) technology to automatically extract the project's objectives, overview, schedule, milestones, resource requirements, etc., from the conversation content, and generates a project plan. The generated project plan is then shared with stakeholders.

[0504] Daily progress reporting and task management

[0505] Users input daily progress and issues into the system through an interface. This data is sent from the terminal to the server. The server analyzes the received data and automatically updates the project progress and any issues that have arisen.

[0506] Emotional analysis using an emotion engine

[0507] The server has an emotion engine built in that analyzes user emotions from meeting audio and daily report data. The server incorporates the analysis results into the project plan and generates progress reports and proposed solutions to issues, taking emotional information into account.

[0508] Generation and notification of proposed solutions to the problem.

[0509] The server generates proposed solutions for the listed issues, incorporating sentiment information before notifying the user. These solutions include optimal solutions based on historical data and a knowledge base.

[0510] Addressing urgent issues

[0511] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[0512] Hardware and software to use

[0513] Hardware: Smart devices (smart glasses, smartphones), conference room recording equipment, servers

[0514] Software: Python, speech_recognition library, transformers library, TextBlob library

[0515] Processing flow

[0516] The system begins by recording meeting audio and transcribing it into text. The text data is then analyzed using natural language processing techniques to generate project plans and progress reports. An emotion engine analyzes user emotions and incorporates the results into the project plans and progress reports. It also provides solutions to issues and notifications for emergencies.

[0517] Examples of specific cases and prompt statements

[0518] Specific example:

[0519] Voice input: "Today's progress was delayed due to a machine malfunction. I'm feeling very stressed."

[0520] Text: "Today's progress was delayed due to a machine malfunction. I'm feeling very stressed."

[0521] Sentiment analysis result: "Negative, 0.85"

[0522] Notification: "Issue: Machine malfunction\nSolution: Call a repair technician and have the malfunctioning machine repaired. Taking a break is recommended."

[0523] Example of a prompt:

[0524] "Analyze the emotion in this text and respond with either positive or negative."

[0525] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0526] Step 1:

[0527] The device records the meeting audio.

[0528] Input: Audio from the project meeting

[0529] Operation: The device (smart device or recording device) records the meeting audio.

[0530] Output: Recorded audio data

[0531] Step 2:

[0532] The device converts the recorded audio data into text data.

[0533] Input: Recorded audio data

[0534] Operation: Converts audio data into text data using speech recognition technology (Google Speech Recognition API).

[0535] Output: Text data

[0536] Step 3:

[0537] The terminal sends text data to the server.

[0538] Input: Text data

[0539] Operation: Sends text data generated by the terminal to the server.

[0540] Output: Text data is sent to the server.

[0541] Step 4:

[0542] The server analyzes the received text data using natural language processing technology and automatically generates a project plan.

[0543] Input: Text data

[0544] Operation: The server uses natural language processing (NLP) techniques to analyze text data and extract project objectives, overview, schedule, milestones, and resource requirements.

[0545] Output: Automated project plan

[0546] Step 5:

[0547] The server shares the project plan with the relevant parties.

[0548] Input: Automated project plan

[0549] Operation: The server notifies and shares the project plan with relevant parties.

[0550] Output: Project plan shared with stakeholders

[0551] Step 6:

[0552] Users input their daily progress and challenges through the interface.

[0553] Input: Information on progress and issues

[0554] Operation: The user inputs progress and issue information into the system interface.

[0555] Output: Progress and issue data entered by the user.

[0556] Step 7:

[0557] The terminal sends daily report data to the server.

[0558] Input: Progress and issue data entered by the user.

[0559] Operation: The device sends progress and task data to the server.

[0560] Output: Daily report data sent to the server

[0561] Step 8:

[0562] The server analyzes the daily report data it receives and automatically updates the project's progress and issues.

[0563] Input: Daily report data

[0564] Operation: The server analyzes progress and issues, updates progress, and generates a list of issues.

[0565] Output: Updated progress and issue list

[0566] Step 9:

[0567] The server uses an emotion engine to analyze the user's emotions and reflects the analysis results in project plans and progress reports.

[0568] Input: Text data and daily report data

[0569] Operation: The server uses an emotion engine to analyze the emotions in the data and reflects the results in the project plan and progress report.

[0570] Output: Project plan and progress report reflecting emotional information

[0571] Step 10:

[0572] The server generates proposed solutions to the problem, incorporates emotional information, and then notifies the user.

[0573] Input: Task list and sentiment analysis results

[0574] Operation: The server generates proposed solutions to the problem based on historical data and a knowledge base, and notifies the user, taking sentiment information into account.

[0575] Output: Proposed course of action notified to the user

[0576] Step 11:

[0577] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[0578] Input: Information on the occurrence of urgent issues and sentiment analysis results

[0579] Operation: The server detects urgent issues, notifies relevant parties, and proposes solutions that take emotional information into account.

[0580] Output: Urgent issues and solutions notified to stakeholders

[0581] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0582] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0583] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0584] [Second Embodiment]

[0585] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0586] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0587] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0588] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0589] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0590] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0591] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0592] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0593] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0594] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0595] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0596] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0597] Modes for carrying out the invention

[0598] This invention relates to a system for efficiently managing projects to build new systems, improving the efficiency of project progress by automating the content of meetings among stakeholders, daily progress reports, and issue management.

[0599] System Components and Their Functions

[0600] Recording and transcribing meeting audio.

[0601] The user begins project planning meetings and progress meetings.

[0602] The terminals (microphones and recording devices in the conference room) acquire the conference audio and convert it into text data in real time.

[0603] All statements made during the meeting are converted into text data and used for subsequent processing. At this stage, speech recognition technology is applied to perform highly accurate speech-to-text conversion.

[0604] Text data analysis and project plan generation

[0605] The terminal sends the generated text data to the server.

[0606] The server analyzes text data using natural language processing (NLP) techniques. It automatically extracts the project's objectives, overview, schedule, milestones, and required resources, and generates a project plan.

[0607] The generated project plan is automatically shared with stakeholders, allowing each project member to review it.

[0608] Daily progress reporting and task management

[0609] Users input daily work reports, progress updates, and any issues that arise as daily reports.

[0610] The terminal sends this input data to the server.

[0611] When the server receives the daily report data, it immediately begins analysis. The analysis results are saved in the database, and the project progress and issue list are updated accordingly.

[0612] Generation and notification of proposed solutions to the problem.

[0613] The server lists newly arising issues and generates proposed solutions. It automatically suggests the optimal solution based on historical data and a knowledge base.

[0614] The generated response plan is notified to the user via their device, prompting them to take the necessary action.

[0615] Progress reports and sharing with upper management

[0616] The server periodically retrieves the latest progress information from the database and updates the project plan. The updated plan is automatically sent as a report to higher levels.

[0617] The server provides this progress report to higher levels in the form of email or dashboards, allowing them to accurately understand the project's current status.

[0618] Addressing urgent issues

[0619] If the server detects an urgent issue, it will immediately notify the relevant parties. The notification will also include specific solutions.

[0620] This notification will be communicated to relevant parties in real time to facilitate a swift response.

[0621] Specific example

[0622] Examples of meeting recording and transcription

[0623] User: "The goal of this project is to improve operational efficiency by revamping our customer management system."

[0624] Device: Recorded the meeting audio and transcribed it as, "The objective of this project is to revamp the customer management system to improve operational efficiency."

[0625] Server: Receives text data, analyzes it, and adds it to the project plan as "Purpose: Renewal of customer management system" and "Goal: Improve operational efficiency."

[0626] Examples of progress reporting and issue management

[0627] User: "Today's progress: 40% of customer data migration completed. There are issues with data format mismatches."

[0628] Terminal: Submit data via input form.

[0629] Server: Updated progress and added "Issue: Data format mismatch" to the list. Proposed "Create a data conversion script" as a solution and notified the user.

[0630] In this way, a system is provided that significantly improves the efficiency of project management by automatically analyzing and managing the statements and daily reports of stakeholders.

[0631] The following describes the processing flow.

[0632] Processing steps

[0633] 1. Initial setup and project plan generation

[0634] Step 1:

[0635] The user initiates a meeting and shares the objectives, overview, and schedule of the new project with stakeholders.

[0636] Step 2:

[0637] The device records meeting audio in real time and converts that audio data into text data.

[0638] Specific example: A microphone or recording device in a conference room acquires audio data, and speech recognition software converts it into text.

[0639] Step 3:

[0640] The terminal sends the converted text data to the server.

[0641] Specific example: A text-based data file is uploaded to a server via the internet.

[0642] Step 4:

[0643] The server analyzes the received text data using natural language processing (NLP) techniques to extract project objectives, overview, schedule, milestones, resource requirements, and other relevant information from the conversation.

[0644] Specific example: An NLP engine analyzes text data to identify important keywords and phrases.

[0645] Step 5:

[0646] The server generates an initial project plan based on the analysis results and automatically shares that plan with the relevant parties.

[0647] Specific example: A project plan is formatted and sent to stakeholders via an email program.

[0648] 2. Daily progress and task management

[0649] Step 6:

[0650] Users input their daily progress and challenges as daily reports and submit them to the system.

[0651] Specific example: Enter progress information and any problems encountered into a dedicated input form.

[0652] Step 7:

[0653] The terminal sends the daily report data to the server.

[0654] Specific example: When you press the submit button on the form, the entered data is uploaded to the server.

[0655] Step 8:

[0656] The server receives the daily report data and performs analysis. Based on the analysis results, it updates the project progress and lists any newly arising issues.

[0657] Specific example: The analysis engine analyzes daily report data and saves progress and issues to a database.

[0658] Step 9:

[0659] The server generates proposed solutions for the issues it has listed and notifies the user of these solutions.

[0660] Specific example: An AI algorithm generates the optimal solution and proposes it to the user via email notification.

[0661] 3. Progress reports and sharing of issues

[0662] Step 10:

[0663] The server periodically retrieves the latest project progress information from the database and updates the project plan.

[0664] Specific example: The clone job is executed according to schedule, and the latest data is used.

[0665] Step 11:

[0666] The server automatically sends the updated project plan as a report to the higher layer.

[0667] Specific example: A program sends an email containing a report summarizing the latest progress.

[0668] Step 12:

[0669] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes a solution.

[0670] Specific example: When an urgent issue arises, the notification system sends real-time alerts to relevant parties and proposes solutions.

[0671] summary

[0672] Through these steps, the project management system operates efficiently through the coordination of users, terminals, and servers, effectively supporting project planning, progress management, and issue resolution. Including specific program processing flows clarifies how the system will be implemented.

[0673] (Example 1)

[0674] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0675] In project management, it is crucial for stakeholders to communicate efficiently, share progress, and resolve issues quickly. However, manually recording project meeting audio, converting it to text, and incorporating it into project plans is a laborious process. Furthermore, delays in reporting progress and issues can impact the overall project schedule. Moreover, the inability to quickly address urgent issues increases project risk. Thus, streamlining the collection, sharing, and updating of information is a major challenge in project management.

[0676] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0677] This invention includes a server that transcribes meeting audio into text in real time and transmits it to the server immediately, a server that notifies relevant parties in real time when it detects an urgent issue and presents specific solutions, and a server that periodically updates the project plan and automatically generates the latest progress status and issue list. This enables rapid transcription and analysis of meeting content, realizing a project management system that can respond quickly to urgent issues.

[0678] "Meeting audio" refers to the audio generated when stakeholders discuss and exchange opinions about a project during its progress.

[0679] "Text data" refers to data obtained by converting recorded meeting audio into text information using speech recognition technology.

[0680] "Natural language processing technology" is a general term for technologies that enable computers to understand, interpret, and generate human language, and is particularly used for analyzing text data.

[0681] A "project plan" is a document that includes information such as the project's objectives, overview, schedule, milestones, and required resources.

[0682] An "interface" is an input method used by users to enter data such as daily work reports, progress status, and issues that have arisen.

[0683] A "server" is a central computer system that receives text data and daily report data sent from each terminal and performs processing such as analysis, storage, and notification.

[0684] "Progress status" refers to information that indicates the current stage of a project's work and how far along it is.

[0685] "Issue data" refers to information that records problems and obstacles that occur during the progress of a project.

[0686] A "list of issues" refers to a compilation of all issues that arose during the project's progress.

[0687] A "proposed solution" refers to the proposed solutions or action plans for the listed issues.

[0688] "Upper management" refers to senior personnel such as managers and executives who receive project progress reports.

[0689] "Real-time" refers to a timeframe in which processing and notifications are carried out immediately the moment an event occurs.

[0690] This invention relates to a system for efficiently managing projects to build new systems. It improves the efficiency of project progress by automating the content of meetings among stakeholders, daily progress reports, and issue management.

[0691] First, users begin project planning and progress meetings. Terminals (such as microphones or recording devices in the meeting room) capture the meeting audio and convert it into text data in real time. At this stage, speech recognition technology is applied, specifically using speech recognition services such as "Google Speech-to-Text API" or "IBM Watson Speech to Text".

[0692] The acquired text data is sent from the terminal to the server. The server analyzes this text data using natural language processing techniques (such as Python's NLTK library or Google Cloud Natural Language API) and automatically extracts the project's objectives, overview, schedule, milestones, and required resources. Based on the extracted information, the server generates a project plan and automatically shares it with stakeholders.

[0693] Users also input daily work reports, progress updates, and issues encountered as daily reports. This daily report data is sent from the terminal to the server. Upon receiving the daily report data, the server immediately begins analysis and saves the analysis results to a database, thereby updating the project progress and issue list.

[0694] The server lists newly arising issues and generates optimal solutions. This process utilizes Elasticsearch and Python's machine learning module (scikit-learn) to generate the best solutions from historical data and a knowledge base. The generated solutions are then communicated to the user via their terminal, and they are required to take action.

[0695] Furthermore, the server periodically retrieves the latest progress information from the database and updates the project plan. This updated plan is automatically sent to higher levels. Reports to higher levels are provided via email or dashboard tools such as Tableau and Power BI.

[0696] If the server detects an urgent issue, it will immediately notify relevant parties in real time and propose specific solutions. This notification will be sent in real time using the Slack API or Twilio API.

[0697] Specific examples include the following:

[0698] User: "The goal of this project is to improve operational efficiency by revamping our customer management system."

[0699] Terminal: Recorded the meeting audio and transcribed it as, "The objective of this project is to revamp the customer management system to improve operational efficiency."

[0700] Server: Receives text data, analyzes it, and adds it to the project plan as "Purpose: Renewal of the customer management system" and "Goal: Improvement of operational efficiency."

[0701] Furthermore, the following specific examples are given for progress reporting and issue management.

[0702] User: "Today's progress: 40% of customer data migration completed. There are issues with data format mismatches."

[0703] Terminal: Submit data from the input form.

[0704] Server: Updated progress and added "Issue: Data format mismatch" to the list. Proposed "Create a data conversion script" as a solution and notified the user.

[0705] Examples of prompt statements to input into the generative AI model are as follows:

[0706] "Automatically generate a project plan that includes the project objectives, overview, schedule, milestones, and required resources."

[0707] "Based on past data, please propose the best solution to the data format mismatch issue."

[0708] This system makes it possible to efficiently analyze and manage conversations and daily reports from stakeholders, significantly improving the efficiency of project management.

[0709] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0710] Step 1:

[0711] The user initiates a project planning or progress meeting. The input is the meeting audio, which serves as the starting point for processing.

[0712] The device activates the microphones and recording devices in the conference room and begins capturing conference audio in real time. Specifically, the device starts recording as soon as it recognizes the voice command "start".

[0713] Step 2:

[0714] The terminal sends the acquired meeting audio data to a speech recognition API (e.g., Google Speech-to-Text API). The input is audio data, and the output returned by the API is text data.

[0715] The terminal receives text data and saves the meeting's statements as text information. Specifically, the audio data is converted to text, such as "The objective of this project is to improve operational efficiency by revamping the customer management system."

[0716] Step 3:

[0717] The terminal sends the generated text data to the server. The input is text data, which is sent to the server for further analysis.

[0718] The server receives and stores the text data. This prepares it for creating the project plan.

[0719] Step 4:

[0720] The server analyzes the received text data using natural language processing techniques (e.g., Python's NLTK library). The input is text data, and the output is extracted information such as the project's objectives, overview, schedule, milestones, and required resources.

[0721] The server automatically generates a project plan based on these analysis results. Specifically, details such as "Objective: Renewal of the customer management system" and "Goal: Improvement of operational efficiency" are added to the plan.

[0722] Step 5:

[0723] The server automatically shares the generated project plan with stakeholders. The input is the automatically generated plan, and the output is the information shared with stakeholders. Sharing is done via email, cloud storage, and project management tools.

[0724] Step 6:

[0725] Users input daily work reports, progress updates, and any issues encountered as daily reports. These inputs consist of user work reports and progress data, which are then recorded on the terminal as daily reports.

[0726] The terminal sends this daily report data to the server. The input here is the daily report data, and the output is the data sent to the server.

[0727] Step 7:

[0728] The server immediately begins analyzing the daily report data upon receiving it and saves the analysis results to the database. The input is the daily report data, and the output is the analysis results and updated database information.

[0729] The server updates the project progress and issue list. Specifically, it updates information such as "Progress: 40% of customer data migration complete" and "Issues: Data format mismatch."

[0730] Step 8:

[0731] The server lists newly arising issues and generates optimal solutions. The input is issue data, and the output is the proposed solutions. Elasticsearch and Python's machine learning module (scikit-learn) are used here.

[0732] The server sends the generated solution to the terminal and notifies the user. Specifically, the user receives a notification stating "Problem: Data format mismatch" and "Suggested solution: Create a data conversion script."

[0733] Step 9:

[0734] The server periodically retrieves the latest progress information from the database and updates the project plan. The input is the latest progress information, and the output is the updated project plan.

[0735] The server automatically sends updated plans to higher levels. Specifically, it provides information to higher levels via email or dashboard tools (Tableau, Power BI).

[0736] Step 10:

[0737] When the server detects an urgent issue, it immediately notifies relevant parties in real time and provides concrete solutions. The input is data on the urgent issue, and the output is the notification and solution.

[0738] Notifications are sent to all relevant parties via the terminal. Specifically, if a "system failure" occurs as an urgent issue, relevant parties will be immediately notified of a solution such as "system restart."

[0739] (Application Example 1)

[0740] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0741] In conventional project management systems, managing meeting content, progress reports, and issues among stakeholders is done manually, requiring a significant amount of time and effort, often resulting in project delays. Furthermore, even in factory settings, manual data entry and management are necessary, hindering efficient project progress. A system is needed to address these challenges and improve the efficiency of project management.

[0742] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0743] In this invention, the server includes means for recording meeting audio conducted by stakeholders during the progress of a project and converting it into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for sharing the project plan with stakeholders; means for providing an interface for inputting daily progress and issues; means for transmitting the progress and issue data to the server; means for analyzing the progress and issue data, updating the progress, and generating an issue list and proposed solutions; means for notifying stakeholders of the issue list and proposed solutions; means for automatically generating a project progress report and transmitting it to a higher level; means for recording meeting audio in real time via a factory robot and converting it into text data; means for transmitting the text data to the server, analyzing it using natural language processing technology, and automatically generating a project plan; and means for automatically inputting the progress and issue data via a factory robot and transmitting it to the server. This makes it possible to automate the work of stakeholders and improve the efficiency of project management.

[0744] "Meeting audio" refers to audio recordings made during meetings held by stakeholders in the course of a project.

[0745] "Text data" refers to data obtained by converting recorded meeting audio into text information.

[0746] "Natural language processing technology" is a technology that interprets and analyzes human language.

[0747] A "project plan" is a document that outlines the project's objectives, overview, schedule, milestones, and required resources.

[0748] "Stakeholders" refers to all members, managers, and other interested parties involved in the project.

[0749] An "interface" refers to an input device or software that users use to record their daily progress and tasks.

[0750] A "server" is a computing system used for analyzing, storing, and transmitting data.

[0751] A "problem list" is a list of issues that arose during the project's progress.

[0752] A "solution plan" is a proposed solution to a problem that has arisen.

[0753] A "progress report" is a report that describes the progress of a project.

[0754] "Upper management" refers to senior managers who monitor project progress and make decisions.

[0755] A "factory robot" is a machine used in a factory to perform automated tasks.

[0756] "Real-time" refers to performing actions or processing in accordance with real-world time.

[0757] "Analysis" is the process of examining data in detail to clarify its structure and meaning.

[0758] "Automatic generation" refers to the process where a system automatically creates documents or data without human intervention.

[0759] This invention relates to a system for streamlining project management in factories, enabling rapid and accurate project progress by automating meeting audio recording, transcription, progress reporting, and issue management.

[0760] System Configuration

[0761] This system consists of the following main elements:

[0762] Meeting audio and text

[0763] The server records meeting audio using microphones and recording devices mounted on factory robots and converts it into text data in real time. It uses speech recognition technology (for example, Google Cloud Speech-to-Text or Amazon Transcribe).

[0764] Text data analysis and project plan generation

[0765] The server analyzes text data using natural language processing (NLP) technology and automatically extracts the project's objectives, overview, schedule, milestones, and required resources to generate a project plan.

[0766] Progress reporting and issue management

[0767] This system provides an interface for inputting daily progress and challenges via factory robots. The data entered by users is sent to a server, where it is automatically analyzed.

[0768] Generating a list of issues and proposed solutions

[0769] Upon receiving a progress report, the server immediately begins analysis, updates the issue list, and generates proposed solutions. These proposed solutions are then notified to the user.

[0770] Project progress report

[0771] The server periodically retrieves the latest progress information from the database, updates the project plan, and reports the latest progress to higher levels.

[0772] Hardware and software

[0773] Hardware:

[0774] Microphones and recording devices mounted on factory robots

[0775] Server (used as a computing resource)

[0776] software:

[0777] Speech recognition technology (Google Cloud Speech-to-Text, Amazon Transcribe, etc.)

[0778] Natural language processing techniques (such as Python's NLTK library)

[0779] Specific example

[0780] For example, suppose a user makes the following statement in a meeting.

[0781] Example of a prompt:

[0782] "The objective of this project is to improve production efficiency by introducing a new manufacturing line."

[0783] The server records the meeting audio and converts it into text data. Then, it analyzes the text data to automatically extract the following information.

[0784] Objective: To introduce a new manufacturing line.

[0785] Objective: Improve production efficiency

[0786] As a concrete example of progress reporting and issue management, the user will input the following information.

[0787] Example of a prompt:

[0788] "Today's progress: 50% of the equipment installation is complete. A new challenge has arisen: a power supply issue."

[0789] The server updates the progress, adds "Issue: Power supply issues" to the list, generates "Inspect and repair the power supply system" as a suggested solution, and notifies the user.

[0790] Thus, the system of the present invention can automate and efficiently and effectively perform project management in a factory.

[0791] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0792] Step 1:

[0793] A microphone and recording device mounted on the factory robot acquire the conference audio. The microphone records all sounds emitted during the conference and converts them to a digital format in real time. This results in the conference audio being acquired as digital audio data.

[0794] Input: Audio from a meeting

[0795] Output: Digital audio data

[0796] Step 2:

[0797] The acquired digital audio data is sent to the server. The server uses speech recognition technology to convert the digital audio data into text data. Google Cloud Speech-to-Text and Amazon Transcribe are used as speech recognition technologies.

[0798] Input: Digital audio data

[0799] Output: Text data

[0800] Step 3:

[0801] Text data is analyzed by the server using natural language processing (NLP) techniques. This analysis automatically extracts information such as the project's objectives, overview, schedule, and required resources. This then generates an initial project plan.

[0802] Input: Text data

[0803] Output: Project plan

[0804] Step 4:

[0805] The generated project plan is shared with stakeholders via the server. Sharing methods include email, display on a dashboard, and other collaboration tools.

[0806] Input: Project plan

[0807] Output: Sharing with stakeholders

[0808] Step 5:

[0809] Users input daily progress and challenges through factory robots. The interface provides progress input forms and challenge reporting forms, and this data is sent to the server in real time.

[0810] Input: Progress status, issue data

[0811] Output: Progress status and issue data sent to the server

[0812] Step 6:

[0813] The server analyzes the received progress and issue data, updates the progress status, and generates an issue list and proposed solutions. Historical data and knowledge bases are used for data analysis to propose the optimal solution.

[0814] Input: Progress status, issue data

[0815] Output: Updated progress, issue list, proposed solutions

[0816] Step 7:

[0817] The server notifies stakeholders of the generated list of issues and proposed solutions. Notification methods include alerts displayed via factory robots, email, and notifications on the dashboard.

[0818] Input: List of issues, proposed solutions

[0819] Output: Notification to relevant parties

[0820] Step 8:

[0821] The server periodically retrieves the latest progress information from the database and updates the project plan. This ensures that stakeholders are always aware of the latest project status.

[0822] Input: Progress information

[0823] Output: Updated project plan

[0824] Step 9:

[0825] The updated project plan is sent to higher levels as an automatically generated progress report by the server. Methods of transmission include periodic emails, dashboard displays, and automated report generation.

[0826] Input: Updated project plan

[0827] Output: Progress report to higher layers

[0828] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0829] Modes for carrying out the invention

[0830] This invention relates to a system that combines an emotion engine with a project management system to more effectively manage project progress and issues. In particular, it aims to increase the success rate of projects by analyzing the emotions of stakeholders and reflecting them in plans and progress reports.

[0831] System Configuration and Functions

[0832] Recording and transcribing meeting audio.

[0833] The user initiates project planning meetings and progress meetings.

[0834] The terminal (the microphone or recording device in the conference room) records the conference audio and converts that audio data into text data.

[0835] All speech during the meeting is converted into text data in real time and used for subsequent processing. Speech recognition technology is used to perform highly accurate speech-to-text conversion.

[0836] Text data analysis and project plan generation

[0837] The terminal sends the generated text data to the server.

[0838] The server analyzes text data using natural language processing (NLP) technology, automatically extracting project objectives, overview, schedule, milestones, resource requirements, and other information from the conversation to generate a project plan.

[0839] The generated project plan is automatically shared with stakeholders and becomes accessible to everyone.

[0840] Daily progress reporting and task management

[0841] Users input their daily progress and challenges into the system as daily reports.

[0842] The terminal sends the daily report data to the server.

[0843] The server analyzes the received daily report data and automatically updates the project progress and any issues that have arisen.

[0844] Emotional analysis using an emotion engine

[0845] The server has an emotion engine built in that analyzes user emotions from meeting audio and daily report data.

[0846] The server incorporates the analysis results into the project plan and generates progress updates and proposed solutions to challenges, taking emotional information into account.

[0847] The emotion engine analyzes emotional tendencies from user statements and input, extracting information such as "stress is increasing" or "satisfaction is high." This allows for early detection of project risks and concrete suggestions for appropriate countermeasures to solve problems.

[0848] Generation and notification of proposed solutions to the problem.

[0849] The server generates proposed solutions for the listed issues, incorporates sentiment information, and then notifies the user.

[0850] The proposed solutions include optimal solutions based on past data and knowledge bases. Solutions that also take emotional information into account are more likely to be accepted by users.

[0851] Progress reports and sharing with upper management

[0852] The server periodically retrieves the latest progress information from the database and updates the project plan.

[0853] Updated project plans and progress reports are automatically sent to higher levels of the system.

[0854] This allows upper management to always have an accurate grasp of the latest project status.

[0855] Addressing urgent issues

[0856] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[0857] Real-time notifications allow for a quick response to urgent issues.

[0858] Specific example

[0859] Examples of meeting recording and transcription

[0860] User: "The goal of this project is to improve operational efficiency by revamping our customer management system."

[0861] Device: Recorded the meeting audio and transcribed it as, "The objective of this project is to revamp the customer management system to improve operational efficiency."

[0862] Server: Receives text data, analyzes it, and adds it to the project plan as "Purpose: Renewal of customer management system" and "Goal: Improve operational efficiency."

[0863] Examples of progress reporting and issue management

[0864] User: "Today we started migrating customer data, but we encountered problems due to data format mismatches. It's very stressful."

[0865] Terminal: Submit data via input form.

[0866] Server: Updates progress and adds "Issue: Data format mismatch" to the list. The emotion engine detects that "stress" is high, suggests "Create a data conversion script" as a solution, and notifies the user.

[0867] By using this system, project management becomes more efficient and effective, leading to increased stakeholder satisfaction and a higher project success rate.

[0868] The following describes the processing flow.

[0869] Processing steps

[0870] 1. Initial setup and project plan generation

[0871] Step 1:

[0872] The user initiates a meeting and shares the objectives, overview, and schedule of the new project with stakeholders.

[0873] Step 2:

[0874] The device records meeting audio in real time and converts that audio data into text data.

[0875] Specific example: A microphone or recording device in a conference room acquires audio data, and speech recognition software converts it into text.

[0876] Step 3:

[0877] The terminal sends the converted text data to the server.

[0878] Specific example: A text-based data file is uploaded to a server via the internet.

[0879] Step 4:

[0880] The server analyzes the received text data using natural language processing (NLP) techniques to extract project objectives, overview, schedule, milestones, resource requirements, and other relevant information from the conversation.

[0881] Specific example: An NLP engine analyzes text data to identify important keywords and phrases.

[0882] Step 5:

[0883] The server generates an initial project plan based on the analysis results and automatically shares that plan with the relevant parties.

[0884] Specific example: A project plan is formatted and sent to stakeholders via an email program.

[0885] 2. Daily progress and task management

[0886] Step 6:

[0887] Users input their daily progress and challenges as daily reports and submit them to the system.

[0888] Specific example: Enter progress information and any problems encountered into a dedicated input form.

[0889] Step 7:

[0890] The terminal sends the daily report data to the server.

[0891] Specific example: When you press the submit button on the form, the entered data is uploaded to the server.

[0892] Step 8:

[0893] The server receives the daily report data and performs analysis. Based on the analysis results, it updates the project progress and lists any newly arising issues.

[0894] Specific example: The analysis engine analyzes daily report data and saves progress and issues to a database.

[0895] Step 9:

[0896] The server generates proposed solutions for the issues it has listed and notifies the user of these solutions.

[0897] Specific example: An AI algorithm generates the optimal solution and proposes it to the user via email notification.

[0898] 3. Emotional analysis through the introduction of an emotion engine

[0899] Step 10:

[0900] The server has an emotion engine built in that analyzes user emotions from meeting audio and daily report data.

[0901] Specific example: Extract emotional indicators such as "stress" and "satisfaction level" from meeting audio and text-based daily reports.

[0902] Step 11:

[0903] The server incorporates the analysis results into the project plan and generates progress updates and proposed solutions to challenges, taking emotional information into account.

[0904] Specific example: For users experiencing high stress levels, we suggest solutions such as "recommending rest" or "enhancing support."

[0905] Step 12:

[0906] The server notifies relevant parties of emotional information, facilitating smooth communication among project members.

[0907] Specific example: Display emotional information on a dashboard so that everyone can share the situation in real time.

[0908] 4. Progress reports and sharing with upper management

[0909] Step 13:

[0910] The server periodically retrieves the latest project progress information from the database and updates the project plan.

[0911] Specific example: Cloning jobs are executed regularly according to schedule, and the latest data is reflected in the project plan.

[0912] Step 14:

[0913] The server automatically sends updated project plans and progress reports to higher levels.

[0914] Specific example: A program sends an email containing a report summarizing the latest progress.

[0915] 5. Addressing urgent issues

[0916] Step 15:

[0917] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[0918] Specific example: When an urgent issue arises, the notification system sends real-time alerts to relevant parties and proposes solutions that take emotional information into account.

[0919] summary

[0920] Through these steps, the project management system operates efficiently through the coordination of users, terminals, and servers, effectively supporting project planning, progress management, and issue resolution. By introducing an emotion engine, emotional information is reflected in project plans and solutions, further improving the project's success rate.

[0921] (Example 2)

[0922] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0923] Traditional project management systems lacked sufficient automation in areas such as recording and transcribing meeting audio, automatically generating project plans, and managing progress and issues. Furthermore, they struggled to provide solutions that considered the feelings of stakeholders. As a result, accurately understanding project progress and efficiently resolving issues was difficult. Moreover, the inability to respond quickly to urgent issues contributed to a lower project success rate.

[0924] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0925] In this invention, the server includes means for recording the voices of stakeholders during a meeting and converting them into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for sharing the project plan with stakeholders; means for providing an interface for inputting daily progress and issues; means for transmitting the progress and issue data to a central processing unit; means for analyzing the progress and issue data, updating the progress, and generating an issue list and proposed solutions; means for notifying stakeholders of the issue list and proposed solutions; means for automatically generating a project progress report and transmitting it to a higher level; means for analyzing emotions from the voice and text data; and means for reflecting the emotion analysis results in the project plan and generating countermeasures that take emotional information into account. This makes project management more efficient and effective, and improves stakeholder satisfaction and the success rate of the project.

[0926] A "meeting" is a place where stakeholders gather to discuss the project's plan, progress, and challenges.

[0927] "Audio" refers to data that includes the words and statements made by participants during a meeting.

[0928] "Text data" refers to information obtained by converting speech into written text.

[0929] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate human language.

[0930] A "project plan" is a document that outlines the project's objectives, overview, schedule, milestones, resource requirements, and other relevant information.

[0931] "Stakeholders" refers to all people involved in the project.

[0932] An "interface" refers to the means by which users input their daily progress and tasks into a system.

[0933] A "central processing unit" refers to a device that processes data for the entire system.

[0934] A "task list" is a document that lists all unresolved issues and tasks within a project.

[0935] A "proposed solution" is a proposal that outlines solutions to a problem.

[0936] A "progress report" is a document that reports on the current progress of a project.

[0937] "Upper management" refers to the people or organizations in a position to oversee and manage the entire project.

[0938] "Emotions" refer to information about the psychological state and feelings of those involved.

[0939] "Emotional analysis" refers to the process of identifying a user's emotions by analyzing text data and audio data.

[0940] "System" refers to a set of computing resources and software that possess the aforementioned functions.

[0941] "Updating the progress status" refers to the act of keeping the project's progress information up to date.

[0942] "To notify" means to convey information to the relevant parties.

[0943] This invention is a system that combines an emotion engine with a project management system. The system includes recording and transcribing meeting audio, analyzing text data, automatically generating and sharing project plans, managing daily progress and issues, sentiment analysis, and generating and notifying solutions to issues.

[0944] Hardware and software usage

[0945] The user initiates project planning meetings and progress meetings.

[0946] The terminal records meeting audio using the conference room's microphone or recording device, and converts the audio data into text using the Google Cloud Speech-to-Text API. It also utilizes an internet connection for data transmission. The recorded audio is transcribed in real time. All statements made during the meeting are recorded as text and used for subsequent processing.

[0947] Processing flow

[0948] The terminal sends the generated text data to the server.

[0949] The server receives text data and performs analysis using natural language processing (NLP) techniques. Here, OpenAI's GPT-4 model is used for text analysis, automatically extracting project objectives, overview, schedule, milestones, resource requirements, etc., and generating a project plan. The generated project plan is automatically shared with stakeholders and made accessible to everyone.

[0950] Progress management and issue management

[0951] Users use an interface to input daily progress and issues into the system as daily reports. The terminal sends the entered daily report data to the server. The server receives the daily report data, analyzes it, and automatically updates the project progress and any issues that have arisen. Programming languages ​​such as Python are used for this analysis.

[0952] Emotion analysis and response plan generation

[0953] The emotion engine built into the server analyzes user emotions from meeting audio and daily report data. For example, it uses the Sentiment Analysis API to detect emotions within text. The server then incorporates these emotion analysis results into project plans, generating progress reports and countermeasures for issues that take emotional information into account.

[0954] The emotion engine can extract information such as "stress is increasing" or "satisfaction is high" from user statements and input. This allows for early detection of project risks and the concrete proposal of appropriate countermeasures for problem solving.

[0955] Notification of proposed response and emergency response

[0956] The server notifies stakeholders of the generated list of issues and proposed solutions. These solutions include optimal solutions based on historical data and a knowledge base. Sentimental information is also taken into account, resulting in solutions that are more likely to be accepted by users.

[0957] The server periodically retrieves project progress reports from the database and updates the project plan. The updated plan and progress reports are automatically sent to higher levels, allowing them to always stay informed about the latest project status.

[0958] In the event of an urgent issue, the server immediately notifies relevant parties and proposes solutions that take emotional information into consideration. This real-time notification enables a rapid response.

[0959] Examples of specific actions

[0960] User: "The goal of this project is to revamp our customer management system to improve operational efficiency."

[0961] Terminal: Records meeting audio, transcribes it into text, and translates it as, "The objective of this project is to improve operational efficiency by revamping the customer management system."

[0962] Server: Receives and analyzes text data, and adds it to the project plan with the objectives "Renewal of customer management system" and the goal "Improvement of operational efficiency."

[0963] Example of a prompt

[0964] "Automatically generate a project plan from the meeting audio. The following is a transcript of the meeting: 'The objective of this project is to improve operational efficiency by revamping the customer management system.'"

[0965] By using this system, project management is expected to be more efficient and effective, leading to increased stakeholder satisfaction and a higher project success rate.

[0966] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0967] Step 1:

[0968] The user initiates project planning meetings and progress meetings.

[0969] Input: Conference audio

[0970] Output: Recorded audio data

[0971] Specific action: The user starts a meeting and makes a statement. That statement is recorded as audio data by the recording device.

[0972] Step 2:

[0973] The device converts recorded meeting audio into text data in real time using the Google Cloud Speech-to-Text API.

[0974] Input: Recorded audio data

[0975] Output: Text converted to character data

[0976] Specific operation: The recording device sends the recorded audio data to the Google Cloud Speech-to-Text API, and the data, converted to text with high accuracy, is returned to the device.

[0977] Step 3:

[0978] The terminal sends text data to the server.

[0979] Input: Text data

[0980] Output: Text data sent to the server

[0981] Specific operation: Text data is sent from the terminal to the server. The data is transmitted in real time using a network connection and received by the server.

[0982] Step 4:

[0983] The server analyzes the received text data using Natural Language Processing (NLP) technology and automatically generates a project plan.

[0984] Input: Text data sent to the server

[0985] Output: Project plan generated based on the analyzed information

[0986] Specific operation: The server analyzes text data, uses the OpenAI GPT-4 model to extract project objectives, overview, schedule, resource requirements, etc., and automatically generates a project plan.

[0987] Step 5:

[0988] The server automatically shares the generated project plan with relevant parties.

[0989] Input: Generated project plan

[0990] Output: Project plan shared with stakeholders

[0991] Specific operation: The server sends the generated project plan to relevant parties' email addresses or cloud systems, making it accessible to everyone.

[0992] Step 6:

[0993] Users input their daily progress and challenges into the system as daily reports.

[0994] Input: Progress status and details of the issues

[0995] Output: Daily report data

[0996] Specific operation: Users input their daily progress and challenges using input forms provided in the system, and this data is saved in the system as daily report data.

[0997] Step 7:

[0998] The terminal sends the daily report data to the server.

[0999] Input: Daily report data

[1000] Output: Daily report data sent to the server

[1001] Specific operation: Daily report data is sent from the terminal to the server. Daily report data is sent over the network and received by the server.

[1002] Step 8:

[1003] The server analyzes the daily report data and automatically updates the project progress and issue list.

[1004] Input: Daily report data sent to the server

[1005] Output: Updated progress and issue list

[1006] Specific operation: The server analyzes the daily report data and updates the progress status and task list using Python or similar tools.

[1007] Step 9:

[1008] The emotion engine built into the server analyzes the user's emotions from daily report data and meeting audio.

[1009] Input: Daily report data and meeting audio data

[1010] Output: User sentiment analysis results

[1011] Specific operation: The emotion engine on the server analyzes the data using the Sentiment Analysis API and extracts emotional information (e.g., "stress is increasing," "satisfaction is high").

[1012] Step 10:

[1013] The server incorporates the sentiment analysis results into the project plan and generates response proposals that take sentiment information into account.

[1014] Input: Sentiment analysis results

[1015] Output: Project plan and response plan reflecting emotional information

[1016] Specific operation: The server updates the project plan based on the sentiment analysis results and generates countermeasures that take sentiment information into account.

[1017] Step 11:

[1018] The server will notify relevant parties of the listed issues and proposed solutions.

[1019] Input: List of issues and proposed solutions

[1020] Output: List of issues and proposed solutions notified to stakeholders.

[1021] Specific operation: The server notifies relevant parties of the issue list and proposed solutions, and sends them via email or messaging system.

[1022] Step 12:

[1023] The server periodically retrieves the latest progress information from the database and updates the project plan.

[1024] Input: Progress information in the database

[1025] Output: Updated project plan

[1026] Specific operation: The server queries the database to retrieve the latest progress information and update the project plan.

[1027] Step 13:

[1028] The server automatically sends updated project plans and progress reports to higher levels.

[1029] Input: Updated project plan and progress report

[1030] Output: Progress report sent to the upper layer

[1031] Specific operation: The server sends updated project plans and progress reports to higher levels via email or cloud systems.

[1032] Step 14:

[1033] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[1034] Input: Urgent issue information and sentiment analysis results

[1035] Output: Urgent issues and solutions notified to stakeholders

[1036] Specific operation: The server detects urgent issues, considers solutions that take emotional information into account, and notifies relevant parties in real time.

[1037] (Application Example 2)

[1038] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1039] Traditional project management systems often manage progress and issues without considering the feelings of stakeholders, leading to decreased stakeholder satisfaction and a lower project success rate. Furthermore, the lack of real-time notifications for urgent issues and analysis results made rapid response difficult.

[1040] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1041] In this invention, the server includes means for recording meeting audio of stakeholders during the project and converting it into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for analyzing the user's emotions using an emotion engine and reflecting the analysis results in the project plan and progress reports; and means for notifying the user of the progress status, issues, and emotion analysis results in real time using a smart device. This enables effective project management that takes into account the emotions of stakeholders, thereby improving the project success rate and stakeholder satisfaction.

[1042] A "project plan" is a document that outlines the project's objectives, overview, schedule, milestones, resource requirements, and other relevant information.

[1043] "Text data" refers to a data format in which audio data is converted into written text.

[1044] "Natural language processing technology" refers to technologies that enable computers to understand, analyze, and generate human language.

[1045] An "emotion engine" is a system that analyzes a user's emotions from text and audio data.

[1046] "Progress status" refers to information indicating the degree of progress of a project.

[1047] "Issue data" refers to information about problems that arise during the progress of a project and matters that require attention.

[1048] A "server" is a computer system that performs specific services or data processing over a network.

[1049] A "smart device" is a portable electronic device with internet connectivity and typically possesses advanced computing capabilities.

[1050] An "interface" is the window or means through which a user interacts with a system.

[1051] "Notification" refers to a means or action used to inform a user of specific information.

[1052] "Real-time" means that information is generated and processed almost instantly.

[1053] "Upper management" refers to managers and leaders who are responsible for overseeing the progress and results of a project.

[1054] An "urgent issue" is a serious problem or obstacle that requires immediate attention.

[1055] A "solution plan" refers to a proposed solution or proposal for a problem that has arisen.

[1056] This invention relates to a system that combines an emotion engine with a project management system to more effectively manage project progress and issues. Specifically, it aims to increase the success rate of projects by analyzing the emotions of stakeholders and reflecting them in project plans and progress reports.

[1057] System Configuration

[1058] The system consists of the following main components. These components are implemented through a combination of hardware and software.

[1059] Recording and transcribing meeting audio.

[1060] A device (such as a smart device or a dedicated recording device) records project planning meetings and progress meetings. The recorded audio data is converted into text data using speech recognition technology. This text data is then used for subsequent processing.

[1061] Text data analysis and project plan generation

[1062] The server receives text data sent from the terminal. This text data is analyzed using natural language processing (NLP) technology to automatically extract the project's objectives, overview, schedule, milestones, resource requirements, etc., from the conversation content, and generates a project plan. The generated project plan is then shared with stakeholders.

[1063] Daily progress reporting and task management

[1064] Users input daily progress and issues into the system through an interface. This data is sent from the terminal to the server. The server analyzes the received data and automatically updates the project progress and any issues that have arisen.

[1065] Emotional analysis using an emotion engine

[1066] The server has an emotion engine built in that analyzes user emotions from meeting audio and daily report data. The server incorporates the analysis results into the project plan and generates progress reports and proposed solutions to issues, taking emotional information into account.

[1067] Generation and notification of proposed solutions to the problem.

[1068] The server generates proposed solutions for the listed issues, incorporating sentiment information before notifying the user. These solutions include optimal solutions based on historical data and a knowledge base.

[1069] Addressing urgent issues

[1070] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[1071] Hardware and software to use

[1072] Hardware: Smart devices (smart glasses, smartphones), conference room recording equipment, servers

[1073] Software: Python, speech_recognition library, transformers library, TextBlob library

[1074] Processing flow

[1075] The system begins by recording meeting audio and transcribing it into text. The text data is then analyzed using natural language processing techniques to generate project plans and progress reports. An emotion engine analyzes user emotions and incorporates the results into the project plans and progress reports. It also provides solutions to issues and notifications for emergencies.

[1076] Examples of specific cases and prompt statements

[1077] Specific example:

[1078] Voice input: "Today's progress was delayed due to a machine malfunction. I'm feeling very stressed."

[1079] Text: "Today's progress was delayed due to a machine malfunction. I'm feeling very stressed."

[1080] Sentiment analysis result: "Negative, 0.85"

[1081] Notification: "Issue: Machine malfunction\nSolution: Call a repair technician and have the malfunctioning machine repaired. Taking a break is recommended."

[1082] Example of a prompt:

[1083] "Analyze the emotion in this text and respond with either positive or negative."

[1084] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1085] Step 1:

[1086] The device records the meeting audio.

[1087] Input: Audio from the project meeting

[1088] Operation: The device (smart device or recording device) records the meeting audio.

[1089] Output: Recorded audio data

[1090] Step 2:

[1091] The device converts the recorded audio data into text data.

[1092] Input: Recorded audio data

[1093] Operation: Converts audio data into text data using speech recognition technology (Google Speech Recognition API).

[1094] Output: Text data

[1095] Step 3:

[1096] The terminal sends text data to the server.

[1097] Input: Text data

[1098] Operation: Sends text data generated by the terminal to the server.

[1099] Output: Text data is sent to the server.

[1100] Step 4:

[1101] The server analyzes the received text data using natural language processing technology and automatically generates a project plan.

[1102] Input: Text data

[1103] Operation: The server uses natural language processing (NLP) techniques to analyze text data and extract project objectives, overview, schedule, milestones, and resource requirements.

[1104] Output: Automated project plan

[1105] Step 5:

[1106] The server shares the project plan with the relevant parties.

[1107] Input: Automated project plan

[1108] Operation: The server notifies and shares the project plan with relevant parties.

[1109] Output: Project plan shared with stakeholders

[1110] Step 6:

[1111] Users input their daily progress and challenges through the interface.

[1112] Input: Information on progress and issues

[1113] Operation: The user inputs progress and issue information into the system interface.

[1114] Output: Progress and issue data entered by the user.

[1115] Step 7:

[1116] The terminal sends daily report data to the server.

[1117] Input: Progress and issue data entered by the user.

[1118] Operation: The device sends progress and task data to the server.

[1119] Output: Daily report data sent to the server

[1120] Step 8:

[1121] The server analyzes the daily report data it receives and automatically updates the project's progress and issues.

[1122] Input: Daily report data

[1123] Operation: The server analyzes progress and issues, updates progress, and generates a list of issues.

[1124] Output: Updated progress and issue list

[1125] Step 9:

[1126] The server uses an emotion engine to analyze the user's emotions and reflects the analysis results in project plans and progress reports.

[1127] Input: Text data and daily report data

[1128] Operation: The server uses an emotion engine to analyze the emotions in the data and reflects the results in the project plan and progress report.

[1129] Output: Project plan and progress report reflecting emotional information

[1130] Step 10:

[1131] The server generates proposed solutions to the problem, incorporates emotional information, and then notifies the user.

[1132] Input: Task list and sentiment analysis results

[1133] Operation: The server generates proposed solutions to the problem based on historical data and a knowledge base, and notifies the user, taking sentiment information into account.

[1134] Output: Proposed course of action notified to the user

[1135] Step 11:

[1136] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[1137] Input: Information on the occurrence of urgent issues and sentiment analysis results

[1138] Operation: The server detects urgent issues, notifies relevant parties, and proposes solutions that take emotional information into account.

[1139] Output: Urgent issues and solutions notified to stakeholders

[1140] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1141] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1142] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1143] [Third Embodiment]

[1144] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1145] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1146] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1147] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1148] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1149] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1150] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1151] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1152] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1153] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1154] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1155] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1156] Modes for carrying out the invention

[1157] This invention relates to a system for efficiently managing projects to build new systems, improving the efficiency of project progress by automating the content of meetings among stakeholders, daily progress reports, and issue management.

[1158] System Components and Their Functions

[1159] Recording and transcribing meeting audio.

[1160] The user begins project planning meetings and progress meetings.

[1161] The terminals (microphones and recording devices in the conference room) acquire the conference audio and convert it into text data in real time.

[1162] All statements made during the meeting are converted into text data and used for subsequent processing. At this stage, speech recognition technology is applied to perform highly accurate speech-to-text conversion.

[1163] Text data analysis and project plan generation

[1164] The terminal sends the generated text data to the server.

[1165] The server analyzes text data using natural language processing (NLP) techniques. It automatically extracts the project's objectives, overview, schedule, milestones, and required resources, and generates a project plan.

[1166] The generated project plan is automatically shared with stakeholders, allowing each project member to review it.

[1167] Daily progress reporting and task management

[1168] Users input daily work reports, progress updates, and any issues that arise as daily reports.

[1169] The terminal sends this input data to the server.

[1170] When the server receives the daily report data, it immediately begins analysis. The analysis results are saved in the database, and the project progress and issue list are updated accordingly.

[1171] Generation and notification of proposed solutions to the problem.

[1172] The server lists newly arising issues and generates proposed solutions. It automatically suggests the optimal solution based on historical data and a knowledge base.

[1173] The generated response plan is notified to the user via their device, prompting them to take the necessary action.

[1174] Progress reports and sharing with upper management

[1175] The server periodically retrieves the latest progress information from the database and updates the project plan. The updated plan is automatically sent as a report to higher levels.

[1176] The server provides this progress report to higher levels in the form of email or dashboards, allowing them to accurately understand the project's current status.

[1177] Addressing urgent issues

[1178] If the server detects an urgent issue, it will immediately notify the relevant parties. The notification will also include specific solutions.

[1179] This notification will be communicated to relevant parties in real time to facilitate a swift response.

[1180] Specific example

[1181] Examples of meeting recording and transcription

[1182] User: "The goal of this project is to improve operational efficiency by revamping our customer management system."

[1183] Device: Recorded the meeting audio and transcribed it as, "The objective of this project is to revamp the customer management system to improve operational efficiency."

[1184] Server: Receives text data, analyzes it, and adds it to the project plan as "Purpose: Renewal of customer management system" and "Goal: Improve operational efficiency."

[1185] Examples of progress reporting and issue management

[1186] User: "Today's progress: 40% of customer data migration completed. There are issues with data format mismatches."

[1187] Terminal: Submit data via input form.

[1188] Server: Updated progress and added "Issue: Data format mismatch" to the list. Proposed "Create a data conversion script" as a solution and notified the user.

[1189] In this way, a system is provided that significantly improves the efficiency of project management by automatically analyzing and managing the statements and daily reports of stakeholders.

[1190] The following describes the processing flow.

[1191] Processing steps

[1192] 1. Initial setup and project plan generation

[1193] Step 1:

[1194] The user initiates a meeting and shares the objectives, overview, and schedule of the new project with stakeholders.

[1195] Step 2:

[1196] The device records meeting audio in real time and converts that audio data into text data.

[1197] Specific example: A microphone or recording device in a conference room acquires audio data, and speech recognition software converts it into text.

[1198] Step 3:

[1199] The terminal sends the converted text data to the server.

[1200] Specific example: A text-based data file is uploaded to a server via the internet.

[1201] Step 4:

[1202] The server analyzes the received text data using natural language processing (NLP) techniques to extract project objectives, overview, schedule, milestones, resource requirements, and other relevant information from the conversation.

[1203] Specific example: An NLP engine analyzes text data to identify important keywords and phrases.

[1204] Step 5:

[1205] The server generates an initial project plan based on the analysis results and automatically shares that plan with the relevant parties.

[1206] Specific example: A project plan is formatted and sent to stakeholders via an email program.

[1207] 2. Daily progress and task management

[1208] Step 6:

[1209] Users input their daily progress and challenges as daily reports and submit them to the system.

[1210] Specific example: Enter progress information and any problems encountered into a dedicated input form.

[1211] Step 7:

[1212] The terminal sends the daily report data to the server.

[1213] Specific example: When you press the submit button on the form, the entered data is uploaded to the server.

[1214] Step 8:

[1215] The server receives the daily report data and performs analysis. Based on the analysis results, it updates the project progress and lists any newly arising issues.

[1216] Specific example: The analysis engine analyzes daily report data and saves progress and issues to a database.

[1217] Step 9:

[1218] The server generates proposed solutions for the issues it has listed and notifies the user of these solutions.

[1219] Specific example: An AI algorithm generates the optimal solution and proposes it to the user via email notification.

[1220] 3. Progress reports and sharing of issues

[1221] Step 10:

[1222] The server periodically retrieves the latest project progress information from the database and updates the project plan.

[1223] Specific example: The clone job is executed according to schedule, and the latest data is used.

[1224] Step 11:

[1225] The server automatically sends the updated project plan as a report to the higher layer.

[1226] Specific example: A program sends an email containing a report summarizing the latest progress.

[1227] Step 12:

[1228] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes a solution.

[1229] Specific example: When an urgent issue arises, the notification system sends real-time alerts to relevant parties and proposes solutions.

[1230] summary

[1231] Through these steps, the project management system operates efficiently through the coordination of users, terminals, and servers, effectively supporting project planning, progress management, and issue resolution. Including specific program processing flows clarifies how the system will be implemented.

[1232] (Example 1)

[1233] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1234] In project management, it is crucial for stakeholders to communicate efficiently, share progress, and resolve issues quickly. However, manually recording project meeting audio, converting it to text, and incorporating it into project plans is a laborious process. Furthermore, delays in reporting progress and issues can impact the overall project schedule. Moreover, the inability to quickly address urgent issues increases project risk. Thus, streamlining the collection, sharing, and updating of information is a major challenge in project management.

[1235] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1236] This invention includes a server that transcribes meeting audio into text in real time and transmits it to the server immediately, a server that notifies relevant parties in real time when it detects an urgent issue and presents specific solutions, and a server that periodically updates the project plan and automatically generates the latest progress status and issue list. This enables rapid transcription and analysis of meeting content, realizing a project management system that can respond quickly to urgent issues.

[1237] "Meeting audio" refers to the audio generated when stakeholders discuss and exchange opinions about a project during its progress.

[1238] "Text data" refers to data obtained by converting recorded meeting audio into text information using speech recognition technology.

[1239] "Natural language processing technology" is a general term for technologies that enable computers to understand, interpret, and generate human language, and is particularly used for analyzing text data.

[1240] A "project plan" is a document that includes information such as the project's objectives, overview, schedule, milestones, and required resources.

[1241] An "interface" is an input method used by users to enter data such as daily work reports, progress status, and issues that have arisen.

[1242] A "server" is a central computer system that receives text data and daily report data sent from each terminal and performs processing such as analysis, storage, and notification.

[1243] "Progress status" refers to information that indicates the current stage of a project's work and how far along it is.

[1244] "Issue data" refers to information that records problems and obstacles that occur during the progress of a project.

[1245] A "list of issues" refers to a compilation of all issues that arose during the project's progress.

[1246] A "proposed solution" refers to the proposed solutions or action plans for the listed issues.

[1247] "Upper management" refers to senior personnel such as managers and executives who receive project progress reports.

[1248] "Real-time" refers to a timeframe in which processing and notifications are carried out immediately the moment an event occurs.

[1249] This invention relates to a system for efficiently managing projects to build new systems. It improves the efficiency of project progress by automating the content of meetings among stakeholders, daily progress reports, and issue management.

[1250] First, users begin project planning and progress meetings. Terminals (such as microphones or recording devices in the meeting room) capture the meeting audio and convert it into text data in real time. At this stage, speech recognition technology is applied, specifically using speech recognition services such as "Google Speech-to-Text API" or "IBM Watson Speech to Text".

[1251] The acquired text data is sent from the terminal to the server. The server analyzes this text data using natural language processing techniques (such as Python's NLTK library or Google Cloud Natural Language API) and automatically extracts the project's objectives, overview, schedule, milestones, and required resources. Based on the extracted information, the server generates a project plan and automatically shares it with stakeholders.

[1252] Users also input daily work reports, progress updates, and issues encountered as daily reports. This daily report data is sent from the terminal to the server. Upon receiving the daily report data, the server immediately begins analysis and saves the analysis results to a database, thereby updating the project progress and issue list.

[1253] The server lists newly arising issues and generates optimal solutions. This process utilizes Elasticsearch and Python's machine learning module (scikit-learn) to generate the best solutions from historical data and a knowledge base. The generated solutions are then communicated to the user via their terminal, and they are required to take action.

[1254] Furthermore, the server periodically retrieves the latest progress information from the database and updates the project plan. This updated plan is automatically sent to higher levels. Reports to higher levels are provided via email or dashboard tools such as Tableau and Power BI.

[1255] If the server detects an urgent issue, it will immediately notify relevant parties in real time and propose specific solutions. This notification will be sent in real time using the Slack API or Twilio API.

[1256] Specific examples include the following:

[1257] User: "The goal of this project is to improve operational efficiency by revamping our customer management system."

[1258] Terminal: Recorded the meeting audio and transcribed it as, "The objective of this project is to revamp the customer management system to improve operational efficiency."

[1259] Server: Receives text data, analyzes it, and adds it to the project plan as "Purpose: Renewal of the customer management system" and "Goal: Improvement of operational efficiency."

[1260] Furthermore, the following specific examples are given for progress reporting and issue management.

[1261] User: "Today's progress: 40% of customer data migration completed. There are issues with data format mismatches."

[1262] Terminal: Submit data from the input form.

[1263] Server: Updated progress and added "Issue: Data format mismatch" to the list. Proposed "Create a data conversion script" as a solution and notified the user.

[1264] Examples of prompt statements to input into the generative AI model are as follows:

[1265] "Automatically generate a project plan that includes the project objectives, overview, schedule, milestones, and required resources."

[1266] "Based on past data, please propose the best solution to the data format mismatch issue."

[1267] This system makes it possible to efficiently analyze and manage conversations and daily reports from stakeholders, significantly improving the efficiency of project management.

[1268] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1269] Step 1:

[1270] The user initiates a project planning or progress meeting. The input is the meeting audio, which serves as the starting point for processing.

[1271] The device activates the microphones and recording devices in the conference room and begins capturing conference audio in real time. Specifically, the device starts recording as soon as it recognizes the voice command "start".

[1272] Step 2:

[1273] The terminal sends the acquired meeting audio data to a speech recognition API (e.g., Google Speech-to-Text API). The input is audio data, and the output returned by the API is text data.

[1274] The terminal receives text data and saves the meeting's statements as text information. Specifically, the audio data is converted to text, such as "The objective of this project is to improve operational efficiency by revamping the customer management system."

[1275] Step 3:

[1276] The terminal sends the generated text data to the server. The input is text data, which is sent to the server for further analysis.

[1277] The server receives and stores the text data. This prepares it for creating the project plan.

[1278] Step 4:

[1279] The server analyzes the received text data using natural language processing techniques (e.g., Python's NLTK library). The input is text data, and the output is extracted information such as the project's objectives, overview, schedule, milestones, and required resources.

[1280] The server automatically generates a project plan based on these analysis results. Specifically, details such as "Objective: Renewal of the customer management system" and "Goal: Improvement of operational efficiency" are added to the plan.

[1281] Step 5:

[1282] The server automatically shares the generated project plan with stakeholders. The input is the automatically generated plan, and the output is the information shared with stakeholders. Sharing is done via email, cloud storage, and project management tools.

[1283] Step 6:

[1284] Users input daily work reports, progress updates, and any issues encountered as daily reports. These inputs consist of user work reports and progress data, which are then recorded on the terminal as daily reports.

[1285] The terminal sends this daily report data to the server. The input here is the daily report data, and the output is the data sent to the server.

[1286] Step 7:

[1287] The server immediately begins analyzing the daily report data upon receiving it and saves the analysis results to the database. The input is the daily report data, and the output is the analysis results and updated database information.

[1288] The server updates the project progress and issue list. Specifically, it updates information such as "Progress: 40% of customer data migration complete" and "Issues: Data format mismatch."

[1289] Step 8:

[1290] The server lists newly arising issues and generates optimal solutions. The input is issue data, and the output is the proposed solutions. Elasticsearch and Python's machine learning module (scikit-learn) are used here.

[1291] The server sends the generated solution to the terminal and notifies the user. Specifically, the user receives a notification stating "Problem: Data format mismatch" and "Suggested solution: Create a data conversion script."

[1292] Step 9:

[1293] The server periodically retrieves the latest progress information from the database and updates the project plan. The input is the latest progress information, and the output is the updated project plan.

[1294] The server automatically sends updated plans to higher levels. Specifically, it provides information to higher levels via email or dashboard tools (Tableau, Power BI).

[1295] Step 10:

[1296] When the server detects an urgent issue, it immediately notifies relevant parties in real time and provides concrete solutions. The input is data on the urgent issue, and the output is the notification and solution.

[1297] Notifications are sent to all relevant parties via the terminal. Specifically, if a "system failure" occurs as an urgent issue, relevant parties will be immediately notified of a solution such as "system restart."

[1298] (Application Example 1)

[1299] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1300] In conventional project management systems, managing meeting content, progress reports, and issues among stakeholders is done manually, requiring a significant amount of time and effort, often resulting in project delays. Furthermore, even in factory settings, manual data entry and management are necessary, hindering efficient project progress. A system is needed to address these challenges and improve the efficiency of project management.

[1301] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1302] In this invention, the server includes means for recording meeting audio conducted by stakeholders during the progress of a project and converting it into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for sharing the project plan with stakeholders; means for providing an interface for inputting daily progress and issues; means for transmitting the progress and issue data to the server; means for analyzing the progress and issue data, updating the progress, and generating an issue list and proposed solutions; means for notifying stakeholders of the issue list and proposed solutions; means for automatically generating a project progress report and transmitting it to a higher level; means for recording meeting audio in real time via a factory robot and converting it into text data; means for transmitting the text data to the server, analyzing it using natural language processing technology, and automatically generating a project plan; and means for automatically inputting the progress and issue data via a factory robot and transmitting it to the server. This makes it possible to automate the work of stakeholders and improve the efficiency of project management.

[1303] "Meeting audio" refers to audio recordings made during meetings held by stakeholders in the course of a project.

[1304] "Text data" refers to data obtained by converting recorded meeting audio into text information.

[1305] "Natural language processing technology" is a technology that interprets and analyzes human language.

[1306] A "project plan" is a document that outlines the project's objectives, overview, schedule, milestones, and required resources.

[1307] "Stakeholders" refers to all members, managers, and other interested parties involved in the project.

[1308] An "interface" refers to an input device or software that users use to record their daily progress and tasks.

[1309] A "server" is a computing system used for analyzing, storing, and transmitting data.

[1310] A "problem list" is a list of issues that arose during the project's progress.

[1311] A "solution plan" is a proposed solution to a problem that has arisen.

[1312] A "progress report" is a report that describes the progress of a project.

[1313] "Upper management" refers to senior managers who monitor project progress and make decisions.

[1314] A "factory robot" is a machine used in a factory to perform automated tasks.

[1315] "Real-time" refers to performing actions or processing in accordance with real-world time.

[1316] "Analysis" is the process of examining data in detail to clarify its structure and meaning.

[1317] "Automatic generation" refers to the process where a system automatically creates documents or data without human intervention.

[1318] This invention relates to a system for streamlining project management in factories, enabling rapid and accurate project progress by automating meeting audio recording, transcription, progress reporting, and issue management.

[1319] System Configuration

[1320] This system consists of the following main elements:

[1321] Meeting audio and text

[1322] The server records meeting audio using microphones and recording devices mounted on factory robots and converts it into text data in real time. It uses speech recognition technology (for example, Google Cloud Speech-to-Text or Amazon Transcribe).

[1323] Text data analysis and project plan generation

[1324] The server analyzes text data using natural language processing (NLP) technology and automatically extracts the project's objectives, overview, schedule, milestones, and required resources to generate a project plan.

[1325] Progress reporting and issue management

[1326] This system provides an interface for inputting daily progress and challenges via factory robots. The data entered by users is sent to a server, where it is automatically analyzed.

[1327] Generating a list of issues and proposed solutions

[1328] Upon receiving a progress report, the server immediately begins analysis, updates the issue list, and generates proposed solutions. These proposed solutions are then notified to the user.

[1329] Project progress report

[1330] The server periodically retrieves the latest progress information from the database, updates the project plan, and reports the latest progress to higher levels.

[1331] Hardware and software

[1332] Hardware:

[1333] Microphones and recording devices mounted on factory robots

[1334] Server (used as a computing resource)

[1335] software:

[1336] Speech recognition technology (Google Cloud Speech-to-Text, Amazon Transcribe, etc.)

[1337] Natural language processing techniques (such as Python's NLTK library)

[1338] Specific example

[1339] For example, suppose a user makes the following statement in a meeting.

[1340] Example of a prompt:

[1341] "The objective of this project is to improve production efficiency by introducing a new manufacturing line."

[1342] The server records the meeting audio and converts it into text data. Then, it analyzes the text data to automatically extract the following information.

[1343] Objective: To introduce a new manufacturing line.

[1344] Objective: Improve production efficiency

[1345] As a concrete example of progress reporting and issue management, the user will input the following information.

[1346] Example of a prompt:

[1347] "Today's progress: 50% of the equipment installation is complete. A new challenge has arisen: a power supply issue."

[1348] The server updates the progress, adds "Issue: Power supply issues" to the list, generates "Inspect and repair the power supply system" as a suggested solution, and notifies the user.

[1349] Thus, the system of the present invention can automate and efficiently and effectively perform project management in a factory.

[1350] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1351] Step 1:

[1352] A microphone and recording device mounted on the factory robot acquire the conference audio. The microphone records all sounds emitted during the conference and converts them to a digital format in real time. This results in the conference audio being acquired as digital audio data.

[1353] Input: Audio from a meeting

[1354] Output: Digital audio data

[1355] Step 2:

[1356] The acquired digital audio data is sent to the server. The server uses speech recognition technology to convert the digital audio data into text data. Google Cloud Speech-to-Text and Amazon Transcribe are used as speech recognition technologies.

[1357] Input: Digital audio data

[1358] Output: Text data

[1359] Step 3:

[1360] Text data is analyzed by the server using natural language processing (NLP) techniques. This analysis automatically extracts information such as the project's objectives, overview, schedule, and required resources. This then generates an initial project plan.

[1361] Input: Text data

[1362] Output: Project plan

[1363] Step 4:

[1364] The generated project plan is shared with stakeholders via the server. Sharing methods include email, display on a dashboard, and other collaboration tools.

[1365] Input: Project plan

[1366] Output: Sharing with stakeholders

[1367] Step 5:

[1368] Users input daily progress and challenges through factory robots. The interface provides progress input forms and challenge reporting forms, and this data is sent to the server in real time.

[1369] Input: Progress status, issue data

[1370] Output: Progress status and issue data sent to the server

[1371] Step 6:

[1372] The server analyzes the received progress and issue data, updates the progress status, and generates an issue list and proposed solutions. Historical data and knowledge bases are used for data analysis to propose the optimal solution.

[1373] Input: Progress status, issue data

[1374] Output: Updated progress, issue list, proposed solutions

[1375] Step 7:

[1376] The server notifies stakeholders of the generated list of issues and proposed solutions. Notification methods include alerts displayed via factory robots, email, and notifications on the dashboard.

[1377] Input: List of issues, proposed solutions

[1378] Output: Notification to relevant parties

[1379] Step 8:

[1380] The server periodically retrieves the latest progress information from the database and updates the project plan. This ensures that stakeholders are always aware of the latest project status.

[1381] Input: Progress information

[1382] Output: Updated project plan

[1383] Step 9:

[1384] The updated project plan is sent to higher levels as an automatically generated progress report by the server. Methods of transmission include periodic emails, dashboard displays, and automated report generation.

[1385] Input: Updated project plan

[1386] Output: Progress report to higher layers

[1387] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1388] Modes for carrying out the invention

[1389] This invention relates to a system that combines an emotion engine with a project management system to more effectively manage project progress and issues. In particular, it aims to increase the success rate of projects by analyzing the emotions of stakeholders and reflecting them in plans and progress reports.

[1390] System Configuration and Functions

[1391] Recording and transcribing meeting audio.

[1392] The user initiates project planning meetings and progress meetings.

[1393] The terminal (the microphone or recording device in the conference room) records the conference audio and converts that audio data into text data.

[1394] All speech during the meeting is converted into text data in real time and used for subsequent processing. Speech recognition technology is used to perform highly accurate speech-to-text conversion.

[1395] Text data analysis and project plan generation

[1396] The terminal sends the generated text data to the server.

[1397] The server analyzes text data using natural language processing (NLP) technology, automatically extracting project objectives, overview, schedule, milestones, resource requirements, and other information from the conversation to generate a project plan.

[1398] The generated project plan is automatically shared with stakeholders and becomes accessible to everyone.

[1399] Daily progress reporting and task management

[1400] Users input their daily progress and challenges into the system as daily reports.

[1401] The terminal sends the daily report data to the server.

[1402] The server analyzes the received daily report data and automatically updates the project progress and any issues that have arisen.

[1403] Emotional analysis using an emotion engine

[1404] The server has an emotion engine built in that analyzes user emotions from meeting audio and daily report data.

[1405] The server incorporates the analysis results into the project plan and generates progress updates and proposed solutions to challenges, taking emotional information into account.

[1406] The emotion engine analyzes emotional tendencies from user statements and input, extracting information such as "stress is increasing" or "satisfaction is high." This allows for early detection of project risks and concrete suggestions for appropriate countermeasures to solve problems.

[1407] Generation and notification of proposed solutions to the problem.

[1408] The server generates proposed solutions for the listed issues, incorporates sentiment information, and then notifies the user.

[1409] The proposed solutions include optimal solutions based on past data and knowledge bases. Solutions that also take emotional information into account are more likely to be accepted by users.

[1410] Progress reports and sharing with upper management

[1411] The server periodically retrieves the latest progress information from the database and updates the project plan.

[1412] Updated project plans and progress reports are automatically sent to higher levels of the system.

[1413] This allows upper management to always have an accurate grasp of the latest project status.

[1414] Addressing urgent issues

[1415] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[1416] Real-time notifications allow for a quick response to urgent issues.

[1417] Specific example

[1418] Examples of meeting recording and transcription

[1419] User: "The goal of this project is to improve operational efficiency by revamping our customer management system."

[1420] Device: Recorded the meeting audio and transcribed it as, "The objective of this project is to revamp the customer management system to improve operational efficiency."

[1421] Server: Receives text data, analyzes it, and adds it to the project plan as "Purpose: Renewal of customer management system" and "Goal: Improve operational efficiency."

[1422] Examples of progress reporting and issue management

[1423] User: "Today we started migrating customer data, but we encountered problems due to data format mismatches. It's very stressful."

[1424] Terminal: Submit data via input form.

[1425] Server: Updates progress and adds "Issue: Data format mismatch" to the list. The emotion engine detects that "stress" is high, suggests "Create a data conversion script" as a solution, and notifies the user.

[1426] By using this system, project management becomes more efficient and effective, leading to increased stakeholder satisfaction and a higher project success rate.

[1427] The following describes the processing flow.

[1428] Processing steps

[1429] 1. Initial setup and project plan generation

[1430] Step 1:

[1431] The user initiates a meeting and shares the objectives, overview, and schedule of the new project with stakeholders.

[1432] Step 2:

[1433] The device records meeting audio in real time and converts that audio data into text data.

[1434] Specific example: A microphone or recording device in a conference room acquires audio data, and speech recognition software converts it into text.

[1435] Step 3:

[1436] The terminal sends the converted text data to the server.

[1437] Specific example: A text-based data file is uploaded to a server via the internet.

[1438] Step 4:

[1439] The server analyzes the received text data using natural language processing (NLP) techniques to extract project objectives, overview, schedule, milestones, resource requirements, and other relevant information from the conversation.

[1440] Specific example: An NLP engine analyzes text data to identify important keywords and phrases.

[1441] Step 5:

[1442] The server generates an initial project plan based on the analysis results and automatically shares that plan with the relevant parties.

[1443] Specific example: A project plan is formatted and sent to stakeholders via an email program.

[1444] 2. Daily progress and task management

[1445] Step 6:

[1446] Users input their daily progress and challenges as daily reports and submit them to the system.

[1447] Specific example: Enter progress information and any problems encountered into a dedicated input form.

[1448] Step 7:

[1449] The terminal sends the daily report data to the server.

[1450] Specific example: When you press the submit button on the form, the entered data is uploaded to the server.

[1451] Step 8:

[1452] The server receives the daily report data and performs analysis. Based on the analysis results, it updates the project progress and lists any newly arising issues.

[1453] Specific example: The analysis engine analyzes daily report data and saves progress and issues to a database.

[1454] Step 9:

[1455] The server generates proposed solutions for the issues it has listed and notifies the user of these solutions.

[1456] Specific example: An AI algorithm generates the optimal solution and proposes it to the user via email notification.

[1457] 3. Emotional analysis through the introduction of an emotion engine

[1458] Step 10:

[1459] The server has an emotion engine built in that analyzes user emotions from meeting audio and daily report data.

[1460] Specific example: Extract emotional indicators such as "stress" and "satisfaction level" from meeting audio and text-based daily reports.

[1461] Step 11:

[1462] The server incorporates the analysis results into the project plan and generates progress updates and proposed solutions to challenges, taking emotional information into account.

[1463] Specific example: For users experiencing high stress levels, we suggest solutions such as "recommending rest" or "enhancing support."

[1464] Step 12:

[1465] The server notifies relevant parties of emotional information, facilitating smooth communication among project members.

[1466] Specific example: Display emotional information on a dashboard so that everyone can share the situation in real time.

[1467] 4. Progress reports and sharing with upper management

[1468] Step 13:

[1469] The server periodically retrieves the latest project progress information from the database and updates the project plan.

[1470] Specific example: Cloning jobs are executed regularly according to schedule, and the latest data is reflected in the project plan.

[1471] Step 14:

[1472] The server automatically sends updated project plans and progress reports to higher levels.

[1473] Specific example: A program sends an email containing a report summarizing the latest progress.

[1474] 5. Addressing urgent issues

[1475] Step 15:

[1476] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[1477] Specific example: When an urgent issue arises, the notification system sends real-time alerts to relevant parties and proposes solutions that take emotional information into account.

[1478] summary

[1479] Through these steps, the project management system operates efficiently through the coordination of users, terminals, and servers, effectively supporting project planning, progress management, and issue resolution. By introducing an emotion engine, emotional information is reflected in project plans and solutions, further improving the project's success rate.

[1480] (Example 2)

[1481] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1482] Traditional project management systems lacked sufficient automation in areas such as recording and transcribing meeting audio, automatically generating project plans, and managing progress and issues. Furthermore, they struggled to provide solutions that considered the feelings of stakeholders. As a result, accurately understanding project progress and efficiently resolving issues was difficult. Moreover, the inability to respond quickly to urgent issues contributed to a lower project success rate.

[1483] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1484] In this invention, the server includes means for recording the voices of stakeholders during a meeting and converting them into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for sharing the project plan with stakeholders; means for providing an interface for inputting daily progress and issues; means for transmitting the progress and issue data to a central processing unit; means for analyzing the progress and issue data, updating the progress, and generating an issue list and proposed solutions; means for notifying stakeholders of the issue list and proposed solutions; means for automatically generating a project progress report and transmitting it to a higher level; means for analyzing emotions from the voice and text data; and means for reflecting the emotion analysis results in the project plan and generating countermeasures that take emotional information into account. This makes project management more efficient and effective, and improves stakeholder satisfaction and the success rate of the project.

[1485] A "meeting" is a place where stakeholders gather to discuss the project's plan, progress, and challenges.

[1486] "Audio" refers to data that includes the words and statements made by participants during a meeting.

[1487] "Text data" refers to information obtained by converting speech into written text.

[1488] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate human language.

[1489] A "project plan" is a document that outlines the project's objectives, overview, schedule, milestones, resource requirements, and other relevant information.

[1490] "Stakeholders" refers to all people involved in the project.

[1491] An "interface" refers to the means by which users input their daily progress and tasks into a system.

[1492] A "central processing unit" refers to a device that processes data for the entire system.

[1493] A "task list" is a document that lists all unresolved issues and tasks within a project.

[1494] A "proposed solution" is a proposal that outlines solutions to a problem.

[1495] A "progress report" is a document that reports on the current progress of a project.

[1496] "Upper management" refers to the people or organizations in a position to oversee and manage the entire project.

[1497] "Emotions" refer to information about the psychological state and feelings of those involved.

[1498] "Emotional analysis" refers to the process of identifying a user's emotions by analyzing text data and audio data.

[1499] "System" refers to a set of computing resources and software that possess the aforementioned functions.

[1500] "Updating the progress status" refers to the act of keeping the project's progress information up to date.

[1501] "To notify" means to convey information to the relevant parties.

[1502] This invention is a system that combines an emotion engine with a project management system. The system includes recording and transcribing meeting audio, analyzing text data, automatically generating and sharing project plans, managing daily progress and issues, sentiment analysis, and generating and notifying solutions to issues.

[1503] Hardware and software usage

[1504] The user initiates project planning meetings and progress meetings.

[1505] The terminal records meeting audio using the conference room's microphone or recording device, and converts the audio data into text using the Google Cloud Speech-to-Text API. It also utilizes an internet connection for data transmission. The recorded audio is transcribed in real time. All statements made during the meeting are recorded as text and used for subsequent processing.

[1506] Processing flow

[1507] The terminal sends the generated text data to the server.

[1508] The server receives text data and performs analysis using natural language processing (NLP) techniques. Here, OpenAI's GPT-4 model is used for text analysis, automatically extracting project objectives, overview, schedule, milestones, resource requirements, etc., and generating a project plan. The generated project plan is automatically shared with stakeholders and made accessible to everyone.

[1509] Progress management and issue management

[1510] Users use an interface to input daily progress and issues into the system as daily reports. The terminal sends the entered daily report data to the server. The server receives the daily report data, analyzes it, and automatically updates the project progress and any issues that have arisen. Programming languages ​​such as Python are used for this analysis.

[1511] Emotion analysis and response plan generation

[1512] The emotion engine built into the server analyzes user emotions from meeting audio and daily report data. For example, it uses the Sentiment Analysis API to detect emotions within text. The server then incorporates these emotion analysis results into project plans, generating progress reports and countermeasures for issues that take emotional information into account.

[1513] The emotion engine can extract information such as "stress is increasing" or "satisfaction is high" from user statements and input. This allows for early detection of project risks and the concrete proposal of appropriate countermeasures for problem solving.

[1514] Notification of proposed response and emergency response

[1515] The server notifies stakeholders of the generated list of issues and proposed solutions. These solutions include optimal solutions based on historical data and a knowledge base. Sentimental information is also taken into account, resulting in solutions that are more likely to be accepted by users.

[1516] The server periodically retrieves project progress reports from the database and updates the project plan. The updated plan and progress reports are automatically sent to higher levels, allowing them to always stay informed about the latest project status.

[1517] In the event of an urgent issue, the server immediately notifies relevant parties and proposes solutions that take emotional information into consideration. This real-time notification enables a rapid response.

[1518] Examples of specific actions

[1519] User: "The goal of this project is to revamp our customer management system to improve operational efficiency."

[1520] Terminal: Records meeting audio, transcribes it into text, and translates it as, "The objective of this project is to improve operational efficiency by revamping the customer management system."

[1521] Server: Receives and analyzes text data, and adds it to the project plan with the objectives "Renewal of customer management system" and the goal "Improvement of operational efficiency."

[1522] Example of a prompt

[1523] "Automatically generate a project plan from the meeting audio. The following is a transcript of the meeting: 'The objective of this project is to improve operational efficiency by revamping the customer management system.'"

[1524] By using this system, project management is expected to be more efficient and effective, leading to increased stakeholder satisfaction and a higher project success rate.

[1525] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1526] Step 1:

[1527] The user initiates project planning meetings and progress meetings.

[1528] Input: Conference audio

[1529] Output: Recorded audio data

[1530] Specific action: The user starts a meeting and makes a statement. That statement is recorded as audio data by the recording device.

[1531] Step 2:

[1532] The device converts recorded meeting audio into text data in real time using the Google Cloud Speech-to-Text API.

[1533] Input: Recorded audio data

[1534] Output: Text converted to character data

[1535] Specific operation: The recording device sends the recorded audio data to the Google Cloud Speech-to-Text API, and the data, converted to text with high accuracy, is returned to the device.

[1536] Step 3:

[1537] The terminal sends text data to the server.

[1538] Input: Text data

[1539] Output: Text data sent to the server

[1540] Specific operation: Text data is sent from the terminal to the server. The data is transmitted in real time using a network connection and received by the server.

[1541] Step 4:

[1542] The server analyzes the received text data using Natural Language Processing (NLP) technology and automatically generates a project plan.

[1543] Input: Text data sent to the server

[1544] Output: Project plan generated based on the analyzed information

[1545] Specific operation: The server analyzes text data, uses the OpenAI GPT-4 model to extract project objectives, overview, schedule, resource requirements, etc., and automatically generates a project plan.

[1546] Step 5:

[1547] The server automatically shares the generated project plan with relevant parties.

[1548] Input: Generated project plan

[1549] Output: Project plan shared with stakeholders

[1550] Specific operation: The server sends the generated project plan to relevant parties' email addresses or cloud systems, making it accessible to everyone.

[1551] Step 6:

[1552] Users input their daily progress and challenges into the system as daily reports.

[1553] Input: Progress status and details of the issues

[1554] Output: Daily report data

[1555] Specific operation: Users input their daily progress and challenges using input forms provided in the system, and this data is saved in the system as daily report data.

[1556] Step 7:

[1557] The terminal sends the daily report data to the server.

[1558] Input: Daily report data

[1559] Output: Daily report data sent to the server

[1560] Specific operation: Daily report data is sent from the terminal to the server. Daily report data is sent over the network and received by the server.

[1561] Step 8:

[1562] The server analyzes the daily report data and automatically updates the project progress and issue list.

[1563] Input: Daily report data sent to the server

[1564] Output: Updated progress and issue list

[1565] Specific operation: The server analyzes the daily report data and updates the progress status and task list using Python or similar tools.

[1566] Step 9:

[1567] The emotion engine built into the server analyzes the user's emotions from daily report data and meeting audio.

[1568] Input: Daily report data and meeting audio data

[1569] Output: User sentiment analysis results

[1570] Specific operation: The emotion engine on the server analyzes the data using the Sentiment Analysis API and extracts emotional information (e.g., "stress is increasing," "satisfaction is high").

[1571] Step 10:

[1572] The server incorporates the sentiment analysis results into the project plan and generates response proposals that take sentiment information into account.

[1573] Input: Sentiment analysis results

[1574] Output: Project plan and response plan reflecting emotional information

[1575] Specific operation: The server updates the project plan based on the sentiment analysis results and generates countermeasures that take sentiment information into account.

[1576] Step 11:

[1577] The server will notify relevant parties of the listed issues and proposed solutions.

[1578] Input: List of issues and proposed solutions

[1579] Output: List of issues and proposed solutions notified to stakeholders.

[1580] Specific operation: The server notifies relevant parties of the issue list and proposed solutions, and sends them via email or messaging system.

[1581] Step 12:

[1582] The server periodically retrieves the latest progress information from the database and updates the project plan.

[1583] Input: Progress information in the database

[1584] Output: Updated project plan

[1585] Specific operation: The server queries the database to retrieve the latest progress information and update the project plan.

[1586] Step 13:

[1587] The server automatically sends updated project plans and progress reports to higher levels.

[1588] Input: Updated project plan and progress report

[1589] Output: Progress report sent to the upper layer

[1590] Specific operation: The server sends updated project plans and progress reports to higher levels via email or cloud systems.

[1591] Step 14:

[1592] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[1593] Input: Urgent issue information and sentiment analysis results

[1594] Output: Urgent issues and solutions notified to stakeholders

[1595] Specific operation: The server detects urgent issues, considers solutions that take emotional information into account, and notifies relevant parties in real time.

[1596] (Application Example 2)

[1597] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1598] Traditional project management systems often manage progress and issues without considering the feelings of stakeholders, leading to decreased stakeholder satisfaction and a lower project success rate. Furthermore, the lack of real-time notifications for urgent issues and analysis results made rapid response difficult.

[1599] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1600] In this invention, the server includes means for recording meeting audio of stakeholders during the project and converting it into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for analyzing the user's emotions using an emotion engine and reflecting the analysis results in the project plan and progress reports; and means for notifying the user of the progress status, issues, and emotion analysis results in real time using a smart device. This enables effective project management that takes into account the emotions of stakeholders, thereby improving the project success rate and stakeholder satisfaction.

[1601] A "project plan" is a document that outlines the project's objectives, overview, schedule, milestones, resource requirements, and other relevant information.

[1602] "Text data" refers to a data format in which audio data is converted into written text.

[1603] "Natural language processing technology" refers to technologies that enable computers to understand, analyze, and generate human language.

[1604] An "emotion engine" is a system that analyzes a user's emotions from text and audio data.

[1605] "Progress status" refers to information indicating the degree of progress of a project.

[1606] "Issue data" refers to information about problems that arise during the progress of a project and matters that require attention.

[1607] A "server" is a computer system that performs specific services or data processing over a network.

[1608] A "smart device" is a portable electronic device with internet connectivity and typically possesses advanced computing capabilities.

[1609] An "interface" is the window or means through which a user interacts with a system.

[1610] "Notification" refers to a means or action used to inform a user of specific information.

[1611] "Real-time" means that information is generated and processed almost instantly.

[1612] "Upper management" refers to managers and leaders who are responsible for overseeing the progress and results of a project.

[1613] An "urgent issue" is a serious problem or obstacle that requires immediate attention.

[1614] A "solution plan" refers to a proposed solution or proposal for a problem that has arisen.

[1615] This invention relates to a system that combines an emotion engine with a project management system to more effectively manage project progress and issues. Specifically, it aims to increase the success rate of projects by analyzing the emotions of stakeholders and reflecting them in project plans and progress reports.

[1616] System Configuration

[1617] The system consists of the following main components. These components are implemented through a combination of hardware and software.

[1618] Recording and transcribing meeting audio.

[1619] A device (such as a smart device or a dedicated recording device) records project planning meetings and progress meetings. The recorded audio data is converted into text data using speech recognition technology. This text data is then used for subsequent processing.

[1620] Text data analysis and project plan generation

[1621] The server receives text data sent from the terminal. This text data is analyzed using natural language processing (NLP) technology to automatically extract the project's objectives, overview, schedule, milestones, resource requirements, etc., from the conversation content, and generates a project plan. The generated project plan is then shared with stakeholders.

[1622] Daily progress reporting and task management

[1623] Users input daily progress and issues into the system through an interface. This data is sent from the terminal to the server. The server analyzes the received data and automatically updates the project progress and any issues that have arisen.

[1624] Emotional analysis using an emotion engine

[1625] The server has an emotion engine built in that analyzes user emotions from meeting audio and daily report data. The server incorporates the analysis results into the project plan and generates progress reports and proposed solutions to issues, taking emotional information into account.

[1626] Generation and notification of proposed solutions to the problem.

[1627] The server generates proposed solutions for the listed issues, incorporating sentiment information before notifying the user. These solutions include optimal solutions based on historical data and a knowledge base.

[1628] Addressing urgent issues

[1629] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[1630] Hardware and software to use

[1631] Hardware: Smart devices (smart glasses, smartphones), conference room recording equipment, servers

[1632] Software: Python, speech_recognition library, transformers library, TextBlob library

[1633] Processing flow

[1634] The system begins by recording meeting audio and transcribing it into text. The text data is then analyzed using natural language processing techniques to generate project plans and progress reports. An emotion engine analyzes user emotions and incorporates the results into the project plans and progress reports. It also provides solutions to issues and notifications for emergencies.

[1635] Examples of specific cases and prompt statements

[1636] Specific example:

[1637] Voice input: "Today's progress was delayed due to a machine malfunction. I'm feeling very stressed."

[1638] Text: "Today's progress was delayed due to a machine malfunction. I'm feeling very stressed."

[1639] Sentiment analysis result: "Negative, 0.85"

[1640] Notification: "Issue: Machine malfunction\nSolution: Call a repair technician and have the malfunctioning machine repaired. Taking a break is recommended."

[1641] Example of a prompt:

[1642] "Analyze the emotion in this text and respond with either positive or negative."

[1643] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1644] Step 1:

[1645] The device records the meeting audio.

[1646] Input: Audio from the project meeting

[1647] Operation: The device (smart device or recording device) records the meeting audio.

[1648] Output: Recorded audio data

[1649] Step 2:

[1650] The device converts the recorded audio data into text data.

[1651] Input: Recorded audio data

[1652] Operation: Converts audio data into text data using speech recognition technology (Google Speech Recognition API).

[1653] Output: Text data

[1654] Step 3:

[1655] The terminal sends text data to the server.

[1656] Input: Text data

[1657] Operation: Sends text data generated by the terminal to the server.

[1658] Output: Text data is sent to the server.

[1659] Step 4:

[1660] The server analyzes the received text data using natural language processing technology and automatically generates a project plan.

[1661] Input: Text data

[1662] Operation: The server uses natural language processing (NLP) techniques to analyze text data and extract project objectives, overview, schedule, milestones, and resource requirements.

[1663] Output: Automated project plan

[1664] Step 5:

[1665] The server shares the project plan with the relevant parties.

[1666] Input: Automated project plan

[1667] Operation: The server notifies and shares the project plan with relevant parties.

[1668] Output: Project plan shared with stakeholders

[1669] Step 6:

[1670] Users input their daily progress and challenges through the interface.

[1671] Input: Information on progress and issues

[1672] Operation: The user inputs progress and issue information into the system interface.

[1673] Output: Progress and issue data entered by the user.

[1674] Step 7:

[1675] The terminal sends daily report data to the server.

[1676] Input: Progress and issue data entered by the user.

[1677] Operation: The device sends progress and task data to the server.

[1678] Output: Daily report data sent to the server

[1679] Step 8:

[1680] The server analyzes the daily report data it receives and automatically updates the project's progress and issues.

[1681] Input: Daily report data

[1682] Operation: The server analyzes progress and issues, updates progress, and generates a list of issues.

[1683] Output: Updated progress and issue list

[1684] Step 9:

[1685] The server uses an emotion engine to analyze the user's emotions and reflects the analysis results in project plans and progress reports.

[1686] Input: Text data and daily report data

[1687] Operation: The server uses an emotion engine to analyze the emotions in the data and reflects the results in the project plan and progress report.

[1688] Output: Project plan and progress report reflecting emotional information

[1689] Step 10:

[1690] The server generates proposed solutions to the problem, incorporates emotional information, and then notifies the user.

[1691] Input: Task list and sentiment analysis results

[1692] Operation: The server generates proposed solutions to the problem based on historical data and a knowledge base, and notifies the user, taking sentiment information into account.

[1693] Output: Proposed course of action notified to the user

[1694] Step 11:

[1695] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[1696] Input: Information on the occurrence of urgent issues and sentiment analysis results

[1697] Operation: The server detects urgent issues, notifies relevant parties, and proposes solutions that take emotional information into account.

[1698] Output: Urgent issues and solutions notified to stakeholders

[1699] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1700] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1701] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1702] [Fourth Embodiment]

[1703] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1704] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1705] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1706] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1707] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1708] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1709] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1710] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1711] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1712] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1713] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1714] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1715] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1716] Modes for carrying out the invention

[1717] This invention relates to a system for efficiently managing projects to build new systems, improving the efficiency of project progress by automating the content of meetings among stakeholders, daily progress reports, and issue management.

[1718] System Components and Their Functions

[1719] Recording and transcribing meeting audio.

[1720] The user begins project planning meetings and progress meetings.

[1721] The terminals (microphones and recording devices in the conference room) acquire the conference audio and convert it into text data in real time.

[1722] All statements made during the meeting are converted into text data and used for subsequent processing. At this stage, speech recognition technology is applied to perform highly accurate speech-to-text conversion.

[1723] Text data analysis and project plan generation

[1724] The terminal sends the generated text data to the server.

[1725] The server analyzes text data using natural language processing (NLP) techniques. It automatically extracts the project's objectives, overview, schedule, milestones, and required resources, and generates a project plan.

[1726] The generated project plan is automatically shared with stakeholders, allowing each project member to review it.

[1727] Daily progress reporting and task management

[1728] Users input daily work reports, progress updates, and any issues that arise as daily reports.

[1729] The terminal sends this input data to the server.

[1730] When the server receives the daily report data, it immediately begins analysis. The analysis results are saved in the database, and the project progress and issue list are updated accordingly.

[1731] Generation and notification of proposed solutions to the problem.

[1732] The server lists newly arising issues and generates proposed solutions. It automatically suggests the optimal solution based on historical data and a knowledge base.

[1733] The generated response plan is notified to the user via their device, prompting them to take the necessary action.

[1734] Progress reports and sharing with upper management

[1735] The server periodically retrieves the latest progress information from the database and updates the project plan. The updated plan is automatically sent as a report to higher levels.

[1736] The server provides this progress report to higher levels in the form of email or dashboards, allowing them to accurately understand the project's current status.

[1737] Addressing urgent issues

[1738] If the server detects an urgent issue, it will immediately notify the relevant parties. The notification will also include specific solutions.

[1739] This notification will be communicated to relevant parties in real time to facilitate a swift response.

[1740] Specific example

[1741] Examples of meeting recording and transcription

[1742] User: "The goal of this project is to improve operational efficiency by revamping our customer management system."

[1743] Device: Recorded the meeting audio and transcribed it as, "The objective of this project is to revamp the customer management system to improve operational efficiency."

[1744] Server: Receives text data, analyzes it, and adds it to the project plan as "Purpose: Renewal of customer management system" and "Goal: Improve operational efficiency."

[1745] Examples of progress reporting and issue management

[1746] User: "Today's progress: 40% of customer data migration completed. There are issues with data format mismatches."

[1747] Terminal: Submit data via input form.

[1748] Server: Updated progress and added "Issue: Data format mismatch" to the list. Proposed "Create a data conversion script" as a solution and notified the user.

[1749] In this way, a system is provided that significantly improves the efficiency of project management by automatically analyzing and managing the statements and daily reports of stakeholders.

[1750] The following describes the processing flow.

[1751] Processing steps

[1752] 1. Initial setup and project plan generation

[1753] Step 1:

[1754] The user initiates a meeting and shares the objectives, overview, and schedule of the new project with stakeholders.

[1755] Step 2:

[1756] The device records meeting audio in real time and converts that audio data into text data.

[1757] Specific example: A microphone or recording device in a conference room acquires audio data, and speech recognition software converts it into text.

[1758] Step 3:

[1759] The terminal sends the converted text data to the server.

[1760] Specific example: A text-based data file is uploaded to a server via the internet.

[1761] Step 4:

[1762] The server analyzes the received text data using natural language processing (NLP) techniques to extract project objectives, overview, schedule, milestones, resource requirements, and other relevant information from the conversation.

[1763] Specific example: An NLP engine analyzes text data to identify important keywords and phrases.

[1764] Step 5:

[1765] The server generates an initial project plan based on the analysis results and automatically shares that plan with the relevant parties.

[1766] Specific example: A project plan is formatted and sent to stakeholders via an email program.

[1767] 2. Daily progress and task management

[1768] Step 6:

[1769] Users input their daily progress and challenges as daily reports and submit them to the system.

[1770] Specific example: Enter progress information and any problems encountered into a dedicated input form.

[1771] Step 7:

[1772] The terminal sends the daily report data to the server.

[1773] Specific example: When you press the submit button on the form, the entered data is uploaded to the server.

[1774] Step 8:

[1775] The server receives the daily report data and performs analysis. Based on the analysis results, it updates the project progress and lists any newly arising issues.

[1776] Specific example: The analysis engine analyzes daily report data and saves progress and issues to a database.

[1777] Step 9:

[1778] The server generates proposed solutions for the issues it has listed and notifies the user of these solutions.

[1779] Specific example: An AI algorithm generates the optimal solution and proposes it to the user via email notification.

[1780] 3. Progress reports and sharing of issues

[1781] Step 10:

[1782] The server periodically retrieves the latest project progress information from the database and updates the project plan.

[1783] Specific example: The clone job is executed according to schedule, and the latest data is used.

[1784] Step 11:

[1785] The server automatically sends the updated project plan as a report to the higher layer.

[1786] Specific example: A program sends an email containing a report summarizing the latest progress.

[1787] Step 12:

[1788] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes a solution.

[1789] Specific example: When an urgent issue arises, the notification system sends real-time alerts to relevant parties and proposes solutions.

[1790] summary

[1791] Through these steps, the project management system operates efficiently through the coordination of users, terminals, and servers, effectively supporting project planning, progress management, and issue resolution. Including specific program processing flows clarifies how the system will be implemented.

[1792] (Example 1)

[1793] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1794] In project management, it is crucial for stakeholders to communicate efficiently, share progress, and resolve issues quickly. However, manually recording project meeting audio, converting it to text, and incorporating it into project plans is a laborious process. Furthermore, delays in reporting progress and issues can impact the overall project schedule. Moreover, the inability to quickly address urgent issues increases project risk. Thus, streamlining the collection, sharing, and updating of information is a major challenge in project management.

[1795] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1796] This invention includes a server that transcribes meeting audio into text in real time and transmits it to the server immediately, a server that notifies relevant parties in real time when it detects an urgent issue and presents specific solutions, and a server that periodically updates the project plan and automatically generates the latest progress status and issue list. This enables rapid transcription and analysis of meeting content, realizing a project management system that can respond quickly to urgent issues.

[1797] "Meeting audio" refers to the audio generated when stakeholders discuss and exchange opinions about a project during its progress.

[1798] "Text data" refers to data obtained by converting recorded meeting audio into text information using speech recognition technology.

[1799] "Natural language processing technology" is a general term for technologies that enable computers to understand, interpret, and generate human language, and is particularly used for analyzing text data.

[1800] A "project plan" is a document that includes information such as the project's objectives, overview, schedule, milestones, and required resources.

[1801] An "interface" is an input method used by users to enter data such as daily work reports, progress status, and issues that have arisen.

[1802] A "server" is a central computer system that receives text data and daily report data sent from each terminal and performs processing such as analysis, storage, and notification.

[1803] "Progress status" refers to information that indicates the current stage of a project's work and how far along it is.

[1804] "Issue data" refers to information that records problems and obstacles that occur during the progress of a project.

[1805] A "list of issues" refers to a compilation of all issues that arose during the project's progress.

[1806] A "proposed solution" refers to the proposed solutions or action plans for the listed issues.

[1807] "Upper management" refers to senior personnel such as managers and executives who receive project progress reports.

[1808] "Real-time" refers to a timeframe in which processing and notifications are carried out immediately the moment an event occurs.

[1809] This invention relates to a system for efficiently managing projects to build new systems. It improves the efficiency of project progress by automating the content of meetings among stakeholders, daily progress reports, and issue management.

[1810] First, users begin project planning and progress meetings. Terminals (such as microphones or recording devices in the meeting room) capture the meeting audio and convert it into text data in real time. At this stage, speech recognition technology is applied, specifically using speech recognition services such as "Google Speech-to-Text API" or "IBM Watson Speech to Text".

[1811] The acquired text data is sent from the terminal to the server. The server analyzes this text data using natural language processing techniques (such as Python's NLTK library or Google Cloud Natural Language API) and automatically extracts the project's objectives, overview, schedule, milestones, and required resources. Based on the extracted information, the server generates a project plan and automatically shares it with stakeholders.

[1812] Users also input daily work reports, progress updates, and issues encountered as daily reports. This daily report data is sent from the terminal to the server. Upon receiving the daily report data, the server immediately begins analysis and saves the analysis results to a database, thereby updating the project progress and issue list.

[1813] The server lists newly arising issues and generates optimal solutions. This process utilizes Elasticsearch and Python's machine learning module (scikit-learn) to generate the best solutions from historical data and a knowledge base. The generated solutions are then communicated to the user via their terminal, and they are required to take action.

[1814] Furthermore, the server periodically retrieves the latest progress information from the database and updates the project plan. This updated plan is automatically sent to higher levels. Reports to higher levels are provided via email or dashboard tools such as Tableau and Power BI.

[1815] If the server detects an urgent issue, it will immediately notify relevant parties in real time and propose specific solutions. This notification will be sent in real time using the Slack API or Twilio API.

[1816] Specific examples include the following:

[1817] User: "The goal of this project is to improve operational efficiency by revamping our customer management system."

[1818] Terminal: Recorded the meeting audio and transcribed it as, "The objective of this project is to revamp the customer management system to improve operational efficiency."

[1819] Server: Receives text data, analyzes it, and adds it to the project plan as "Purpose: Renewal of the customer management system" and "Goal: Improvement of operational efficiency."

[1820] Furthermore, the following specific examples are given for progress reporting and issue management.

[1821] User: "Today's progress: 40% of customer data migration completed. There are issues with data format mismatches."

[1822] Terminal: Submit data from the input form.

[1823] Server: Updated progress and added "Issue: Data format mismatch" to the list. Proposed "Create a data conversion script" as a solution and notified the user.

[1824] Examples of prompt statements to input into the generative AI model are as follows:

[1825] "Automatically generate a project plan that includes the project objectives, overview, schedule, milestones, and required resources."

[1826] "Based on past data, please propose the best solution to the data format mismatch issue."

[1827] This system makes it possible to efficiently analyze and manage conversations and daily reports from stakeholders, significantly improving the efficiency of project management.

[1828] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1829] Step 1:

[1830] The user initiates a project planning or progress meeting. The input is the meeting audio, which serves as the starting point for processing.

[1831] The device activates the microphones and recording devices in the conference room and begins capturing conference audio in real time. Specifically, the device starts recording as soon as it recognizes the voice command "start".

[1832] Step 2:

[1833] The terminal sends the acquired meeting audio data to a speech recognition API (e.g., Google Speech-to-Text API). The input is audio data, and the output returned by the API is text data.

[1834] The terminal receives text data and saves the meeting's statements as text information. Specifically, the audio data is converted to text, such as "The objective of this project is to improve operational efficiency by revamping the customer management system."

[1835] Step 3:

[1836] The terminal sends the generated text data to the server. The input is text data, which is sent to the server for further analysis.

[1837] The server receives and stores the text data. This prepares it for creating the project plan.

[1838] Step 4:

[1839] The server analyzes the received text data using natural language processing techniques (e.g., Python's NLTK library). The input is text data, and the output is extracted information such as the project's objectives, overview, schedule, milestones, and required resources.

[1840] The server automatically generates a project plan based on these analysis results. Specifically, details such as "Objective: Renewal of the customer management system" and "Goal: Improvement of operational efficiency" are added to the plan.

[1841] Step 5:

[1842] The server automatically shares the generated project plan with stakeholders. The input is the automatically generated plan, and the output is the information shared with stakeholders. Sharing is done via email, cloud storage, and project management tools.

[1843] Step 6:

[1844] Users input daily work reports, progress updates, and any issues encountered as daily reports. These inputs consist of user work reports and progress data, which are then recorded on the terminal as daily reports.

[1845] The terminal sends this daily report data to the server. The input here is the daily report data, and the output is the data sent to the server.

[1846] Step 7:

[1847] The server immediately begins analyzing the daily report data upon receiving it and saves the analysis results to the database. The input is the daily report data, and the output is the analysis results and updated database information.

[1848] The server updates the project progress and issue list. Specifically, it updates information such as "Progress: 40% of customer data migration complete" and "Issues: Data format mismatch."

[1849] Step 8:

[1850] The server lists newly arising issues and generates optimal solutions. The input is issue data, and the output is the proposed solutions. Elasticsearch and Python's machine learning module (scikit-learn) are used here.

[1851] The server sends the generated solution to the terminal and notifies the user. Specifically, the user receives a notification stating "Problem: Data format mismatch" and "Suggested solution: Create a data conversion script."

[1852] Step 9:

[1853] The server periodically retrieves the latest progress information from the database and updates the project plan. The input is the latest progress information, and the output is the updated project plan.

[1854] The server automatically sends updated plans to higher levels. Specifically, it provides information to higher levels via email or dashboard tools (Tableau, Power BI).

[1855] Step 10:

[1856] When the server detects an urgent issue, it immediately notifies relevant parties in real time and provides concrete solutions. The input is data on the urgent issue, and the output is the notification and solution.

[1857] Notifications are sent to all relevant parties via the terminal. Specifically, if a "system failure" occurs as an urgent issue, relevant parties will be immediately notified of a solution such as "system restart."

[1858] (Application Example 1)

[1859] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1860] In conventional project management systems, managing meeting content, progress reports, and issues among stakeholders is done manually, requiring a significant amount of time and effort, often resulting in project delays. Furthermore, even in factory settings, manual data entry and management are necessary, hindering efficient project progress. A system is needed to address these challenges and improve the efficiency of project management.

[1861] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1862] In this invention, the server includes means for recording meeting audio conducted by stakeholders during the progress of a project and converting it into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for sharing the project plan with stakeholders; means for providing an interface for inputting daily progress and issues; means for transmitting the progress and issue data to the server; means for analyzing the progress and issue data, updating the progress, and generating an issue list and proposed solutions; means for notifying stakeholders of the issue list and proposed solutions; means for automatically generating a project progress report and transmitting it to a higher level; means for recording meeting audio in real time via a factory robot and converting it into text data; means for transmitting the text data to the server, analyzing it using natural language processing technology, and automatically generating a project plan; and means for automatically inputting the progress and issue data via a factory robot and transmitting it to the server. This makes it possible to automate the work of stakeholders and improve the efficiency of project management.

[1863] "Meeting audio" refers to audio recordings made during meetings held by stakeholders in the course of a project.

[1864] "Text data" refers to data obtained by converting recorded meeting audio into text information.

[1865] "Natural language processing technology" is a technology that interprets and analyzes human language.

[1866] A "project plan" is a document that outlines the project's objectives, overview, schedule, milestones, and required resources.

[1867] "Stakeholders" refers to all members, managers, and other interested parties involved in the project.

[1868] An "interface" refers to an input device or software that users use to record their daily progress and tasks.

[1869] A "server" is a computing system used for analyzing, storing, and transmitting data.

[1870] A "problem list" is a list of issues that arose during the project's progress.

[1871] A "solution plan" is a proposed solution to a problem that has arisen.

[1872] A "progress report" is a report that describes the progress of a project.

[1873] "Upper management" refers to senior managers who monitor project progress and make decisions.

[1874] A "factory robot" is a machine used in a factory to perform automated tasks.

[1875] "Real-time" refers to performing actions or processing in accordance with real-world time.

[1876] "Analysis" is the process of examining data in detail to clarify its structure and meaning.

[1877] "Automatic generation" refers to the process where a system automatically creates documents or data without human intervention.

[1878] This invention relates to a system for streamlining project management in factories, enabling rapid and accurate project progress by automating meeting audio recording, transcription, progress reporting, and issue management.

[1879] System Configuration

[1880] This system consists of the following main elements:

[1881] Meeting audio and text

[1882] The server records meeting audio using microphones and recording devices mounted on factory robots and converts it into text data in real time. It uses speech recognition technology (for example, Google Cloud Speech-to-Text or Amazon Transcribe).

[1883] Text data analysis and project plan generation

[1884] The server analyzes text data using natural language processing (NLP) technology and automatically extracts the project's objectives, overview, schedule, milestones, and required resources to generate a project plan.

[1885] Progress reporting and issue management

[1886] This system provides an interface for inputting daily progress and challenges via factory robots. The data entered by users is sent to a server, where it is automatically analyzed.

[1887] Generating a list of issues and proposed solutions

[1888] Upon receiving a progress report, the server immediately begins analysis, updates the issue list, and generates proposed solutions. These proposed solutions are then notified to the user.

[1889] Project progress report

[1890] The server periodically retrieves the latest progress information from the database, updates the project plan, and reports the latest progress to higher levels.

[1891] Hardware and software

[1892] Hardware:

[1893] Microphones and recording devices mounted on factory robots

[1894] Server (used as a computing resource)

[1895] software:

[1896] Speech recognition technology (Google Cloud Speech-to-Text, Amazon Transcribe, etc.)

[1897] Natural language processing techniques (such as Python's NLTK library)

[1898] Specific example

[1899] For example, suppose a user makes the following statement in a meeting.

[1900] Example of a prompt:

[1901] "The objective of this project is to improve production efficiency by introducing a new manufacturing line."

[1902] The server records the meeting audio and converts it into text data. Then, it analyzes the text data to automatically extract the following information.

[1903] Objective: To introduce a new manufacturing line.

[1904] Objective: Improve production efficiency

[1905] As a concrete example of progress reporting and issue management, the user will input the following information.

[1906] Example of a prompt:

[1907] "Today's progress: 50% of the equipment installation is complete. A new challenge has arisen: a power supply issue."

[1908] The server updates the progress, adds "Issue: Power supply issues" to the list, generates "Inspect and repair the power supply system" as a suggested solution, and notifies the user.

[1909] Thus, the system of the present invention can automate and efficiently and effectively perform project management in a factory.

[1910] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1911] Step 1:

[1912] A microphone and recording device mounted on the factory robot acquire the conference audio. The microphone records all sounds emitted during the conference and converts them to a digital format in real time. This results in the conference audio being acquired as digital audio data.

[1913] Input: Audio from a meeting

[1914] Output: Digital audio data

[1915] Step 2:

[1916] The acquired digital audio data is sent to the server. The server uses speech recognition technology to convert the digital audio data into text data. Google Cloud Speech-to-Text and Amazon Transcribe are used as speech recognition technologies.

[1917] Input: Digital audio data

[1918] Output: Text data

[1919] Step 3:

[1920] Text data is analyzed by the server using natural language processing (NLP) techniques. This analysis automatically extracts information such as the project's objectives, overview, schedule, and required resources. This then generates an initial project plan.

[1921] Input: Text data

[1922] Output: Project plan

[1923] Step 4:

[1924] The generated project plan is shared with stakeholders via the server. Sharing methods include email, display on a dashboard, and other collaboration tools.

[1925] Input: Project plan

[1926] Output: Sharing with stakeholders

[1927] Step 5:

[1928] Users input daily progress and challenges through factory robots. The interface provides progress input forms and challenge reporting forms, and this data is sent to the server in real time.

[1929] Input: Progress status, issue data

[1930] Output: Progress status and issue data sent to the server

[1931] Step 6:

[1932] The server analyzes the received progress and issue data, updates the progress status, and generates an issue list and proposed solutions. Historical data and knowledge bases are used for data analysis to propose the optimal solution.

[1933] Input: Progress status, issue data

[1934] Output: Updated progress, issue list, proposed solutions

[1935] Step 7:

[1936] The server notifies stakeholders of the generated list of issues and proposed solutions. Notification methods include alerts displayed via factory robots, email, and notifications on the dashboard.

[1937] Input: List of issues, proposed solutions

[1938] Output: Notification to relevant parties

[1939] Step 8:

[1940] The server periodically retrieves the latest progress information from the database and updates the project plan. This ensures that stakeholders are always aware of the latest project status.

[1941] Input: Progress information

[1942] Output: Updated project plan

[1943] Step 9:

[1944] The updated project plan is sent to higher levels as an automatically generated progress report by the server. Methods of transmission include periodic emails, dashboard displays, and automated report generation.

[1945] Input: Updated project plan

[1946] Output: Progress report to higher layers

[1947] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1948] Modes for carrying out the invention

[1949] This invention relates to a system that combines an emotion engine with a project management system to more effectively manage project progress and issues. In particular, it aims to increase the success rate of projects by analyzing the emotions of stakeholders and reflecting them in plans and progress reports.

[1950] System Configuration and Functions

[1951] Recording and transcribing meeting audio.

[1952] The user initiates project planning meetings and progress meetings.

[1953] The terminal (the microphone or recording device in the conference room) records the conference audio and converts that audio data into text data.

[1954] All speech during the meeting is converted into text data in real time and used for subsequent processing. Speech recognition technology is used to perform highly accurate speech-to-text conversion.

[1955] Text data analysis and project plan generation

[1956] The terminal sends the generated text data to the server.

[1957] The server analyzes text data using natural language processing (NLP) technology, automatically extracting project objectives, overview, schedule, milestones, resource requirements, and other information from the conversation to generate a project plan.

[1958] The generated project plan is automatically shared with stakeholders and becomes accessible to everyone.

[1959] Daily progress reporting and task management

[1960] Users input their daily progress and challenges into the system as daily reports.

[1961] The terminal sends the daily report data to the server.

[1962] The server analyzes the received daily report data and automatically updates the project progress and any issues that have arisen.

[1963] Emotional analysis using an emotion engine

[1964] The server has an emotion engine built in that analyzes user emotions from meeting audio and daily report data.

[1965] The server incorporates the analysis results into the project plan and generates progress updates and proposed solutions to challenges, taking emotional information into account.

[1966] The emotion engine analyzes emotional tendencies from user statements and input, extracting information such as "stress is increasing" or "satisfaction is high." This allows for early detection of project risks and concrete suggestions for appropriate countermeasures to solve problems.

[1967] Generation and notification of proposed solutions to the problem.

[1968] The server generates proposed solutions for the listed issues, incorporates sentiment information, and then notifies the user.

[1969] The proposed solutions include optimal solutions based on past data and knowledge bases. Solutions that also take emotional information into account are more likely to be accepted by users.

[1970] Progress reports and sharing with upper management

[1971] The server periodically retrieves the latest progress information from the database and updates the project plan.

[1972] Updated project plans and progress reports are automatically sent to higher levels of the system.

[1973] This allows upper management to always have an accurate grasp of the latest project status.

[1974] Addressing urgent issues

[1975] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[1976] Real-time notifications allow for a quick response to urgent issues.

[1977] Specific example

[1978] Examples of meeting recording and transcription

[1979] User: "The goal of this project is to improve operational efficiency by revamping our customer management system."

[1980] Device: Recorded the meeting audio and transcribed it as, "The objective of this project is to revamp the customer management system to improve operational efficiency."

[1981] Server: Receives text data, analyzes it, and adds it to the project plan as "Purpose: Renewal of customer management system" and "Goal: Improve operational efficiency."

[1982] Examples of progress reporting and issue management

[1983] User: "Today we started migrating customer data, but we encountered problems due to data format mismatches. It's very stressful."

[1984] Terminal: Submit data via input form.

[1985] Server: Updates progress and adds "Issue: Data format mismatch" to the list. The emotion engine detects that "stress" is high, suggests "Create a data conversion script" as a solution, and notifies the user.

[1986] By using this system, project management becomes more efficient and effective, leading to increased stakeholder satisfaction and a higher project success rate.

[1987] The following describes the processing flow.

[1988] Processing steps

[1989] 1. Initial setup and project plan generation

[1990] Step 1:

[1991] The user initiates a meeting and shares the objectives, overview, and schedule of the new project with stakeholders.

[1992] Step 2:

[1993] The device records meeting audio in real time and converts that audio data into text data.

[1994] Specific example: A microphone or recording device in a conference room acquires audio data, and speech recognition software converts it into text.

[1995] Step 3:

[1996] The terminal sends the converted text data to the server.

[1997] Specific example: A text-based data file is uploaded to a server via the internet.

[1998] Step 4:

[1999] The server analyzes the received text data using natural language processing (NLP) techniques to extract project objectives, overview, schedule, milestones, resource requirements, and other relevant information from the conversation.

[2000] Specific example: An NLP engine analyzes text data to identify important keywords and phrases.

[2001] Step 5:

[2002] The server generates an initial project plan based on the analysis results and automatically shares that plan with the relevant parties.

[2003] Specific example: A project plan is formatted and sent to stakeholders via an email program.

[2004] 2. Daily progress and task management

[2005] Step 6:

[2006] Users input their daily progress and challenges as daily reports and submit them to the system.

[2007] Specific example: Enter progress information and any problems encountered into a dedicated input form.

[2008] Step 7:

[2009] The terminal sends the daily report data to the server.

[2010] Specific example: When you press the submit button on the form, the entered data is uploaded to the server.

[2011] Step 8:

[2012] The server receives the daily report data and performs analysis. Based on the analysis results, it updates the project progress and lists any newly arising issues.

[2013] Specific example: The analysis engine analyzes daily report data and saves progress and issues to a database.

[2014] Step 9:

[2015] The server generates proposed solutions for the issues it has listed and notifies the user of these solutions.

[2016] Specific example: An AI algorithm generates the optimal solution and proposes it to the user via email notification.

[2017] 3. Emotional analysis through the introduction of an emotion engine

[2018] Step 10:

[2019] The server has an emotion engine built in that analyzes user emotions from meeting audio and daily report data.

[2020] Specific example: Extract emotional indicators such as "stress" and "satisfaction level" from meeting audio and text-based daily reports.

[2021] Step 11:

[2022] The server incorporates the analysis results into the project plan and generates progress updates and proposed solutions to challenges, taking emotional information into account.

[2023] Specific example: For users experiencing high stress levels, we suggest solutions such as "recommending rest" or "enhancing support."

[2024] Step 12:

[2025] The server notifies relevant parties of emotional information, facilitating smooth communication among project members.

[2026] Specific example: Display emotional information on a dashboard so that everyone can share the situation in real time.

[2027] 4. Progress reports and sharing with upper management

[2028] Step 13:

[2029] The server periodically retrieves the latest project progress information from the database and updates the project plan.

[2030] Specific example: Cloning jobs are executed regularly according to schedule, and the latest data is reflected in the project plan.

[2031] Step 14:

[2032] The server automatically sends updated project plans and progress reports to higher levels.

[2033] Specific example: A program sends an email containing a report summarizing the latest progress.

[2034] 5. Addressing urgent issues

[2035] Step 15:

[2036] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[2037] Specific example: When an urgent issue arises, the notification system sends real-time alerts to relevant parties and proposes solutions that take emotional information into account.

[2038] summary

[2039] Through these steps, the project management system operates efficiently through the coordination of users, terminals, and servers, effectively supporting project planning, progress management, and issue resolution. By introducing an emotion engine, emotional information is reflected in project plans and solutions, further improving the project's success rate.

[2040] (Example 2)

[2041] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2042] Traditional project management systems lacked sufficient automation in areas such as recording and transcribing meeting audio, automatically generating project plans, and managing progress and issues. Furthermore, they struggled to provide solutions that considered the feelings of stakeholders. As a result, accurately understanding project progress and efficiently resolving issues was difficult. Moreover, the inability to respond quickly to urgent issues contributed to a lower project success rate.

[2043] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[2044] In this invention, the server includes means for recording the voices of stakeholders during a meeting and converting them into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for sharing the project plan with stakeholders; means for providing an interface for inputting daily progress and issues; means for transmitting the progress and issue data to a central processing unit; means for analyzing the progress and issue data, updating the progress, and generating an issue list and proposed solutions; means for notifying stakeholders of the issue list and proposed solutions; means for automatically generating a project progress report and transmitting it to a higher level; means for analyzing emotions from the voice and text data; and means for reflecting the emotion analysis results in the project plan and generating countermeasures that take emotional information into account. This makes project management more efficient and effective, and improves stakeholder satisfaction and the success rate of the project.

[2045] A "meeting" is a place where stakeholders gather to discuss the project's plan, progress, and challenges.

[2046] "Audio" refers to data that includes the words and statements made by participants during a meeting.

[2047] "Text data" refers to information obtained by converting speech into written text.

[2048] "Natural language processing technology" refers to the technology that enables computers to understand, analyze, and generate human language.

[2049] A "project plan" is a document that outlines the project's objectives, overview, schedule, milestones, resource requirements, and other relevant information.

[2050] "Stakeholders" refers to all people involved in the project.

[2051] An "interface" refers to the means by which users input their daily progress and tasks into a system.

[2052] A "central processing unit" refers to a device that processes data for the entire system.

[2053] A "task list" is a document that lists all unresolved issues and tasks within a project.

[2054] A "proposed solution" is a proposal that outlines solutions to a problem.

[2055] A "progress report" is a document that reports on the current progress of a project.

[2056] "Upper management" refers to the people or organizations in a position to oversee and manage the entire project.

[2057] "Emotions" refer to information about the psychological state and feelings of those involved.

[2058] "Emotional analysis" refers to the process of identifying a user's emotions by analyzing text data and audio data.

[2059] "System" refers to a set of computing resources and software that possess the aforementioned functions.

[2060] "Updating the progress status" refers to the act of keeping the project's progress information up to date.

[2061] "To notify" means to convey information to the relevant parties.

[2062] This invention is a system that combines an emotion engine with a project management system. The system includes recording and transcribing meeting audio, analyzing text data, automatically generating and sharing project plans, managing daily progress and issues, sentiment analysis, and generating and notifying solutions to issues.

[2063] Hardware and software usage

[2064] The user initiates project planning meetings and progress meetings.

[2065] The terminal records meeting audio using the conference room's microphone or recording device, and converts the audio data into text using the Google Cloud Speech-to-Text API. It also utilizes an internet connection for data transmission. The recorded audio is transcribed in real time. All statements made during the meeting are recorded as text and used for subsequent processing.

[2066] Processing flow

[2067] The terminal sends the generated text data to the server.

[2068] The server receives text data and performs analysis using natural language processing (NLP) techniques. Here, OpenAI's GPT-4 model is used for text analysis, automatically extracting project objectives, overview, schedule, milestones, resource requirements, etc., and generating a project plan. The generated project plan is automatically shared with stakeholders and made accessible to everyone.

[2069] Progress management and issue management

[2070] Users use an interface to input daily progress and issues into the system as daily reports. The terminal sends the entered daily report data to the server. The server receives the daily report data, analyzes it, and automatically updates the project progress and any issues that have arisen. Programming languages ​​such as Python are used for this analysis.

[2071] Emotion analysis and response plan generation

[2072] The emotion engine built into the server analyzes user emotions from meeting audio and daily report data. For example, it uses the Sentiment Analysis API to detect emotions within text. The server then incorporates these emotion analysis results into project plans, generating progress reports and countermeasures for issues that take emotional information into account.

[2073] The emotion engine can extract information such as "stress is increasing" or "satisfaction is high" from user statements and input. This allows for early detection of project risks and the concrete proposal of appropriate countermeasures for problem solving.

[2074] Notification of proposed response and emergency response

[2075] The server notifies stakeholders of the generated list of issues and proposed solutions. These solutions include optimal solutions based on historical data and a knowledge base. Sentimental information is also taken into account, resulting in solutions that are more likely to be accepted by users.

[2076] The server periodically retrieves project progress reports from the database and updates the project plan. The updated plan and progress reports are automatically sent to higher levels, allowing them to always stay informed about the latest project status.

[2077] In the event of an urgent issue, the server immediately notifies relevant parties and proposes solutions that take emotional information into consideration. This real-time notification enables a rapid response.

[2078] Examples of specific actions

[2079] User: "The goal of this project is to revamp our customer management system to improve operational efficiency."

[2080] Terminal: Records meeting audio, transcribes it into text, and translates it as, "The objective of this project is to improve operational efficiency by revamping the customer management system."

[2081] Server: Receives and analyzes text data, and adds it to the project plan with the objectives "Renewal of customer management system" and the goal "Improvement of operational efficiency."

[2082] Example of a prompt

[2083] "Automatically generate a project plan from the meeting audio. The following is a transcript of the meeting: 'The objective of this project is to improve operational efficiency by revamping the customer management system.'"

[2084] By using this system, project management is expected to be more efficient and effective, leading to increased stakeholder satisfaction and a higher project success rate.

[2085] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2086] Step 1:

[2087] The user initiates project planning meetings and progress meetings.

[2088] Input: Conference audio

[2089] Output: Recorded audio data

[2090] Specific action: The user starts a meeting and makes a statement. That statement is recorded as audio data by the recording device.

[2091] Step 2:

[2092] The device converts recorded meeting audio into text data in real time using the Google Cloud Speech-to-Text API.

[2093] Input: Recorded audio data

[2094] Output: Text converted to character data

[2095] Specific operation: The recording device sends the recorded audio data to the Google Cloud Speech-to-Text API, and the data, converted to text with high accuracy, is returned to the device.

[2096] Step 3:

[2097] The terminal sends text data to the server.

[2098] Input: Text data

[2099] Output: Text data sent to the server

[2100] Specific operation: Text data is sent from the terminal to the server. The data is transmitted in real time using a network connection and received by the server.

[2101] Step 4:

[2102] The server analyzes the received text data using Natural Language Processing (NLP) technology and automatically generates a project plan.

[2103] Input: Text data sent to the server

[2104] Output: Project plan generated based on the analyzed information

[2105] Specific operation: The server analyzes text data, uses the OpenAI GPT-4 model to extract project objectives, overview, schedule, resource requirements, etc., and automatically generates a project plan.

[2106] Step 5:

[2107] The server automatically shares the generated project plan with relevant parties.

[2108] Input: Generated project plan

[2109] Output: Project plan shared with stakeholders

[2110] Specific operation: The server sends the generated project plan to relevant parties' email addresses or cloud systems, making it accessible to everyone.

[2111] Step 6:

[2112] Users input their daily progress and challenges into the system as daily reports.

[2113] Input: Progress status and details of the issues

[2114] Output: Daily report data

[2115] Specific operation: Users input their daily progress and challenges using input forms provided in the system, and this data is saved in the system as daily report data.

[2116] Step 7:

[2117] The terminal sends the daily report data to the server.

[2118] Input: Daily report data

[2119] Output: Daily report data sent to the server

[2120] Specific operation: Daily report data is sent from the terminal to the server. Daily report data is sent over the network and received by the server.

[2121] Step 8:

[2122] The server analyzes the daily report data and automatically updates the project progress and issue list.

[2123] Input: Daily report data sent to the server

[2124] Output: Updated progress and issue list

[2125] Specific operation: The server analyzes the daily report data and updates the progress status and task list using Python or similar tools.

[2126] Step 9:

[2127] The emotion engine built into the server analyzes the user's emotions from daily report data and meeting audio.

[2128] Input: Daily report data and meeting audio data

[2129] Output: User sentiment analysis results

[2130] Specific operation: The emotion engine on the server analyzes the data using the Sentiment Analysis API and extracts emotional information (e.g., "stress is increasing," "satisfaction is high").

[2131] Step 10:

[2132] The server incorporates the sentiment analysis results into the project plan and generates response proposals that take sentiment information into account.

[2133] Input: Sentiment analysis results

[2134] Output: Project plan and response plan reflecting emotional information

[2135] Specific operation: The server updates the project plan based on the sentiment analysis results and generates countermeasures that take sentiment information into account.

[2136] Step 11:

[2137] The server will notify relevant parties of the listed issues and proposed solutions.

[2138] Input: List of issues and proposed solutions

[2139] Output: List of issues and proposed solutions notified to stakeholders.

[2140] Specific operation: The server notifies relevant parties of the issue list and proposed solutions, and sends them via email or messaging system.

[2141] Step 12:

[2142] The server periodically retrieves the latest progress information from the database and updates the project plan.

[2143] Input: Progress information in the database

[2144] Output: Updated project plan

[2145] Specific operation: The server queries the database to retrieve the latest progress information and update the project plan.

[2146] Step 13:

[2147] The server automatically sends updated project plans and progress reports to higher levels.

[2148] Input: Updated project plan and progress report

[2149] Output: Progress report sent to the upper layer

[2150] Specific operation: The server sends updated project plans and progress reports to higher levels via email or cloud systems.

[2151] Step 14:

[2152] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[2153] Input: Urgent issue information and sentiment analysis results

[2154] Output: Urgent issues and solutions notified to stakeholders

[2155] Specific operation: The server detects urgent issues, considers solutions that take emotional information into account, and notifies relevant parties in real time.

[2156] (Application Example 2)

[2157] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2158] Traditional project management systems often manage progress and issues without considering the feelings of stakeholders, leading to decreased stakeholder satisfaction and a lower project success rate. Furthermore, the lack of real-time notifications for urgent issues and analysis results made rapid response difficult.

[2159] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[2160] In this invention, the server includes means for recording meeting audio of stakeholders during the project and converting it into text data; means for analyzing the text data using natural language processing technology and automatically generating a project plan; means for analyzing the user's emotions using an emotion engine and reflecting the analysis results in the project plan and progress reports; and means for notifying the user of the progress status, issues, and emotion analysis results in real time using a smart device. This enables effective project management that takes into account the emotions of stakeholders, thereby improving the project success rate and stakeholder satisfaction.

[2161] A "project plan" is a document that outlines the project's objectives, overview, schedule, milestones, resource requirements, and other relevant information.

[2162] "Text data" refers to a data format in which audio data is converted into written text.

[2163] "Natural language processing technology" refers to technologies that enable computers to understand, analyze, and generate human language.

[2164] An "emotion engine" is a system that analyzes a user's emotions from text and audio data.

[2165] "Progress status" refers to information indicating the degree of progress of a project.

[2166] "Issue data" refers to information about problems that arise during the progress of a project and matters that require attention.

[2167] A "server" is a computer system that performs specific services or data processing over a network.

[2168] A "smart device" is a portable electronic device with internet connectivity and typically possesses advanced computing capabilities.

[2169] An "interface" is the window or means through which a user interacts with a system.

[2170] "Notification" refers to a means or action used to inform a user of specific information.

[2171] "Real-time" means that information is generated and processed almost instantly.

[2172] "Upper management" refers to managers and leaders who are responsible for overseeing the progress and results of a project.

[2173] An "urgent issue" is a serious problem or obstacle that requires immediate attention.

[2174] A "solution plan" refers to a proposed solution or proposal for a problem that has arisen.

[2175] This invention relates to a system that combines an emotion engine with a project management system to more effectively manage project progress and issues. Specifically, it aims to increase the success rate of projects by analyzing the emotions of stakeholders and reflecting them in project plans and progress reports.

[2176] System Configuration

[2177] The system consists of the following main components. These components are implemented through a combination of hardware and software.

[2178] Recording and transcribing meeting audio.

[2179] A device (such as a smart device or a dedicated recording device) records project planning meetings and progress meetings. The recorded audio data is converted into text data using speech recognition technology. This text data is then used for subsequent processing.

[2180] Text data analysis and project plan generation

[2181] The server receives text data sent from the terminal. This text data is analyzed using natural language processing (NLP) technology to automatically extract the project's objectives, overview, schedule, milestones, resource requirements, etc., from the conversation content, and generates a project plan. The generated project plan is then shared with stakeholders.

[2182] Daily progress reporting and task management

[2183] Users input daily progress and issues into the system through an interface. This data is sent from the terminal to the server. The server analyzes the received data and automatically updates the project progress and any issues that have arisen.

[2184] Emotional analysis using an emotion engine

[2185] The server has an emotion engine built in that analyzes user emotions from meeting audio and daily report data. The server incorporates the analysis results into the project plan and generates progress reports and proposed solutions to issues, taking emotional information into account.

[2186] Generation and notification of proposed solutions to the problem.

[2187] The server generates proposed solutions for the listed issues, incorporating sentiment information before notifying the user. These solutions include optimal solutions based on historical data and a knowledge base.

[2188] Addressing urgent issues

[2189] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[2190] Hardware and software to use

[2191] Hardware: Smart devices (smart glasses, smartphones), conference room recording equipment, servers

[2192] Software: Python, speech_recognition library, transformers library, TextBlob library

[2193] Processing flow

[2194] The system begins by recording meeting audio and transcribing it into text. The text data is then analyzed using natural language processing techniques to generate project plans and progress reports. An emotion engine analyzes user emotions and incorporates the results into the project plans and progress reports. It also provides solutions to issues and notifications for emergencies.

[2195] Examples of specific cases and prompt statements

[2196] Specific example:

[2197] Voice input: "Today's progress was delayed due to a machine malfunction. I'm feeling very stressed."

[2198] Text: "Today's progress was delayed due to a machine malfunction. I'm feeling very stressed."

[2199] Sentiment analysis result: "Negative, 0.85"

[2200] Notification: "Issue: Machine malfunction\nSolution: Call a repair technician and have the malfunctioning machine repaired. Taking a break is recommended."

[2201] Example of a prompt:

[2202] "Analyze the emotion in this text and respond with either positive or negative."

[2203] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2204] Step 1:

[2205] The device records the meeting audio.

[2206] Input: Audio from the project meeting

[2207] Operation: The device (smart device or recording device) records the meeting audio.

[2208] Output: Recorded audio data

[2209] Step 2:

[2210] The device converts the recorded audio data into text data.

[2211] Input: Recorded audio data

[2212] Operation: Converts audio data into text data using speech recognition technology (Google Speech Recognition API).

[2213] Output: Text data

[2214] Step 3:

[2215] The terminal sends text data to the server.

[2216] Input: Text data

[2217] Operation: Sends text data generated by the terminal to the server.

[2218] Output: Text data is sent to the server.

[2219] Step 4:

[2220] The server analyzes the received text data using natural language processing technology and automatically generates a project plan.

[2221] Input: Text data

[2222] Operation: The server uses natural language processing (NLP) techniques to analyze text data and extract project objectives, overview, schedule, milestones, and resource requirements.

[2223] Output: Automated project plan

[2224] Step 5:

[2225] The server shares the project plan with the relevant parties.

[2226] Input: Automated project plan

[2227] Operation: The server notifies and shares the project plan with relevant parties.

[2228] Output: Project plan shared with stakeholders

[2229] Step 6:

[2230] Users input their daily progress and challenges through the interface.

[2231] Input: Information on progress and issues

[2232] Operation: The user inputs progress and issue information into the system interface.

[2233] Output: Progress and issue data entered by the user.

[2234] Step 7:

[2235] The terminal sends daily report data to the server.

[2236] Input: Progress and issue data entered by the user.

[2237] Operation: The device sends progress and task data to the server.

[2238] Output: Daily report data sent to the server

[2239] Step 8:

[2240] The server analyzes the daily report data it receives and automatically updates the project's progress and issues.

[2241] Input: Daily report data

[2242] Operation: The server analyzes progress and issues, updates progress, and generates a list of issues.

[2243] Output: Updated progress and issue list

[2244] Step 9:

[2245] The server uses an emotion engine to analyze the user's emotions and reflects the analysis results in project plans and progress reports.

[2246] Input: Text data and daily report data

[2247] Operation: The server uses an emotion engine to analyze the emotions in the data and reflects the results in the project plan and progress report.

[2248] Output: Project plan and progress report reflecting emotional information

[2249] Step 10:

[2250] The server generates proposed solutions to the problem, incorporates emotional information, and then notifies the user.

[2251] Input: Task list and sentiment analysis results

[2252] Operation: The server generates proposed solutions to the problem based on historical data and a knowledge base, and notifies the user, taking sentiment information into account.

[2253] Output: Proposed course of action notified to the user

[2254] Step 11:

[2255] When the server detects an urgent issue, it immediately notifies the relevant parties and proposes solutions that take emotional information into consideration.

[2256] Input: Information on the occurrence of urgent issues and sentiment analysis results

[2257] Operation: The server detects urgent issues, notifies relevant parties, and proposes solutions that take emotional information into account.

[2258] Output: Urgent issues and solutions notified to stakeholders

[2259] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2260] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2261] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2262] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2263] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[2264] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[2265] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[2266] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[2267] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such a...

Claims

1. A means of recording the audio of meetings held by stakeholders during the project and converting it into text data, A means for analyzing the aforementioned text data using natural language processing technology and automatically generating a project plan, Means of sharing the project plan with stakeholders, A means of providing an interface for inputting daily progress and challenges, The means for sending the aforementioned progress status and issue data to the server, A means for analyzing the aforementioned progress status and issue data, updating the progress status, and generating an issue list and proposed solutions, A means of notifying the relevant parties of the aforementioned list of issues and proposed solutions, A means to automatically generate project progress reports and send them to higher levels, A system that includes this.

2. A method for regularly updating the project plan and automatically generating the latest progress and issue list, A means of sharing the updated project plan with stakeholders, including The system according to claim 1.

3. This includes means of notifying stakeholders of an urgent issue and proposing solutions when one arises. The system according to claim 1.

Images