system

Abstract

We provide the system. [Solution] A means of generating meeting minutes, Means for analyzing the project status, A means to visualize the contributions of members, Means for generating feedback and praise messages, Methods for conducting sentiment analysis, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] This invention aims to solve the problems that in a remote work environment, communication between team members is insufficient and individual contributions are difficult to recognize. In particular, a mechanism is required for members to have confidence in their work and enhance the sense of achievement as a team.

Means for Solving the Problems

[0005] To solve this problem, the invention provides the following means. First, it has means for automatically summarizing the content of online meetings using recorded data and generating meeting minutes. Second, it provides means for analyzing information from project management tools and evaluating and visualizing the contribution of each team member. It also includes means for collecting and analyzing communication data and generating appropriate feedback and praise messages. Furthermore, it includes means for conducting sentiment analysis and generating information to understand and support the emotional state of members. Through these means, the invention promotes respect within the team and creates an environment where people can feel joy in their work.

[0006] A "meeting minutes generation tool" is a function that automatically summarizes the content of an online meeting, compiles the important points, and generates them as meeting minutes.

[0007] "Means for analyzing project status" refers to a function that allows users to understand the progress and priority of tasks based on information obtained from project management tools.

[0008] "Methods for visualizing member contributions" refer to functions that evaluate the contributions of each team member and visually display them as graphs or charts.

[0009] "Means for generating feedback and praise messages" refers to a function that analyzes communication among members and creates appropriate feedback and praise for individual work.

[0010] "Methods for performing emotional analysis" refer to functions that analyze members' communication data and evaluate their emotional and stress levels. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] This invention is a system that facilitates communication and collaboration within teams in a remote work environment. Specific embodiments are described below.

[0033] This system operates based on two-way communication between the server, terminal, and user. First, the server retrieves the audio recording or transcript of an online meeting after it has ended. Next, the server uses generative AI to summarize this data and extract key points to generate meeting minutes. These minutes are then delivered to the user's terminal, allowing the user to quickly review the meeting content.

[0034] For project status analysis, the server retrieves data on task progress and priority from project management tools. Based on this data, the server uses a generative AI to evaluate each member's contribution, generates a dashboard that visually displays this information as graphs and charts, and delivers it to the user's device. This allows the user to understand the overall team progress and see the contributions of each member.

[0035] In generating feedback and praise messages, the server collects communication data and analyzes the context and sentiment of the messages. By using generative AI to generate appropriate feedback and praise messages and sending them to the user's device, the user can directly feel that their work is being appreciated.

[0036] Furthermore, to perform sentiment analysis, the server detects emotions and tone of voice from communication data between members, and uses sentiment analysis AI to evaluate members' emotions and stress levels. This information is provided to the user as feedback, enabling them to take appropriate support measures.

[0037] This system fosters respect and collaboration within teams, creating a workplace environment where employees can feel energized and accomplished, even in a remote work setting.

[0038] The following describes the processing flow.

[0039] Step 1:

[0040] The server retrieves audio recordings or transcripts from the meeting system after the online meeting has ended. This ensures that everything discussed during the meeting is accurately captured.

[0041] Step 2:

[0042] The server uses a generative AI to summarize the acquired audio data and create meeting minutes that extract key topics and decisions from the meeting. These minutes include a brief explanation of each agenda item.

[0043] Step 3:

[0044] The server delivers the generated meeting minutes to the user's terminal. The user opens the meeting minutes on their terminal and reviews the meeting content.

[0045] Step 4:

[0046] The server automatically retrieves task progress data from the project management tool and analyzes each member's contribution. In this process, emphasis is placed on task completion and the role of the assigned person.

[0047] Step 5:

[0048] Based on the analysis results, the server generates a dashboard that visually displays each member's contribution. This dashboard uses graphs and charts to make it easy to understand visually.

[0049] Step 6:

[0050] The server delivers the generated dashboard to the user's device, allowing the user to monitor their own and their team members' performance.

[0051] Step 7:

[0052] The server collects communication data from messaging platforms and email within the team. This data is in text format and is used for analysis.

[0053] Step 8:

[0054] The server analyzes the collected communication data and uses generative AI to automatically generate feedback and praise messages. This allows team members to identify what aspects of their work were praised.

[0055] Step 9:

[0056] The server sends the generated feedback and praise messages to the user's device, allowing the user to receive recognition from the team for their contributions.

[0057] Step 10:

[0058] The server uses emotion analysis AI to analyze the emotions and stress levels of members from the acquired communication data, particularly focusing on word choice and context.

[0059] Step 11:

[0060] The server provides feedback to the user based on the sentiment analysis results. This feedback includes suggestions and points to be aware of.

[0061] Step 12:

[0062] Users review the feedback provided through their devices and adjust their behavior and communication methods as needed.

[0063] (Example 1)

[0064] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0065] In remote work environments, it is essential to organize information and provide feedback to team members in an efficient and objective manner to facilitate communication and collaboration within teams, as well as to manage project progress and evaluate individual contributions. In particular, it is necessary to improve team dynamics through summarizing discussions in online meetings, visualizing each member's contribution to tasks, generating appropriate feedback, and assessing emotions.

[0066] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0067] In this invention, the server operates based on bidirectional communication and includes means for acquiring meeting data and generating summaries, means for analyzing project progress information and visualizing structured data, means for analyzing communication information and generating appropriate feedback, and means for performing sentiment evaluation and analyzing the emotional state of team members. This makes it possible to facilitate communication within the team even in a remote work environment, maintain transparency in project progress, and improve the motivation of team members.

[0068] "Two-way communication" is a communication method in which information travels back and forth between the sender and receiver.

[0069] "Meeting data" refers to information recorded in various formats, such as audio, text, and video, during online meetings.

[0070] A "summary" is a format that extracts the main points of information and presents them concisely.

[0071] "Project progress information" refers to data regarding the current progress and completion status of tasks related to a project.

[0072] "Structured data" refers to data arranged according to a clear format and rules, making it easy to analyze and visualize.

[0073] "Visualization" is the act of representing data and information visually, making it easier to understand.

[0074] "Feedback" refers to opinions and evaluations given regarding actions or work, including areas for improvement and points of praise.

[0075] "Communication information" refers to linguistic or non-linguistic data generated through dialogue and message exchange.

[0076] "Emotional assessment" is the process of analyzing an individual's emotional state and evaluating specific indicators based on that analysis.

[0077] A "member" refers to a person who belongs to a specific group or team and works together on a project.

[0078] This invention is a system for improving the efficiency of team communication and project management in a remote work environment. Specifically, it generates summaries of online meetings, visualizes project progress, provides feedback, and analyzes the sentiment of team members.

[0079] The server retrieves meeting data via an online meeting tool (e.g., a common communication platform API). This data is stored on the server in audio or text format. The server then uses a generative AI model (e.g., a common natural language processing model) to generate a summary from the meeting data. An example of this prompt would be: "Summarize the following meeting content and extract the key points: {meeting data}".

[0080] The server also retrieves progress information from project management tools (e.g., general project management software) and evaluates each member's contribution to their tasks. This data is generated as a dashboard using visualization software (e.g., general data visualization tools) and delivered to the user's terminal. Users can then monitor the overall team progress and their own contributions in a timely manner on their terminal.

[0081] Furthermore, the server collects communication information from the communication platform and uses a generative AI model to create feedback and praise messages. This allows users to quickly feel that their work is being properly evaluated.

[0082] For sentiment analysis, the server utilizes natural language processing libraries (e.g., common sentiment analysis APIs) to assess emotions and stress levels from the communication of team members. This information is provided to users as feedback, forming the basis for improving the team's emotional health.

[0083] Through these systems, users can communicate effectively even in a remote work environment, have their individual contributions recognized, and contribute to the success of projects.

[0084] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0085] Step 1:

[0086] After an online meeting ends, the server automatically retrieves the meeting audio or text transcript using the communication platform's API. The input is the meeting data provided by the communication platform, and the output is the raw data stored on the server. This raw data is used for subsequent processing.

[0087] Step 2:

[0088] The server inputs raw data into a generation AI model to create a summary. This process uses the prompt "Summarize the following meeting content and extract the key points: {meeting data}". The input is the raw data stored on the server, and the output is the generated meeting summary. The server saves this in text format.

[0089] Step 3:

[0090] The server retrieves progress data from project management software. The input is task progress and priority data obtained from the project management software, and the output is project progress information organized within the server. The server uses this as foundational data for evaluation.

[0091] Step 4:

[0092] The server uses a generative AI model to analyze progress data and evaluate the contribution of each team member. The input is project progress information, and the output is numerical data showing the contribution of each team member. The server creates graphs and charts based on this evaluation.

[0093] Step 5:

[0094] The server uses visualization software to create a dashboard based on the generated data. The input is numerical data showing the contribution of each member, and the output is a visual dashboard delivered to the user's terminal. Users view this dashboard on their terminal to check their progress.

[0095] Step 6:

[0096] The server inputs communication information obtained from the communication platform into a generating AI model, which then generates feedback and praise messages. The input is the communication information stored on the server, and the output is the generated message. The generated message is sent to the user's terminal.

[0097] Step 7:

[0098] The server utilizes a natural language processing library to perform sentiment analysis based on communication information. The input is communication information, and the output is data indicating the emotional state of the team members. This data is fed back to the user, contributing to improving the team's emotional health.

[0099] (Application Example 1)

[0100] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0101] In remote work environments and smart cities, communication and collaboration among residents and team members are not efficient, leading to challenges in information sharing and project management. In particular, organizing information after online meetings, evaluating members' contributions, and providing appropriate feedback to improve motivation are difficult. Furthermore, the lack of support for stress management using sentiment analysis makes it difficult to maintain the mental health of residents and team members.

[0102] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0103] In this invention, the server includes means for generating meeting minutes, means for analyzing the project status, means for visualizing the contributions of members, means for generating feedback and praise messages, means for performing sentiment analysis, means for supporting communication among residents, and means for displaying information using a smart device. This enables faster information sharing, more efficient project management, and smoother communication among members.

[0104] A "meeting minutes generation system" is a system that has the function of acquiring records of online meetings, summarizing them, and providing them to participants.

[0105] "Means for analyzing project status" refers to functions for collecting project progress data, evaluating and analyzing its status, and visualizing it.

[0106] "Methods for visualizing member contributions" refers to a function that displays each member's contribution to a task as numbers or graphs, presenting it in an easy-to-understand visual format.

[0107] "Means for generating feedback and praise messages" refers to a function that analyzes communication data and results within a team and automatically generates appropriate feedback and praise.

[0108] "Methods for performing emotional analysis" refer to functions that analyze the tone of voice and the content of communication to evaluate a person's emotions and stress levels.

[0109] "Means of supporting communication among residents" refers to functions that facilitate more effective and smoother dialogue among residents of a smart city.

[0110] "Information display methods using smart devices" refer to functions that utilize digital terminals such as smartphones and smart glasses to intuitively display necessary information to the user.

[0111] This invention is a system for facilitating smooth communication in remote work environments and smart cities. The server uses a multi-functional processing program to process various types of data. Details are provided below.

[0112] The server first acquires the audio recording of the online meeting. Next, it uses this data to perform summarization processing with a generative AI model to generate meeting minutes. These summarized minutes are then delivered to the user's device. This allows members who were unable to attend the meeting to quickly understand the meeting content.

[0113] Furthermore, the server collects task progress information from project management tools and uses a generated AI model to evaluate each member's contribution to the task. It then visualizes the results and delivers them to the user's device as an easy-to-understand dashboard. This allows users to grasp the overall project status and individual contributions.

[0114] Furthermore, the server analyzes communication data and automatically generates appropriate feedback and praise messages. These messages are sent to the user's device, contributing to increased motivation among members.

[0115] From an emotion analysis perspective, the server uses the Google® Cloud Natural Language API to analyze the tone of communication and assess stress levels. Based on this information, users can receive support to maintain their mental health.

[0116] By using smart devices, information generated on the server can be visually displayed to residents and users via smartphones and smart glasses, enabling rapid information retrieval. This system promotes cooperation throughout the community.

[0117] For example, by inputting a prompt message such as "Summarize the contents of the residents' meeting and create minutes" into the AI ​​model after a residents' meeting, it is possible to automatically create the minutes. Also, by using prompt messages such as "Analyze the project's progress and generate a graph," it becomes easier to visually understand the project's status.

[0118] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0119] Step 1:

[0120] The server retrieves audio data or transcripts after an online meeting ends. This serves as the input. The server uses a generative AI model to summarize this data and generate meeting minutes. These meeting minutes serve as the output.

[0121] Step 2:

[0122] The server collects task progress information from project management tools. This data is used as input. The server analyzes the data using a generative AI model and evaluates each member's contribution to the task. As output, it generates a dashboard visualizing the evaluation results and delivers it to the user.

[0123] Step 3:

[0124] The server retrieves messages sent from the communication platform. It uses message data as input. A generative AI model analyzes the data and automatically generates feedback and praise messages. These generated messages become the output.

[0125] Step 4:

[0126] The server uses the Google Cloud Natural Language API to perform sentiment analysis. Message data is used as input. The server analyzes this data and evaluates the members' emotions and stress levels. The resulting evaluation data is output and notified to the user.

[0127] Step 5:

[0128] The user's smart device receives information generated by the server and displays it visually on the device. The input is data sent from the server, and the output is the information displayed to the user. The user can then decide on an action based on the information.

[0129] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0130] This invention provides an advanced system for improving communication among team members in a remote work environment and for appropriately evaluating members' emotions. This system utilizes a server, terminals, and an emotion engine responsible for emotion recognition.

[0131] First, the server retrieves the online meeting audio and transcript data and automatically generates meeting minutes. The generated minutes are summarized and delivered to the user's device. This allows the user to easily understand the content of the meeting.

[0132] Next, the server retrieves task progress from the project management tool and evaluates each member's contribution to the project. The server creates a dashboard that visualizes these contributions as graphs and charts and delivers it to the user's terminal. The user can use this dashboard to visually understand the overall team progress and their own role.

[0133] The emotion engine analyzes real-time communication data between members to recognize the user's emotional state. Furthermore, based on this analysis, it evaluates the user's stress level. This information is provided as feedback from the server to the user's terminal. By obtaining information about their emotional tendencies and states of heightened stress, users can take appropriate action.

[0134] For example, if a user experiences stress during an online meeting due to a heated discussion, the emotion engine will detect this change in emotion. If the analysis reveals an increase in stress levels, the server will communicate this information to the user and offer suggestions and countermeasures to help them relax.

[0135] In this way, the entire system can improve the user's work efficiency while also providing support that takes their mental health into consideration.

[0136] The following describes the processing flow.

[0137] Step 1:

[0138] The server retrieves the audio or transcript from the meeting platform after the online meeting has ended. This data is recorded to accurately reflect the content of the meeting.

[0139] Step 2:

[0140] The server passes the acquired meeting data to a generation AI, which performs summarization and generates meeting minutes. The minutes include the main topics discussed, decisions made, and participants' opinions from the meeting.

[0141] Step 3:

[0142] The server delivers the generated meeting minutes to the user's terminal. The user can open the meeting minutes on their terminal and quickly review the meeting's content.

[0143] Step 4:

[0144] The server retrieves the progress of current tasks from the project management tool. This includes the task completion status of each team member and the overall project progress.

[0145] Step 5:

[0146] The server analyzes the acquired task data and calculates each member's contribution. It then generates a dashboard in a visually easy-to-understand format.

[0147] Step 6:

[0148] The generated dashboard is delivered to the user's device. Through this, the user can understand the overall team situation and their own contribution.

[0149] Step 7:

[0150] A server equipped with an emotion engine monitors users' text communications in real time. Text is collected from sources such as chat messages and emails.

[0151] Step 8:

[0152] The emotion engine allows the server to analyze the user's text data and determine their emotional state based on the context and tone. This enables the evaluation of the user's stress level and emotional changes.

[0153] Step 9:

[0154] Based on sentiment analysis, the server provides feedback to the user's device. This feedback includes the user's emotional progression and, if necessary, suggestions for relaxation.

[0155] Step 10:

[0156] Users can receive feedback through their devices and choose actions to reduce mental stress. This helps to alleviate the mental burden of daily work and maintain a healthy work style.

[0157] (Example 2)

[0158] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0159] In a remote work environment, improving the quality of communication among team members, as well as appropriately evaluating and providing feedback on members' emotional states and contributions to projects, are challenges. Current communication and project management tools make it difficult to grasp emotions and contributions, and do not adequately support work efficiency or mental health.

[0160] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0161] In this invention, the server includes means for collecting meeting data and converting speech to text, means for analyzing conversation content using natural language processing technology, and means for acquiring project data and evaluating the contribution of members to the work. This makes it possible to evaluate the emotional state of members in real time, understand their stress levels, and provide appropriate feedback.

[0162] "Methods for collecting meeting data and converting audio to text" refers to technologies that acquire recordings or live data of audio conferences, encode them using speech recognition technology, and convert them into text format.

[0163] "Methods for analyzing conversation content using natural language processing technology" refer to technologies that perform semantic analysis and contextual understanding on generated text data and structure the information.

[0164] "A means of acquiring project data and evaluating the contribution of team members" refers to a technology that analyzes task management information and calculates the contribution of each team member based on the role they played and the tasks they completed within the project.

[0165] "Means of visually displaying contribution using data visualization technology" refers to technologies that display evaluated contribution data in a graph or chart format so that it can be intuitively understood.

[0166] "A means of analyzing communication data in real time and evaluating emotional states" refers to a technology that uses algorithms to analyze text data from conversations and communications in real time and estimate emotional states.

[0167] "A method for creating supportive messages based on emotion analysis" refers to a technology that automatically generates messages to support or encourage improvement of a person's emotions, based on the results of emotion analysis.

[0168] "Methods for measuring stress levels and providing countermeasures" refer to technologies that quantify a user's psychological stress state and then present relaxation techniques and action plans accordingly.

[0169] This invention is a system designed to improve effective communication and work efficiency among team members in a remote work environment. The system's main components are a server, user terminals, and an emotion analysis engine.

[0170] The server collects meeting data via the online meeting platform's API and uses "speech recognition technology" to transcribe the recordings into text. The specific software used is a "speech recognition engine." The generated text is then analyzed using "natural language processing technology" and summarized as meeting minutes. The generated minutes are then condensed using a summarization algorithm and sent to the user's device. This process allows the user to instantly understand the important meeting content.

[0171] Furthermore, the server retrieves project status from the "project management tool" and evaluates the contribution of members to their tasks. Using data visualization tools, the evaluation results are visually displayed as a dashboard and shown on the user's terminal. This allows users to visually check project progress and their own roles.

[0172] The sentiment analysis engine uses "sentiment analysis technology" to analyze real-time communication data and recognize the user's emotional state. For example, when a specific emotional state or stress level is detected, the server generates feedback and sends a message to the user suggesting an appropriate action plan.

[0173] For example, if a user experiences stress during a meeting, the emotion analysis engine will detect this change and provide feedback such as, "Your stress level is rising; we recommend taking a 5-minute break." An example of a prompt to the generative AI model would be, "Analyze the emotional state of this conversation and assess the stress level." This allows users to manage their own emotional state appropriately and take steps to reduce stress.

[0174] This entire system aims to provide a comfortable remote work environment by supporting users' mental health and promoting efficient communication.

[0175] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0176] Step 1:

[0177] The server retrieves meeting data using the API of the online meeting platform. The input data consists of audio files and text transcripts. The server converts this data into text using speech recognition technology to obtain text data of the meeting content. Specifically, it analyzes the audio files to generate text information and prepares it for the next processing step.

[0178] Step 2:

[0179] The server performs natural language processing on text data using a generative AI model. The input is the text of the meeting generated in step 1, and the output is a summarized meeting transcript. Natural language processing technology extracts the key points of the text and summarizes them in a format that is easy for the user to understand.

[0180] Step 3:

[0181] The server retrieves project data using the project management tool's API. Inputs include task information and progress data, while output is an evaluation of each member's contribution. The server analyzes the collected data and generates metrics for quantitatively evaluating contributions. This is based on task completion and progress.

[0182] Step 4:

[0183] The server uses data visualization technology to visually display the contribution level evaluated in step 3. The input is contribution level data, and the output is visualized data in the form of graphs and charts. Specifically, it creates a dashboard in a user-friendly format based on the evaluation results and delivers it to the user's terminal.

[0184] Step 5:

[0185] The server sends real-time communication data to the sentiment analysis engine. The input is text messages and conversation data, and the output is analyzed data indicating the user's emotional state. The sentiment analysis engine evaluates the emotional state in real time and provides the results to the server.

[0186] Step 6:

[0187] The server generates and presents a feedback message to the user based on the results of the emotion analysis. The input is the emotional state data obtained in step 5, and the output is a message containing specific countermeasures and suggestions. Based on this prompt, the server generates advice for stress reduction and sends it to the user's terminal.

[0188] (Application Example 2)

[0189] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0190] In a remote work environment, there is a need to facilitate smooth communication among team members and provide appropriate feedback tailored to each member's contribution and emotional state. Furthermore, improving work efficiency and managing mental health are also necessary, and building a comprehensive system to support these aspects remains an unresolved challenge.

[0191] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0192] In this invention, the server includes means for generating meeting minutes, means for analyzing project status, means for generating feedback and praise messages, means for performing sentiment analysis, means for providing ergonomic support, means for processing audio and image data, means for displaying progress information, and means for presenting stress-based relaxation methods. This enables efficient communication among team members and appropriate support that takes into account their individual emotional states, even in a remote environment.

[0193] A "meeting minutes generation method" is a technique that acquires recording data from online meetings and automatically generates meeting minutes that summarize the key points of the conversation.

[0194] "Methods for analyzing project status" refer to methods for incorporating project management information and analyzing progress and task status.

[0195] "Methods for visualizing member contributions" refer to methods that visually display each team member's contribution to a project and the progress of their tasks.

[0196] "Means for generating feedback and praise messages" refers to methods for providing members with specific feedback and praise messages based on analyzed data.

[0197] "Means of performing sentiment analysis" refers to methods that analyze voice and image data to evaluate the user's emotional state.

[0198] "Means of providing ergonomic support" refers to methods that analyze the user's work environment and posture and propose the optimal work method and environment.

[0199] "Means for processing audio and image data" refers to methods for processing collected audio and image data using a computer and extracting necessary information.

[0200] "Means of displaying progress information" refers to methods of displaying the progress of a project or task to the user in real time.

[0201] "A means of suggesting stress-based relaxation methods" refers to a method that recommends appropriate relaxation methods according to the user's stress level, as detected through emotion analysis.

[0202] To implement this invention, a server plays a central role. The server acquires audio data from online meetings and converts the audio to text using the Google Cloud Speech-to-Text API. This text data is then analyzed using the natural language processing library spaCy to automatically generate and summarize meeting minutes.

[0203] Next, the server uses project management information to analyze the project's progress. This analysis is then visually displayed to the user using the Django framework. Each member's contribution is evaluated based on the time allocation and completion rate of each task, and this is displayed on the user's terminal as a visualized dashboard.

[0204] For emotion analysis, the server utilizes IBM Watson® and Microsoft® Azure® Emotion Recognition APIs to analyze voice and image data and evaluate the user's emotional state in real time. If the user's stress level is detected, relaxation methods are presented to the device. These include music playback and breathing exercises.

[0205] For example, if a user experiences stress during an online meeting, the server detects the emotional shift and immediately notifies the user's device with a message such as, "Let's take a break. How about a deep breath?" By using a generative AI model, real-time emotional analysis and feedback become possible, supporting the user's mental well-being.

[0206] An example of a prompt would be, "Please provide three key conclusions from this meeting. Also, analyze participants' emotions in real time and suggest relaxation techniques as needed."

[0207] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0208] Step 1:

[0209] The server receives audio data from an online meeting. It takes the audio data as input and converts it to text data using the Google Cloud Speech-to-Text API. As output, it generates meeting minutes data in text format.

[0210] Step 2:

[0211] The server analyzes the acquired text data using the natural language processing library spaCy. It uses the text data as input to extract context and important keywords, thereby creating a summary. The output is a summarized meeting minutes document.

[0212] Step 3:

[0213] The server analyzes project management information. Using data obtained from project management tools as input, it calculates the task progress of each member. The output generates data including each member's contribution.

[0214] Step 4:

[0215] The server uses the Django framework to visualize the analyzed member contributions. Using the data from step 3 as input, it builds a real-time dashboard. As output, the visualized progress dashboard is displayed on the user's terminal.

[0216] Step 5:

[0217] The server performs emotion analysis using IBM Watson and the Microsoft Azure Emotion Recognition API. It processes audio and image data from meetings as input to determine emotional states. Emotional information, such as the user's stress level, is generated as output.

[0218] Step 6:

[0219] The device receives emotional data from the server and suggests relaxation methods tailored to the user's stress level. It uses emotional information as input and a generative AI model to create appropriate feedback. The output is a notification to the user recommending relaxation methods.

[0220] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0221] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0222] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0223] [Second Embodiment]

[0224] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0225] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0226] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0227] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0228] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0229] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0230] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0231] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0232] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0233] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0234] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0235] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0236] This invention is a system that facilitates communication and collaboration within teams in a remote work environment. Specific embodiments are described below.

[0237] This system operates based on two-way communication between the server, terminal, and user. First, the server retrieves the audio recording or transcript of an online meeting after it has ended. Next, the server uses generative AI to summarize this data and extract key points to generate meeting minutes. These minutes are then delivered to the user's terminal, allowing the user to quickly review the meeting content.

[0238] For project status analysis, the server retrieves data on task progress and priority from project management tools. Based on this data, the server uses a generative AI to evaluate each member's contribution, generates a dashboard that visually displays this information as graphs and charts, and delivers it to the user's device. This allows the user to understand the overall team progress and see the contributions of each member.

[0239] In generating feedback and praise messages, the server collects communication data and analyzes the context and sentiment of the messages. By using generative AI to generate appropriate feedback and praise messages and sending them to the user's device, the user can directly feel that their work is being appreciated.

[0240] Furthermore, to perform sentiment analysis, the server detects emotions and tone of voice from communication data between members, and uses sentiment analysis AI to evaluate members' emotions and stress levels. This information is provided to the user as feedback, enabling them to take appropriate support measures.

[0241] This system fosters respect and collaboration within teams, creating a workplace environment where employees can feel energized and accomplished, even in a remote work setting.

[0242] The following describes the processing flow.

[0243] Step 1:

[0244] The server retrieves audio recordings or transcripts from the meeting system after the online meeting has ended. This ensures that everything discussed during the meeting is accurately captured.

[0245] Step 2:

[0246] The server uses a generative AI to summarize the acquired audio data and create meeting minutes that extract key topics and decisions from the meeting. These minutes include a brief explanation of each agenda item.

[0247] Step 3:

[0248] The server delivers the generated meeting minutes to the user's terminal. The user opens the meeting minutes on their terminal and reviews the meeting content.

[0249] Step 4:

[0250] The server automatically retrieves task progress data from the project management tool and analyzes each member's contribution. In this process, emphasis is placed on task completion and the role of the assigned person.

[0251] Step 5:

[0252] Based on the analysis results, the server generates a dashboard that visually displays each member's contribution. This dashboard uses graphs and charts to make it easy to understand visually.

[0253] Step 6:

[0254] The server delivers the generated dashboard to the user's device, allowing the user to monitor their own and their team members' performance.

[0255] Step 7:

[0256] The server collects communication data from messaging platforms and email within the team. This data is in text format and is used for analysis.

[0257] Step 8:

[0258] The server analyzes the collected communication data and uses generative AI to automatically generate feedback and praise messages. This allows team members to identify what aspects of their work were praised.

[0259] Step 9:

[0260] The server sends the generated feedback and praise messages to the user's device, allowing the user to receive recognition from the team for their contributions.

[0261] Step 10:

[0262] The server uses emotion analysis AI to analyze the emotions and stress levels of members from the acquired communication data, particularly focusing on word choice and context.

[0263] Step 11:

[0264] The server provides feedback to the user based on the sentiment analysis results. This feedback includes suggestions and points to be aware of.

[0265] Step 12:

[0266] Users review the feedback provided through their devices and adjust their behavior and communication methods as needed.

[0267] (Example 1)

[0268] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0269] In remote work environments, it is essential to organize information and provide feedback to team members in an efficient and objective manner to facilitate communication and collaboration within teams, as well as to manage project progress and evaluate individual contributions. In particular, it is necessary to improve team dynamics through summarizing discussions in online meetings, visualizing each member's contribution to tasks, generating appropriate feedback, and assessing emotions.

[0270] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0271] In this invention, the server operates based on bidirectional communication and includes means for acquiring meeting data and generating summaries, means for analyzing project progress information and visualizing structured data, means for analyzing communication information and generating appropriate feedback, and means for performing sentiment evaluation and analyzing the emotional state of team members. This makes it possible to facilitate communication within the team even in a remote work environment, maintain transparency in project progress, and improve the motivation of team members.

[0272] "Two-way communication" is a communication method in which information travels back and forth between the sender and receiver.

[0273] "Meeting data" refers to information recorded in various formats, such as audio, text, and video, during online meetings.

[0274] A "summary" is a format that extracts the main points of information and presents them concisely.

[0275] "Project progress information" refers to data regarding the current progress and completion status of tasks related to a project.

[0276] "Structured data" refers to data arranged according to a clear format and rules, making it easy to analyze and visualize.

[0277] "Visualization" is the act of representing data and information visually, making it easier to understand.

[0278] "Feedback" refers to opinions and evaluations given regarding actions or work, including areas for improvement and points of praise.

[0279] "Communication information" refers to linguistic or non-linguistic data generated through dialogue and message exchange.

[0280] "Emotional assessment" is the process of analyzing an individual's emotional state and evaluating specific indicators based on that analysis.

[0281] A "member" refers to a person who belongs to a specific group or team and works together on a project.

[0282] This invention is a system for improving the efficiency of team communication and project management in a remote work environment. Specifically, it generates summaries of online meetings, visualizes project progress, provides feedback, and analyzes the sentiment of team members.

[0283] The server retrieves meeting data via an online meeting tool (e.g., a common communication platform API). This data is stored on the server in audio or text format. The server then uses a generative AI model (e.g., a common natural language processing model) to generate a summary from the meeting data. An example of this prompt would be: "Summarize the following meeting content and extract the key points: {meeting data}".

[0284] The server also retrieves progress information from project management tools (e.g., general project management software) and evaluates each member's contribution to their tasks. This data is generated as a dashboard using visualization software (e.g., general data visualization tools) and delivered to the user's terminal. Users can then monitor the overall team progress and their own contributions in a timely manner on their terminal.

[0285] Furthermore, the server collects communication information from the communication platform and creates feedback and praise messages using the generated AI model. As a result, users can quickly feel that their work is being properly evaluated.

[0286] Regarding sentiment analysis, the server utilizes a natural language processing library (e.g., a common sentiment analysis API) to evaluate the sentiment and stress level from the communication of team members. This information is provided to the users as feedback and serves as a basis for improving the emotional health of the team.

[0287] Through these systems, users can communicate effectively even in a remote work environment, while their individual contributions are recognized and can lead to the success of the project.

[0288] The flow of the specific process in Example 1 will be described using FIG. 11.

[0289] Step 1:

[0290] After the online meeting ends, the server automatically obtains the recording data or text transcript of the meeting using the API of the communication platform. The input is the meeting data provided by the communication platform, and the output is the raw data stored in the server. This raw data is used in subsequent processing.

[0291] Step 2:

[0292] The server inputs the raw data into the generated AI model to create a summary. In this process, the prompt sentence "Please summarize the following meeting content and extract the important points: {meeting data}" is used. The input is the raw data stored in the server, and the output is the generated summary information of the meeting. The server saves this in text format.

[0293] Step 3:

[0294] The server retrieves progress data from project management software. The input is task progress and priority data obtained from the project management software, and the output is project progress information organized within the server. The server uses this as foundational data for evaluation.

[0295] Step 4:

[0296] The server uses a generative AI model to analyze progress data and evaluate the contribution of each team member. The input is project progress information, and the output is numerical data showing the contribution of each team member. The server creates graphs and charts based on this evaluation.

[0297] Step 5:

[0298] The server uses visualization software to create a dashboard based on the generated data. The input is numerical data showing the contribution of each member, and the output is a visual dashboard delivered to the user's terminal. Users view this dashboard on their terminal to check their progress.

[0299] Step 6:

[0300] The server inputs communication information obtained from the communication platform into a generating AI model, which then generates feedback and praise messages. The input is the communication information stored on the server, and the output is the generated message. The generated message is sent to the user's terminal.

[0301] Step 7:

[0302] The server utilizes a natural language processing library to perform sentiment analysis based on communication information. The input is communication information, and the output is data indicating the emotional states of the team members. This data is fed back to the user and contributes to improving the emotional health of the team.

[0303] (Application Example 1)

[0304] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0305] In remote work environments and smart cities, communication and cooperation among residents and team members are not being carried out efficiently, resulting in problems in information sharing and project management. In particular, it is difficult to organize information after online meetings, evaluate the contributions of members, and provide appropriate feedback for motivation improvement. Also, due to insufficient support for stress management using sentiment analysis, it is difficult to maintain the mental health of residents and members.

[0306] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following respective means.

[0307] In this invention, the server includes a minutes generation means, a means for analyzing the project situation, a means for visualizing the contribution degree of members, a means for generating feedback and praise messages, a means for performing sentiment analysis, a means for assisting communication among residents, and an information display means using a smart device. Thereby, it becomes possible to speed up information sharing, improve the efficiency of project management, and enable smooth communication among members.

[0308] The "minutes generation means" is a system having a function of acquiring the record of an online meeting, summarizing it, and providing it to the participants.

[0309] "Means for analyzing project status" refers to functions for collecting project progress data, evaluating and analyzing its status, and visualizing it.

[0310] "Methods for visualizing member contributions" refers to a function that displays each member's contribution to a task as numbers or graphs, presenting it in an easy-to-understand visual format.

[0311] "Means for generating feedback and praise messages" refers to a function that analyzes communication data and results within a team and automatically generates appropriate feedback and praise.

[0312] "Methods for performing emotional analysis" refer to functions that analyze the tone of voice and the content of communication to evaluate a person's emotions and stress levels.

[0313] "Means of supporting communication among residents" refers to functions that facilitate more effective and smoother dialogue among residents of a smart city.

[0314] "Information display methods using smart devices" refer to functions that utilize digital terminals such as smartphones and smart glasses to intuitively display necessary information to the user.

[0315] This invention is a system for facilitating smooth communication in remote work environments and smart cities. The server uses a multi-functional processing program to process various types of data. Details are provided below.

[0316] The server first acquires the audio recording of the online meeting. Next, it uses this data to perform summarization processing with a generative AI model to generate meeting minutes. These summarized minutes are then delivered to the user's device. This allows members who were unable to attend the meeting to quickly understand the meeting content.

[0317] Furthermore, the server collects task progress information from project management tools and uses a generated AI model to evaluate each member's contribution to the task. It then visualizes the results and delivers them to the user's device as an easy-to-understand dashboard. This allows users to grasp the overall project status and individual contributions.

[0318] Furthermore, the server analyzes communication data and automatically generates appropriate feedback and praise messages. These messages are sent to the user's device, contributing to increased motivation among members.

[0319] From an emotion analysis perspective, the server uses the Google Cloud Natural Language API to analyze the tone of communication and assess stress levels. Based on this information, users can receive support to maintain their mental health.

[0320] By using smart devices, information generated on the server can be visually displayed to residents and users via smartphones and smart glasses, enabling rapid information retrieval. This system promotes cooperation throughout the community.

[0321] For example, by inputting a prompt message such as "Summarize the contents of the residents' meeting and create minutes" into the AI ​​model after a residents' meeting, it is possible to automatically create the minutes. Also, by using prompt messages such as "Analyze the project's progress and generate a graph," it becomes easier to visually understand the project's status.

[0322] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0323] Step 1:

[0324] The server retrieves audio data or transcripts after an online meeting ends. This serves as the input. The server uses a generative AI model to summarize this data and generate meeting minutes. These meeting minutes serve as the output.

[0325] Step 2:

[0326] The server collects task progress information from project management tools. This data is used as input. The server analyzes the data using a generative AI model and evaluates each member's contribution to the task. As output, it generates a dashboard visualizing the evaluation results and delivers it to the user.

[0327] Step 3:

[0328] The server retrieves messages sent from the communication platform. It uses message data as input. A generative AI model analyzes the data and automatically generates feedback and praise messages. These generated messages become the output.

[0329] Step 4:

[0330] The server uses the Google Cloud Natural Language API to perform sentiment analysis. Message data is used as input. The server analyzes this data and evaluates the members' emotions and stress levels. The resulting evaluation data is output and notified to the user.

[0331] Step 5:

[0332] The user's smart device receives information generated by the server and displays it visually on the device. The input is data sent from the server, and the output is the information displayed to the user. The user can then decide on an action based on the information.

[0333] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0334] This invention provides an advanced system for improving communication among team members in a remote work environment and for appropriately evaluating members' emotions. This system utilizes a server, terminals, and an emotion engine responsible for emotion recognition.

[0335] First, the server retrieves the online meeting audio and transcript data and automatically generates meeting minutes. The generated minutes are summarized and delivered to the user's device. This allows the user to easily understand the content of the meeting.

[0336] Next, the server retrieves task progress from the project management tool and evaluates each member's contribution to the project. The server creates a dashboard that visualizes these contributions as graphs and charts and delivers it to the user's terminal. The user can use this dashboard to visually understand the overall team progress and their own role.

[0337] The emotion engine analyzes real-time communication data between members to recognize the user's emotional state. Furthermore, based on this analysis, it evaluates the user's stress level. This information is provided as feedback from the server to the user's terminal. By obtaining information about their emotional tendencies and states of heightened stress, users can take appropriate action.

[0338] For example, if a user experiences stress during an online meeting due to a heated discussion, the emotion engine will detect this change in emotion. If the analysis reveals an increase in stress levels, the server will communicate this information to the user and offer suggestions and countermeasures to help them relax.

[0339] In this way, the entire system can improve the user's work efficiency while also providing support that takes their mental health into consideration.

[0340] The following describes the processing flow.

[0341] Step 1:

[0342] The server retrieves the audio or transcript from the meeting platform after the online meeting has ended. This data is recorded to accurately reflect the content of the meeting.

[0343] Step 2:

[0344] The server passes the acquired meeting data to a generation AI, which performs summarization and generates meeting minutes. The minutes include the main topics discussed, decisions made, and participants' opinions from the meeting.

[0345] Step 3:

[0346] The server delivers the generated meeting minutes to the user's terminal. The user can open the meeting minutes on their terminal and quickly review the meeting's content.

[0347] Step 4:

[0348] The server retrieves the progress of current tasks from the project management tool. This includes the task completion status of each team member and the overall project progress.

[0349] Step 5:

[0350] The server analyzes the acquired task data and calculates each member's contribution. It then generates a dashboard in a visually easy-to-understand format.

[0351] Step 6:

[0352] The generated dashboard is delivered to the user's device. Through this, the user can understand the overall team situation and their own contribution.

[0353] Step 7:

[0354] A server equipped with an emotion engine monitors users' text communications in real time. Text is collected from sources such as chat messages and emails.

[0355] Step 8:

[0356] The emotion engine allows the server to analyze the user's text data and determine their emotional state based on the context and tone. This enables the evaluation of the user's stress level and emotional changes.

[0357] Step 9:

[0358] Based on sentiment analysis, the server provides feedback to the user's device. This feedback includes the user's emotional progression and, if necessary, suggestions for relaxation.

[0359] Step 10:

[0360] Users can receive feedback through their devices and choose actions to reduce mental stress. This helps to alleviate the mental burden of daily work and maintain a healthy work style.

[0361] (Example 2)

[0362] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0363] In a remote work environment, improving the quality of communication among team members, as well as appropriately evaluating and providing feedback on members' emotional states and contributions to projects, are challenges. Current communication and project management tools make it difficult to grasp emotions and contributions, and do not adequately support work efficiency or mental health.

[0364] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0365] In this invention, the server includes means for collecting meeting data and converting speech to text, means for analyzing conversation content using natural language processing technology, and means for acquiring project data and evaluating the contribution of members to the work. This makes it possible to evaluate the emotional state of members in real time, understand their stress levels, and provide appropriate feedback.

[0366] "Methods for collecting meeting data and converting audio to text" refers to technologies that acquire recordings or live data of audio conferences, encode them using speech recognition technology, and convert them into text format.

[0367] "Methods for analyzing conversation content using natural language processing technology" refer to technologies that perform semantic analysis and contextual understanding on generated text data and structure the information.

[0368] "A means of acquiring project data and evaluating the contribution of team members" refers to a technology that analyzes task management information and calculates the contribution of each team member based on the role they played and the tasks they completed within the project.

[0369] "Means of visually displaying contribution using data visualization technology" refers to technologies that display evaluated contribution data in a graph or chart format so that it can be intuitively understood.

[0370] "A means of analyzing communication data in real time and evaluating emotional states" refers to a technology that uses algorithms to analyze text data from conversations and communications in real time and estimate emotional states.

[0371] "A method for creating supportive messages based on emotion analysis" refers to a technology that automatically generates messages to support or encourage improvement of a person's emotions, based on the results of emotion analysis.

[0372] "Methods for measuring stress levels and providing countermeasures" refer to technologies that quantify a user's psychological stress state and then present relaxation techniques and action plans accordingly.

[0373] This invention is a system designed to improve effective communication and work efficiency among team members in a remote work environment. The system's main components are a server, user terminals, and an emotion analysis engine.

[0374] The server collects meeting data via the online meeting platform's API and uses "speech recognition technology" to transcribe the recordings into text. The specific software used is a "speech recognition engine." The generated text is then analyzed using "natural language processing technology" and summarized as meeting minutes. The generated minutes are then condensed using a summarization algorithm and sent to the user's device. This process allows the user to instantly understand the important meeting content.

[0375] Furthermore, the server retrieves project status from the "project management tool" and evaluates the contribution of members to their tasks. Using data visualization tools, the evaluation results are visually displayed as a dashboard and shown on the user's terminal. This allows users to visually check project progress and their own roles.

[0376] The sentiment analysis engine uses "sentiment analysis technology" to analyze real-time communication data and recognize the user's emotional state. For example, when a specific emotional state or stress level is detected, the server generates feedback and sends a message to the user suggesting an appropriate action plan.

[0377] For example, if a user experiences stress during a meeting, the emotion analysis engine will detect this change and provide feedback such as, "Your stress level is rising; we recommend taking a 5-minute break." An example of a prompt to the generative AI model would be, "Analyze the emotional state of this conversation and assess the stress level." This allows users to manage their own emotional state appropriately and take steps to reduce stress.

[0378] This entire system aims to provide a comfortable remote work environment by supporting users' mental health and promoting efficient communication.

[0379] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0380] Step 1:

[0381] The server retrieves meeting data using the API of the online meeting platform. The input data consists of audio files and text transcripts. The server converts this data into text using speech recognition technology to obtain text data of the meeting content. Specifically, it analyzes the audio files to generate text information and prepares it for the next processing step.

[0382] Step 2:

[0383] The server performs natural language processing on text data using a generative AI model. The input is the text of the meeting generated in step 1, and the output is a summarized meeting transcript. Natural language processing technology extracts the key points of the text and summarizes them in a format that is easy for the user to understand.

[0384] Step 3:

[0385] The server retrieves project data using the project management tool's API. Inputs include task information and progress data, while output is an evaluation of each member's contribution. The server analyzes the collected data and generates metrics for quantitatively evaluating contributions. This is based on task completion and progress.

[0386] Step 4:

[0387] The server uses data visualization technology to visually display the contribution level evaluated in step 3. The input is contribution level data, and the output is visualized data in the form of graphs and charts. Specifically, it creates a dashboard in a user-friendly format based on the evaluation results and delivers it to the user's terminal.

[0388] Step 5:

[0389] The server sends real-time communication data to the sentiment analysis engine. The input is text messages and conversation data, and the output is analyzed data indicating the user's emotional state. The sentiment analysis engine evaluates the emotional state in real time and provides the results to the server.

[0390] Step 6:

[0391] The server generates and presents a feedback message to the user based on the results of the emotion analysis. The input is the emotional state data obtained in step 5, and the output is a message containing specific countermeasures and suggestions. Based on this prompt, the server generates advice for stress reduction and sends it to the user's terminal.

[0392] (Application Example 2)

[0393] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0394] In a remote work environment, there is a need to facilitate smooth communication among team members and provide appropriate feedback tailored to each member's contribution and emotional state. Furthermore, improving work efficiency and managing mental health are also necessary, and building a comprehensive system to support these aspects remains an unresolved challenge.

[0395] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0396] In this invention, the server includes means for generating meeting minutes, means for analyzing project status, means for generating feedback and praise messages, means for performing sentiment analysis, means for providing ergonomic support, means for processing audio and image data, means for displaying progress information, and means for presenting stress-based relaxation methods. This enables efficient communication among team members and appropriate support that takes into account their individual emotional states, even in a remote environment.

[0397] A "meeting minutes generation method" is a technique that acquires recording data from online meetings and automatically generates meeting minutes that summarize the key points of the conversation.

[0398] "Methods for analyzing project status" refer to methods for incorporating project management information and analyzing progress and task status.

[0399] "Methods for visualizing member contributions" refer to methods that visually display each team member's contribution to a project and the progress of their tasks.

[0400] "Means for generating feedback and praise messages" refers to methods for providing members with specific feedback and praise messages based on analyzed data.

[0401] "Means of performing sentiment analysis" refers to methods that analyze voice and image data to evaluate the user's emotional state.

[0402] "Means of providing ergonomic support" refers to methods that analyze the user's work environment and posture and propose the optimal work method and environment.

[0403] "Means for processing audio and image data" refers to methods for processing collected audio and image data using a computer and extracting necessary information.

[0404] "Means of displaying progress information" refers to methods of displaying the progress of a project or task to the user in real time.

[0405] "A means of suggesting stress-based relaxation methods" refers to a method that recommends appropriate relaxation methods according to the user's stress level, as detected through emotion analysis.

[0406] To implement this invention, a server plays a central role. The server acquires audio data from online meetings and converts the audio to text using the Google Cloud Speech-to-Text API. This text data is then analyzed using the natural language processing library spaCy to automatically generate and summarize meeting minutes.

[0407] Next, the server uses project management information to analyze the project's progress. This analysis is then visually displayed to the user using the Django framework. Each member's contribution is evaluated based on the time allocation and completion rate of each task, and this is displayed on the user's terminal as a visualized dashboard.

[0408] For emotion analysis, the server utilizes IBM Watson and the Microsoft Azure Emotion Recognition API to analyze voice and image data and assess the user's emotional state in real time. If the user's stress level is detected, the device will suggest relaxation methods, which may include playing music or providing breathing exercises.

[0409] For example, if a user experiences stress during an online meeting, the server detects the emotional shift and immediately notifies the user's device with a message such as, "Let's take a break. How about a deep breath?" By using a generative AI model, real-time emotional analysis and feedback become possible, supporting the user's mental well-being.

[0410] An example of a prompt would be, "Please provide three key conclusions from this meeting. Also, analyze participants' emotions in real time and suggest relaxation techniques as needed."

[0411] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0412] Step 1:

[0413] The server receives audio data from an online meeting. It takes the audio data as input and converts it to text data using the Google Cloud Speech-to-Text API. As output, it generates meeting minutes data in text format.

[0414] Step 2:

[0415] The server analyzes the acquired text data using the natural language processing library spaCy. It uses the text data as input to extract context and important keywords, thereby creating a summary. The output is a summarized meeting minutes document.

[0416] Step 3:

[0417] The server analyzes project management information. Using data obtained from project management tools as input, it calculates the task progress of each member. The output generates data including each member's contribution.

[0418] Step 4:

[0419] The server uses the Django framework to visualize the analyzed member contributions. Using the data from step 3 as input, it builds a real-time dashboard. As output, the visualized progress dashboard is displayed on the user's terminal.

[0420] Step 5:

[0421] The server performs emotion analysis using IBM Watson and the Microsoft Azure Emotion Recognition API. It processes audio and image data from meetings as input to determine emotional states. Emotional information, such as the user's stress level, is generated as output.

[0422] Step 6:

[0423] The device receives emotional data from the server and suggests relaxation methods tailored to the user's stress level. It uses emotional information as input and a generative AI model to create appropriate feedback. The output is a notification to the user recommending relaxation methods.

[0424] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0425] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0426] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0427] [Third Embodiment]

[0428] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0429] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0430] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0431] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0432] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0433] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0434] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0435] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0436] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0437] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0438] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0439] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0440] This invention is a system that facilitates communication and collaboration within teams in a remote work environment. Specific embodiments are described below.

[0441] This system operates based on two-way communication between the server, terminal, and user. First, the server retrieves the audio recording or transcript of an online meeting after it has ended. Next, the server uses generative AI to summarize this data and extract key points to generate meeting minutes. These minutes are then delivered to the user's terminal, allowing the user to quickly review the meeting content.

[0442] For project status analysis, the server retrieves data on task progress and priority from project management tools. Based on this data, the server uses a generative AI to evaluate each member's contribution, generates a dashboard that visually displays this information as graphs and charts, and delivers it to the user's device. This allows the user to understand the overall team progress and see the contributions of each member.

[0443] In generating feedback and praise messages, the server collects communication data and analyzes the context and sentiment of the messages. By using generative AI to generate appropriate feedback and praise messages and sending them to the user's device, the user can directly feel that their work is being appreciated.

[0444] Furthermore, to perform sentiment analysis, the server detects emotions and tone of voice from communication data between members, and uses sentiment analysis AI to evaluate members' emotions and stress levels. This information is provided to the user as feedback, enabling them to take appropriate support measures.

[0445] This system fosters respect and collaboration within teams, creating a workplace environment where employees can feel energized and accomplished, even in a remote work setting.

[0446] The following describes the processing flow.

[0447] Step 1:

[0448] The server retrieves audio recordings or transcripts from the meeting system after the online meeting has ended. This ensures that everything discussed during the meeting is accurately captured.

[0449] Step 2:

[0450] The server uses a generative AI to summarize the acquired audio data and create meeting minutes that extract key topics and decisions from the meeting. These minutes include a brief explanation of each agenda item.

[0451] Step 3:

[0452] The server delivers the generated meeting minutes to the user's terminal. The user opens the meeting minutes on their terminal and reviews the meeting content.

[0453] Step 4:

[0454] The server automatically retrieves task progress data from the project management tool and analyzes each member's contribution. In this process, emphasis is placed on task completion and the role of the assigned person.

[0455] Step 5:

[0456] Based on the analysis results, the server generates a dashboard that visually displays each member's contribution. This dashboard uses graphs and charts to make it easy to understand visually.

[0457] Step 6:

[0458] The server delivers the generated dashboard to the user's device, allowing the user to monitor their own and their team members' performance.

[0459] Step 7:

[0460] The server collects communication data from messaging platforms and email within the team. This data is in text format and is used for analysis.

[0461] Step 8:

[0462] The server analyzes the collected communication data and uses generative AI to automatically generate feedback and praise messages. This allows team members to identify what aspects of their work were praised.

[0463] Step 9:

[0464] The server sends the generated feedback and praise messages to the user's device, allowing the user to receive recognition from the team for their contributions.

[0465] Step 10:

[0466] The server uses emotion analysis AI to analyze the emotions and stress levels of members from the acquired communication data, particularly focusing on word choice and context.

[0467] Step 11:

[0468] The server provides feedback to the user based on the sentiment analysis results. This feedback includes suggestions and points to be aware of.

[0469] Step 12:

[0470] Users review the feedback provided through their devices and adjust their behavior and communication methods as needed.

[0471] (Example 1)

[0472] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0473] In remote work environments, it is essential to organize information and provide feedback to team members in an efficient and objective manner to facilitate communication and collaboration within teams, as well as to manage project progress and evaluate individual contributions. In particular, it is necessary to improve team dynamics through summarizing discussions in online meetings, visualizing each member's contribution to tasks, generating appropriate feedback, and assessing emotions.

[0474] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0475] In this invention, the server operates based on bidirectional communication and includes means for acquiring meeting data and generating summaries, means for analyzing project progress information and visualizing structured data, means for analyzing communication information and generating appropriate feedback, and means for performing sentiment evaluation and analyzing the emotional state of team members. This makes it possible to facilitate communication within the team even in a remote work environment, maintain transparency in project progress, and improve the motivation of team members.

[0476] "Two-way communication" is a communication method in which information travels back and forth between the sender and receiver.

[0477] "Meeting data" refers to information recorded in various formats, such as audio, text, and video, during online meetings.

[0478] A "summary" is a format that extracts the main points of information and presents them concisely.

[0479] "Project progress information" refers to data regarding the current progress and completion status of tasks related to a project.

[0480] "Structured data" refers to data arranged according to a clear format and rules, making it easy to analyze and visualize.

[0481] "Visualization" is the act of representing data and information visually, making it easier to understand.

[0482] "Feedback" refers to opinions and evaluations given regarding actions or work, including areas for improvement and points of praise.

[0483] "Communication information" refers to linguistic or non-linguistic data generated through dialogue and message exchange.

[0484] "Emotional assessment" is the process of analyzing an individual's emotional state and evaluating specific indicators based on that analysis.

[0485] A "member" refers to a person who belongs to a specific group or team and works together on a project.

[0486] This invention is a system for improving the efficiency of team communication and project management in a remote work environment. Specifically, it generates summaries of online meetings, visualizes project progress, provides feedback, and analyzes the sentiment of team members.

[0487] The server retrieves meeting data via an online meeting tool (e.g., a common communication platform API). This data is stored on the server in audio or text format. The server then uses a generative AI model (e.g., a common natural language processing model) to generate a summary from the meeting data. An example of this prompt would be: "Summarize the following meeting content and extract the key points: {meeting data}".

[0488] The server also retrieves progress information from project management tools (e.g., general project management software) and evaluates each member's contribution to their tasks. This data is generated as a dashboard using visualization software (e.g., general data visualization tools) and delivered to the user's terminal. Users can then monitor the overall team progress and their own contributions in a timely manner on their terminal.

[0489] Furthermore, the server collects communication information from the communication platform and uses a generative AI model to create feedback and praise messages. This allows users to quickly feel that their work is being properly evaluated.

[0490] For sentiment analysis, the server utilizes natural language processing libraries (e.g., common sentiment analysis APIs) to assess emotions and stress levels from the communication of team members. This information is provided to users as feedback, forming the basis for improving the team's emotional health.

[0491] Through these systems, users can communicate effectively even in a remote work environment, have their individual contributions recognized, and contribute to the success of projects.

[0492] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0493] Step 1:

[0494] After an online meeting ends, the server automatically retrieves the meeting audio or text transcript using the communication platform's API. The input is the meeting data provided by the communication platform, and the output is the raw data stored on the server. This raw data is used for subsequent processing.

[0495] Step 2:

[0496] The server inputs raw data into a generation AI model to create a summary. This process uses the prompt "Summarize the following meeting content and extract the key points: {meeting data}". The input is the raw data stored on the server, and the output is the generated meeting summary. The server saves this in text format.

[0497] Step 3:

[0498] The server retrieves progress data from project management software. The input is task progress and priority data obtained from the project management software, and the output is project progress information organized within the server. The server uses this as foundational data for evaluation.

[0499] Step 4:

[0500] The server uses a generative AI model to analyze progress data and evaluate the contribution of each team member. The input is project progress information, and the output is numerical data showing the contribution of each team member. The server creates graphs and charts based on this evaluation.

[0501] Step 5:

[0502] The server uses visualization software to create a dashboard based on the generated data. The input is numerical data showing the contribution of each member, and the output is a visual dashboard delivered to the user's terminal. Users view this dashboard on their terminal to check their progress.

[0503] Step 6:

[0504] The server inputs communication information obtained from the communication platform into a generating AI model, which then generates feedback and praise messages. The input is the communication information stored on the server, and the output is the generated message. The generated message is sent to the user's terminal.

[0505] Step 7:

[0506] The server utilizes a natural language processing library to perform sentiment analysis based on communication information. The input is communication information, and the output is data indicating the emotional state of the team members. This data is fed back to the user, contributing to improving the team's emotional health.

[0507] (Application Example 1)

[0508] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0509] In remote work environments and smart cities, communication and collaboration among residents and team members are not efficient, leading to challenges in information sharing and project management. In particular, organizing information after online meetings, evaluating members' contributions, and providing appropriate feedback to improve motivation are difficult. Furthermore, the lack of support for stress management using sentiment analysis makes it difficult to maintain the mental health of residents and team members.

[0510] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0511] In this invention, the server includes means for generating meeting minutes, means for analyzing the project status, means for visualizing the contributions of members, means for generating feedback and praise messages, means for performing sentiment analysis, means for supporting communication among residents, and means for displaying information using a smart device. This enables faster information sharing, more efficient project management, and smoother communication among members.

[0512] A "meeting minutes generation system" is a system that has the function of acquiring records of online meetings, summarizing them, and providing them to participants.

[0513] "Means for analyzing project status" refers to functions for collecting project progress data, evaluating and analyzing its status, and visualizing it.

[0514] "Methods for visualizing member contributions" refers to a function that displays each member's contribution to a task as numbers or graphs, presenting it in an easy-to-understand visual format.

[0515] "Means for generating feedback and praise messages" refers to a function that analyzes communication data and results within a team and automatically generates appropriate feedback and praise.

[0516] "Methods for performing emotional analysis" refer to functions that analyze the tone of voice and the content of communication to evaluate a person's emotions and stress levels.

[0517] "Means of supporting communication among residents" refers to functions that facilitate more effective and smoother dialogue among residents of a smart city.

[0518] "Information display methods using smart devices" refer to functions that utilize digital terminals such as smartphones and smart glasses to intuitively display necessary information to the user.

[0519] This invention is a system for facilitating smooth communication in remote work environments and smart cities. The server uses a multi-functional processing program to process various types of data. Details are provided below.

[0520] The server first acquires the audio recording of the online meeting. Next, it uses this data to perform summarization processing with a generative AI model to generate meeting minutes. These summarized minutes are then delivered to the user's device. This allows members who were unable to attend the meeting to quickly understand the meeting content.

[0521] Furthermore, the server collects task progress information from project management tools and uses a generated AI model to evaluate each member's contribution to the task. It then visualizes the results and delivers them to the user's device as an easy-to-understand dashboard. This allows users to grasp the overall project status and individual contributions.

[0522] Furthermore, the server analyzes communication data and automatically generates appropriate feedback and praise messages. These messages are sent to the user's device, contributing to increased motivation among members.

[0523] From an emotion analysis perspective, the server uses the Google Cloud Natural Language API to analyze the tone of communication and assess stress levels. Based on this information, users can receive support to maintain their mental health.

[0524] By using smart devices, information generated on the server can be visually displayed to residents and users via smartphones and smart glasses, enabling rapid information retrieval. This system promotes cooperation throughout the community.

[0525] For example, by inputting a prompt message such as "Summarize the contents of the residents' meeting and create minutes" into the AI ​​model after a residents' meeting, it is possible to automatically create the minutes. Also, by using prompt messages such as "Analyze the project's progress and generate a graph," it becomes easier to visually understand the project's status.

[0526] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0527] Step 1:

[0528] The server retrieves audio data or transcripts after an online meeting ends. This serves as the input. The server uses a generative AI model to summarize this data and generate meeting minutes. These meeting minutes serve as the output.

[0529] Step 2:

[0530] The server collects task progress information from project management tools. This data is used as input. The server analyzes the data using a generative AI model and evaluates each member's contribution to the task. As output, it generates a dashboard visualizing the evaluation results and delivers it to the user.

[0531] Step 3:

[0532] The server retrieves messages sent from the communication platform. It uses message data as input. A generative AI model analyzes the data and automatically generates feedback and praise messages. These generated messages become the output.

[0533] Step 4:

[0534] The server uses the Google Cloud Natural Language API to perform sentiment analysis. Message data is used as input. The server analyzes this data and evaluates the members' emotions and stress levels. The resulting evaluation data is output and notified to the user.

[0535] Step 5:

[0536] The user's smart device receives information generated by the server and displays it visually on the device. The input is data sent from the server, and the output is the information displayed to the user. The user can then decide on an action based on the information.

[0537] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0538] This invention provides an advanced system for improving communication among team members in a remote work environment and for appropriately evaluating members' emotions. This system utilizes a server, terminals, and an emotion engine responsible for emotion recognition.

[0539] First, the server retrieves the online meeting audio and transcript data and automatically generates meeting minutes. The generated minutes are summarized and delivered to the user's device. This allows the user to easily understand the content of the meeting.

[0540] Next, the server retrieves task progress from the project management tool and evaluates each member's contribution to the project. The server creates a dashboard that visualizes these contributions as graphs and charts and delivers it to the user's terminal. The user can use this dashboard to visually understand the overall team progress and their own role.

[0541] The emotion engine analyzes real-time communication data between members to recognize the user's emotional state. Furthermore, based on this analysis, it evaluates the user's stress level. This information is provided as feedback from the server to the user's terminal. By obtaining information about their emotional tendencies and states of heightened stress, users can take appropriate action.

[0542] For example, if a user experiences stress during an online meeting due to a heated discussion, the emotion engine will detect this change in emotion. If the analysis reveals an increase in stress levels, the server will communicate this information to the user and offer suggestions and countermeasures to help them relax.

[0543] In this way, the entire system can improve the user's work efficiency while also providing support that takes their mental health into consideration.

[0544] The following describes the processing flow.

[0545] Step 1:

[0546] The server retrieves the audio or transcript from the meeting platform after the online meeting has ended. This data is recorded to accurately reflect the content of the meeting.

[0547] Step 2:

[0548] The server passes the acquired meeting data to a generation AI, which performs summarization and generates meeting minutes. The minutes include the main topics discussed, decisions made, and participants' opinions from the meeting.

[0549] Step 3:

[0550] The server delivers the generated meeting minutes to the user's terminal. The user can open the meeting minutes on their terminal and quickly review the meeting's content.

[0551] Step 4:

[0552] The server retrieves the progress of current tasks from the project management tool. This includes the task completion status of each team member and the overall project progress.

[0553] Step 5:

[0554] The server analyzes the acquired task data and calculates each member's contribution. It then generates a dashboard in a visually easy-to-understand format.

[0555] Step 6:

[0556] The generated dashboard is delivered to the user's device. Through this, the user can understand the overall team situation and their own contribution.

[0557] Step 7:

[0558] A server equipped with an emotion engine monitors users' text communications in real time. Text is collected from sources such as chat messages and emails.

[0559] Step 8:

[0560] The emotion engine allows the server to analyze the user's text data and determine their emotional state based on the context and tone. This enables the evaluation of the user's stress level and emotional changes.

[0561] Step 9:

[0562] Based on sentiment analysis, the server provides feedback to the user's device. This feedback includes the user's emotional progression and, if necessary, suggestions for relaxation.

[0563] Step 10:

[0564] Users can receive feedback through their devices and choose actions to reduce mental stress. This helps to alleviate the mental burden of daily work and maintain a healthy work style.

[0565] (Example 2)

[0566] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0567] In a remote work environment, improving the quality of communication among team members, as well as appropriately evaluating and providing feedback on members' emotional states and contributions to projects, are challenges. Current communication and project management tools make it difficult to grasp emotions and contributions, and do not adequately support work efficiency or mental health.

[0568] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0569] In this invention, the server includes means for collecting meeting data and converting speech to text, means for analyzing conversation content using natural language processing technology, and means for acquiring project data and evaluating the contribution of members to the work. This makes it possible to evaluate the emotional state of members in real time, understand their stress levels, and provide appropriate feedback.

[0570] "Methods for collecting meeting data and converting audio to text" refers to technologies that acquire recordings or live data of audio conferences, encode them using speech recognition technology, and convert them into text format.

[0571] "Methods for analyzing conversation content using natural language processing technology" refer to technologies that perform semantic analysis and contextual understanding on generated text data and structure the information.

[0572] "A means of acquiring project data and evaluating the contribution of team members" refers to a technology that analyzes task management information and calculates the contribution of each team member based on the role they played and the tasks they completed within the project.

[0573] "Means of visually displaying contribution using data visualization technology" refers to technologies that display evaluated contribution data in a graph or chart format so that it can be intuitively understood.

[0574] "A means of analyzing communication data in real time and evaluating emotional states" refers to a technology that uses algorithms to analyze text data from conversations and communications in real time and estimate emotional states.

[0575] "A method for creating supportive messages based on emotion analysis" refers to a technology that automatically generates messages to support or encourage improvement of a person's emotions, based on the results of emotion analysis.

[0576] "Methods for measuring stress levels and providing countermeasures" refer to technologies that quantify a user's psychological stress state and then present relaxation techniques and action plans accordingly.

[0577] This invention is a system designed to improve effective communication and work efficiency among team members in a remote work environment. The system's main components are a server, user terminals, and an emotion analysis engine.

[0578] The server collects meeting data via the online meeting platform's API and uses "speech recognition technology" to transcribe the recordings into text. The specific software used is a "speech recognition engine." The generated text is then analyzed using "natural language processing technology" and summarized as meeting minutes. The generated minutes are then condensed using a summarization algorithm and sent to the user's device. This process allows the user to instantly understand the important meeting content.

[0579] Furthermore, the server retrieves project status from the "project management tool" and evaluates the contribution of members to their tasks. Using data visualization tools, the evaluation results are visually displayed as a dashboard and shown on the user's terminal. This allows users to visually check project progress and their own roles.

[0580] The sentiment analysis engine uses "sentiment analysis technology" to analyze real-time communication data and recognize the user's emotional state. For example, when a specific emotional state or stress level is detected, the server generates feedback and sends a message to the user suggesting an appropriate action plan.

[0581] For example, if a user experiences stress during a meeting, the emotion analysis engine will detect this change and provide feedback such as, "Your stress level is rising; we recommend taking a 5-minute break." An example of a prompt to the generative AI model would be, "Analyze the emotional state of this conversation and assess the stress level." This allows users to manage their own emotional state appropriately and take steps to reduce stress.

[0582] This entire system aims to provide a comfortable remote work environment by supporting users' mental health and promoting efficient communication.

[0583] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0584] Step 1:

[0585] The server retrieves meeting data using the API of the online meeting platform. The input data consists of audio files and text transcripts. The server converts this data into text using speech recognition technology to obtain text data of the meeting content. Specifically, it analyzes the audio files to generate text information and prepares it for the next processing step.

[0586] Step 2:

[0587] The server performs natural language processing on text data using a generative AI model. The input is the text of the meeting generated in step 1, and the output is a summarized meeting transcript. Natural language processing technology extracts the key points of the text and summarizes them in a format that is easy for the user to understand.

[0588] Step 3:

[0589] The server retrieves project data using the project management tool's API. Inputs include task information and progress data, while output is an evaluation of each member's contribution. The server analyzes the collected data and generates metrics for quantitatively evaluating contributions. This is based on task completion and progress.

[0590] Step 4:

[0591] The server uses data visualization technology to visually display the contribution level evaluated in step 3. The input is contribution level data, and the output is visualized data in the form of graphs and charts. Specifically, it creates a dashboard in a user-friendly format based on the evaluation results and delivers it to the user's terminal.

[0592] Step 5:

[0593] The server sends real-time communication data to the sentiment analysis engine. The input is text messages and conversation data, and the output is analyzed data indicating the user's emotional state. The sentiment analysis engine evaluates the emotional state in real time and provides the results to the server.

[0594] Step 6:

[0595] The server generates and presents a feedback message to the user based on the results of the emotion analysis. The input is the emotional state data obtained in step 5, and the output is a message containing specific countermeasures and suggestions. Based on this prompt, the server generates advice for stress reduction and sends it to the user's terminal.

[0596] (Application Example 2)

[0597] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0598] In a remote work environment, there is a need to facilitate smooth communication among team members and provide appropriate feedback tailored to each member's contribution and emotional state. Furthermore, improving work efficiency and managing mental health are also necessary, and building a comprehensive system to support these aspects remains an unresolved challenge.

[0599] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0600] In this invention, the server includes means for generating meeting minutes, means for analyzing project status, means for generating feedback and praise messages, means for performing sentiment analysis, means for providing ergonomic support, means for processing audio and image data, means for displaying progress information, and means for presenting stress-based relaxation methods. This enables efficient communication among team members and appropriate support that takes into account their individual emotional states, even in a remote environment.

[0601] A "meeting minutes generation method" is a technique that acquires recording data from online meetings and automatically generates meeting minutes that summarize the key points of the conversation.

[0602] "Methods for analyzing project status" refer to methods for incorporating project management information and analyzing progress and task status.

[0603] "Methods for visualizing member contributions" refer to methods that visually display each team member's contribution to a project and the progress of their tasks.

[0604] "Means for generating feedback and praise messages" refers to methods for providing members with specific feedback and praise messages based on analyzed data.

[0605] "Means of performing sentiment analysis" refers to methods that analyze voice and image data to evaluate the user's emotional state.

[0606] "Means of providing ergonomic support" refers to methods that analyze the user's work environment and posture and propose the optimal work method and environment.

[0607] "Means for processing audio and image data" refers to methods for processing collected audio and image data using a computer and extracting necessary information.

[0608] "Means of displaying progress information" refers to methods of displaying the progress of a project or task to the user in real time.

[0609] "A means of suggesting stress-based relaxation methods" refers to a method that recommends appropriate relaxation methods according to the user's stress level, as detected through emotion analysis.

[0610] To implement this invention, a server plays a central role. The server acquires audio data from online meetings and converts the audio to text using the Google Cloud Speech-to-Text API. This text data is then analyzed using the natural language processing library spaCy to automatically generate and summarize meeting minutes.

[0611] Next, the server uses project management information to analyze the project's progress. This analysis is then visually displayed to the user using the Django framework. Each member's contribution is evaluated based on the time allocation and completion rate of each task, and this is displayed on the user's terminal as a visualized dashboard.

[0612] For emotion analysis, the server utilizes IBM Watson and the Microsoft Azure Emotion Recognition API to analyze voice and image data and assess the user's emotional state in real time. If the user's stress level is detected, the device will suggest relaxation methods, which may include playing music or providing breathing exercises.

[0613] For example, if a user experiences stress during an online meeting, the server detects the emotional shift and immediately notifies the user's device with a message such as, "Let's take a break. How about a deep breath?" By using a generative AI model, real-time emotional analysis and feedback become possible, supporting the user's mental well-being.

[0614] An example of a prompt would be, "Please provide three key conclusions from this meeting. Also, analyze participants' emotions in real time and suggest relaxation techniques as needed."

[0615] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0616] Step 1:

[0617] The server receives audio data from an online meeting. It takes the audio data as input and converts it to text data using the Google Cloud Speech-to-Text API. As output, it generates meeting minutes data in text format.

[0618] Step 2:

[0619] The server analyzes the acquired text data using the natural language processing library spaCy. It uses the text data as input to extract context and important keywords, thereby creating a summary. The output is a summarized meeting minutes document.

[0620] Step 3:

[0621] The server analyzes project management information. Using data obtained from project management tools as input, it calculates the task progress of each member. The output generates data including each member's contribution.

[0622] Step 4:

[0623] The server uses the Django framework to visualize the analyzed member contributions. Using the data from step 3 as input, it builds a real-time dashboard. As output, the visualized progress dashboard is displayed on the user's terminal.

[0624] Step 5:

[0625] The server performs emotion analysis using IBM Watson and the Microsoft Azure Emotion Recognition API. It processes audio and image data from meetings as input to determine emotional states. Emotional information, such as the user's stress level, is generated as output.

[0626] Step 6:

[0627] The device receives emotional data from the server and suggests relaxation methods tailored to the user's stress level. It uses emotional information as input and a generative AI model to create appropriate feedback. The output is a notification to the user recommending relaxation methods.

[0628] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0629] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0630] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0631] [Fourth Embodiment]

[0632] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0633] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0634] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0635] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0636] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0637] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0638] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0639] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0640] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0641] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0642] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0643] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0644] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0645] This invention is a system that facilitates communication and collaboration within teams in a remote work environment. Specific embodiments are described below.

[0646] This system operates based on two-way communication between the server, terminal, and user. First, the server retrieves the audio recording or transcript of an online meeting after it has ended. Next, the server uses generative AI to summarize this data and extract key points to generate meeting minutes. These minutes are then delivered to the user's terminal, allowing the user to quickly review the meeting content.

[0647] For project status analysis, the server retrieves data on task progress and priority from project management tools. Based on this data, the server uses a generative AI to evaluate each member's contribution, generates a dashboard that visually displays this information as graphs and charts, and delivers it to the user's device. This allows the user to understand the overall team progress and see the contributions of each member.

[0648] In generating feedback and praise messages, the server collects communication data and analyzes the context and sentiment of the messages. By using generative AI to generate appropriate feedback and praise messages and sending them to the user's device, the user can directly feel that their work is being appreciated.

[0649] Furthermore, to perform sentiment analysis, the server detects emotions and tone of voice from communication data between members, and uses sentiment analysis AI to evaluate members' emotions and stress levels. This information is provided to the user as feedback, enabling them to take appropriate support measures.

[0650] This system fosters respect and collaboration within teams, creating a workplace environment where employees can feel energized and accomplished, even in a remote work setting.

[0651] The following describes the processing flow.

[0652] Step 1:

[0653] The server retrieves audio recordings or transcripts from the meeting system after the online meeting has ended. This ensures that everything discussed during the meeting is accurately captured.

[0654] Step 2:

[0655] The server uses a generative AI to summarize the acquired audio data and create meeting minutes that extract key topics and decisions from the meeting. These minutes include a brief explanation of each agenda item.

[0656] Step 3:

[0657] The server delivers the generated meeting minutes to the user's terminal. The user opens the meeting minutes on their terminal and reviews the meeting content.

[0658] Step 4:

[0659] The server automatically retrieves task progress data from the project management tool and analyzes each member's contribution. In this process, emphasis is placed on task completion and the role of the assigned person.

[0660] Step 5:

[0661] Based on the analysis results, the server generates a dashboard that visually displays each member's contribution. This dashboard uses graphs and charts to make it easy to understand visually.

[0662] Step 6:

[0663] The server delivers the generated dashboard to the user's device, allowing the user to monitor their own and their team members' performance.

[0664] Step 7:

[0665] The server collects communication data from messaging platforms and email within the team. This data is in text format and is used for analysis.

[0666] Step 8:

[0667] The server analyzes the collected communication data and uses generative AI to automatically generate feedback and praise messages. This allows team members to identify what aspects of their work were praised.

[0668] Step 9:

[0669] The server sends the generated feedback and praise messages to the user's device, allowing the user to receive recognition from the team for their contributions.

[0670] Step 10:

[0671] The server uses emotion analysis AI to analyze the emotions and stress levels of members from the acquired communication data, particularly focusing on word choice and context.

[0672] Step 11:

[0673] The server provides feedback to the user based on the sentiment analysis results. This feedback includes suggestions and points to be aware of.

[0674] Step 12:

[0675] Users review the feedback provided through their devices and adjust their behavior and communication methods as needed.

[0676] (Example 1)

[0677] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0678] In remote work environments, it is essential to organize information and provide feedback to team members in an efficient and objective manner to facilitate communication and collaboration within teams, as well as to manage project progress and evaluate individual contributions. In particular, it is necessary to improve team dynamics through summarizing discussions in online meetings, visualizing each member's contribution to tasks, generating appropriate feedback, and assessing emotions.

[0679] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0680] In this invention, the server operates based on bidirectional communication and includes means for acquiring meeting data and generating summaries, means for analyzing project progress information and visualizing structured data, means for analyzing communication information and generating appropriate feedback, and means for performing sentiment evaluation and analyzing the emotional state of team members. This makes it possible to facilitate communication within the team even in a remote work environment, maintain transparency in project progress, and improve the motivation of team members.

[0681] "Two-way communication" is a communication method in which information travels back and forth between the sender and receiver.

[0682] "Meeting data" refers to information recorded in various formats, such as audio, text, and video, during online meetings.

[0683] A "summary" is a format that extracts the main points of information and presents them concisely.

[0684] "Project progress information" refers to data regarding the current progress and completion status of tasks related to a project.

[0685] "Structured data" refers to data arranged according to a clear format and rules, making it easy to analyze and visualize.

[0686] "Visualization" is the act of representing data and information visually, making it easier to understand.

[0687] "Feedback" refers to opinions and evaluations given regarding actions or work, including areas for improvement and points of praise.

[0688] "Communication information" refers to linguistic or non-linguistic data generated through dialogue and message exchange.

[0689] "Emotional assessment" is the process of analyzing an individual's emotional state and evaluating specific indicators based on that analysis.

[0690] A "member" refers to a person who belongs to a specific group or team and works together on a project.

[0691] This invention is a system for improving the efficiency of team communication and project management in a remote work environment. Specifically, it generates summaries of online meetings, visualizes project progress, provides feedback, and analyzes the sentiment of team members.

[0692] The server retrieves meeting data via an online meeting tool (e.g., a common communication platform API). This data is stored on the server in audio or text format. The server then uses a generative AI model (e.g., a common natural language processing model) to generate a summary from the meeting data. An example of this prompt would be: "Summarize the following meeting content and extract the key points: {meeting data}".

[0693] The server also retrieves progress information from project management tools (e.g., general project management software) and evaluates each member's contribution to their tasks. This data is generated as a dashboard using visualization software (e.g., general data visualization tools) and delivered to the user's terminal. Users can then monitor the overall team progress and their own contributions in a timely manner on their terminal.

[0694] Furthermore, the server collects communication information from the communication platform and uses a generative AI model to create feedback and praise messages. This allows users to quickly feel that their work is being properly evaluated.

[0695] For sentiment analysis, the server utilizes natural language processing libraries (e.g., common sentiment analysis APIs) to assess emotions and stress levels from the communication of team members. This information is provided to users as feedback, forming the basis for improving the team's emotional health.

[0696] Through these systems, users can communicate effectively even in a remote work environment, have their individual contributions recognized, and contribute to the success of projects.

[0697] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0698] Step 1:

[0699] After an online meeting ends, the server automatically retrieves the meeting audio or text transcript using the communication platform's API. The input is the meeting data provided by the communication platform, and the output is the raw data stored on the server. This raw data is used for subsequent processing.

[0700] Step 2:

[0701] The server inputs raw data into a generation AI model to create a summary. This process uses the prompt "Summarize the following meeting content and extract the key points: {meeting data}". The input is the raw data stored on the server, and the output is the generated meeting summary. The server saves this in text format.

[0702] Step 3:

[0703] The server retrieves progress data from project management software. The input is task progress and priority data obtained from the project management software, and the output is project progress information organized within the server. The server uses this as foundational data for evaluation.

[0704] Step 4:

[0705] The server uses a generative AI model to analyze progress data and evaluate the contribution of each team member. The input is project progress information, and the output is numerical data showing the contribution of each team member. The server creates graphs and charts based on this evaluation.

[0706] Step 5:

[0707] The server uses visualization software to create a dashboard based on the generated data. The input is numerical data showing the contribution of each member, and the output is a visual dashboard delivered to the user's terminal. Users view this dashboard on their terminal to check their progress.

[0708] Step 6:

[0709] The server inputs communication information obtained from the communication platform into a generating AI model, which then generates feedback and praise messages. The input is the communication information stored on the server, and the output is the generated message. The generated message is sent to the user's terminal.

[0710] Step 7:

[0711] The server utilizes a natural language processing library to perform sentiment analysis based on communication information. The input is communication information, and the output is data indicating the emotional state of the team members. This data is fed back to the user, contributing to improving the team's emotional health.

[0712] (Application Example 1)

[0713] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0714] In remote work environments and smart cities, communication and collaboration among residents and team members are not efficient, leading to challenges in information sharing and project management. In particular, organizing information after online meetings, evaluating members' contributions, and providing appropriate feedback to improve motivation are difficult. Furthermore, the lack of support for stress management using sentiment analysis makes it difficult to maintain the mental health of residents and team members.

[0715] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0716] In this invention, the server includes means for generating meeting minutes, means for analyzing the project status, means for visualizing the contributions of members, means for generating feedback and praise messages, means for performing sentiment analysis, means for supporting communication among residents, and means for displaying information using a smart device. This enables faster information sharing, more efficient project management, and smoother communication among members.

[0717] A "meeting minutes generation system" is a system that has the function of acquiring records of online meetings, summarizing them, and providing them to participants.

[0718] "Means for analyzing project status" refers to functions for collecting project progress data, evaluating and analyzing its status, and visualizing it.

[0719] "Methods for visualizing member contributions" refers to a function that displays each member's contribution to a task as numbers or graphs, presenting it in an easy-to-understand visual format.

[0720] "Means for generating feedback and praise messages" refers to a function that analyzes communication data and results within a team and automatically generates appropriate feedback and praise.

[0721] "Methods for performing emotional analysis" refer to functions that analyze the tone of voice and the content of communication to evaluate a person's emotions and stress levels.

[0722] "Means of supporting communication among residents" refers to functions that facilitate more effective and smoother dialogue among residents of a smart city.

[0723] "Information display methods using smart devices" refer to functions that utilize digital terminals such as smartphones and smart glasses to intuitively display necessary information to the user.

[0724] This invention is a system for facilitating smooth communication in remote work environments and smart cities. The server uses a multi-functional processing program to process various types of data. Details are provided below.

[0725] The server first acquires the audio recording of the online meeting. Next, it uses this data to perform summarization processing with a generative AI model to generate meeting minutes. These summarized minutes are then delivered to the user's device. This allows members who were unable to attend the meeting to quickly understand the meeting content.

[0726] Furthermore, the server collects task progress information from project management tools and uses a generated AI model to evaluate each member's contribution to the task. It then visualizes the results and delivers them to the user's device as an easy-to-understand dashboard. This allows users to grasp the overall project status and individual contributions.

[0727] Furthermore, the server analyzes communication data and automatically generates appropriate feedback and praise messages. These messages are sent to the user's device, contributing to increased motivation among members.

[0728] From an emotion analysis perspective, the server uses the Google Cloud Natural Language API to analyze the tone of communication and assess stress levels. Based on this information, users can receive support to maintain their mental health.

[0729] By using smart devices, information generated on the server can be visually displayed to residents and users via smartphones and smart glasses, enabling rapid information retrieval. This system promotes cooperation throughout the community.

[0730] For example, by inputting a prompt message such as "Summarize the contents of the residents' meeting and create minutes" into the AI ​​model after a residents' meeting, it is possible to automatically create the minutes. Also, by using prompt messages such as "Analyze the project's progress and generate a graph," it becomes easier to visually understand the project's status.

[0731] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0732] Step 1:

[0733] The server retrieves audio data or transcripts after an online meeting ends. This serves as the input. The server uses a generative AI model to summarize this data and generate meeting minutes. These meeting minutes serve as the output.

[0734] Step 2:

[0735] The server collects task progress information from project management tools. This data is used as input. The server analyzes the data using a generative AI model and evaluates each member's contribution to the task. As output, it generates a dashboard visualizing the evaluation results and delivers it to the user.

[0736] Step 3:

[0737] The server retrieves messages sent from the communication platform. It uses message data as input. A generative AI model analyzes the data and automatically generates feedback and praise messages. These generated messages become the output.

[0738] Step 4:

[0739] The server uses the Google Cloud Natural Language API to perform sentiment analysis. Message data is used as input. The server analyzes this data and evaluates the members' emotions and stress levels. The resulting evaluation data is output and notified to the user.

[0740] Step 5:

[0741] The user's smart device receives information generated by the server and displays it visually on the device. The input is data sent from the server, and the output is the information displayed to the user. The user can then decide on an action based on the information.

[0742] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0743] This invention provides an advanced system for improving communication among team members in a remote work environment and for appropriately evaluating members' emotions. This system utilizes a server, terminals, and an emotion engine responsible for emotion recognition.

[0744] First, the server retrieves the online meeting audio and transcript data and automatically generates meeting minutes. The generated minutes are summarized and delivered to the user's device. This allows the user to easily understand the content of the meeting.

[0745] Next, the server retrieves task progress from the project management tool and evaluates each member's contribution to the project. The server creates a dashboard that visualizes these contributions as graphs and charts and delivers it to the user's terminal. The user can use this dashboard to visually understand the overall team progress and their own role.

[0746] The emotion engine analyzes real-time communication data between members to recognize the user's emotional state. Furthermore, based on this analysis, it evaluates the user's stress level. This information is provided as feedback from the server to the user's terminal. By obtaining information about their emotional tendencies and states of heightened stress, users can take appropriate action.

[0747] For example, if a user experiences stress during an online meeting due to a heated discussion, the emotion engine will detect this change in emotion. If the analysis reveals an increase in stress levels, the server will communicate this information to the user and offer suggestions and countermeasures to help them relax.

[0748] In this way, the entire system can improve the user's work efficiency while also providing support that takes their mental health into consideration.

[0749] The following describes the processing flow.

[0750] Step 1:

[0751] The server retrieves the audio or transcript from the meeting platform after the online meeting has ended. This data is recorded to accurately reflect the content of the meeting.

[0752] Step 2:

[0753] The server passes the acquired meeting data to a generation AI, which performs summarization and generates meeting minutes. The minutes include the main topics discussed, decisions made, and participants' opinions from the meeting.

[0754] Step 3:

[0755] The server delivers the generated meeting minutes to the user's terminal. The user can open the meeting minutes on their terminal and quickly review the meeting's content.

[0756] Step 4:

[0757] The server retrieves the progress of current tasks from the project management tool. This includes the task completion status of each team member and the overall project progress.

[0758] Step 5:

[0759] The server analyzes the acquired task data and calculates each member's contribution. It then generates a dashboard in a visually easy-to-understand format.

[0760] Step 6:

[0761] The generated dashboard is delivered to the user's device. Through this, the user can understand the overall team situation and their own contribution.

[0762] Step 7:

[0763] A server equipped with an emotion engine monitors users' text communications in real time. Text is collected from sources such as chat messages and emails.

[0764] Step 8:

[0765] The emotion engine allows the server to analyze the user's text data and determine their emotional state based on the context and tone. This enables the evaluation of the user's stress level and emotional changes.

[0766] Step 9:

[0767] Based on sentiment analysis, the server provides feedback to the user's device. This feedback includes the user's emotional progression and, if necessary, suggestions for relaxation.

[0768] Step 10:

[0769] Users can receive feedback through their devices and choose actions to reduce mental stress. This helps to alleviate the mental burden of daily work and maintain a healthy work style.

[0770] (Example 2)

[0771] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0772] In a remote work environment, improving the quality of communication among team members, as well as appropriately evaluating and providing feedback on members' emotional states and contributions to projects, are challenges. Current communication and project management tools make it difficult to grasp emotions and contributions, and do not adequately support work efficiency or mental health.

[0773] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0774] In this invention, the server includes means for collecting meeting data and converting speech to text, means for analyzing conversation content using natural language processing technology, and means for acquiring project data and evaluating the contribution of members to the work. This makes it possible to evaluate the emotional state of members in real time, understand their stress levels, and provide appropriate feedback.

[0775] "Methods for collecting meeting data and converting audio to text" refers to technologies that acquire recordings or live data of audio conferences, encode them using speech recognition technology, and convert them into text format.

[0776] "Methods for analyzing conversation content using natural language processing technology" refer to technologies that perform semantic analysis and contextual understanding on generated text data and structure the information.

[0777] "A means of acquiring project data and evaluating the contribution of team members" refers to a technology that analyzes task management information and calculates the contribution of each team member based on the role they played and the tasks they completed within the project.

[0778] "Means of visually displaying contribution using data visualization technology" refers to technologies that display evaluated contribution data in a graph or chart format so that it can be intuitively understood.

[0779] "A means of analyzing communication data in real time and evaluating emotional states" refers to a technology that uses algorithms to analyze text data from conversations and communications in real time and estimate emotional states.

[0780] "A method for creating supportive messages based on emotion analysis" refers to a technology that automatically generates messages to support or encourage improvement of a person's emotions, based on the results of emotion analysis.

[0781] "Methods for measuring stress levels and providing countermeasures" refer to technologies that quantify a user's psychological stress state and then present relaxation techniques and action plans accordingly.

[0782] This invention is a system designed to improve effective communication and work efficiency among team members in a remote work environment. The system's main components are a server, user terminals, and an emotion analysis engine.

[0783] The server collects meeting data via the online meeting platform's API and uses "speech recognition technology" to transcribe the recordings into text. The specific software used is a "speech recognition engine." The generated text is then analyzed using "natural language processing technology" and summarized as meeting minutes. The generated minutes are then condensed using a summarization algorithm and sent to the user's device. This process allows the user to instantly understand the important meeting content.

[0784] Furthermore, the server retrieves project status from the "project management tool" and evaluates the contribution of members to their tasks. Using data visualization tools, the evaluation results are visually displayed as a dashboard and shown on the user's terminal. This allows users to visually check project progress and their own roles.

[0785] The sentiment analysis engine uses "sentiment analysis technology" to analyze real-time communication data and recognize the user's emotional state. For example, when a specific emotional state or stress level is detected, the server generates feedback and sends a message to the user suggesting an appropriate action plan.

[0786] For example, if a user experiences stress during a meeting, the emotion analysis engine will detect this change and provide feedback such as, "Your stress level is rising; we recommend taking a 5-minute break." An example of a prompt to the generative AI model would be, "Analyze the emotional state of this conversation and assess the stress level." This allows users to manage their own emotional state appropriately and take steps to reduce stress.

[0787] This entire system aims to provide a comfortable remote work environment by supporting users' mental health and promoting efficient communication.

[0788] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0789] Step 1:

[0790] The server retrieves meeting data using the API of the online meeting platform. The input data consists of audio files and text transcripts. The server converts this data into text using speech recognition technology to obtain text data of the meeting content. Specifically, it analyzes the audio files to generate text information and prepares it for the next processing step.

[0791] Step 2:

[0792] The server performs natural language processing on text data using a generative AI model. The input is the text of the meeting generated in step 1, and the output is a summarized meeting transcript. Natural language processing technology extracts the key points of the text and summarizes them in a format that is easy for the user to understand.

[0793] Step 3:

[0794] The server retrieves project data using the project management tool's API. Inputs include task information and progress data, while output is an evaluation of each member's contribution. The server analyzes the collected data and generates metrics for quantitatively evaluating contributions. This is based on task completion and progress.

[0795] Step 4:

[0796] The server uses data visualization technology to visually display the contribution level evaluated in step 3. The input is contribution level data, and the output is visualized data in the form of graphs and charts. Specifically, it creates a dashboard in a user-friendly format based on the evaluation results and delivers it to the user's terminal.

[0797] Step 5:

[0798] The server sends real-time communication data to the sentiment analysis engine. The input is text messages and conversation data, and the output is analyzed data indicating the user's emotional state. The sentiment analysis engine evaluates the emotional state in real time and provides the results to the server.

[0799] Step 6:

[0800] The server generates and presents a feedback message to the user based on the results of the emotion analysis. The input is the emotional state data obtained in step 5, and the output is a message containing specific countermeasures and suggestions. Based on this prompt, the server generates advice for stress reduction and sends it to the user's terminal.

[0801] (Application Example 2)

[0802] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0803] In a remote work environment, there is a need to facilitate smooth communication among team members and provide appropriate feedback tailored to each member's contribution and emotional state. Furthermore, improving work efficiency and managing mental health are also necessary, and building a comprehensive system to support these aspects remains an unresolved challenge.

[0804] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0805] In this invention, the server includes means for generating meeting minutes, means for analyzing project status, means for generating feedback and praise messages, means for performing sentiment analysis, means for providing ergonomic support, means for processing audio and image data, means for displaying progress information, and means for presenting stress-based relaxation methods. This enables efficient communication among team members and appropriate support that takes into account their individual emotional states, even in a remote environment.

[0806] A "meeting minutes generation method" is a technique that acquires recording data from online meetings and automatically generates meeting minutes that summarize the key points of the conversation.

[0807] "Methods for analyzing project status" refer to methods for incorporating project management information and analyzing progress and task status.

[0808] "Methods for visualizing member contributions" refer to methods that visually display each team member's contribution to a project and the progress of their tasks.

[0809] "Means for generating feedback and praise messages" refers to methods for providing members with specific feedback and praise messages based on analyzed data.

[0810] "Means of performing sentiment analysis" refers to methods that analyze voice and image data to evaluate the user's emotional state.

[0811] "Means of providing ergonomic support" refers to methods that analyze the user's work environment and posture and propose the optimal work method and environment.

[0812] "Means for processing audio and image data" refers to methods for processing collected audio and image data using a computer and extracting necessary information.

[0813] "Means of displaying progress information" refers to methods of displaying the progress of a project or task to the user in real time.

[0814] "A means of suggesting stress-based relaxation methods" refers to a method that recommends appropriate relaxation methods according to the user's stress level, as detected through emotion analysis.

[0815] To implement this invention, a server plays a central role. The server acquires audio data from online meetings and converts the audio to text using the Google Cloud Speech-to-Text API. This text data is then analyzed using the natural language processing library spaCy to automatically generate and summarize meeting minutes.

[0816] Next, the server uses project management information to analyze the project's progress. This analysis is then visually displayed to the user using the Django framework. Each member's contribution is evaluated based on the time allocation and completion rate of each task, and this is displayed on the user's terminal as a visualized dashboard.

[0817] For emotion analysis, the server utilizes IBM Watson and the Microsoft Azure Emotion Recognition API to analyze voice and image data and assess the user's emotional state in real time. If the user's stress level is detected, the device will suggest relaxation methods, which may include playing music or providing breathing exercises.

[0818] For example, if a user experiences stress during an online meeting, the server detects the emotional shift and immediately notifies the user's device with a message such as, "Let's take a break. How about a deep breath?" By using a generative AI model, real-time emotional analysis and feedback become possible, supporting the user's mental well-being.

[0819] An example of a prompt would be, "Please provide three key conclusions from this meeting. Also, analyze participants' emotions in real time and suggest relaxation techniques as needed."

[0820] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0821] Step 1:

[0822] The server receives audio data from an online meeting. It takes the audio data as input and converts it to text data using the Google Cloud Speech-to-Text API. As output, it generates meeting minutes data in text format.

[0823] Step 2:

[0824] The server analyzes the acquired text data using the natural language processing library spaCy. It uses the text data as input to extract context and important keywords, thereby creating a summary. The output is a summarized meeting minutes document.

[0825] Step 3:

[0826] The server analyzes project management information. Using data obtained from project management tools as input, it calculates the task progress of each member. The output generates data including each member's contribution.

[0827] Step 4:

[0828] The server uses the Django framework to visualize the analyzed member contributions. Using the data from step 3 as input, it builds a real-time dashboard. As output, the visualized progress dashboard is displayed on the user's terminal.

[0829] Step 5:

[0830] The server performs emotion analysis using IBM Watson and the Microsoft Azure Emotion Recognition API. It processes audio and image data from meetings as input to determine emotional states. Emotional information, such as the user's stress level, is generated as output.

[0831] Step 6:

[0832] The device receives emotional data from the server and suggests relaxation methods tailored to the user's stress level. It uses emotional information as input and a generative AI model to create appropriate feedback. The output is a notification to the user recommending relaxation methods.

[0833] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0834] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0835] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0836] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0837] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0838] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0839] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0840] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0841] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0842] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0843] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0844] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0845] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0846] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0847] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0848] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0849] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0850] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0851] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0852] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0853] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0854] The following is further disclosed regarding the embodiments described above.

[0855] (Claim 1)

[0856] A means of generating meeting minutes,

[0857] Means for analyzing the project status,

[0858] A means to visualize the contributions of members,

[0859] Means for generating feedback and praise messages,

[0860] Methods for conducting sentiment analysis,

[0861] A system that includes this.

[0862] (Claim 2)

[0863] The system according to claim 1, which acquires online meeting data and automatically generates a summary.

[0864] (Claim 3)

[0865] The system according to claim 1, which analyzes project management information and visually displays the task contribution of each member.

[0866] "Example 1"

[0867] (Claim 1)

[0868] A means of operating based on bidirectional communication, acquiring meeting data and generating summaries,

[0869] A means of analyzing project progress information and visualizing structured data,

[0870] A means of analyzing communication information and generating appropriate feedback,

[0871] A means of conducting emotional evaluations and analyzing the emotional states of the members,

[0872] A system that includes this.

[0873] (Claim 2)

[0874] The system according to claim 1, which acquires audio recordings or text records after the meeting has ended and automatically generates a summary.

[0875] (Claim 3)

[0876] The system according to claim 1, which analyzes project management information and visually displays the contribution of each member.

[0877] "Application Example 1"

[0878] (Claim 1)

[0879] A means of generating meeting minutes,

[0880] Means for analyzing the project status,

[0881] A means to visualize the contributions of members,

[0882] Means for generating feedback and praise messages,

[0883] Methods for conducting sentiment analysis,

[0884] Means to support communication among residents,

[0885] Information display means using smart devices,

[0886] A system that includes this.

[0887] (Claim 2)

[0888] The system according to claim 1, which acquires online meeting data and automatically generates a summary.

[0889] (Claim 3)

[0890] The system according to claim 1, which analyzes project management information and visually displays the task contribution of each member.

[0891] "Example 2 of combining an emotion engine"

[0892] (Claim 1)

[0893] A means of collecting meeting data and converting audio to text,

[0894] A means of analyzing conversation content using natural language processing technology,

[0895] A means of acquiring project data and evaluating the contribution of team members to the work,

[0896] A means of visually displaying the degree of contribution using data visualization technology,

[0897] A means of analyzing communication data in real time and evaluating emotional states,

[0898] A method for creating supportive messages based on emotion analysis,

[0899] A means of measuring stress levels and providing countermeasures,

[0900] A system that includes this.

[0901] (Claim 2)

[0902] The system according to claim 1, which processes meeting data for summary and distributes it to the terminals of the members.

[0903] (Claim 3)

[0904] The system according to claim 1, which analyzes project management data and uses visualization techniques to show the contribution of each member.

[0905] "Application example 2 when combining with an emotional engine"

[0906] (Claim 1)

[0907] A means of generating meeting minutes,

[0908] Means for analyzing the project status,

[0909] A means to visualize the contributions of members,

[0910] Means for generating feedback and praise messages,

[0911] Methods for conducting sentiment analysis,

[0912] Means of providing ergonomic support,

[0913] Means for processing audio and image data,

[0914] A means of displaying information on the progress status,

[0915] A means of presenting stress-based relaxation methods,

[0916] A system that includes this.

[0917] (Claim 2)

[0918] The system according to claim 1, which acquires online meeting data and automatically generates a summary.

[0919] (Claim 3)

[0920] The system according to claim 1, which analyzes project management information, visually displays the task contribution of each member, and provides relaxation suggestions according to the individual's mental state. [Explanation of Symbols]

[0921] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of generating meeting minutes, Means for analyzing the project status, A means to visualize the contributions of members, Means for generating feedback and praise messages, Methods for conducting sentiment analysis, A system that includes this.

2. The system according to claim 1, which acquires online meeting data and automatically generates a summary.

3. The system according to claim 1, which analyzes project management information and visually displays the task contribution of each member.

Images