system

Abstract

The system according to this embodiment aims to grasp the progress of a meeting and any deviations in the points of discussion in real time, and to correct them appropriately. [Solution] The system according to the embodiment comprises an analysis unit, a visualization unit, an identification unit, and a guide unit. The analysis unit analyzes the progress of the meeting in real time. The visualization unit visualizes the progress toward the goal and the issues based on the data analyzed by the analysis unit. The identification unit points out discrepancies in the discussion based on the information visualized by the visualization unit. The guide unit provides a guide for correcting the discrepancies pointed out by the identification unit.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, it is difficult to grasp the progress of a meeting and the deviation of arguments in real time and make appropriate corrections, and there is room for improvement to enhance the quality of the meeting.

[0005] The system according to the embodiment aims to grasp the progress of a meeting and the deviation of arguments in real time and make appropriate corrections.

Means for Solving the Problems

[0006] The system according to this embodiment comprises an analysis unit, a visualization unit, an identification unit, and a guide unit. The analysis unit analyzes the progress of the meeting in real time. The visualization unit visualizes the progress toward the goal and the issues based on the data analyzed by the analysis unit. The identification unit points out discrepancies in the discussion based on the information visualized by the visualization unit. The guide unit provides guidance for correcting the discrepancies pointed out by the identification unit. [Effects of the Invention]

[0007] The system according to this embodiment can grasp the progress of a meeting and any deviations from the main points in real time, and correct them appropriately. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applicable to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 comprises a computer 36, a receiving device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The receiving device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The meeting facilitation support system according to an embodiment of the present invention is a system that analyzes the progress of a meeting in real time and visualizes the progress toward the set goals, the current discussion points, the status of responses to those points, and unresolved issues. This system provides guidance to prevent the discussion from deviating from the main topic and to prevent important points from being overlooked. For example, in the preparation stage of a meeting, the system proposes specific goal settings according to the purpose. Next, the system automatically creates the optimal order of discussions and timeline, and automatically links past related discussions. During the meeting, the system automatically detects deviations from the main topic of discussion and issues warnings, automatically organizes key points, and supports participants in focusing on the progress. It also has a function to visualize unresolved issues and navigate to conclusions. Furthermore, the system can adjust the priority of discussions according to the remaining time and can also search the company database and the web based on the context. This system has basic functions such as agenda optimization, meeting efficiency analysis, automatic linking of related meetings, and graphing of organizational knowledge. In the future, the aim is to optimize decision-making across the entire organization and to form an autonomous meeting improvement cycle. This allows the meeting facilitation support system to improve the quality of meetings by analyzing the progress of the meeting in real time, visualizing progress toward the goal and the points of discussion, pointing out deviations in the discussion, and providing guidance for correction.

[0029] The meeting facilitation support system according to this embodiment comprises an analysis unit, a visualization unit, an identification unit, and a guide unit. The analysis unit analyzes the progress of the meeting in real time. For example, the analysis unit monitors the progress of the meeting in real time and collects data. Based on the collected data, the analysis unit analyzes the progress of the meeting. For example, the analysis unit analyzes the content and frequency of comments and evaluates the progress of the meeting. The visualization unit visualizes the progress toward the goal and the issues based on the data analyzed by the analysis unit. For example, the visualization unit displays the progress in graphs or charts. The visualization unit can also display the issues in list format. For example, the visualization unit can display the progress in different colors to improve visibility. The identification unit points out deviations in the discussion based on the information visualized by the visualization unit. For example, the identification unit issues a warning when the discussion deviates from the main topic. The identification unit can also make suggestions to prevent important issues from being overlooked. For example, the identification unit points out deviations in the discussion in real time and encourages corrections. The guiding unit provides guidance for correcting discrepancies pointed out by the identification unit. The guiding unit, for example, proposes specific actions to support the progress of the discussion. The guiding unit can also provide reference materials. The guiding unit provides guidance for smoother discussion progress. As a result, the meeting facilitation support system according to this embodiment can improve the quality of the meeting by analyzing the progress of the meeting in real time, visualizing progress toward the goal and points of discussion, identifying discrepancies in the discussion, and providing guidance for correction.

[0030] The analysis department analyzes the progress of meetings in real time. Specifically, it monitors the progress of meetings in real time and collects audio and text data. Audio data is obtained by recording what is said during the meeting and converting it into text using speech recognition technology. Text data is obtained from meeting minutes, chat logs, etc. Based on this data, the analysis department analyzes the content and frequency of statements, as well as the emotions and tone of the speakers. For example, it extracts keywords from the content of statements and compiles the speaking time for each speaker. It also uses natural language processing technology to analyze the sentiment of statements and identify positive and negative statements. This allows for a quantitative and qualitative evaluation of the progress of meetings. Furthermore, by accumulating past meeting data and learning meeting progress patterns and trends, the analysis department can also predict the progress of future meetings. For example, it can predict how easily discussions on specific topics will gain momentum and where discussions are likely to stagnate, allowing for proactive measures to be taken. In this way, the analysis department can analyze the progress of meetings in real time and provide data to improve the quality of meetings.

[0031] The visualization unit visualizes progress toward goals and issues based on data analyzed by the analysis unit. Specifically, it displays progress using graphs and charts for easy visual understanding. For example, it displays a timeline showing the progress of the meeting and a pie chart showing the percentage of speaking time for each agenda item. It also displays issues in a list format, clearly indicating the content and speakers for each issue. Furthermore, it enhances visibility by displaying progress using color coding. For example, it displays green if progress is on track and red if it is behind schedule. The visualization unit updates this information in real time, so that the progress of the meeting is always displayed in its most up-to-date state. In addition, the visualization unit provides a customizable dashboard for users, allowing them to see necessary information at a glance. For example, widgets can be placed to display the progress of a specific agenda item or a list of important issues. In this way, the visualization unit can visually display the progress and issues of a meeting in an easy-to-understand manner, improving the efficiency of the meeting.

[0032] The feedback unit points out deviations in discussions based on information visualized by the visualization unit. Specifically, it issues warnings when the discussion deviates from the main topic. For example, it monitors the progress of the meeting and displays a warning if there is a continuous stream of comments unrelated to the agenda. It also provides feedback to prevent important points from being overlooked. For example, it issues a warning if there are few comments on a particular point or if the discussion is biased. The feedback unit displays these warnings in real time and prompts action to correct the progress of the meeting. Furthermore, the feedback unit can learn points where discussions tend to deviate based on past meeting data and issue warnings in advance. For example, if discussions on a particular agenda item have deviated many times in the past, it will issue a warning when that agenda item is discussed again. In this way, the feedback unit can point out deviations in discussions in real time and ensure smooth meeting progress.

[0033] The Guidance Department provides guidance to correct discrepancies pointed out by the Critique Department. Specifically, it proposes concrete actions to support the progress of the discussion. For example, it asks specific questions or makes suggestions to return to the agenda when the discussion deviates from the main topic. It also facilitates the smooth progress of the discussion by providing reference materials. For example, it provides relevant data or past meeting minutes to be used as references for the discussion. Furthermore, the Guidance Department provides templates and frameworks to support the progress of the discussion. For example, it provides a progress schedule for each agenda item and a framework for organizing the points of discussion. In this way, the Guidance Department can correct discrepancies in the discussion and provide concrete support to ensure the smooth progress of the meeting. In addition, the Guidance Department can collect user feedback and continuously improve the accuracy and effectiveness of the guide content. For example, it collects user evaluations and comments on the guide content and incorporates them into the next meeting. In this way, the Guidance Department can provide effective support to users and improve the quality of the meeting.

[0034] The proposal team proposes setting specific goals tailored to the meeting's objectives during the preparation phase. For example, the proposal team clarifies the objectives to be achieved based on the meeting's purpose. The proposal team can also set evaluation criteria. For example, the proposal team proposes specific goals according to the meeting's purpose and shares them with the participants. By proposing specific goal setting during the preparation phase, the proposal team clarifies the meeting's purpose and supports efficient progress.

[0035] The creation function automatically generates the optimal discussion order and timeline. For example, it evaluates the importance and relevance of agenda items and determines the optimal order. The creation function can also set time allocation criteria and set the time for each agenda item. For example, it determines the order based on the importance of the agenda items and creates the timeline. This streamlines the meeting by automatically creating the optimal discussion order and timeline.

[0036] The linking function automatically connects related past discussions. For example, it evaluates the similarity and relevance of discussion content and links related discussions. The linking function can also search the content of past discussions and provide information relevant to the current discussion. For example, it automatically links highly relevant past discussions and presents them to participants. By automatically linking related past discussions, it facilitates a deeper understanding of the meeting content and promotes more effective discussion.

[0037] The summarization function automatically organizes key points. For example, it extracts and summarizes important points from a discussion. The summarization function can also set how information is summarized and organize important information. For example, it can automatically organize particularly important points from a discussion and present them to participants. This automatically organizes key points, making the meeting run more smoothly and ensuring that important points are not overlooked.

[0038] The coordination unit adjusts the priority of discussions according to the remaining time. For example, the coordination unit evaluates the importance of agenda items and time constraints to determine priority. The coordination unit can also adjust the time allocation and allocate time to important agenda items. For example, the coordination unit adjusts the priority of discussions according to the remaining time to support efficient progress. This allows discussions to proceed efficiently within the time limit by adjusting the priority of discussions according to the remaining time.

[0039] The search unit searches internal databases and the web based on context. For example, it analyzes preceding and succeeding statements and related topics to find the necessary information. The search unit can also search relevant literature and historical data to provide information useful for discussion. For example, it searches internal databases and the web based on context to quickly obtain the necessary information. This allows for the rapid acquisition of necessary information by searching internal databases and the web based on context, supporting the discussion.

[0040] The analytics department analyzes the progress of meetings in real time and automatically determines the priority of important issues. For example, the analytics department analyzes the frequency and content of comments during the meeting to extract important issues. The analytics department can also analyze participants' reactions and prioritize important issues. The analytics department can also refer to past meeting data to determine the priority of important issues. This promotes efficient discussion by analyzing the progress of meetings in real time and automatically determining the priority of important issues.

[0041] The analysis department improves the accuracy of its analysis by referring to past meeting data when analyzing the progress of meetings. For example, the analysis department refers to past meeting data and analyzes the progress of similar discussions. The analysis department can also extract discussion patterns and incorporate them into the analysis. The analysis department can also identify factors that influence the progress of discussions and incorporate them into the analysis. In this way, by referring to past meeting data, the accuracy of the analysis is improved and more effective discussions are supported.

[0042] The analysis department considers the frequency and content of participants' contributions when analyzing the progress of a meeting. For example, the analysis department analyzes the frequency of participants' contributions and gives more weight to the opinions of those who speak frequently. The analysis department can also analyze the content of participants' contributions, extract important keywords, and incorporate them into the analysis. The analysis department can also comprehensively analyze the frequency and content of participants' contributions to identify important points of discussion. By considering the frequency and content of participants' contributions, a more accurate analysis becomes possible.

[0043] The analysis department applies different analytical methods depending on the purpose and theme of the meeting when analyzing the progress of the meeting. For example, if the purpose of the meeting is decision-making, the analysis department will focus on analyzing the information necessary for decision-making. If the theme of the meeting is brainstorming, the analysis department can also analyze the number and quality of ideas. If the purpose of the meeting is information sharing, the analysis department can also analyze how information is conveyed. By applying analytical methods appropriate to the purpose and theme of the meeting, more effective analysis becomes possible.

[0044] The visualization unit updates the visualization content in real time according to the progress of the meeting. For example, the visualization unit updates the visualization content in real time according to what is said and the progress of the meeting. The visualization unit can also update the visualization content when important points are raised. The visualization unit can also update the visualization content when unresolved issues arise. In this way, by updating the visualization content in real time according to the progress of the meeting, the latest information can be provided.

[0045] The visualization section highlights important points and unresolved issues during visualization. For example, the visualization section can highlight important points to ensure participants don't miss them. The visualization section can also highlight unresolved issues to encourage discussion toward their resolution. The visualization section can also color-code important points and unresolved issues to improve visibility. This highlights important points and unresolved issues, ensuring participants don't miss them.

[0046] The visualization unit applies different visualization methods depending on the progress of the meeting. For example, it uses graphs and charts depending on the progress of the meeting. It can also use text and images. The visualization unit can change the visualization method in real time. This allows for more effective visualization by applying visualization methods according to the progress of the meeting.

[0047] The visualization unit customizes the visualization content by considering the attribute information of the meeting participants. For example, the visualization unit highlights important information according to the participant's job title. The visualization unit can also highlight relevant information according to the participant's field of expertise. The visualization unit can also customize and display important information according to the participant's interests. In this way, by considering the attribute information of the meeting participants, it can provide the most optimal visualization content for each participant.

[0048] The feedback team updates its feedback in real time according to the progress of the meeting. For example, the feedback team can point out discrepancies in the discussion in real time as the meeting progresses. The feedback team can also point out overlooked important points in real time. The feedback team can also point out unresolved issues in real time. This allows for the provision of the latest information by updating feedback in real time according to the progress of the meeting.

[0049] The section that points out issues prioritizes highlighting important points and unresolved issues. For example, it can prioritize highlighting important points to facilitate the progress of the discussion. It can also prioritize highlighting unresolved issues to encourage discussion toward their resolution. The section that points out issues can also use color coding to highlight important points and unresolved issues, thereby improving visibility. This prioritizes highlighting important points and unresolved issues, thus facilitating the progress of the discussion.

[0050] The feedback team applies different feedback methods depending on the progress of the meeting. For example, they may use verbal or written feedback depending on the meeting's progress. They can also use detailed or concise feedback. The feedback team can even change their feedback method in real time. This allows for more effective feedback by applying feedback methods appropriate to the meeting's progress.

[0051] The feedback team customizes its feedback by considering the attribute information of the meeting participants. For example, it can highlight important points based on the participant's position. It can also highlight relevant points based on the participant's area of ​​expertise. Furthermore, it can customize important points based on the participant's interests. This allows the feedback team to provide feedback that is optimal for each participant by considering their attribute information.

[0052] The guiding team updates the guide content in real time according to the progress of the meeting. For example, the guiding team can guide participants in real time to identify discrepancies in the discussion during the meeting. The guiding team can also guide participants in real time to identify any overlooked important points. The guiding team can also guide participants in real time to identify any unresolved issues. By updating the guide content in real time according to the progress of the meeting, the team can provide the most up-to-date information.

[0053] The guiding function prioritizes guiding participants through important points and unresolved issues. For example, it can prioritize guiding participants through important points to facilitate the progress of the discussion. It can also prioritize guiding participants through unresolved issues to encourage discussion toward their resolution. The guiding function can also color-code important points and unresolved issues to improve visibility. This allows for the discussion to progress by prioritizing the guidance of important points and unresolved issues.

[0054] The guiding team applies different guiding methods depending on the progress of the meeting. For example, the guiding team may switch between verbal and written guidance depending on the meeting's progress. The guiding team can also switch between detailed and concise guidance. The guiding team can even change guiding methods in real time. This allows for more effective guiding by applying guiding methods that are appropriate to the meeting's progress.

[0055] The guiding department customizes the guide content by considering the attribute information of the meeting participants. For example, the guiding department may highlight important guides based on the participant's job title. The guiding department may also highlight relevant guides based on the participant's area of ​​expertise. The guiding department may also customize important guides based on the participant's interests. In this way, by considering the attribute information of the meeting participants, the guiding department can provide the most appropriate guide content for each participant.

[0056] The proposal team, when proposing goal setting, refers to past meeting data to suggest the optimal goal. For example, the proposal team may refer to past meeting data and suggest the optimal goal based on the goals of similar meetings. The proposal team can also suggest the optimal goal based on the goals of successful meetings. The proposal team can also suggest the optimal goal by reflecting the goals of unsuccessful meetings. In this way, by referring to past meeting data, the optimal goal setting can be proposed.

[0057] The proposal team applies different proposal methods depending on the purpose and theme of the meeting when proposing goal setting. For example, if the purpose of the meeting is decision-making, the proposal team will propose goals necessary for decision-making. If the theme of the meeting is brainstorming, the proposal team may also propose goals that emphasize the quantity and quality of ideas. If the purpose of the meeting is information sharing, the proposal team may also propose goals that emphasize the effectiveness of information dissemination. By applying proposal methods appropriate to the purpose and theme of the meeting, more effective goal setting becomes possible.

[0058] The creation process, when creating the discussion order and timeline, refers to past meeting data to create the optimal order and timeline. For example, it refers to past meeting data and creates the optimal order and timeline based on the order and timeline of similar meetings. It can also create the optimal order and timeline based on the order and timeline of successful meetings. It can also create the optimal order and timeline by reflecting the order and timeline of unsuccessful meetings. In this way, the optimal discussion order and timeline can be created by referring to past meeting data.

[0059] The planning team applies different methods to create the discussion order and timeline depending on the purpose and theme of the meeting. For example, if the purpose of the meeting is decision-making, the planning team will create the order and timeline necessary for decision-making. If the theme of the meeting is brainstorming, the planning team can also create an order and timeline that emphasizes the quantity and quality of ideas. If the purpose of the meeting is information sharing, the planning team can also create an order and timeline that emphasizes the transmission of information. By applying a planning method appropriate to the purpose and theme of the meeting, a more effective discussion order and timeline can be created.

[0060] The linking function prioritizes linking highly relevant discussions when connecting past discussions. For example, it prioritizes linking past discussions that are relevant to the current discussion. It can also prioritize linking discussions that contain important points. It can also prioritize linking discussions that contain unresolved issues. By prioritizing the linking of highly relevant discussions, it can provide information useful for the current discussion.

[0061] The linking function applies different linking methods depending on the purpose and theme of the meeting when linking past discussions. For example, if the purpose of the meeting is decision-making, the linking function will prioritize linking discussions related to decision-making. If the theme of the meeting is brainstorming, the linking function can also prioritize linking discussions that emphasize the quantity and quality of ideas. If the purpose of the meeting is information sharing, the linking function can also prioritize linking discussions that emphasize the status of information transmission. By applying a linking method appropriate to the purpose and theme of the meeting, it becomes possible to link discussions more effectively.

[0062] The summarizing team prioritizes organizing important points and unresolved issues when summarizing key points. For example, the summarizing team can prioritize organizing important points to facilitate the progress of the discussion. The summarizing team can also prioritize organizing unresolved issues to facilitate discussion toward their resolution. The summarizing team can also improve visibility by using color coding to distinguish between important points and unresolved issues. This allows them to prioritize organizing important points and unresolved issues, thereby facilitating the progress of the discussion.

[0063] The summarization team applies different summarization methods depending on the progress of the meeting. For example, they prioritize summarizing important points as the meeting progresses. They can also prioritize summarizing unresolved issues. The summarization team can also change the summarization method in real time. This allows for more effective summarization of key points by applying a summarization method that is appropriate to the progress of the meeting.

[0064] The coordination unit updates the prioritization of discussions in real time according to the progress of the meeting. For example, the coordination unit prioritizes important discussions according to the progress of the meeting. The coordination unit can also prioritize unresolved issues. The coordination unit can also update the prioritization in real time. This allows for the provision of the latest information by updating the prioritization in real time according to the progress of the meeting.

[0065] The coordination team applies different coordination methods depending on the purpose and theme of the meeting when adjusting the priority of discussions. For example, if the purpose of the meeting is decision-making, the coordination team will prioritize discussions necessary for decision-making. If the theme of the meeting is brainstorming, the coordination team may also prioritize discussions that emphasize the quantity and quality of ideas. If the purpose of the meeting is information sharing, the coordination team may also prioritize discussions that emphasize the effectiveness of information dissemination. By applying coordination methods appropriate to the purpose and theme of the meeting, it becomes possible to adjust the priority of discussions more effectively.

[0066] The search function updates search results in real time according to the progress of the meeting. For example, the search function prioritizes displaying important information in the search results according to the progress of the meeting. The search function can also prioritize displaying information related to unresolved issues in the search results. The search function can also update search results in real time. This allows for the provision of the latest information by updating search results in real time according to the progress of the meeting.

[0067] The search function applies different display methods depending on the purpose and theme of the meeting when displaying search results. For example, if the purpose of the meeting is decision-making, the search function will prioritize displaying information necessary for decision-making. If the theme of the meeting is brainstorming, the search function can also prioritize displaying information that emphasizes the quantity and quality of ideas. If the purpose of the meeting is information sharing, the search function can also prioritize displaying information that emphasizes the status of information transmission. By applying display methods tailored to the purpose and theme of the meeting, more effective search results can be displayed.

[0068] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0069] The meeting facilitation support system can also include a summarization unit that automatically summarizes participants' comments and displays the summarized content in real time. The summarization unit can, for example, analyze the comments, extract key points, and summarize them. It can also prioritize summaries based on the frequency and importance of each comment. The summarization unit can visually display the summarized content, making it easier for participants to grasp the overall picture of the discussion. This allows participants to quickly understand the main points of the discussion and promote efficient discussion by automatically summarizing the comments.

[0070] The meeting facilitation support system can also include a classification unit that automatically categorizes participants' comments and displays the categorized content in real time. The classification unit, for example, analyzes the comments and categorizes them by topic. It can also determine the priority of the classification based on the content and relevance of the comments. The classification unit can visually display the categorized content, making it easier for participants to understand the progress of the discussion. This allows participants to quickly understand the progress of the discussion and promote efficient discussion by automatically categorizing the comments.

[0071] The meeting facilitation support system can also include an evaluation unit that automatically evaluates participants' contributions and displays the evaluation results in real time. For example, the evaluation unit analyzes the content of the contributions and assesses their quality and importance. The evaluation unit can also prioritize evaluations based on the content and relevance of the contributions. The evaluation unit can also visually display the evaluation results, making it easier for participants to understand the quality and importance of their contributions. This allows participants to quickly understand the quality and importance of their contributions by automatically evaluating their contributions, thereby promoting efficient discussion.

[0072] The meeting facilitation support system can also include a recording unit that automatically records participants' statements and displays the recorded content in real time. The recording unit can, for example, analyze the statements and record key points. It can also prioritize recordings based on the frequency and importance of each statement. The recording unit can visually display the recorded content, making it easier for participants to grasp the overall picture of the discussion. This allows participants to quickly understand the main points of the discussion and promote efficient discussion by automatically recording what is said.

[0073] The meeting facilitation support system can also include a summarization unit that automatically summarizes participants' comments and displays the summarized content in real time. The summarization unit can, for example, analyze the comments, extract key points, and summarize them. It can also prioritize summaries based on the frequency and importance of each comment. The summarization unit can visually display the summarized content, making it easier for participants to grasp the overall picture of the discussion. This allows participants to quickly understand the main points of the discussion and promote efficient discussion by automatically summarizing the comments.

[0074] The meeting facilitation support system can also include a classification unit that automatically categorizes participants' comments and displays the categorized content in real time. The classification unit, for example, analyzes the comments and categorizes them by topic. It can also determine the priority of the classification based on the content and relevance of the comments. The classification unit can visually display the categorized content, making it easier for participants to understand the progress of the discussion. This allows participants to quickly understand the progress of the discussion and promote efficient discussion by automatically categorizing the comments.

[0075] The meeting facilitation support system can also include an evaluation unit that automatically evaluates participants' contributions and displays the evaluation results in real time. For example, the evaluation unit analyzes the content of the contributions and assesses their quality and importance. The evaluation unit can also prioritize evaluations based on the content and relevance of the contributions. The evaluation unit can also visually display the evaluation results, making it easier for participants to understand the quality and importance of their contributions. This allows participants to quickly understand the quality and importance of their contributions by automatically evaluating their contributions, thereby promoting efficient discussion.

[0076] The meeting facilitation support system can also include a recording unit that automatically records participants' statements and displays the recorded content in real time. The recording unit can, for example, analyze the statements and record key points. It can also prioritize recordings based on the frequency and importance of each statement. The recording unit can visually display the recorded content, making it easier for participants to grasp the overall picture of the discussion. This allows participants to quickly understand the main points of the discussion and promote efficient discussion by automatically recording what is said.

[0077] The following briefly describes the processing flow for example form 1.

[0078] Step 1: The analysis department analyzes the meeting's progress in real time. Specifically, they monitor the meeting's progress in real time and collect data. Based on the collected data, they analyze the content and frequency of contributions to evaluate the meeting's progress. Step 2: The visualization unit visualizes progress toward the goal and issues based on the data analyzed by the analysis unit. Specifically, it displays progress using graphs and charts, and issues in list format. Visualization is enhanced by color-coding the progress. Step 3: The feedback unit points out deviations in the discussion based on the information visualized by the visualization unit. Specifically, it issues warnings when the discussion deviates from the main topic and provides feedback to prevent important points from being overlooked. It points out deviations in the discussion in real time and encourages corrections. Step 4: The guide section provides guidance for correcting the discrepancies pointed out by the criticism section. Specifically, it proposes concrete actions to support the progress of the discussion and provides reference materials. It provides guidance to ensure the smooth progress of the discussion.

[0079] (Example of form 2) The meeting facilitation support system according to an embodiment of the present invention is a system that analyzes the progress of a meeting in real time and visualizes the progress toward the set goals, the current discussion points, the status of responses to those points, and unresolved issues. This system provides guidance to prevent the discussion from deviating from the main topic and to prevent important points from being overlooked. For example, in the preparation stage of a meeting, the system proposes specific goal settings according to the purpose. Next, the system automatically creates the optimal order of discussions and timeline, and automatically links past related discussions. During the meeting, the system automatically detects deviations from the main topic of discussion and issues warnings, automatically organizes key points, and supports participants in focusing on the progress. It also has a function to visualize unresolved issues and navigate to conclusions. Furthermore, the system can adjust the priority of discussions according to the remaining time and can also search the company database and the web based on the context. This system has basic functions such as agenda optimization, meeting efficiency analysis, automatic linking of related meetings, and graphing of organizational knowledge. In the future, the aim is to optimize decision-making across the entire organization and to form an autonomous meeting improvement cycle. This allows the meeting facilitation support system to improve the quality of meetings by analyzing the progress of the meeting in real time, visualizing progress toward the goal and the points of discussion, pointing out deviations in the discussion, and providing guidance for correction.

[0080] The meeting facilitation support system according to this embodiment comprises an analysis unit, a visualization unit, an identification unit, and a guide unit. The analysis unit analyzes the progress of the meeting in real time. For example, the analysis unit monitors the progress of the meeting in real time and collects data. Based on the collected data, the analysis unit analyzes the progress of the meeting. For example, the analysis unit analyzes the content and frequency of comments and evaluates the progress of the meeting. The visualization unit visualizes the progress toward the goal and the issues based on the data analyzed by the analysis unit. For example, the visualization unit displays the progress in graphs or charts. The visualization unit can also display the issues in list format. For example, the visualization unit can display the progress in different colors to improve visibility. The identification unit points out deviations in the discussion based on the information visualized by the visualization unit. For example, the identification unit issues a warning when the discussion deviates from the main topic. The identification unit can also make suggestions to prevent important issues from being overlooked. For example, the identification unit points out deviations in the discussion in real time and encourages corrections. The guiding unit provides guidance for correcting discrepancies pointed out by the identification unit. The guiding unit, for example, proposes specific actions to support the progress of the discussion. The guiding unit can also provide reference materials. The guiding unit provides guidance for smoother discussion progress. As a result, the meeting facilitation support system according to this embodiment can improve the quality of the meeting by analyzing the progress of the meeting in real time, visualizing progress toward the goal and points of discussion, identifying discrepancies in the discussion, and providing guidance for correction.

[0081] The analysis department analyzes the progress of meetings in real time. Specifically, it monitors the progress of meetings in real time and collects audio and text data. Audio data is obtained by recording what is said during the meeting and converting it into text using speech recognition technology. Text data is obtained from meeting minutes, chat logs, etc. Based on this data, the analysis department analyzes the content and frequency of statements, as well as the emotions and tone of the speakers. For example, it extracts keywords from the content of statements and compiles the speaking time for each speaker. It also uses natural language processing technology to analyze the sentiment of statements and identify positive and negative statements. This allows for a quantitative and qualitative evaluation of the progress of meetings. Furthermore, by accumulating past meeting data and learning meeting progress patterns and trends, the analysis department can also predict the progress of future meetings. For example, it can predict how easily discussions on specific topics will gain momentum and where discussions are likely to stagnate, allowing for proactive measures to be taken. In this way, the analysis department can analyze the progress of meetings in real time and provide data to improve the quality of meetings.

[0082] The visualization unit visualizes progress toward goals and issues based on data analyzed by the analysis unit. Specifically, it displays progress using graphs and charts for easy visual understanding. For example, it displays a timeline showing the progress of the meeting and a pie chart showing the percentage of speaking time for each agenda item. It also displays issues in a list format, clearly indicating the content and speakers for each issue. Furthermore, it enhances visibility by displaying progress using color coding. For example, it displays green if progress is on track and red if it is behind schedule. The visualization unit updates this information in real time, so that the progress of the meeting is always displayed in its most up-to-date state. In addition, the visualization unit provides a customizable dashboard for users, allowing them to see necessary information at a glance. For example, widgets can be placed to display the progress of a specific agenda item or a list of important issues. In this way, the visualization unit can visually display the progress and issues of a meeting in an easy-to-understand manner, improving the efficiency of the meeting.

[0083] The feedback unit points out deviations in discussions based on information visualized by the visualization unit. Specifically, it issues warnings when the discussion deviates from the main topic. For example, it monitors the progress of the meeting and displays a warning if there is a continuous stream of comments unrelated to the agenda. It also provides feedback to prevent important points from being overlooked. For example, it issues a warning if there are few comments on a particular point or if the discussion is biased. The feedback unit displays these warnings in real time and prompts action to correct the progress of the meeting. Furthermore, the feedback unit can learn points where discussions tend to deviate based on past meeting data and issue warnings in advance. For example, if discussions on a particular agenda item have deviated many times in the past, it will issue a warning when that agenda item is discussed again. In this way, the feedback unit can point out deviations in discussions in real time and ensure smooth meeting progress.

[0084] The Guidance Department provides guidance to correct discrepancies pointed out by the Critique Department. Specifically, it proposes concrete actions to support the progress of the discussion. For example, it asks specific questions or makes suggestions to return to the agenda when the discussion deviates from the main topic. It also facilitates the smooth progress of the discussion by providing reference materials. For example, it provides relevant data or past meeting minutes to be used as references for the discussion. Furthermore, the Guidance Department provides templates and frameworks to support the progress of the discussion. For example, it provides a progress schedule for each agenda item and a framework for organizing the points of discussion. In this way, the Guidance Department can correct discrepancies in the discussion and provide concrete support to ensure the smooth progress of the meeting. In addition, the Guidance Department can collect user feedback and continuously improve the accuracy and effectiveness of the guide content. For example, it collects user evaluations and comments on the guide content and incorporates them into the next meeting. In this way, the Guidance Department can provide effective support to users and improve the quality of the meeting.

[0085] The proposal team proposes setting specific goals tailored to the meeting's objectives during the preparation phase. For example, the proposal team clarifies the objectives to be achieved based on the meeting's purpose. The proposal team can also set evaluation criteria. For example, the proposal team proposes specific goals according to the meeting's purpose and shares them with the participants. By proposing specific goal setting during the preparation phase, the proposal team clarifies the meeting's purpose and supports efficient progress.

[0086] The creation function automatically generates the optimal discussion order and timeline. For example, it evaluates the importance and relevance of agenda items and determines the optimal order. The creation function can also set time allocation criteria and set the time for each agenda item. For example, it determines the order based on the importance of the agenda items and creates the timeline. This streamlines the meeting by automatically creating the optimal discussion order and timeline.

[0087] The linking function automatically connects related past discussions. For example, it evaluates the similarity and relevance of discussion content and links related discussions. The linking function can also search the content of past discussions and provide information relevant to the current discussion. For example, it automatically links highly relevant past discussions and presents them to participants. By automatically linking related past discussions, it facilitates a deeper understanding of the meeting content and promotes more effective discussion.

[0088] The summarization function automatically organizes key points. For example, it extracts and summarizes important points from a discussion. The summarization function can also set how information is summarized and organize important information. For example, it can automatically organize particularly important points from a discussion and present them to participants. This automatically organizes key points, making the meeting run more smoothly and ensuring that important points are not overlooked.

[0089] The coordination unit adjusts the priority of discussions according to the remaining time. For example, the coordination unit evaluates the importance of agenda items and time constraints to determine priority. The coordination unit can also adjust the time allocation and allocate time to important agenda items. For example, the coordination unit adjusts the priority of discussions according to the remaining time to support efficient progress. This allows discussions to proceed efficiently within the time limit by adjusting the priority of discussions according to the remaining time.

[0090] The search unit searches internal databases and the web based on context. For example, it analyzes preceding and succeeding statements and related topics to find the necessary information. The search unit can also search relevant literature and historical data to provide information useful for discussion. For example, it searches internal databases and the web based on context to quickly obtain the necessary information. This allows for the rapid acquisition of necessary information by searching internal databases and the web based on context, supporting the discussion.

[0091] The analysis unit estimates the user's emotions and analyzes the progress of the meeting based on the estimated emotions. For example, if the user is nervous, the analysis unit suggests a way to facilitate the meeting to help them relax. If the user is excited, the analysis unit can also encourage a calm approach to prevent the discussion from becoming heated. If the user is tired, the analysis unit can also simplify the proceedings and focus on the important points. In this way, by analyzing the progress of the meeting based on the user's emotions, a more appropriate approach can be suggested. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0092] The analytics department analyzes the progress of meetings in real time and automatically determines the priority of important issues. For example, the analytics department analyzes the frequency and content of comments during the meeting to extract important issues. The analytics department can also analyze participants' reactions and prioritize important issues. The analytics department can also refer to past meeting data to determine the priority of important issues. This promotes efficient discussion by analyzing the progress of meetings in real time and automatically determining the priority of important issues.

[0093] The analysis department improves the accuracy of its analysis by referring to past meeting data when analyzing the progress of meetings. For example, the analysis department refers to past meeting data and analyzes the progress of similar discussions. The analysis department can also extract discussion patterns and incorporate them into the analysis. The analysis department can also identify factors that influence the progress of discussions and incorporate them into the analysis. In this way, by referring to past meeting data, the accuracy of the analysis is improved and more effective discussions are supported.

[0094] The analysis unit estimates the user's emotions and displays an analysis of the meeting's progress based on the estimated emotions. For example, if the user is nervous, the analysis unit displays a simple and easy-to-understand analysis. If the user is relaxed, the analysis unit can also display a detailed analysis. If the user is in a hurry, the analysis unit can also display a concise analysis. This allows the system to provide the user with the most relevant information by displaying analysis results based on their emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0095] The analysis department considers the frequency and content of participants' contributions when analyzing the progress of a meeting. For example, the analysis department analyzes the frequency of participants' contributions and gives more weight to the opinions of those who speak frequently. The analysis department can also analyze the content of participants' contributions, extract important keywords, and incorporate them into the analysis. The analysis department can also comprehensively analyze the frequency and content of participants' contributions to identify important points of discussion. By considering the frequency and content of participants' contributions, a more accurate analysis becomes possible.

[0096] The analysis department applies different analytical methods depending on the purpose and theme of the meeting when analyzing the progress of the meeting. For example, if the purpose of the meeting is decision-making, the analysis department will focus on analyzing the information necessary for decision-making. If the theme of the meeting is brainstorming, the analysis department can also analyze the number and quality of ideas. If the purpose of the meeting is information sharing, the analysis department can also analyze how information is conveyed. By applying analytical methods appropriate to the purpose and theme of the meeting, more effective analysis becomes possible.

[0097] The visualization unit estimates the user's emotions and adjusts the visualization method based on the estimated emotions. For example, if the user is tense, the visualization unit provides a simple and highly visible visualization method. If the user is relaxed, the visualization unit can also provide a visualization method that includes detailed information. If the user is in a hurry, the visualization unit can also provide a visualization method that gets straight to the point. In this way, by adjusting the visualization method based on the user's emotions, the optimal visualization method can be provided for the user. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0098] The visualization unit updates the visualization content in real time according to the progress of the meeting. For example, the visualization unit updates the visualization content in real time according to what is said and the progress of the meeting. The visualization unit can also update the visualization content when important points are raised. The visualization unit can also update the visualization content when unresolved issues arise. In this way, by updating the visualization content in real time according to the progress of the meeting, the latest information can be provided.

[0099] The visualization section highlights important points and unresolved issues during visualization. For example, the visualization section can highlight important points to ensure participants don't miss them. The visualization section can also highlight unresolved issues to encourage discussion toward their resolution. The visualization section can also color-code important points and unresolved issues to improve visibility. This highlights important points and unresolved issues, ensuring participants don't miss them.

[0100] The visualization unit estimates the user's emotions and determines the visualization priority based on the estimated emotions. For example, if the user is stressed, the visualization unit will prioritize visualizing important information. If the user is relaxed, the visualization unit can also prioritize visualizing detailed information. If the user is in a hurry, the visualization unit can also prioritize visualizing key points. In this way, by determining the visualization priority based on the user's emotions, information that is important to the user can be provided preferentially. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0101] The visualization unit applies different visualization methods depending on the progress of the meeting. For example, it uses graphs and charts depending on the progress of the meeting. It can also use text and images. The visualization unit can change the visualization method in real time. This allows for more effective visualization by applying visualization methods according to the progress of the meeting.

[0102] The visualization unit customizes the visualization content by considering the attribute information of the meeting participants. For example, the visualization unit highlights important information according to the participant's job title. The visualization unit can also highlight relevant information according to the participant's field of expertise. The visualization unit can also customize and display important information according to the participant's interests. In this way, by considering the attribute information of the meeting participants, it can provide the most optimal visualization content for each participant.

[0103] The feedback function estimates the user's emotions and adjusts its feedback method based on the estimated emotions. For example, if the user is tense, the feedback function will give feedback in a calm tone. If the user is relaxed, the feedback function can also give detailed feedback. If the user is in a hurry, the feedback function can also give concise feedback. In this way, by adjusting the feedback method based on the user's emotions, the system can provide the most appropriate feedback method for the user. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0104] The feedback team updates its feedback in real time according to the progress of the meeting. For example, the feedback team can point out discrepancies in the discussion in real time as the meeting progresses. The feedback team can also point out overlooked important points in real time. The feedback team can also point out unresolved issues in real time. This allows for the provision of the latest information by updating feedback in real time according to the progress of the meeting.

[0105] The section that points out issues prioritizes highlighting important points and unresolved issues. For example, it can prioritize highlighting important points to facilitate the progress of the discussion. It can also prioritize highlighting unresolved issues to encourage discussion toward their resolution. The section that points out issues can also use color coding to highlight important points and unresolved issues, thereby improving visibility. This prioritizes highlighting important points and unresolved issues, thus facilitating the progress of the discussion.

[0106] The feedback function estimates the user's emotions and prioritizes feedback based on those emotions. For example, if the user is stressed, the feedback function will prioritize important feedback. If the user is relaxed, the feedback function may also prioritize detailed feedback. If the user is in a hurry, the feedback function may also prioritize key points. This ensures that feedback is prioritized to the user by prioritizing it based on their emotions. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0107] The feedback team applies different feedback methods depending on the progress of the meeting. For example, they may use verbal or written feedback depending on the meeting's progress. They can also use detailed or concise feedback. The feedback team can even change their feedback method in real time. This allows for more effective feedback by applying feedback methods appropriate to the meeting's progress.

[0108] The feedback team customizes its feedback by considering the attribute information of the meeting participants. For example, it can highlight important points based on the participant's position. It can also highlight relevant points based on the participant's area of ​​expertise. Furthermore, it can customize important points based on the participant's interests. This allows the feedback team to provide feedback that is optimal for each participant by considering their attribute information.

[0109] The guiding unit estimates the user's emotions and adjusts its guiding method based on the estimated emotions. For example, if the user is nervous, the guiding unit will guide in a calm tone. If the user is relaxed, the guiding unit may provide detailed guidance. If the user is in a hurry, the guiding unit may provide concise guidance. In this way, by adjusting the guiding method based on the user's emotions, the system can provide the most suitable guiding method for the user. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0110] The guiding team updates the guide content in real time according to the progress of the meeting. For example, the guiding team can guide participants in real time to identify discrepancies in the discussion during the meeting. The guiding team can also guide participants in real time to identify any overlooked important points. The guiding team can also guide participants in real time to identify any unresolved issues. By updating the guide content in real time according to the progress of the meeting, the team can provide the most up-to-date information.

[0111] The guiding function prioritizes guiding participants through important points and unresolved issues. For example, it can prioritize guiding participants through important points to facilitate the progress of the discussion. It can also prioritize guiding participants through unresolved issues to encourage discussion toward their resolution. The guiding function can also color-code important points and unresolved issues to improve visibility. This allows for the discussion to progress by prioritizing the guidance of important points and unresolved issues.

[0112] The guide unit estimates the user's emotions and prioritizes the guide based on those emotions. For example, if the user is nervous, the guide unit will prioritize important guidance. If the user is relaxed, the guide unit may also prioritize detailed guidance. If the user is in a hurry, the guide unit may also prioritize guiding to the key points. This ensures that the guide is prioritized for the user by prioritizing it based on their emotions. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0113] The guiding team applies different guiding methods depending on the progress of the meeting. For example, the guiding team may switch between verbal and written guidance depending on the meeting's progress. The guiding team can also switch between detailed and concise guidance. The guiding team can even change guiding methods in real time. This allows for more effective guiding by applying guiding methods that are appropriate to the meeting's progress.

[0114] The guiding department customizes the guide content by considering the attribute information of the meeting participants. For example, the guiding department may highlight important guides based on the participant's job title. The guiding department may also highlight relevant guides based on the participant's area of ​​expertise. The guiding department may also customize important guides based on the participant's interests. In this way, by considering the attribute information of the meeting participants, the guiding department can provide the most appropriate guide content for each participant.

[0115] The suggestion function estimates the user's emotions and adjusts the goal-setting suggestion method based on the estimated emotions. For example, if the user is nervous, the suggestion function suggests simple and clear goal settings. If the user is relaxed, the suggestion function may suggest detailed goal settings. If the user is in a hurry, the suggestion function may suggest concise goal settings. In this way, by adjusting the goal-setting suggestion method based on the user's emotions, the optimal goal settings can be provided for the user. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0116] The proposal team, when proposing goal setting, refers to past meeting data to suggest the optimal goal. For example, the proposal team may refer to past meeting data and suggest the optimal goal based on the goals of similar meetings. The proposal team can also suggest the optimal goal based on the goals of successful meetings. The proposal team can also suggest the optimal goal by reflecting the goals of unsuccessful meetings. In this way, by referring to past meeting data, the optimal goal setting can be proposed.

[0117] The suggestion function estimates the user's emotions and determines the priority of goal setting based on the estimated emotions. For example, if the user is stressed, the suggestion function will prioritize important goals. If the user is relaxed, the suggestion function may also prioritize detailed goals. If the user is in a hurry, the suggestion function may also prioritize key points. This allows for prioritizing goals based on the user's emotions, thereby prioritizing goals that are important to the user. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0118] The proposal team applies different proposal methods depending on the purpose and theme of the meeting when proposing goal setting. For example, if the purpose of the meeting is decision-making, the proposal team will propose goals necessary for decision-making. If the theme of the meeting is brainstorming, the proposal team may also propose goals that emphasize the quantity and quality of ideas. If the purpose of the meeting is information sharing, the proposal team may also propose goals that emphasize the effectiveness of information dissemination. By applying proposal methods appropriate to the purpose and theme of the meeting, more effective goal setting becomes possible.

[0119] The creation unit estimates the user's emotions and adjusts the order and timeline of the discussion based on the estimated emotions. For example, if the user is nervous, the creation unit will create a simple and clear discussion order and timeline. If the user is relaxed, the creation unit can also create a detailed discussion order and timeline. If the user is in a hurry, the creation unit can also create a concise discussion order and timeline. In this way, by adjusting the order and timeline of the discussion based on the user's emotions, the optimal discussion order and timeline can be provided to the user. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0120] The creation process, when creating the discussion order and timeline, refers to past meeting data to create the optimal order and timeline. For example, it refers to past meeting data and creates the optimal order and timeline based on the order and timeline of similar meetings. It can also create the optimal order and timeline based on the order and timeline of successful meetings. It can also create the optimal order and timeline by reflecting the order and timeline of unsuccessful meetings. In this way, the optimal discussion order and timeline can be created by referring to past meeting data.

[0121] The creation unit estimates the user's emotions and determines the order of discussions and the priority of the timeline based on the estimated emotions. For example, if the user is tense, the creation unit will prioritize important discussions. If the user is relaxed, the creation unit may also prioritize detailed discussions. If the user is in a hurry, the creation unit may also prioritize key points. In this way, by determining the order of discussions and the priority of the timeline based on the user's emotions, discussions that are important to the user can be prioritized. Emotion estimation is achieved using emotion estimation functions, such as emotion engines or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0122] The planning team applies different methods to create the discussion order and timeline depending on the purpose and theme of the meeting. For example, if the purpose of the meeting is decision-making, the planning team will create the order and timeline necessary for decision-making. If the theme of the meeting is brainstorming, the planning team can also create an order and timeline that emphasizes the quantity and quality of ideas. If the purpose of the meeting is information sharing, the planning team can also create an order and timeline that emphasizes the transmission of information. By applying a planning method appropriate to the purpose and theme of the meeting, a more effective discussion order and timeline can be created.

[0123] The linking unit estimates the user's emotions and adjusts the linking method of past discussions based on the estimated user emotions. For example, if the user is nervous, the linking unit provides a simple and clear linking method. If the user is relaxed, the linking unit can also provide a detailed linking method. If the user is in a hurry, the linking unit can also provide a concise linking method. In this way, by adjusting the linking method of past discussions based on the user's emotions, the optimal linking method can be provided for the user. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0124] The linking function prioritizes linking highly relevant discussions when connecting past discussions. For example, it prioritizes linking past discussions that are relevant to the current discussion. It can also prioritize linking discussions that contain important points. It can also prioritize linking discussions that contain unresolved issues. By prioritizing the linking of highly relevant discussions, it can provide information useful for the current discussion.

[0125] The linking unit estimates the user's emotions and determines the priority of linking past discussions based on the estimated emotions. For example, if the user is tense, the linking unit will prioritize linking important discussions. If the user is relaxed, the linking unit may also prioritize linking detailed discussions. If the user is in a hurry, the linking unit may also prioritize linking key points. In this way, by determining the priority of linking past discussions based on the user's emotions, discussions that are important to the user can be prioritized. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0126] The linking function applies different linking methods depending on the purpose and theme of the meeting when linking past discussions. For example, if the purpose of the meeting is decision-making, the linking function will prioritize linking discussions related to decision-making. If the theme of the meeting is brainstorming, the linking function can also prioritize linking discussions that emphasize the quantity and quality of ideas. If the purpose of the meeting is information sharing, the linking function can also prioritize linking discussions that emphasize the status of information transmission. By applying a linking method appropriate to the purpose and theme of the meeting, it becomes possible to link discussions more effectively.

[0127] The organization unit estimates the user's emotions and adjusts the way the key points are organized based on the estimated emotions. For example, if the user is nervous, the organization unit provides a simple and clear way to organize the key points. If the user is relaxed, the organization unit can also provide a detailed way to organize the key points. If the user is in a hurry, the organization unit can also provide a concise way to organize the key points. In this way, by adjusting the way the key points are organized based on the user's emotions, the optimal organization method can be provided for the user. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0128] The summarizing team prioritizes organizing important points and unresolved issues when summarizing key points. For example, the summarizing team can prioritize organizing important points to facilitate the progress of the discussion. The summarizing team can also prioritize organizing unresolved issues to facilitate discussion toward their resolution. The summarizing team can also improve visibility by using color coding to distinguish between important points and unresolved issues. This allows them to prioritize organizing important points and unresolved issues, thereby facilitating the progress of the discussion.

[0129] The organization unit estimates the user's emotions and determines the priority of summarizing the key points based on the estimated emotions. For example, if the user is tense, the organization unit will prioritize summarizing the important points. If the user is relaxed, the organization unit may also prioritize summarizing the detailed points. If the user is in a hurry, the organization unit may also prioritize summarizing the key points. In this way, by determining the priority of summarizing the key points based on the user's emotions, the key points that are important to the user can be prioritized. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0130] The summarization team applies different summarization methods depending on the progress of the meeting. For example, they prioritize summarizing important points as the meeting progresses. They can also prioritize summarizing unresolved issues. The summarization team can also change the summarization method in real time. This allows for more effective summarization of key points by applying a summarization method that is appropriate to the progress of the meeting.

[0131] The adjustment unit estimates the user's emotions and adjusts the method of prioritizing discussions based on the estimated emotions. For example, if the user is tense, the adjustment unit will prioritize important discussions. If the user is relaxed, the adjustment unit may also prioritize detailed discussions. If the user is in a hurry, the adjustment unit may also prioritize key points. In this way, by adjusting the method of prioritizing discussions based on the user's emotions, the system can provide the user with the most optimal adjustment method. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may include, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0132] The coordination unit updates the prioritization of discussions in real time according to the progress of the meeting. For example, the coordination unit prioritizes important discussions according to the progress of the meeting. The coordination unit can also prioritize unresolved issues. The coordination unit can also update the prioritization in real time. This allows for the provision of the latest information by updating the prioritization in real time according to the progress of the meeting.

[0133] The adjustment unit estimates the user's emotions and determines the priority of discussions based on the estimated emotions. For example, if the user is tense, the adjustment unit will prioritize important discussions. If the user is relaxed, the adjustment unit may also prioritize detailed discussions. If the user is in a hurry, the adjustment unit may also prioritize key points. In this way, by determining the priority of discussions based on the user's emotions, discussions that are important to the user can be prioritized. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI may be, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0134] The coordination team applies different coordination methods depending on the purpose and theme of the meeting when adjusting the priority of discussions. For example, if the purpose of the meeting is decision-making, the coordination team will prioritize discussions necessary for decision-making. If the theme of the meeting is brainstorming, the coordination team may also prioritize discussions that emphasize the quantity and quality of ideas. If the purpose of the meeting is information sharing, the coordination team may also prioritize discussions that emphasize the effectiveness of information dissemination. By applying coordination methods appropriate to the purpose and theme of the meeting, it becomes possible to adjust the priority of discussions more effectively.

[0135] The search engine estimates the user's emotions and adjusts how search results are displayed based on that estimation. For example, if the user is stressed, the search engine displays simple, easy-to-read search results. If the user is relaxed, it can also display detailed search results. If the user is in a hurry, it can display concise search results. By adjusting how search results are displayed based on the user's emotions, the system can provide the most optimal display method for the user. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0136] The search function updates search results in real time according to the progress of the meeting. For example, the search function prioritizes displaying important information in the search results according to the progress of the meeting. The search function can also prioritize displaying information related to unresolved issues in the search results. The search function can also update search results in real time. This allows for the provision of the latest information by updating search results in real time according to the progress of the meeting.

[0137] The search engine estimates the user's emotions and prioritizes search results based on those emotions. For example, if the user is stressed, the search engine will prioritize displaying important information. If the user is relaxed, the search engine can also prioritize displaying detailed information. If the user is in a hurry, the search engine can also prioritize displaying key points. By prioritizing search results based on the user's emotions, the system can prioritize providing the user with information that is important to them. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI.

[0138] The search function applies different display methods depending on the purpose and theme of the meeting when displaying search results. For example, if the purpose of the meeting is decision-making, the search function will prioritize displaying information necessary for decision-making. If the theme of the meeting is brainstorming, the search function can also prioritize displaying information that emphasizes the quantity and quality of ideas. If the purpose of the meeting is information sharing, the search function can also prioritize displaying information that emphasizes the status of information transmission. By applying display methods tailored to the purpose and theme of the meeting, more effective search results can be displayed.

[0139] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0140] The meeting facilitation support system can also include a summarization unit that automatically summarizes participants' comments and displays the summarized content in real time. The summarization unit can, for example, analyze the comments, extract key points, and summarize them. It can also prioritize summaries based on the frequency and importance of each comment. The summarization unit can visually display the summarized content, making it easier for participants to grasp the overall picture of the discussion. This allows participants to quickly understand the main points of the discussion and promote efficient discussion by automatically summarizing the comments.

[0141] The meeting facilitation support system can also include an emotion analysis unit that analyzes the sentiment of participants' statements and displays changes in emotion in real time. For example, the emotion analysis unit analyzes the content of statements and tone of voice to estimate participants' emotions. The emotion analysis unit can also visually display the estimated emotions, making it easier for participants to understand the emotions of other participants. By displaying changes in emotion in real time, the emotion analysis unit can also help grasp the atmosphere of the discussion. This allows for a better understanding of the discussion's atmosphere and appropriate responses by displaying participants' emotions in real time.

[0142] The meeting facilitation support system can also include a classification unit that automatically categorizes participants' comments and displays the categorized content in real time. The classification unit, for example, analyzes the comments and categorizes them by topic. It can also determine the priority of the classification based on the content and relevance of the comments. The classification unit can visually display the categorized content, making it easier for participants to understand the progress of the discussion. This allows participants to quickly understand the progress of the discussion and promote efficient discussion by automatically categorizing the comments.

[0143] The meeting facilitation support system can also include an evaluation unit that automatically evaluates participants' contributions and displays the evaluation results in real time. For example, the evaluation unit analyzes the content of the contributions and assesses their quality and importance. The evaluation unit can also prioritize evaluations based on the content and relevance of the contributions. The evaluation unit can also visually display the evaluation results, making it easier for participants to understand the quality and importance of their contributions. This allows participants to quickly understand the quality and importance of their contributions by automatically evaluating their contributions, thereby promoting efficient discussion.

[0144] The meeting facilitation support system can also include a recording unit that automatically records participants' statements and displays the recorded content in real time. The recording unit can, for example, analyze the statements and record key points. It can also prioritize recordings based on the frequency and importance of each statement. The recording unit can visually display the recorded content, making it easier for participants to grasp the overall picture of the discussion. This allows participants to quickly understand the main points of the discussion and promote efficient discussion by automatically recording what is said.

[0145] The meeting facilitation support system can also include an emotion analysis unit that analyzes the sentiment of participants' statements and displays changes in emotion in real time. For example, the emotion analysis unit analyzes the content of statements and tone of voice to estimate participants' emotions. The emotion analysis unit can also visually display the estimated emotions, making it easier for participants to understand the emotions of other participants. By displaying changes in emotion in real time, the emotion analysis unit can also help grasp the atmosphere of the discussion. This allows for a better understanding of the discussion's atmosphere and appropriate responses by displaying participants' emotions in real time.

[0146] The meeting facilitation support system can also include a summarization unit that automatically summarizes participants' comments and displays the summarized content in real time. The summarization unit can, for example, analyze the comments, extract key points, and summarize them. It can also prioritize summaries based on the frequency and importance of each comment. The summarization unit can visually display the summarized content, making it easier for participants to grasp the overall picture of the discussion. This allows participants to quickly understand the main points of the discussion and promote efficient discussion by automatically summarizing the comments.

[0147] The meeting facilitation support system can also include a classification unit that automatically categorizes participants' comments and displays the categorized content in real time. The classification unit, for example, analyzes the comments and categorizes them by topic. It can also determine the priority of the classification based on the content and relevance of the comments. The classification unit can visually display the categorized content, making it easier for participants to understand the progress of the discussion. This allows participants to quickly understand the progress of the discussion and promote efficient discussion by automatically categorizing the comments.

[0148] The meeting facilitation support system can also include an evaluation unit that automatically evaluates participants' contributions and displays the evaluation results in real time. For example, the evaluation unit analyzes the content of the contributions and assesses their quality and importance. The evaluation unit can also prioritize evaluations based on the content and relevance of the contributions. The evaluation unit can also visually display the evaluation results, making it easier for participants to understand the quality and importance of their contributions. This allows participants to quickly understand the quality and importance of their contributions by automatically evaluating their contributions, thereby promoting efficient discussion.

[0149] The meeting facilitation support system can also include a recording unit that automatically records participants' statements and displays the recorded content in real time. The recording unit can, for example, analyze the statements and record key points. It can also prioritize recordings based on the frequency and importance of each statement. The recording unit can visually display the recorded content, making it easier for participants to grasp the overall picture of the discussion. This allows participants to quickly understand the main points of the discussion and promote efficient discussion by automatically recording what is said.

[0150] The following briefly describes the processing flow for example form 2.

[0151] Step 1: The analysis department analyzes the meeting's progress in real time. Specifically, they monitor the meeting's progress in real time and collect data. Based on the collected data, they analyze the content and frequency of contributions to evaluate the meeting's progress. Step 2: The visualization unit visualizes progress toward the goal and issues based on the data analyzed by the analysis unit. Specifically, it displays progress using graphs and charts, and issues in list format. Visualization is enhanced by color-coding the progress. Step 3: The feedback unit points out deviations in the discussion based on the information visualized by the visualization unit. Specifically, it issues warnings when the discussion deviates from the main topic and provides feedback to prevent important points from being overlooked. It points out deviations in the discussion in real time and encourages corrections. Step 4: The guide section provides guidance for correcting the discrepancies pointed out by the criticism section. Specifically, it proposes concrete actions to support the progress of the discussion and provides reference materials. It provides guidance to ensure the smooth progress of the discussion.

[0152] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0153] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0154] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0155] Each of the multiple elements described above, including the analysis unit, visualization unit, identification unit, guide unit, proposal unit, creation unit, linking unit, organization unit, adjustment unit, and search unit, is implemented in at least one of the smart device 14 and the data processing unit 12. For example, the analysis unit is implemented by the processor 46 of the smart device 14, which monitors the progress of the meeting in real time and collects data. The visualization unit is implemented by the specific processing unit 290 of the data processing unit 12, which displays the progress and issues in graphs and charts. The identification unit is implemented by the control unit 46A of the smart device 14, which identifies discrepancies in the discussion in real time. The guide unit is implemented by the specific processing unit 290 of the data processing unit 12, which proposes specific actions to support the progress of the discussion. The proposal unit is implemented by the processor 46 of the smart device 14, which proposes specific goal settings according to the purpose of the meeting. The creation unit is implemented by the specific processing unit 290 of the data processing unit 12, which automatically creates the optimal discussion order and timeline. The linking unit is implemented by the control unit 46A of the smart device 14, which automatically links past related discussions. The organization unit is implemented by the specific processing unit 290 of the data processing device 12, which automatically organizes the key points. The adjustment unit is implemented by the processor 46 of the smart device 14, which adjusts the priority of discussions according to the remaining time. The search unit is implemented by the specific processing unit 290 of the data processing device 12, which searches the company database and the web based on the context. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0156] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0157] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0158] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0159] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0160] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0161] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0162] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0163] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0164] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0165] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0166] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0167] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0168] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0169] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0170] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0171] Each of the multiple elements described above, including the analysis unit, visualization unit, identification unit, guide unit, proposal unit, creation unit, linking unit, organization unit, adjustment unit, and search unit, is implemented by at least one of the smart glasses 214 and the data processing unit 12. For example, the analysis unit is implemented by the processor 46 of the smart glasses 214, which monitors the progress of the meeting in real time and collects data. The visualization unit is implemented by the identification processing unit 290 of the data processing unit 12, which displays the progress and points of discussion in graphs and charts. The identification unit is implemented by the control unit 46A of the smart glasses 214, which identifies deviations in the discussion in real time. The guide unit is implemented by the identification processing unit 290 of the data processing unit 12, which proposes specific actions to support the progress of the discussion. The proposal unit is implemented by the processor 46 of the smart glasses 214, which proposes specific goal settings according to the purpose of the meeting. The creation unit is implemented by the identification processing unit 290 of the data processing unit 12, which automatically creates the optimal discussion order and timeline. The linking function is implemented by the control unit 46A of the smart glasses 214, which automatically links related past discussions. The organization function is implemented by the identification processing unit 290 of the data processing device 12, which automatically organizes the key points. The adjustment function is implemented by the processor 46 of the smart glasses 214, which adjusts the priority of discussions according to the remaining time. The search function is implemented by the identification processing unit 290 of the data processing device 12, which searches the company database and the web based on the context. The correspondence between each function and the device or control unit is not limited to the example described above, and various changes are possible.

[0172] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0173] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0174] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0175] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0176] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0177] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0178] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0179] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0180] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0181] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0182] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0183] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0184] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0185] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0186] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0187] Each of the multiple elements described above, including the analysis unit, visualization unit, identification unit, guide unit, proposal unit, creation unit, linking unit, organization unit, adjustment unit, and search unit, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the analysis unit is implemented by the processor 46 of the headset terminal 314, which monitors the progress of the meeting in real time and collects data. The visualization unit is implemented by the specific processing unit 290 of the data processing unit 12, which displays the progress and points of discussion in graphs and charts. The identification unit is implemented by the control unit 46A of the headset terminal 314, which identifies discrepancies in the discussion in real time. The guide unit is implemented by the specific processing unit 290 of the data processing unit 12, which proposes specific actions to support the progress of the discussion. The proposal unit is implemented by the processor 46 of the headset terminal 314, which proposes specific goal settings according to the purpose of the meeting. The creation unit is implemented by the specific processing unit 290 of the data processing unit 12, which automatically creates the optimal discussion order and timeline. The linking function is implemented by the control unit 46A of the headset terminal 314, which automatically links past related discussions. The organization function is implemented by the specific processing unit 290 of the data processing device 12, which automatically organizes the key points. The adjustment function is implemented by the processor 46 of the headset terminal 314, which adjusts the priority of discussions according to the remaining time. The search function is implemented by the specific processing unit 290 of the data processing device 12, which searches the company database and the web based on the context. The correspondence between each function and the devices and control units is not limited to the example described above, and various changes are possible.

[0188] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0189] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0190] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0191] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0192] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0193] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0194] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0195] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0196] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0197] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0198] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0199] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0200] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0201] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0202] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0203] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0204] Each of the multiple elements described above, including the analysis unit, visualization unit, identification unit, guide unit, proposal unit, creation unit, linking unit, organization unit, adjustment unit, and search unit, is implemented by at least one of the robot 414 and the data processing unit 12. For example, the analysis unit is implemented by the processor 46 of the robot 414, which monitors the progress of the meeting in real time and collects data. The visualization unit is implemented by the specific processing unit 290 of the data processing unit 12, which displays the progress and points of discussion in graphs and charts. The identification unit is implemented by the control unit 46A of the robot 414, which identifies deviations in the discussion in real time. The guide unit is implemented by the specific processing unit 290 of the data processing unit 12, which proposes specific actions to support the progress of the discussion. The proposal unit is implemented by the processor 46 of the robot 414, which proposes specific goal settings according to the purpose of the meeting. The creation unit is implemented by the specific processing unit 290 of the data processing unit 12, which automatically creates the optimal discussion order and timeline. The linking unit is implemented by the control unit 46A of the robot 414, which automatically links past related discussions. The organization unit is implemented by the specific processing unit 290 of the data processing device 12, which automatically organizes the key points. The adjustment unit is implemented by the processor 46 of the robot 414, which adjusts the priority of discussions according to the remaining time. The search unit is implemented by the specific processing unit 290 of the data processing device 12, which searches the company database and the web based on the context. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0205] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0206] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0207] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0208] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0209] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0210] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0211] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0212] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0213] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0214] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0215] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0216] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0217] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0218] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0219] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0220] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0221] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0222] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0223] (Note 1) The analysis department analyzes the progress of the meeting in real time, A visualization unit visualizes the progress toward the goal and issues based on the data analyzed by the aforementioned analysis unit, Based on the information visualized by the aforementioned visualization unit, the identification unit points out discrepancies in the discussion, The system includes a guide section that provides a guide for correcting the misalignment pointed out by the aforementioned pointing section. A system characterized by the following features. (Note 2) The meeting preparation phase includes a proposal department that suggests specific goal settings tailored to the purpose of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 3) It includes a creation section that automatically generates the optimal discussion order and timeline. The system described in Appendix 1, characterized by the features described herein. (Note 4) It includes a linking function that automatically connects past related discussions. The system described in Appendix 1, characterized by the features described herein. (Note 5) It features an organization section that automatically organizes key points. The system described in Appendix 1, characterized by the features described herein. (Note 6) It includes an adjustment unit that adjusts the priority of discussions according to the remaining time. The system described in Appendix 1, characterized by the features described herein. (Note 7) It features a search function that searches internal databases and the web based on context. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned analysis unit is It estimates user emotions and analyzes the progress of the meeting based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned analysis unit is It analyzes the progress of meetings in real time and automatically determines the priority of important discussion points. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned analysis unit is When analyzing the progress of a meeting, referencing past meeting data improves the accuracy of the analysis. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned analysis unit is It estimates the user's emotions and displays an analysis of the meeting's progress based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned analysis unit is When analyzing the progress of a meeting, the frequency and content of participants' contributions should be taken into consideration. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned analysis unit is When analyzing the progress of a meeting, different analytical methods should be applied depending on the purpose and theme of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned visualization unit, It estimates the user's emotions and adjusts the visualization method based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned visualization unit, When visualizing the meeting, the visualization content is updated in real time according to the progress of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned visualization unit, When visualizing, highlight important points and unresolved issues. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned visualization unit, It estimates the user's emotions and determines the visualization priority based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned visualization unit, When visualizing the meeting, different visualization methods are applied depending on the progress of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned visualization unit, When visualizing data, customize the visualization content by considering the attribute information of the meeting participants. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned point is, It estimates the user's emotions and adjusts the method of giving feedback based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 21) The aforementioned point is, When making comments, the comments will be updated in real time according to the progress of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 22) The aforementioned point is, When pointing out issues, prioritize highlighting important points and unresolved problems. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned point is, It estimates the user's emotions and prioritizes the feedback based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 24) The aforementioned point is, When giving feedback, different feedback methods should be applied depending on the progress of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 25) The aforementioned point is, When providing feedback, customize the feedback by considering the attribute information of the meeting participants. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned guide section is It estimates the user's emotions and adjusts the guidance method based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned guide section is When providing guidance, the guidance content will be updated in real time according to the progress of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned guide section is When providing guidance, prioritize guiding through important points and unresolved issues. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned guide section is It estimates the user's emotions and determines the priority of the guide based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 30) The aforementioned guide section is When guiding, apply different guiding methods depending on the progress of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 31) The aforementioned guide section is When providing guidance, customize the guide content by taking into account the attribute information of the meeting participants. The system described in Appendix 1, characterized by the features described herein. (Note 32) The aforementioned proposal section is, We estimate the user's emotions and adjust the goal setting suggestions based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 33) The aforementioned proposal section is, When proposing goal setting, we refer to past meeting data to suggest the most suitable goals. The system described in Appendix 1, characterized by the features described herein. (Note 34) The aforementioned proposal section is, The system estimates user emotions and prioritizes goal setting based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 35) The aforementioned proposal section is, When proposing goal setting, apply different proposal methods depending on the purpose and theme of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 36) The aforementioned creation unit, It estimates user sentiment and adjusts the order of discussions and how the timeline is created based on the estimated user sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 37) The aforementioned creation unit, When creating the order and timeline of discussions, we refer to past meeting data to create the optimal order and timeline. The system described in Appendix 1, characterized by the features described herein. (Note 38) The aforementioned creation unit, It estimates user sentiment and determines the order of discussions and the priority of the timeline based on the estimated user sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 39) The aforementioned creation unit, When creating the order and timeline of discussions, different creation methods are applied depending on the purpose and theme of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 40) The aforementioned tying part is, It estimates the user's emotions and adjusts how past discussions are linked based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 41) The aforementioned tying part is, When linking past discussions, prioritize linking discussions that are highly relevant. The system described in Appendix 1, characterized by the features described herein. (Note 42) The linking unit estimates the user's emotion and determines the priority order of linking past discussions based on the estimated user's emotion The system according to Appendix 1, characterized in that (Appendix 43) The linking unit When linking past discussions, applies different linking methods according to the purpose and theme of the meeting The system according to Appendix 1, characterized in that (Appendix 44) The organizing unit estimates the user's emotion and adjusts the method of organizing key points based on the estimated user's emotion The system according to Appendix 1, characterized in that (Appendix 45) The organizing unit When organizing key points, preferentially organizes important arguments and unsolved problems The system according to Appendix 1, characterized in that (Appendix 46) The organizing unit estimates the user's emotion and determines the priority order of organizing key points based on the estimated user's emotion The system according to Appendix 1, characterized in that (Appendix 47) The organizing unit When organizing key points, applies different organizing methods according to the progress of the meeting The system according to Appendix​​​​​​​​​​​​​​​​​​The adjustment unit is, It estimates the user's emotions and determines the priority of discussions based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 51) The adjustment unit is, When adjusting the priorities of discussions, different adjustment methods are applied depending on the purpose and topic of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 52) The aforementioned search unit, It estimates the user's sentiment and adjusts how search results are displayed based on that estimated sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 53) The aforementioned search unit, When displaying search results, the search results are updated in real time according to the progress of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 54) The aforementioned search unit, It estimates the user's emotions and determines the priority of search results based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 55) The aforementioned search unit, When displaying search results, different display methods are applied depending on the purpose and theme of the meeting. The system described in Appendix 1, characterized by the features described herein. [Explanation of Symbols]

[0224] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. The analysis department analyzes the progress of the meeting in real time, A visualization unit visualizes the progress toward the goal and issues based on the data analyzed by the aforementioned analysis unit, Based on the information visualized by the aforementioned visualization unit, the identification unit points out discrepancies in the discussion, The system includes a guide section that provides a guide for correcting the misalignment pointed out by the aforementioned pointing section. A system characterized by the following features.

2. The meeting preparation phase includes a proposal department that suggests specific goal settings tailored to the purpose of the meeting. The system according to feature 1.

3. It includes a creation section that automatically generates the optimal discussion order and timeline. The system according to feature 1.

4. It includes a linking function that automatically connects past related discussions. The system according to feature 1.

5. It features an organization section that automatically organizes key points. The system according to feature 1.

6. It includes an adjustment unit that adjusts the priority of discussions according to the remaining time. The system according to feature 1.

7. It features a search function that searches internal databases and the web based on context. The system according to feature 1.

8. The aforementioned analysis unit is It estimates user emotions and analyzes the progress of the meeting based on the estimated user emotions. The system according to feature 1.

9. The aforementioned analysis unit is It analyzes the progress of meetings in real time and automatically determines the priority of important discussion points. The system according to feature 1.

10. The aforementioned analysis unit is When analyzing the progress of a meeting, referencing past meeting data improves the accuracy of the analysis. The system according to feature 1.

Images