system

Abstract

The system according to this embodiment aims to ensure that discussions during a meeting proceed in accordance with the agenda. [Solution] The system according to the embodiment comprises a summarization unit, an analysis unit, a quantification unit, and a visualization unit. The summarization unit summarizes the gist of the meeting materials. The analysis unit analyzes the statements made during the meeting based on the gist summarized by the summarization unit. The quantification unit quantifies the degree to which the statements analyzed by the analysis unit match the agenda. The visualization unit visualizes the results obtained by the quantification unit.

Description

Technical Field

[0006] , ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, there is a risk that the discussion during a meeting deviates and the progress of the topic is delayed.

[0005] The system according to the embodiment aims to make the discussion during a meeting proceed along the topic.

Means for Solving the Problems

[0006] The system according to the embodiment includes a summarization unit, an analysis unit, a quantification unit, and a visualization unit. The summarization unit summarizes the gist of meeting materials. The analysis unit analyzes the utterances during the meeting based on the gist summarized by the summarization unit. The quantification unit quantifies the degree of coincidence between the utterance content analyzed by the analysis unit and the topic. The visualization unit visualizes the results obtained by the quantification unit.

Effects of the Invention

[0007] The system according to this embodiment allows discussions during a meeting to proceed in accordance with the agenda. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. Also, the reception device 38, the output device 40, and the camera 42 are connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The meeting efficiency system according to an embodiment of the present invention is a system that improves meeting efficiency by utilizing generative AI. This meeting efficiency system uses generative AI to automatically summarize the main points from meeting materials, analyzes comments made during the meeting in real time, and quantifies the degree of agreement with the agenda. This visualizes the progress of the meeting, allowing all participants to objectively understand it. This mechanism enables efficient meeting management that does not rely on subjective opinions and strongly supports corporate decision-making. For example, the generative AI automatically summarizes the main points from meeting materials. For example, the generative AI analyzes slide materials created with presentation applications and extracts the main points of the meeting. This clarifies the purpose and agenda of the meeting. Next, the generative AI analyzes comments made during the meeting in real time. The generative AI compares the content of the comments with the main points of the meeting and quantifies the degree of agreement. For example, it quantifies how relevant the comments are to the agenda and determines whether they deviate from the agenda. Furthermore, the generative AI visualizes the results of its analysis. This allows all participants to objectively understand the progress of the meeting. For example, the degree of alignment with the agenda can be displayed in graphs and charts, allowing for a quick overview of the meeting's progress. This system enables efficient meeting management that is not subjective. Participants will be more mindful of making comments relevant to the agenda, leading to smoother meeting progress. Furthermore, even if comments deviate from the agenda, the generating AI will point it out in real time, allowing for quick correction. For example, companies using online meeting tools can improve meeting efficiency by implementing this system. Because the meeting's progress can be grasped in real time, discussions will be aligned with the agenda, and decision-making will be accelerated. In this way, by utilizing generating AI, we provide a system that improves meeting efficiency and strongly supports corporate decision-making. As a result, the meeting efficiency system can objectively grasp the progress of meetings and achieve efficient meeting management.

[0029] The meeting efficiency system according to this embodiment comprises a summarization unit, an analysis unit, a quantification unit, and a visualization unit. The summarization unit summarizes the main points of the meeting materials. The summarization unit analyzes the meeting materials using, for example, a generative AI and extracts the main points. For example, the summarization unit analyzes slide materials created with a presentation application or the like to clarify the purpose and agenda of the meeting. The summarization unit can also summarize the content of the meeting materials using a generative AI. For example, the summarization unit uses a generative AI to extract the key points of the meeting materials and generates a concise summary. The analysis unit analyzes the statements made during the meeting based on the main points summarized by the summarization unit. The analysis unit uses, for example, a generative AI to analyze the content of the statements and compares them to the main points of the meeting. For example, the analysis unit analyzes the content of the statements and quantifies the degree of agreement. The analysis unit can also analyze the content of the statements using a generative AI. For example, the analysis unit uses a generative AI to analyze the content of the statements and quantifies the degree of agreement with the agenda. The quantification unit quantifies the degree of agreement between the content of the statements analyzed by the analysis unit and the agenda. The quantification unit quantifies the degree of agreement between statements using, for example, a generative AI. For example, the quantification unit quantifies how relevant a statement is to the agenda. The quantification unit can also quantify the degree of agreement between statements using a generative AI. For example, the quantification unit uses a generative AI to quantify the degree of agreement between statements and determines whether or not they deviate from the agenda. The visualization unit visualizes the results obtained by the quantification unit. The visualization unit displays the degree of agreement in graphs or charts using, for example, a generative AI. For example, the visualization unit makes it possible to check the progress of the meeting at a glance. The visualization unit can also visualize the progress of the meeting using a generative AI. For example, the visualization unit uses a generative AI to display the degree of agreement in graphs or charts, making it possible to visually grasp the progress of the meeting. As a result, the meeting efficiency system according to the embodiment can objectively grasp the progress of the meeting and realize efficient meeting management.

[0030] The summarization unit summarizes the main points of the meeting materials. For example, the summarization unit analyzes the meeting materials using generative AI and extracts the key points. Specifically, the generative AI uses natural language processing technology to analyze the text data of the meeting materials. For example, when analyzing slide materials created with presentation applications, it focuses on extracting slide titles, bullet points, and figure captions to clarify the purpose and agenda of the meeting. The generative AI understands the context and identifies important keywords and phrases to grasp the essence of the entire document. Furthermore, the generative AI can also summarize the content of the meeting materials. For example, the generative AI extracts the key points of the meeting materials and generates a concise summary. This allows participants to grasp the key points of the materials before the meeting and prepare efficiently. The summarization unit can improve its accuracy by using past meeting materials and summarization results as training data for the generative AI. As a result, the summarization unit can quickly and accurately summarize the content of the meeting materials and provide it to participants.

[0031] The analysis unit analyzes the statements made during the meeting based on the main points summarized by the summarization unit. For example, the analysis unit uses generative AI to analyze the content of the statements and compare them to the main points of the meeting. Specifically, the generative AI uses speech recognition technology to convert the statements made during the meeting into text data. The converted text data is analyzed using natural language processing technology to evaluate how well the statements match the main points and agenda of the meeting. For example, the analysis unit analyzes the statements and quantifies the degree of match. The generative AI extracts keywords and phrases from the statements and compares them with the content of the meeting materials summarized by the summarization unit. This allows the analysis unit to determine whether the statements are relevant to the agenda. The analysis unit can also use generative AI to analyze the statements. For example, the generative AI analyzes the statements and quantifies the degree of match with the agenda. This allows the analysis unit to objectively evaluate how relevant the statements made during the meeting are to the agenda. Furthermore, the analysis unit can provide the analysis results of the statements in real time, providing information to help understand the progress of the meeting.

[0032] The quantification unit quantifies the degree to which the spoken content analyzed by the analysis unit matches the agenda. For example, the quantification unit uses a generative AI to quantify the degree of matching of spoken content. Specifically, the generative AI uses an algorithm that extracts keywords and phrases from the spoken content and quantifies their importance in relation to the agenda. For example, to quantify how relevant a spoken statement is to the agenda, the generative AI calculates the degree of matching between the spoken content and the keywords of the agenda and assigns a score. The quantification unit can also use the generative AI to quantify the degree of matching of spoken content. For example, the generative AI quantifies the degree of matching of spoken content and determines whether it deviates from the agenda. This allows the quantification unit to objectively evaluate how relevant spoken content is to the agenda during a meeting and display it numerically. Furthermore, the quantification unit can calculate the degree of matching of spoken content in real time and provide information to understand the progress of the meeting. This allows the quantification unit to contribute to the efficiency of meetings and promote discussion focused on the agenda.

[0033] The visualization unit visualizes the results obtained by the quantification unit. For example, the visualization unit uses generative AI to display the degree of agreement in graphs and charts. Specifically, the generative AI uses an algorithm that displays the degree of agreement calculated by the quantification unit in a visually easy-to-understand format. For example, the degree of agreement can be displayed in bar graphs, pie charts, or line graphs, allowing participants to see the progress of the meeting at a glance. The visualization unit can also visualize the progress of the meeting using generative AI. For example, the generative AI can display the degree of agreement in graphs and charts, allowing participants to visually grasp the progress of the meeting. This allows participants to understand the progress of the meeting in real time and engage in focused discussions on the agenda. Furthermore, the visualization unit can save the visualized data as a meeting record for later reference. In this way, the visualization unit can contribute to the efficiency of meetings and provide information for objectively evaluating the progress of discussions.

[0034] The summarization unit can analyze slide materials created with presentation applications and extract the main points of a meeting. For example, the summarization unit can use generative AI to analyze the slide materials and extract the main points. For example, the summarization unit can analyze the content of the slide materials to clarify the purpose and agenda of the meeting. The summarization unit can also use generative AI to summarize the content of the slide materials. For example, the summarization unit can have the generative AI extract the key points of the slide materials and generate a concise summary. This clarifies the purpose and agenda of the meeting by extracting the main points. The main points include, but are not limited to, important agenda items, conclusions, and action items. Some or all of the above processing in the summarization unit may be performed using, for example, generative AI, or without generative AI. For example, the summarization unit can input the slide materials into the generative AI and have the generative AI perform the extraction of the main points.

[0035] The analysis unit can compare the content of the statements with the purpose of the meeting and quantify the degree of agreement. The analysis unit can, for example, use a generative AI to analyze the content of the statements and compare it with the purpose of the meeting. For example, the analysis unit can analyze the content of the statements and quantify the degree of agreement. The analysis unit can also use a generative AI to analyze the content of the statements. For example, the analysis unit can have a generative AI analyze the content of the statements and quantify the degree of agreement with the agenda. By quantifying the degree of agreement of the content of the statements, it is possible to determine whether or not there is a deviation from the agenda. The degree of agreement includes, but is not limited to, the relevance and importance of the content of the statements. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or without a generative AI. For example, the analysis unit can input the content of the statements into a generative AI and have the generative AI perform the quantification of the degree of agreement.

[0036] The quantification unit can quantify how relevant a statement is to the agenda and determine whether it deviates from the agenda. The quantification unit can quantify the degree of relevance of the statement content using, for example, a generative AI. For example, the quantification unit can quantify how relevant a statement is to the agenda. The quantification unit can also quantify the degree of relevance of the statement content using a generative AI. For example, the quantification unit can use a generative AI to quantify the degree of relevance of the statement content and determine whether it deviates from the agenda. In this way, by quantifying the relevance of the statement, it is possible to determine whether it deviates from the agenda. Relevance includes, but is not limited to, the degree of relevance of the statement content and importance. Some or all of the above processing in the quantification unit may be performed using, for example, a generative AI, or without a generative AI. For example, the quantification unit can input the statement content into a generative AI and have the generative AI perform the quantification of relevance.

[0037] The visualization unit can display the degree of agreement with the agenda in graphs and charts, allowing for a quick overview of the meeting's progress. The visualization unit can, for example, use generative AI to display the degree of agreement in graphs and charts. For example, the visualization unit can make the meeting's progress immediately apparent. The visualization unit can also visualize the meeting's progress using generative AI. For example, the visualization unit can use generative AI to display the degree of agreement in graphs and charts, allowing for a visual understanding of the meeting's progress. This allows for a quick overview of the meeting's progress by visualizing the degree of agreement with the agenda. The progress includes, but is not limited to, the progress of agenda items and the frequency of contributions. Some or all of the above-described processes in the visualization unit may be performed, for example, using generative AI, or without using generative AI. For example, the visualization unit can input the degree of agreement data into the generative AI and have the generative AI perform the visualization.

[0038] The visualization unit enables companies using online meeting tools to understand the progress of meetings in real time. The visualization unit can, for example, use generative AI to display the progress of meetings in real time. For example, the visualization unit enables companies using online meeting tools to understand the progress of meetings in real time. The visualization unit can also visualize the progress of meetings in real time using generative AI. For example, the visualization unit has generative AI display the progress of meetings in real time, so that all participants can understand the progress. This allows companies using online meeting tools to understand the progress of meetings in real time. Real time includes, but is not limited to, data update frequency and display delay time. Some or all of the above processing in the visualization unit may be performed using, for example, generative AI, or without generative AI. For example, the visualization unit can input real-time data into generative AI and have the generative AI perform the visualization.

[0039] The summarization unit can adjust the level of detail in the summary based on the importance of the meeting during summary generation. For example, the summarization unit can use a generation AI to evaluate the importance of the meeting and adjust the level of detail in the summary. For example, for highly important meetings, the generation AI will provide a detailed summary. For less important meetings, the generation AI may provide a concise summary. For meetings of moderate importance, the generation AI may provide a summary with an appropriate level of detail. This allows for the provision of an appropriate summary by adjusting the level of detail in the summary according to the importance of the meeting. Importance includes, but is not limited to, the purpose of the meeting and the positions of the participants. Some or all of the above processing in the summarization unit may be performed using a generation AI, for example, or without a generation AI. For example, the summarization unit can input meeting importance data into a generation AI and have the generation AI perform the adjustment of the level of detail in the summary.

[0040] The summarization unit can apply different summarization algorithms depending on the meeting category when generating summaries. For example, the summarization unit uses a generation AI to evaluate the meeting category and apply a different summarization algorithm. For example, for technical meetings, the generation AI can provide a summary that emphasizes technical details. For sales meetings, the generation AI can also provide a summary that emphasizes sales and customer information. For management meetings, the generation AI can also provide a summary that emphasizes strategic points. This allows for the provision of appropriate summaries by applying a summarization algorithm according to the meeting category. Categories include, but are not limited to, business meetings and technical meetings. Some or all of the above processing in the summarization unit may be performed using, for example, a generation AI, or without a generation AI. For example, the summarization unit can input meeting category data into a generation AI and have the generation AI apply the summarization algorithm.

[0041] The summarization unit can determine the priority of summaries based on the timing of meetings when generating summaries. For example, the summarization unit uses a generation AI to evaluate the timing of meetings and determine the priority of summaries. For example, the generation AI will prioritize summaries for upcoming meetings. The summarization unit can also provide summaries for past meetings as needed. The summarization unit can also provide summaries for future meetings in advance. This allows for the provision of appropriate summaries by determining the priority of summaries based on the timing of meetings. Timing includes, but is not limited to, the frequency and importance of meetings. Some or all of the above processing in the summarization unit may be performed using a generation AI, or not. For example, the summarization unit can input meeting timing data into the generation AI and have the generation AI determine the priority of summaries.

[0042] The summarization unit can adjust the order of summaries based on the relevance of the meeting during summary generation. For example, the summarization unit may use a generation AI to evaluate the relevance of the meeting and adjust the order of the summaries. For example, the generation AI may provide summaries in order of the most relevant topics. The summarization unit may also postpone the provision of summaries for less relevant topics. The generation AI may also provide summaries for topics of moderate relevance in an appropriate order. This allows for the provision of appropriate summaries by adjusting the order of summaries based on the relevance of the meeting. Relevance includes, but is not limited to, the relevance of the topics and the importance of the statements. Some or all of the above processing in the summarization unit may be performed using a generation AI, for example, or without a generation AI. For example, the summarization unit may input meeting relevance data into a generation AI and have the generation AI adjust the order of the summaries.

[0043] The analysis unit can improve the accuracy of its analysis by considering the interrelationships between statements during the analysis process. For example, the analysis unit can use a generative AI to evaluate the interrelationships between statements and improve the accuracy of the analysis. For example, the analysis unit can analyze the relationships between speakers, and the generative AI can evaluate their impact on the agenda. The analysis unit can also analyze the contextual relationships between statements, and the generative AI can evaluate their impact on the progress of the agenda. The analysis unit can also analyze the relationship between the frequency and content of statements, and the generative AI can evaluate the importance of the agenda. In this way, the accuracy of the analysis is improved by considering the interrelationships between statements. Interrelationships include, but are not limited to, the relationships between statements and the relationships between speakers. Some or all of the above-described processes in the analysis unit may be performed using, for example, a generative AI, or without a generative AI. For example, the analysis unit can input interrelationship data of statements into a generative AI and have the generative AI perform the improvement of the analysis accuracy.

[0044] The analysis unit can perform analysis while considering the speaker's attribute information. For example, the analysis unit can use a generative AI to evaluate the speaker's attribute information and perform the analysis. For example, the analysis unit can have the generative AI perform the analysis while considering the speaker's job title and field of expertise. The analysis unit can also have the generative AI perform the analysis while considering the speaker's past speech history. The analysis unit can also have the generative AI perform the analysis while considering the speaker's department and team. This improves the accuracy of the analysis by considering the speaker's attribute information. Attribute information includes, but is not limited to, the speaker's job title and field of expertise. Some or all of the above-described processes in the analysis unit may be performed using, for example, a generative AI, or without using a generative AI. For example, the analysis unit can input speaker attribute information data into a generative AI and have the generative AI perform the analysis.

[0045] The analysis unit can perform analysis while considering the geographical distribution of statements. For example, the analysis unit can use a generative AI to evaluate the geographical distribution of statements and perform the analysis. For example, the analysis unit can have the generative AI perform the analysis while considering the location of the speaker. The analysis unit can also have the generative AI perform the analysis while considering the geographical influence of the statements. The analysis unit can also have the generative AI evaluate the regional trends of statements by analyzing them. This improves the accuracy of the analysis by considering the geographical distribution of statements. Geographical distribution includes, but is not limited to, the location of the speaker and the regional characteristics of the statements. Some or all of the above-described processes in the analysis unit may be performed using a generative AI, for example, or without using a generative AI. For example, the analysis unit can input geographical distribution data of statements into a generative AI and have the generative AI perform the analysis.

[0046] The analysis unit can improve the accuracy of its analysis by referring to relevant literature related to the statement during the analysis. For example, the analysis unit can use a generative AI to evaluate relevant literature related to the statement and improve the accuracy of the analysis. For example, the analysis unit can have the generative AI perform the analysis by referring to literature related to the content of the statement. The analysis unit can also have the generative AI perform the analysis by referring to literature that forms the basis of the statement. The analysis unit can also have the generative AI perform the analysis by obtaining background information on the statement from literature. In this way, the accuracy of the analysis is improved by referring to relevant literature related to the statement. Relevant literature includes, but is not limited to, academic papers and technical reports. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or without using a generative AI. For example, the analysis unit can input relevant literature data for the statement into a generative AI and have the generative AI perform the analysis.

[0047] The quantification unit can adjust the level of detail of the quantification based on the importance of the statement during the quantification process. For example, the quantification unit uses a generation AI to evaluate the importance of the statement and adjust the level of detail of the quantification. For example, the generation AI provides detailed quantification for statements of high importance. The generation AI can also provide concise quantification for statements of low importance. The generation AI can also provide quantification with an appropriate level of detail for statements of moderate importance. In this way, appropriate quantification can be provided by adjusting the level of detail of the quantification according to the importance of the statement. Importance includes, but is not limited to, the content of the statement and the position of the speaker. Some or all of the above processing in the quantification unit may be performed using, for example, a generation AI, or without using a generation AI. For example, the quantification unit can input statement importance data into a generation AI and have the generation AI perform the adjustment of the level of detail of the quantification.

[0048] The quantification unit can apply different quantification algorithms depending on the category of the statement during quantification. For example, the quantification unit uses a generative AI to evaluate the category of the statement and apply a different quantification algorithm. For example, for technical statements, the generative AI provides quantification that emphasizes technical details. For sales statements, the generative AI can also provide quantification that emphasizes sales and customer information. For management statements, the generative AI can also provide quantification that emphasizes strategic points. In this way, appropriate quantification can be provided by applying a quantification algorithm according to the category of the statement. Categories include, but are not limited to, technical statements and business statements. Some or all of the above processing in the quantification unit may be performed using, for example, a generative AI, or without a generative AI. For example, the quantification unit can input statement category data into a generative AI and have the generative AI execute the application of a quantification algorithm.

[0049] The quantification unit can perform quantification while considering the geographical distribution of statements. For example, the quantification unit can use a generation AI to evaluate the geographical distribution of statements and perform quantification. For example, the quantification unit can have the generation AI perform quantification while considering the location of the speaker. The quantification unit can also have the generation AI perform quantification while considering the geographical impact of statements. The quantification unit can also have the generation AI evaluate the regional trends of statements by quantifying them. This improves the accuracy of quantification by considering the geographical distribution of statements. Geographical distribution includes, but is not limited to, the location of the speaker and the regionality of the statements. Some or all of the above processing in the quantification unit may be performed using a generation AI, for example, or without using a generation AI. For example, the quantification unit can input geographical distribution data of statements into a generation AI and have the generation AI perform quantification.

[0050] The quantification unit can improve the accuracy of quantification by referring to relevant literature related to the statement during the quantification process. For example, the quantification unit can use a generation AI to evaluate relevant literature related to the statement and improve the accuracy of quantification. For example, the quantification unit can have the generation AI perform quantification by referring to literature related to the content of the statement. The quantification unit can also have the generation AI perform quantification by referring to literature that forms the basis of the statement. The quantification unit can also have the generation AI perform quantification by obtaining background information on the statement from literature. In this way, the accuracy of quantification is improved by referring to relevant literature related to the statement. Relevant literature includes, but is not limited to, academic papers and technical reports. Some or all of the above processing in the quantification unit may be performed using, for example, a generation AI, or without using a generation AI. For example, the quantification unit can input relevant literature data for the statement into a generation AI and have the generation AI perform quantification.

[0051] The visualization unit can adjust the level of detail of the visualization based on the importance of the meeting during visualization. For example, the visualization unit uses a generative AI to evaluate the importance of the meeting and adjust the level of detail of the visualization. For example, the visualization unit can use the generative AI to provide a detailed visualization for high-importance meetings. The visualization unit can also use the generative AI to provide a concise visualization for low-importance meetings. The visualization unit can also use the generative AI to provide a visualization with a moderate level of detail for meetings of moderate importance. In this way, appropriate visualizations can be provided by adjusting the level of detail of the visualization according to the importance of the meeting. Importance includes, but is not limited to, the purpose of the meeting and the positions of the participants. Some or all of the above processing in the visualization unit may be performed using, for example, a generative AI, or without a generative AI. For example, the visualization unit can input meeting importance data into the generative AI and have the generative AI perform the adjustment of the level of detail of the visualization.

[0052] The visualization unit can apply different visualization algorithms depending on the meeting category during visualization. For example, the visualization unit uses a generative AI to evaluate the meeting category and apply a different visualization algorithm. For example, for a technical meeting, the generative AI provides a visualization that emphasizes technical details. For a sales meeting, the generative AI can also provide a visualization that emphasizes sales and customer information. For a management meeting, the generative AI can also provide a visualization that emphasizes strategic points. In this way, appropriate visualizations can be provided by applying a visualization algorithm according to the meeting category. Categories include, but are not limited to, business meetings and technical meetings. Some or all of the above processing in the visualization unit may be performed using, for example, a generative AI, or not using a generative AI. For example, the visualization unit can input meeting category data into a generative AI and have the generative AI execute the application of a visualization algorithm.

[0053] The visualization unit can determine the priority of visualizations based on the timing of meetings during the visualization process. For example, the visualization unit uses a generative AI to evaluate the timing of meetings and determine the priority of visualizations. For example, the visualization unit prioritizes providing visualizations to the generative AI for upcoming meetings. For past meetings, the visualization unit can also provide visualizations to the generative AI as needed. For future meetings, the visualization unit can also provide visualizations to the generative AI in advance. This allows for the provision of appropriate visualizations by determining the priority of visualizations based on the timing of meetings. Timing includes, but is not limited to, the frequency and importance of meetings. Some or all of the above processing in the visualization unit may be performed using, for example, a generative AI, or without a generative AI. For example, the visualization unit can input meeting timing data into a generative AI and have the generative AI determine the priority of visualizations.

[0054] The visualization unit can adjust the order of visualizations based on the relevance of the meeting during visualization. For example, the visualization unit may use a generative AI to evaluate the relevance of the meeting and adjust the order of visualizations. For example, the visualization unit may have the generative AI provide visualizations in order of the most relevance of the agenda items. The visualization unit may also have the generative AI provide visualizations for less relevance items later. The visualization unit may also have the generative AI provide visualizations for moderately relevance items in an appropriate order. In this way, appropriate visualizations can be provided by adjusting the order of visualizations based on the relevance of the meeting. Relevance includes, but is not limited to, the relevance of the agenda items and the importance of the statements. Some or all of the above processing in the visualization unit may be performed using, for example, a generative AI, or without a generative AI. For example, the visualization unit may input meeting relevance data into a generative AI and have the generative AI perform the adjustment of the order of visualizations.

[0055] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0056] The meeting efficiency system can also include a notification function. This notification function can inform participants in real time about the meeting's progress and important changes to the agenda. For example, the notification function can send an alert to participants if the meeting is behind schedule. It can also notify participants if the agenda changes. Furthermore, it can notify participants when important statements are made. This allows participants to stay informed about the meeting's progress and important changes in real time, enabling more efficient meeting management.

[0057] The meeting efficiency system can also include a feedback section. This section can collect feedback from participants after a meeting and incorporate it into future meetings. For example, the feedback section can gather opinions from participants regarding the appropriateness of the meeting's progress and agenda. It can also collect suggestions from participants for improving meeting efficiency. Furthermore, the feedback section can evaluate participant satisfaction and identify areas for improvement in future meetings. This can improve the quality of meetings and increase participant satisfaction.

[0058] Meeting efficiency systems can also include a recording function. This function can automatically record the meeting content for later reference. For example, it can record speech during the meeting as text. It can also record the meeting's audio. Furthermore, it can record the meeting's video. This allows for later review of the meeting's content, ensuring that important information is not missed.

[0059] The meeting efficiency system can also be equipped with a translation function. This function can translate speech during a meeting in real time, enabling smooth communication among participants who speak different languages. For example, the translation function can translate speech in English into Japanese. It can also translate speech in Japanese into English. The translation function can support other languages ​​as well. This allows for efficient discussions even in international meetings, overcoming language barriers.

[0060] The meeting efficiency system can also include a scheduling unit. This unit can automatically adjust participants' schedules and suggest the optimal meeting date and time. For example, it can refer to participants' calendars to identify a time when everyone is available. It can also prioritize meetings according to their importance. Furthermore, it can send meeting reminders to participants. This ensures efficient meeting scheduling and makes it easier for all participants to attend.

[0061] The following briefly describes the processing flow for example form 1.

[0062] Step 1: The summarization unit summarizes the main points of the meeting materials. For example, it analyzes the meeting materials using generation AI and extracts the key points. The summarization unit also analyzes slide materials created with presentation applications to clarify the purpose and agenda of the meeting. Step 2: The analysis unit analyzes the statements made during the meeting based on the main points summarized by the summarization unit. For example, it uses a generation AI to analyze the content of the statements and compares them to the main points of the meeting. Step 3: The quantification unit quantifies the degree to which the utterances analyzed by the analysis unit match the agenda. For example, it uses a generation AI to quantify the degree of match of the utterances and quantifies how relevant the utterances are to the agenda. Step 4: The visualization unit visualizes the results obtained by the quantification unit. For example, it uses a generation AI to display the degree of agreement in a graph or chart, allowing the progress of the meeting to be seen at a glance.

[0063] (Example of form 2) The meeting efficiency system according to an embodiment of the present invention is a system that improves meeting efficiency by utilizing generative AI. This meeting efficiency system uses generative AI to automatically summarize the main points from meeting materials, analyzes comments made during the meeting in real time, and quantifies the degree of agreement with the agenda. This visualizes the progress of the meeting, allowing all participants to objectively understand it. This mechanism enables efficient meeting management that does not rely on subjective opinions and strongly supports corporate decision-making. For example, the generative AI automatically summarizes the main points from meeting materials. For example, the generative AI analyzes slide materials created with presentation applications and extracts the main points of the meeting. This clarifies the purpose and agenda of the meeting. Next, the generative AI analyzes comments made during the meeting in real time. The generative AI compares the content of the comments with the main points of the meeting and quantifies the degree of agreement. For example, it quantifies how relevant the comments are to the agenda and determines whether they deviate from the agenda. Furthermore, the generative AI visualizes the results of its analysis. This allows all participants to objectively understand the progress of the meeting. For example, the degree of alignment with the agenda can be displayed in graphs and charts, allowing for a quick overview of the meeting's progress. This system enables efficient meeting management that is not subjective. Participants will be more mindful of making comments relevant to the agenda, leading to smoother meeting progress. Furthermore, even if comments deviate from the agenda, the generating AI will point it out in real time, allowing for quick correction. For example, companies using online meeting tools can improve meeting efficiency by implementing this system. Because the meeting's progress can be grasped in real time, discussions will be aligned with the agenda, and decision-making will be accelerated. In this way, by utilizing generating AI, we provide a system that improves meeting efficiency and strongly supports corporate decision-making. As a result, the meeting efficiency system can objectively grasp the progress of meetings and achieve efficient meeting management.

[0064] The meeting efficiency system according to this embodiment comprises a summarization unit, an analysis unit, a quantification unit, and a visualization unit. The summarization unit summarizes the main points of the meeting materials. The summarization unit analyzes the meeting materials using, for example, a generative AI and extracts the main points. For example, the summarization unit analyzes slide materials created with a presentation application or the like to clarify the purpose and agenda of the meeting. The summarization unit can also summarize the content of the meeting materials using a generative AI. For example, the summarization unit uses a generative AI to extract the key points of the meeting materials and generates a concise summary. The analysis unit analyzes the statements made during the meeting based on the main points summarized by the summarization unit. The analysis unit uses, for example, a generative AI to analyze the content of the statements and compares them to the main points of the meeting. For example, the analysis unit analyzes the content of the statements and quantifies the degree of agreement. The analysis unit can also analyze the content of the statements using a generative AI. For example, the analysis unit uses a generative AI to analyze the content of the statements and quantifies the degree of agreement with the agenda. The quantification unit quantifies the degree of agreement between the content of the statements analyzed by the analysis unit and the agenda. The quantification unit quantifies the degree of agreement between statements using, for example, a generative AI. For example, the quantification unit quantifies how relevant a statement is to the agenda. The quantification unit can also quantify the degree of agreement between statements using a generative AI. For example, the quantification unit uses a generative AI to quantify the degree of agreement between statements and determines whether or not they deviate from the agenda. The visualization unit visualizes the results obtained by the quantification unit. The visualization unit displays the degree of agreement in graphs or charts using, for example, a generative AI. For example, the visualization unit makes it possible to check the progress of the meeting at a glance. The visualization unit can also visualize the progress of the meeting using a generative AI. For example, the visualization unit uses a generative AI to display the degree of agreement in graphs or charts, making it possible to visually grasp the progress of the meeting. As a result, the meeting efficiency system according to the embodiment can objectively grasp the progress of the meeting and realize efficient meeting management.

[0065] The summarization unit summarizes the main points of the meeting materials. For example, the summarization unit analyzes the meeting materials using generative AI and extracts the key points. Specifically, the generative AI uses natural language processing technology to analyze the text data of the meeting materials. For example, when analyzing slide materials created with presentation applications, it focuses on extracting slide titles, bullet points, and figure captions to clarify the purpose and agenda of the meeting. The generative AI understands the context and identifies important keywords and phrases to grasp the essence of the entire document. Furthermore, the generative AI can also summarize the content of the meeting materials. For example, the generative AI extracts the key points of the meeting materials and generates a concise summary. This allows participants to grasp the key points of the materials before the meeting and prepare efficiently. The summarization unit can improve its accuracy by using past meeting materials and summarization results as training data for the generative AI. As a result, the summarization unit can quickly and accurately summarize the content of the meeting materials and provide it to participants.

[0066] The analysis unit analyzes the statements made during the meeting based on the main points summarized by the summarization unit. For example, the analysis unit uses generative AI to analyze the content of the statements and compare them to the main points of the meeting. Specifically, the generative AI uses speech recognition technology to convert the statements made during the meeting into text data. The converted text data is analyzed using natural language processing technology to evaluate how well the statements match the main points and agenda of the meeting. For example, the analysis unit analyzes the statements and quantifies the degree of match. The generative AI extracts keywords and phrases from the statements and compares them with the content of the meeting materials summarized by the summarization unit. This allows the analysis unit to determine whether the statements are relevant to the agenda. The analysis unit can also use generative AI to analyze the statements. For example, the generative AI analyzes the statements and quantifies the degree of match with the agenda. This allows the analysis unit to objectively evaluate how relevant the statements made during the meeting are to the agenda. Furthermore, the analysis unit can provide the analysis results of the statements in real time, providing information to help understand the progress of the meeting.

[0067] The quantification unit quantifies the degree to which the spoken content analyzed by the analysis unit matches the agenda. For example, the quantification unit uses a generative AI to quantify the degree of matching of spoken content. Specifically, the generative AI uses an algorithm that extracts keywords and phrases from the spoken content and quantifies their importance in relation to the agenda. For example, to quantify how relevant a spoken statement is to the agenda, the generative AI calculates the degree of matching between the spoken content and the keywords of the agenda and assigns a score. The quantification unit can also use the generative AI to quantify the degree of matching of spoken content. For example, the generative AI quantifies the degree of matching of spoken content and determines whether it deviates from the agenda. This allows the quantification unit to objectively evaluate how relevant spoken content is to the agenda during a meeting and display it numerically. Furthermore, the quantification unit can calculate the degree of matching of spoken content in real time and provide information to understand the progress of the meeting. This allows the quantification unit to contribute to the efficiency of meetings and promote discussion focused on the agenda.

[0068] The visualization unit visualizes the results obtained by the quantification unit. For example, the visualization unit uses generative AI to display the degree of agreement in graphs and charts. Specifically, the generative AI uses an algorithm that displays the degree of agreement calculated by the quantification unit in a visually easy-to-understand format. For example, the degree of agreement can be displayed in bar graphs, pie charts, or line graphs, allowing participants to see the progress of the meeting at a glance. The visualization unit can also visualize the progress of the meeting using generative AI. For example, the generative AI can display the degree of agreement in graphs and charts, allowing participants to visually grasp the progress of the meeting. This allows participants to understand the progress of the meeting in real time and engage in focused discussions on the agenda. Furthermore, the visualization unit can save the visualized data as a meeting record for later reference. In this way, the visualization unit can contribute to the efficiency of meetings and provide information for objectively evaluating the progress of discussions.

[0069] The summarization unit can analyze slide materials created with presentation applications and extract the main points of a meeting. For example, the summarization unit can use generative AI to analyze the slide materials and extract the main points. For example, the summarization unit can analyze the content of the slide materials to clarify the purpose and agenda of the meeting. The summarization unit can also use generative AI to summarize the content of the slide materials. For example, the summarization unit can have the generative AI extract the key points of the slide materials and generate a concise summary. This clarifies the purpose and agenda of the meeting by extracting the main points. The main points include, but are not limited to, important agenda items, conclusions, and action items. Some or all of the above processing in the summarization unit may be performed using, for example, generative AI, or without generative AI. For example, the summarization unit can input the slide materials into the generative AI and have the generative AI perform the extraction of the main points.

[0070] The analysis unit can compare the content of the statements with the purpose of the meeting and quantify the degree of agreement. The analysis unit can, for example, use a generative AI to analyze the content of the statements and compare it with the purpose of the meeting. For example, the analysis unit can analyze the content of the statements and quantify the degree of agreement. The analysis unit can also use a generative AI to analyze the content of the statements. For example, the analysis unit can have a generative AI analyze the content of the statements and quantify the degree of agreement with the agenda. By quantifying the degree of agreement of the content of the statements, it is possible to determine whether or not there is a deviation from the agenda. The degree of agreement includes, but is not limited to, the relevance and importance of the content of the statements. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or without a generative AI. For example, the analysis unit can input the content of the statements into a generative AI and have the generative AI perform the quantification of the degree of agreement.

[0071] The quantification unit can quantify how relevant a statement is to the agenda and determine whether it deviates from the agenda. The quantification unit can quantify the degree of relevance of the statement content using, for example, a generative AI. For example, the quantification unit can quantify how relevant a statement is to the agenda. The quantification unit can also quantify the degree of relevance of the statement content using a generative AI. For example, the quantification unit can use a generative AI to quantify the degree of relevance of the statement content and determine whether it deviates from the agenda. In this way, by quantifying the relevance of the statement, it is possible to determine whether it deviates from the agenda. Relevance includes, but is not limited to, the degree of relevance of the statement content and importance. Some or all of the above processing in the quantification unit may be performed using, for example, a generative AI, or without a generative AI. For example, the quantification unit can input the statement content into a generative AI and have the generative AI perform the quantification of relevance.

[0072] The visualization unit can display the degree of agreement with the agenda in graphs and charts, allowing for a quick overview of the meeting's progress. The visualization unit can, for example, use generative AI to display the degree of agreement in graphs and charts. For example, the visualization unit can make the meeting's progress immediately apparent. The visualization unit can also visualize the meeting's progress using generative AI. For example, the visualization unit can use generative AI to display the degree of agreement in graphs and charts, allowing for a visual understanding of the meeting's progress. This allows for a quick overview of the meeting's progress by visualizing the degree of agreement with the agenda. The progress includes, but is not limited to, the progress of agenda items and the frequency of contributions. Some or all of the above-described processes in the visualization unit may be performed, for example, using generative AI, or without using generative AI. For example, the visualization unit can input the degree of agreement data into the generative AI and have the generative AI perform the visualization.

[0073] The visualization unit enables companies using online meeting tools to understand the progress of meetings in real time. The visualization unit can, for example, use generative AI to display the progress of meetings in real time. For example, the visualization unit enables companies using online meeting tools to understand the progress of meetings in real time. The visualization unit can also visualize the progress of meetings in real time using generative AI. For example, the visualization unit has generative AI display the progress of meetings in real time, so that all participants can understand the progress. This allows companies using online meeting tools to understand the progress of meetings in real time. Real time includes, but is not limited to, data update frequency and display delay time. Some or all of the above processing in the visualization unit may be performed using, for example, generative AI, or without generative AI. For example, the visualization unit can input real-time data into generative AI and have the generative AI perform the visualization.

[0074] The summarization unit can estimate the user's emotions and adjust the way the summary is presented based on the estimated emotions. For example, the summarization unit can use a generative AI to estimate the user's emotions and adjust the way the summary is presented. For example, if the user is stressed, the generative AI can provide a concise and to-the-point summary. If the user is relaxed, the generative AI can also provide a summary that includes detailed explanations. If the user is excited, the generative AI can also provide a visually appealing summary. This allows for the provision of a more appropriate summary by adjusting the way the summary is presented according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processes in the summarization unit may be performed using a generative AI, or not. For example, the summarization unit can input user emotion data into a generative AI and have the generative AI adjust the way the summary is presented.

[0075] The summarization unit can adjust the level of detail in the summary based on the importance of the meeting during summary generation. For example, the summarization unit can use a generation AI to evaluate the importance of the meeting and adjust the level of detail in the summary. For example, for highly important meetings, the generation AI will provide a detailed summary. For less important meetings, the generation AI may provide a concise summary. For meetings of moderate importance, the generation AI may provide a summary with an appropriate level of detail. This allows for the provision of an appropriate summary by adjusting the level of detail in the summary according to the importance of the meeting. Importance includes, but is not limited to, the purpose of the meeting and the positions of the participants. Some or all of the above processing in the summarization unit may be performed using a generation AI, for example, or without a generation AI. For example, the summarization unit can input meeting importance data into a generation AI and have the generation AI perform the adjustment of the level of detail in the summary.

[0076] The summarization unit can apply different summarization algorithms depending on the meeting category when generating summaries. For example, the summarization unit uses a generation AI to evaluate the meeting category and apply a different summarization algorithm. For example, for technical meetings, the generation AI can provide a summary that emphasizes technical details. For sales meetings, the generation AI can also provide a summary that emphasizes sales and customer information. For management meetings, the generation AI can also provide a summary that emphasizes strategic points. This allows for the provision of appropriate summaries by applying a summarization algorithm according to the meeting category. Categories include, but are not limited to, business meetings and technical meetings. Some or all of the above processing in the summarization unit may be performed using, for example, a generation AI, or without a generation AI. For example, the summarization unit can input meeting category data into a generation AI and have the generation AI apply the summarization algorithm.

[0077] The summarization unit can estimate the user's emotions and adjust the length of the summary based on the estimated emotions. For example, the summarization unit can use generative AI to estimate the user's emotions and adjust the length of the summary. For example, if the user is in a hurry, the generative AI can provide a short, concise summary. If the user is relaxed, the generative AI can provide a longer summary that includes detailed explanations. If the user is excited, the generative AI can provide a visually appealing summary. By adjusting the length of the summary according to the user's emotions, a more appropriate summary can be provided. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the summarization unit may be performed using generative AI, or not. For example, the summarization unit can input user emotion data into the generative AI and have the generative AI adjust the length of the summary.

[0078] The summarization unit can determine the priority of summaries based on the timing of meetings when generating summaries. For example, the summarization unit uses a generation AI to evaluate the timing of meetings and determine the priority of summaries. For example, the generation AI will prioritize summaries for upcoming meetings. The summarization unit can also provide summaries for past meetings as needed. The summarization unit can also provide summaries for future meetings in advance. This allows for the provision of appropriate summaries by determining the priority of summaries based on the timing of meetings. Timing includes, but is not limited to, the frequency and importance of meetings. Some or all of the above processing in the summarization unit may be performed using a generation AI, or not. For example, the summarization unit can input meeting timing data into the generation AI and have the generation AI determine the priority of summaries.

[0079] The summarization unit can adjust the order of summaries based on the relevance of the meeting during summary generation. For example, the summarization unit may use a generation AI to evaluate the relevance of the meeting and adjust the order of the summaries. For example, the generation AI may provide summaries in order of the most relevant topics. The summarization unit may also postpone the provision of summaries for less relevant topics. The generation AI may also provide summaries for topics of moderate relevance in an appropriate order. This allows for the provision of appropriate summaries by adjusting the order of summaries based on the relevance of the meeting. Relevance includes, but is not limited to, the relevance of the topics and the importance of the statements. Some or all of the above processing in the summarization unit may be performed using a generation AI, for example, or without a generation AI. For example, the summarization unit may input meeting relevance data into a generation AI and have the generation AI adjust the order of the summaries.

[0080] The analysis unit can estimate the user's emotions and adjust the analysis criteria based on the estimated user emotions. For example, the analysis unit can use a generative AI to estimate the user's emotions and adjust the analysis criteria. For example, if the user is stressed, the generative AI can provide a concise and to-the-point analysis. If the user is relaxed, the generative AI can provide a detailed analysis. If the user is excited, the generative AI can provide a visually appealing analysis. By adjusting the analysis criteria according to the user's emotions, a more appropriate analysis can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processes in the analysis unit may be performed using a generative AI, or not. For example, the analysis unit can input user emotion data into a generative AI and have the generative AI adjust the analysis criteria.

[0081] The analysis unit can improve the accuracy of its analysis by considering the interrelationships between statements during the analysis process. For example, the analysis unit can use a generative AI to evaluate the interrelationships between statements and improve the accuracy of the analysis. For example, the analysis unit can analyze the relationships between speakers, and the generative AI can evaluate their impact on the agenda. The analysis unit can also analyze the contextual relationships between statements, and the generative AI can evaluate their impact on the progress of the agenda. The analysis unit can also analyze the relationship between the frequency and content of statements, and the generative AI can evaluate the importance of the agenda. In this way, the accuracy of the analysis is improved by considering the interrelationships between statements. Interrelationships include, but are not limited to, the relationships between statements and the relationships between speakers. Some or all of the above-described processes in the analysis unit may be performed using, for example, a generative AI, or without a generative AI. For example, the analysis unit can input interrelationship data of statements into a generative AI and have the generative AI perform the improvement of the analysis accuracy.

[0082] The analysis unit can perform analysis while considering the speaker's attribute information. For example, the analysis unit can use a generative AI to evaluate the speaker's attribute information and perform the analysis. For example, the analysis unit can have the generative AI perform the analysis while considering the speaker's job title and field of expertise. The analysis unit can also have the generative AI perform the analysis while considering the speaker's past speech history. The analysis unit can also have the generative AI perform the analysis while considering the speaker's department and team. This improves the accuracy of the analysis by considering the speaker's attribute information. Attribute information includes, but is not limited to, the speaker's job title and field of expertise. Some or all of the above-described processes in the analysis unit may be performed using, for example, a generative AI, or without using a generative AI. For example, the analysis unit can input speaker attribute information data into a generative AI and have the generative AI perform the analysis.

[0083] The analysis unit can estimate the user's emotions and adjust the order in which the analysis results are displayed based on the estimated emotions. For example, the analysis unit can use a generative AI to estimate the user's emotions and adjust the order in which the analysis results are displayed. For example, if the user is in a hurry, the analysis unit can prioritize displaying important analysis results. If the user is relaxed, the analysis unit can also display detailed analysis results sequentially. If the user is excited, the analysis unit can also display visually appealing analysis results. By adjusting the display order of the analysis results according to the user's emotions, more appropriate analysis results can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. The generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the analysis unit may be performed using a generative AI, or not using a generative AI. For example, the analysis unit can input user emotion data into a generative AI and have the generative AI adjust the display order of the analysis results.

[0084] The analysis unit can perform analysis while considering the geographical distribution of statements. For example, the analysis unit can use a generative AI to evaluate the geographical distribution of statements and perform the analysis. For example, the analysis unit can have the generative AI perform the analysis while considering the location of the speaker. The analysis unit can also have the generative AI perform the analysis while considering the geographical influence of the statements. The analysis unit can also have the generative AI evaluate the regional trends of statements by analyzing them. This improves the accuracy of the analysis by considering the geographical distribution of statements. Geographical distribution includes, but is not limited to, the location of the speaker and the regional characteristics of the statements. Some or all of the above-described processes in the analysis unit may be performed using a generative AI, for example, or without using a generative AI. For example, the analysis unit can input geographical distribution data of statements into a generative AI and have the generative AI perform the analysis.

[0085] The analysis unit can improve the accuracy of its analysis by referring to relevant literature related to the statement during the analysis. For example, the analysis unit can use a generative AI to evaluate relevant literature related to the statement and improve the accuracy of the analysis. For example, the analysis unit can have the generative AI perform the analysis by referring to literature related to the content of the statement. The analysis unit can also have the generative AI perform the analysis by referring to literature that forms the basis of the statement. The analysis unit can also have the generative AI perform the analysis by obtaining background information on the statement from literature. In this way, the accuracy of the analysis is improved by referring to relevant literature related to the statement. Relevant literature includes, but is not limited to, academic papers and technical reports. Some or all of the above processing in the analysis unit may be performed using, for example, a generative AI, or without using a generative AI. For example, the analysis unit can input relevant literature data for the statement into a generative AI and have the generative AI perform the analysis.

[0086] The quantification unit can estimate the user's emotions and adjust the quantification criteria based on the estimated user emotions. For example, the quantification unit can use a generative AI to estimate the user's emotions and adjust the quantification criteria. For example, if the user is stressed, the generative AI can provide a concise and to-the-point quantification. If the user is relaxed, the generative AI can also provide a detailed quantification. If the user is excited, the generative AI can also provide a visually appealing quantification. This allows for more appropriate quantification by adjusting the quantification criteria according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processes in the quantification unit may be performed using a generative AI, or not. For example, the quantification unit can input user emotion data into a generative AI and have the generative AI adjust the quantification criteria.

[0087] The quantification unit can adjust the level of detail of the quantification based on the importance of the statement during the quantification process. For example, the quantification unit uses a generation AI to evaluate the importance of the statement and adjust the level of detail of the quantification. For example, the generation AI provides detailed quantification for statements of high importance. The generation AI can also provide concise quantification for statements of low importance. The generation AI can also provide quantification with an appropriate level of detail for statements of moderate importance. In this way, appropriate quantification can be provided by adjusting the level of detail of the quantification according to the importance of the statement. Importance includes, but is not limited to, the content of the statement and the position of the speaker. Some or all of the above processing in the quantification unit may be performed using, for example, a generation AI, or without using a generation AI. For example, the quantification unit can input statement importance data into a generation AI and have the generation AI perform the adjustment of the level of detail of the quantification.

[0088] The quantification unit can apply different quantification algorithms depending on the category of the statement during quantification. For example, the quantification unit uses a generative AI to evaluate the category of the statement and apply a different quantification algorithm. For example, for technical statements, the generative AI provides quantification that emphasizes technical details. For sales statements, the generative AI can also provide quantification that emphasizes sales and customer information. For management statements, the generative AI can also provide quantification that emphasizes strategic points. In this way, appropriate quantification can be provided by applying a quantification algorithm according to the category of the statement. Categories include, but are not limited to, technical statements and business statements. Some or all of the above processing in the quantification unit may be performed using, for example, a generative AI, or without a generative AI. For example, the quantification unit can input statement category data into a generative AI and have the generative AI execute the application of a quantification algorithm.

[0089] The quantification unit can estimate the user's emotions and adjust the order in which the quantification results are displayed based on the estimated user emotions. For example, the quantification unit can use a generative AI to estimate the user's emotions and adjust the order in which the quantification results are displayed. For example, if the user is in a hurry, the quantification unit can prioritize displaying important quantification results. If the user is relaxed, the quantification unit can also sequentially display detailed quantification results. If the user is excited, the quantification unit can also display visually appealing quantification results. This allows for the provision of more appropriate quantification results by adjusting the display order of the quantification results according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the quantification unit may be performed using a generative AI, or not. For example, the quantification unit can input user emotion data into a generative AI and have the generative AI adjust the display order of the quantification results.

[0090] The quantification unit can perform quantification while considering the geographical distribution of statements. For example, the quantification unit can use a generation AI to evaluate the geographical distribution of statements and perform quantification. For example, the quantification unit can have the generation AI perform quantification while considering the location of the speaker. The quantification unit can also have the generation AI perform quantification while considering the geographical impact of statements. The quantification unit can also have the generation AI evaluate the regional trends of statements by quantifying them. This improves the accuracy of quantification by considering the geographical distribution of statements. Geographical distribution includes, but is not limited to, the location of the speaker and the regionality of the statements. Some or all of the above processing in the quantification unit may be performed using a generation AI, for example, or without using a generation AI. For example, the quantification unit can input geographical distribution data of statements into a generation AI and have the generation AI perform quantification.

[0091] The quantification unit can improve the accuracy of quantification by referring to relevant literature related to the statement during the quantification process. For example, the quantification unit can use a generation AI to evaluate relevant literature related to the statement and improve the accuracy of quantification. For example, the quantification unit can have the generation AI perform quantification by referring to literature related to the content of the statement. The quantification unit can also have the generation AI perform quantification by referring to literature that forms the basis of the statement. The quantification unit can also have the generation AI perform quantification by obtaining background information on the statement from literature. In this way, the accuracy of quantification is improved by referring to relevant literature related to the statement. Relevant literature includes, but is not limited to, academic papers and technical reports. Some or all of the above processing in the quantification unit may be performed using, for example, a generation AI, or without using a generation AI. For example, the quantification unit can input relevant literature data for the statement into a generation AI and have the generation AI perform quantification.

[0092] The visualization unit can estimate the user's emotions and adjust the visualization method based on the estimated emotions. For example, the visualization unit can use a generative AI to estimate the user's emotions and adjust the visualization method. For example, if the user is stressed, the generative AI can provide a simple and highly visible visualization. If the user is relaxed, the generative AI can also provide a visualization that includes detailed information. If the user is excited, the generative AI can also provide a visually appealing visualization. This allows for the provision of more appropriate visualizations by adjusting the visualization method according to the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processes in the visualization unit may be performed using a generative AI, or not. For example, the visualization unit can input user emotion data into a generative AI and have the generative AI adjust the visualization method.

[0093] The visualization unit can adjust the level of detail of the visualization based on the importance of the meeting during visualization. For example, the visualization unit uses a generative AI to evaluate the importance of the meeting and adjust the level of detail of the visualization. For example, the visualization unit can use the generative AI to provide a detailed visualization for high-importance meetings. The visualization unit can also use the generative AI to provide a concise visualization for low-importance meetings. The visualization unit can also use the generative AI to provide a visualization with a moderate level of detail for meetings of moderate importance. In this way, appropriate visualizations can be provided by adjusting the level of detail of the visualization according to the importance of the meeting. Importance includes, but is not limited to, the purpose of the meeting and the positions of the participants. Some or all of the above processing in the visualization unit may be performed using, for example, a generative AI, or without a generative AI. For example, the visualization unit can input meeting importance data into the generative AI and have the generative AI perform the adjustment of the level of detail of the visualization.

[0094] The visualization unit can apply different visualization algorithms depending on the meeting category during visualization. For example, the visualization unit uses a generative AI to evaluate the meeting category and apply a different visualization algorithm. For example, for a technical meeting, the generative AI provides a visualization that emphasizes technical details. For a sales meeting, the generative AI can also provide a visualization that emphasizes sales and customer information. For a management meeting, the generative AI can also provide a visualization that emphasizes strategic points. In this way, appropriate visualizations can be provided by applying a visualization algorithm according to the meeting category. Categories include, but are not limited to, business meetings and technical meetings. Some or all of the above processing in the visualization unit may be performed using, for example, a generative AI, or not using a generative AI. For example, the visualization unit can input meeting category data into a generative AI and have the generative AI execute the application of a visualization algorithm.

[0095] The visualization unit can estimate the user's emotions and adjust the length of the visualization based on the estimated emotions. For example, the visualization unit can use a generative AI to estimate the user's emotions and adjust the length of the visualization. For example, if the user is in a hurry, the generative AI can provide a short, concise visualization. If the user is relaxed, the generative AI can also provide a longer visualization containing detailed information. If the user is excited, the generative AI can also provide a visually appealing visualization. By adjusting the length of the visualization according to the user's emotions, a more appropriate visualization can be provided. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above processing in the visualization unit may be performed using a generative AI, or not. For example, the visualization unit can input user emotion data into a generative AI and have the generative AI adjust the length of the visualization.

[0096] The visualization unit can determine the priority of visualizations based on the timing of meetings during the visualization process. For example, the visualization unit uses a generative AI to evaluate the timing of meetings and determine the priority of visualizations. For example, the visualization unit prioritizes providing visualizations to the generative AI for upcoming meetings. For past meetings, the visualization unit can also provide visualizations to the generative AI as needed. For future meetings, the visualization unit can also provide visualizations to the generative AI in advance. This allows for the provision of appropriate visualizations by determining the priority of visualizations based on the timing of meetings. Timing includes, but is not limited to, the frequency and importance of meetings. Some or all of the above processing in the visualization unit may be performed using, for example, a generative AI, or without a generative AI. For example, the visualization unit can input meeting timing data into a generative AI and have the generative AI determine the priority of visualizations.

[0097] The visualization unit can adjust the order of visualizations based on the relevance of the meeting during visualization. For example, the visualization unit may use a generative AI to evaluate the relevance of the meeting and adjust the order of visualizations. For example, the visualization unit may have the generative AI provide visualizations in order of the most relevance of the agenda items. The visualization unit may also have the generative AI provide visualizations for less relevance items later. The visualization unit may also have the generative AI provide visualizations for moderately relevance items in an appropriate order. In this way, appropriate visualizations can be provided by adjusting the order of visualizations based on the relevance of the meeting. Relevance includes, but is not limited to, the relevance of the agenda items and the importance of the statements. Some or all of the above processing in the visualization unit may be performed using, for example, a generative AI, or without a generative AI. For example, the visualization unit may input meeting relevance data into a generative AI and have the generative AI perform the adjustment of the order of visualizations.

[0098] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0099] The meeting efficiency system can also include a notification function. This notification function can inform participants in real time about the meeting's progress and important changes to the agenda. For example, the notification function can send an alert to participants if the meeting is behind schedule. It can also notify participants if the agenda changes. Furthermore, it can notify participants when important statements are made. This allows participants to stay informed about the meeting's progress and important changes in real time, enabling more efficient meeting management.

[0100] The meeting efficiency system can also include a feedback section. This section can collect feedback from participants after a meeting and incorporate it into future meetings. For example, the feedback section can gather opinions from participants regarding the appropriateness of the meeting's progress and agenda. It can also collect suggestions from participants for improving meeting efficiency. Furthermore, the feedback section can evaluate participant satisfaction and identify areas for improvement in future meetings. This can improve the quality of meetings and increase participant satisfaction.

[0101] Meeting efficiency systems can also include a recording function. This function can automatically record the meeting content for later reference. For example, it can record speech during the meeting as text. It can also record the meeting's audio. Furthermore, it can record the meeting's video. This allows for later review of the meeting's content, ensuring that important information is not missed.

[0102] The meeting efficiency system can also be equipped with a translation function. This function can translate speech during a meeting in real time, enabling smooth communication among participants who speak different languages. For example, the translation function can translate speech in English into Japanese. It can also translate speech in Japanese into English. The translation function can support other languages ​​as well. This allows for efficient discussions even in international meetings, overcoming language barriers.

[0103] The meeting efficiency system can also include a scheduling unit. This unit can automatically adjust participants' schedules and suggest the optimal meeting date and time. For example, it can refer to participants' calendars to identify a time when everyone is available. It can also prioritize meetings according to their importance. Furthermore, it can send meeting reminders to participants. This ensures efficient meeting scheduling and makes it easier for all participants to attend.

[0104] The analysis unit can estimate the user's emotions and evaluate the importance of their statements based on those emotions. For example, if the user is excited, the analysis unit will evaluate their statements as highly important. If the user is relaxed, the analysis unit may evaluate their statements as less important. If the user is stressed, the analysis unit may evaluate their statements as moderately important. By evaluating the importance of statements according to the user's emotions, the analysis unit can provide more appropriate results.

[0105] The quantification unit can estimate the user's emotions and adjust the degree of agreement with the statement based on the estimated emotions. For example, if the user is excited, the quantification unit will rate the degree of agreement with the statement highly. If the user is relaxed, the quantification unit may rate the degree of agreement with the statement less highly. If the user is stressed, the quantification unit may rate the degree of agreement with the statement moderately. By adjusting the degree of agreement with the statement according to the user's emotions, more appropriate quantification results can be provided.

[0106] The visualization unit can estimate the user's emotions and adjust the colors of the visualization based on those emotions. For example, if the user is stressed, the visualization unit can provide a visualization with calming colors. If the user is relaxed, the visualization unit can also provide a visualization with bright colors. If the user is excited, the visualization unit can also provide a visualization with vivid colors. By adjusting the colors of the visualization according to the user's emotions, a more appropriate visualization can be provided.

[0107] The summary section can estimate the user's emotions and adjust the summary format based on those emotions. For example, if the user is stressed, the summary section may provide a bulleted list summary. If the user is relaxed, it may provide a paragraph-style summary. If the user is excited, it may provide a visually appealing summary. This allows for a more appropriate summary to be provided by adjusting the summary format according to the user's emotions.

[0108] The analysis unit can estimate the user's emotions and adjust the timing of the analysis based on those emotions. For example, if the user is in a hurry, the analysis unit can provide the results quickly. If the user is relaxed, the analysis unit can also provide detailed results. If the user is excited, the analysis unit can also provide visually appealing results. By adjusting the timing of the analysis according to the user's emotions, more appropriate results can be provided.

[0109] The following briefly describes the processing flow for example form 2.

[0110] Step 1: The summarization unit summarizes the main points of the meeting materials. For example, it analyzes the meeting materials using generation AI and extracts the key points. The summarization unit also analyzes slide materials created with presentation applications to clarify the purpose and agenda of the meeting. Step 2: The analysis unit analyzes the statements made during the meeting based on the main points summarized by the summarization unit. For example, it uses a generation AI to analyze the content of the statements and compares them to the main points of the meeting. Step 3: The quantification unit quantifies the degree to which the utterances analyzed by the analysis unit match the agenda. For example, it uses a generation AI to quantify the degree of match of the utterances and quantifies how relevant the utterances are to the agenda. Step 4: The visualization unit visualizes the results obtained by the quantification unit. For example, it uses a generation AI to display the degree of agreement in a graph or chart, allowing the progress of the meeting to be seen at a glance.

[0111] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0112] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0113] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0114] Each of the multiple elements described above, including the summarization unit, analysis unit, quantification unit, and visualization unit, is implemented in at least one of the smart device 14 and the data processing device 12. For example, the summarization unit is implemented by the control unit 46A of the smart device 14, which analyzes meeting materials using generation AI and extracts key points. The analysis unit is implemented by the identification processing unit 290 of the data processing device 12, which analyzes the content of statements and compares it to the purpose of the meeting. The quantification unit is implemented by the identification processing unit 290 of the data processing device 12, which quantifies the degree of agreement of the content of statements. The visualization unit is implemented by the control unit 46A of the smart device 14, which displays the degree of agreement in a graph or chart. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0115] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0116] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0117] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0118] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0119] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0120] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0121] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0122] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0123] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0124] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0125] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0126] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0127] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0128] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0129] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0130] Each of the multiple elements described above, including the summarization unit, analysis unit, quantification unit, and visualization unit, is implemented in at least one of the smart glasses 214 and the data processing device 12. For example, the summarization unit is implemented by the control unit 46A of the smart glasses 214, which analyzes meeting materials using generation AI and extracts key points. The analysis unit is implemented by the identification processing unit 290 of the data processing device 12, which analyzes the content of statements and compares it to the purpose of the meeting. The quantification unit is implemented by the identification processing unit 290 of the data processing device 12, which quantifies the degree of agreement of the content of statements. The visualization unit is implemented by the control unit 46A of the smart glasses 214, which displays the degree of agreement in a graph or chart. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0131] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0132] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0133] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0134] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0135] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0136] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0137] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0138] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0139] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0140] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0141] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0142] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0143] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0144] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0145] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0146] Each of the multiple elements described above, including the summarization unit, analysis unit, quantification unit, and visualization unit, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the summarization unit is implemented by the control unit 46A of the headset terminal 314, which analyzes meeting materials using generation AI and extracts key points. The analysis unit is implemented by the identification processing unit 290 of the data processing unit 12, which analyzes the content of statements and compares it to the purpose of the meeting. The quantification unit is implemented by the identification processing unit 290 of the data processing unit 12, which quantifies the degree of agreement of the content of statements. The visualization unit is implemented by the control unit 46A of the headset terminal 314, which displays the degree of agreement in a graph or chart. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0147] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0148] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0149] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0150] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0151] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0152] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0153] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0154] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0155] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0156] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0157] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0158] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0159] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0160] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0161] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0162] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0163] Each of the multiple elements described above, including the summarization unit, analysis unit, quantification unit, and visualization unit, is implemented by, for example, at least one of the robot 414 and the data processing unit 12. For example, the summarization unit is implemented by the control unit 46A of the robot 414, which analyzes meeting materials using generating AI and extracts key points. The analysis unit is implemented by, for example, the identification processing unit 290 of the data processing unit 12, which analyzes the content of statements and compares it to the purpose of the meeting. The quantification unit is implemented by, for example, the identification processing unit 290 of the data processing unit 12, which quantifies the degree of agreement of the content of statements. The visualization unit is implemented by, for example, the control unit 46A of the robot 414, which displays the degree of agreement in a graph or chart. The correspondence between each unit and the device or control unit is not limited to the example described above, and various changes are possible.

[0164] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0165] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0166] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0167] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0168] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0169] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0170] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0171] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0172] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0173] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0174] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0175] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0176] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0177] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0178] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0179] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0180] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0181] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0182] (Note 1) The meeting materials consist of a summary section that summarizes the main points, An analysis unit analyzes the statements made during the meeting based on the gist summarized by the aforementioned summarization unit, A quantification unit quantifies the degree to which the content of the statements analyzed by the aforementioned analysis unit matches the topic, The system includes a visualization unit that visualizes the results obtained by the quantification unit. A system characterized by the following features. (Note 2) The summary section above is, Analyze the slide presentation and extract the key points of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 3) The aforementioned analysis unit, The content of the statements will be compared against the purpose of the meeting, and the degree of agreement will be quantified. The system described in Appendix 1, characterized by the features described herein. (Note 4) The quantification unit is, The degree to which a statement is relevant to the agenda is quantified, and it is determined whether or not it deviates from the agenda. The system described in Appendix 1, characterized by the features described herein. (Note 5) The visualization unit, The degree of agreement with the agenda is displayed using graphs and charts, allowing for a quick overview of the meeting's progress. The system described in Appendix 1, characterized by the features described herein. (Note 6) The visualization unit, For companies using online meeting tools, this allows them to monitor the progress of meetings in real time. The system described in Appendix 1, characterized by the features described herein. (Note 7) The summary section above is, It estimates the user's emotions and adjusts the way the summary is presented based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 8) The summary section above is, When generating a summary, adjust the level of detail in the summary based on the importance of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 9) The summary section above is, When generating summaries, different summarization algorithms are applied depending on the meeting category. The system described in Appendix 1, characterized by the features described herein. (Note 10) The summary section above is, It estimates the user's sentiment and adjusts the length of the summary based on the estimated user sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 11) The summary section above is, When generating summaries, prioritize the summaries based on when the meetings took place. The system described in Appendix 1, characterized by the features described herein. (Note 12) The summary section above is, When generating summaries, the order of the summaries is adjusted based on the relevance of the meetings. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned analysis unit, We estimate the user's emotions and adjust the analysis criteria based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned analysis unit, During analysis, the interrelationships between statements are taken into consideration to improve the accuracy of the analysis. The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned analysis unit, During analysis, the speaker's attribute information is taken into consideration. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned analysis unit, It estimates the user's emotions and adjusts the order in which the analysis results are displayed based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned analysis unit, During the analysis, the geographical distribution of the statements will be taken into consideration. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned analysis unit, During analysis, we refer to relevant literature related to the statements to improve the accuracy of the analysis. The system described in Appendix 1, characterized by the features described herein. (Note 19) The quantification unit is, We estimate the user's emotions and adjust the quantification criteria based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 20) The quantification unit is, When quantifying, adjust the level of detail based on the importance of the statement. The system described in Appendix 1, characterized by the features described herein. (Note 21) The quantification unit is, When quantifying, different quantification algorithms are applied depending on the category of the statement. The system described in Appendix 1, characterized by the features described herein. (Note 22) The quantification unit is, It estimates the user's emotions and adjusts the order in which the quantification results are displayed based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 23) The quantification unit is, When quantifying, the geographical distribution of the statements is taken into consideration. The system described in Appendix 1, characterized by the features described herein. (Note 24) The quantification unit is, When quantifying, refer to relevant literature related to the statements to improve the accuracy of the quantification. The system described in Appendix 1, characterized by the features described herein. (Note 25) The visualization unit, It estimates the user's emotions and adjusts the visualization method based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 26) The visualization unit, When creating visualizations, adjust the level of detail based on the importance of the meeting. The system described in Appendix 1, characterized by the features described herein. (Note 27) The visualization unit, When creating visualizations, different visualization algorithms are applied depending on the meeting category. The system described in Appendix 1, characterized by the features described herein. (Note 28) The visualization unit, It estimates the user's emotions and adjusts the length of the visualization based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 29) The visualization unit, When creating visualizations, prioritize visualizations based on when the meetings took place. The system described in Appendix 1, characterized by the features described herein. (Note 30) The visualization unit, When creating visualizations, adjust the order of visualizations based on the relevance of the meetings. The system described in Appendix 1, characterized by the features described herein. [Explanation of Symbols]

[0183] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. The meeting materials consist of a summary section that summarizes the main points, An analysis unit analyzes the statements made during the meeting based on the gist summarized by the aforementioned summarization unit, A quantification unit quantifies the degree to which the content of the statements analyzed by the aforementioned analysis unit matches the topic, The system includes a visualization unit that visualizes the results obtained by the quantification unit. A system characterized by the following features.

2. The summary section above is, Analyze the slide presentation and extract the key points of the meeting. The system according to feature 1.

3. The aforementioned analysis unit, The content of the statements will be compared against the purpose of the meeting, and the degree of agreement will be quantified. The system according to feature 1.

4. The quantification unit is, The degree to which a statement is relevant to the agenda is quantified, and it is determined whether or not it deviates from the agenda. The system according to feature 1.

5. The visualization unit, The degree of agreement with the agenda is displayed using graphs and charts, allowing for a quick overview of the meeting's progress. The system according to feature 1.

6. The visualization unit, For companies using online meeting tools, this allows them to monitor the progress of meetings in real time. The system according to feature 1.

7. The summary section above is, It estimates the user's emotions and adjusts the way the summary is presented based on those estimated emotions. The system according to feature 1.

8. The summary section above is, When generating a summary, adjust the level of detail in the summary based on the importance of the meeting. The system according to feature 1.

9. The summary section above is, When generating summaries, different summarization algorithms are applied depending on the meeting category. The system according to feature 1.

Images