System

Abstract

To provide a system which automatically generates minutes on the basis of voice data of a meeting and sends the minutes to participants.SOLUTION: The system according to an embodiment includes a collection unit, an analysis unit, a generation unit, and a sending unit. The collection unit collects voice data of a meeting. The analysis unit analyzes the voice data collected by the collection unit. The generation unit generates minutes on the basis of the data analyzed by the analysis unit. The sending unit sends the minutes generated by the generation unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, the process of creating meeting minutes based on audio data and automatically sending them to participants is not sufficiently automated, leaving room for improvement.

[0005] The system according to the embodiment aims to automatically generate minutes of a meeting based on audio data and transmit them to the participants. [Means for solving the problem]

[0006] The system according to the embodiment includes a collection unit, an analysis unit, a generation unit, and a transmission unit. The collection unit collects audio data of a conference. The analysis unit analyzes the audio data collected by the collection unit. The generation unit generates minutes of a meeting based on the data analyzed by the analysis unit. The transmission unit transmits the minutes of a meeting generated by the generation unit. Effect of the Invention

[0007] The system according to the embodiment can automatically generate minutes of a meeting based on audio data and transmit them to the participants. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The automatic minutes creation system according to the embodiment of the present invention is a system that automatically creates minutes of meetings held daily and automatically sends them to participating members. This automatic minutes creation system is a mechanism that collects audio data of a meeting, analyzes it with a generation AI, automatically generates minutes, and automatically sends them to participating members. For example, when collecting audio data of a meeting, collection of audio data begins at the same time as the start of the meeting. Audio data can be collected using a microphone installed in a conference room or a recording function of an online conference system. Next, the generation AI analyzes the collected audio data. The generation AI converts the audio data into text data and automatically summarizes the contents of the meeting as minutes. For example, the minutes are created by organizing the remarks of each speaker, the main points of the discussion, and the decisions made. The generated minutes are automatically sent to participating members. For example, it can be set to be automatically sent by email after the meeting ends. This makes it easier for participating members to check the contents of the meeting, and prevents omissions and oversights. This mechanism makes it more efficient to create minutes of a meeting, and all participating members can share accurate information. For example, even if a person receives a phone call during a meeting and has to temporarily leave their seat, they can later check the minutes and understand the contents of the meeting. In addition, the automatic transmission of minutes ensures that all participants have the same information, making information sharing smoother. This allows the automatic minutes creation system to automatically create meeting minutes and quickly provide participants with accurate information.

[0029] The automatic minutes creation system according to the embodiment includes a collection unit, an analysis unit, a generation unit, and a transmission unit. The collection unit collects audio data of the conference. The audio data of the conference includes, for example, audio data collected using a microphone installed in a conference room or a recording function of an online conference system, but is not limited to such examples. The collection unit starts collecting audio data, for example, at the same time as the conference starts. The collection unit can also continuously collect audio data while the conference is in progress. Furthermore, the collection unit can stop collecting audio data at the same time as the conference ends. The analysis unit uses a generation AI to analyze the audio data collected by the collection unit. The analysis unit, for example, converts the audio data into text data and automatically summarizes the contents of the conference as minutes. The generation AI analyzes the audio data using a text generation AI (for example, LLM) and creates minutes by organizing the remarks of each speaker, the main points of the discussion, and decisions. The generation unit generates minutes based on the data analyzed by the analysis unit. The generation unit generates minutes based on text data generated from the audio data, for example, using a generation AI. The generation AI understands the contents of the voice data, extracts important information, and creates minutes. The transmission unit transmits the minutes generated by the generation unit. The transmission unit, for example, automatically transmits the generated minutes by email. The transmission unit can also upload the generated minutes to cloud storage. Furthermore, the transmission unit can also transmit the generated minutes by a chat tool. As a result, the automatic minutes creation system according to the embodiment can accurately record the contents of the meeting and share them with participating members by collecting and analyzing the voice data of the meeting, generating minutes, and transmitting them.

[0030] The collection unit collects audio data of the conference. The audio data of the conference includes, for example, a microphone installed in a conference room and audio data collected using a recording function of an online conference system, but is not limited to such examples. Specifically, a high-sensitivity microphone installed in a conference room can clearly collect the remarks of multiple participants. As a result, all remarks are accurately recorded regardless of the volume or position of the speaker's voice. In addition, when the recording function of an online conference system is used, audio data of a conference held via the Internet can be collected with high quality. For example, the collection unit starts collecting audio data at the same time as the start of the conference. As a result, all remarks are recorded from the beginning of the conference, and important information is not leaked. Furthermore, the collection unit can also continuously collect audio data while the conference is in progress. As a result, all remarks are consistently recorded without interruption in the middle of the conference. The collection unit can also stop collecting audio data at the same time as the end of the conference. As a result, unnecessary audio data is prevented from being collected after the end of the conference, making it easier to manage the data. The collection unit can transmit the collected audio data to the analysis unit in real time. As a result, the analysis unit can instantly analyze the collected audio data and quickly create minutes. In addition, the collection unit can use noise canceling technology to improve the quality of the audio data, eliminating background noise and other distracting sounds, allowing the spoken content to be recorded more clearly.

[0031] The analysis unit uses the generation AI to analyze the voice data collected by the collection unit. For example, the analysis unit converts the voice data into text data and automatically compiles the contents of the meeting as minutes. Specifically, the generation AI analyzes the voice data using voice recognition technology and converts the contents of each speaker's remarks into text data. The generation AI can identify the voice of the speaker from the voice data and record each speaker's remarks individually. This makes it possible to accurately grasp who said what. Furthermore, the generation AI uses a text generation AI (for example, LLM) to analyze the remarks and organize the main points of the discussion and decisions. The generation AI can use natural language processing technology to understand the meaning of the remarks and extract important information. For example, it can detect important keywords such as "decisions" and "action items" from the remarks and reflect them in the minutes. The analysis unit can analyze the collected voice data in real time and update the minutes as the meeting progresses. This allows the minutes to be completed immediately after the end of the meeting and shared with participants. Furthermore, the analysis unit can refer to past meeting data and automatically associate similar discussions and decisions. This allows for a deeper understanding of the content of the meeting and a grasp of its relevance to past discussions. The analysis unit sends the results of the analysis of the audio data to the generation unit to assist in the generation of minutes.

[0032] The generation unit generates minutes based on the data analyzed by the analysis unit. The generation unit generates minutes based on text data generated from voice data, for example, using a generation AI. The generation AI understands the contents of the voice data and extracts important information to create minutes. Specifically, the generation AI organizes the remarks of each speaker and clearly describes the main points of the discussion and the decisions. The generation unit automatically determines the structure of the minutes based on the information extracted by the generation AI. For example, the minutes begin with the date and time of the meeting and information about the participants, followed by a bullet list of the main points of the discussion and the decisions. The generation unit can flexibly change the format of the minutes. For example, the format of the minutes can be customized according to the type and purpose of the meeting, and the necessary information can be appropriately described. The generation unit can automatically save the generated minutes and edit them as necessary. For example, if there is an error in the generated minutes, it can be manually corrected. The generation unit transmits the generated minutes to the transmission unit and prepares to share them with the participants. In addition, the generation department can compare the generated minutes with past minutes to grasp the progress and unresolved issues, which allows for consistent management of meeting content and continuous improvement.

[0033] The transmission unit transmits the minutes generated by the generation unit. For example, the transmission unit automatically transmits the generated minutes by email. Specifically, the transmission unit can transmit the generated minutes as an attachment to all participants of the meeting by email. This allows the participants to quickly check the contents of the meeting and take necessary actions. The transmission unit can also upload the generated minutes to cloud storage. This allows the participants to access the minutes anytime and anywhere and refer to the contents of past meetings. Furthermore, the transmission unit can transmit the generated minutes by a chat tool. For example, a link to the minutes can be shared in a team chat group so that participants can easily access it. The transmission unit can automatically manage a list of destinations and transmit the minutes appropriately to participants and related parties of the meeting. This allows important information to be shared with all related parties without leaking out. Furthermore, the transmission unit can record the transmission history and check the transmission status of past minutes. This prevents the minutes from being sent out or sent in duplicate, realizing efficient information sharing. The transmission unit can collect feedback on the transmitted minutes and provide the feedback to the generation unit. This allows us to continually improve the quality of our minutes and provide more accurate and useful information.

[0034] The analysis unit may include an identification unit that identifies the speaker. The identification unit uses, for example, an algorithm that identifies the speaker from the voice data. For example, the identification unit uses a voice recognition technology to analyze the characteristics of the speaker's voice and identify the speaker. The identification unit can also learn the speaker's voice pattern and identify the speaker. Furthermore, the identification unit can organize the remarks of each speaker based on the characteristics of the speaker's voice. In this way, by identifying the speaker, the remarks of each speaker can be accurately recorded in the minutes. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI, or may be performed without using AI. For example, the identification unit can input the voice data to the generation AI and cause the generation AI to identify the speaker.

[0035] The analysis unit may include an extraction unit that extracts the main points of the discussion. The extraction unit uses, for example, an algorithm that extracts the main points of the discussion from the voice data. For example, the extraction unit analyzes the voice data using natural language processing technology and extracts important information. The extraction unit can also understand the content of the voice data and extract the main points of the discussion. Furthermore, the extraction unit can also create minutes based on the main points of the discussion. In this way, by extracting the main points of the discussion, important information can be efficiently recorded in the minutes. Some or all of the above-mentioned processing in the extraction unit may be performed, for example, using AI, or may be performed without using AI. For example, the extraction unit may input the voice data to a generation AI and cause the generation AI to extract the main points of the discussion.

[0036] The transmission unit may include an upload unit that uploads the minutes to the cloud storage. The upload unit uploads the generated minutes to the cloud storage, for example. For example, the upload unit uploads the minutes using the API of the cloud storage. The upload unit can also automatically upload the minutes to the cloud storage. Furthermore, the upload unit can monitor the upload status of the minutes and confirm that the upload is complete. This allows the participating members to access the minutes at any time by uploading the minutes to the cloud storage. A part or all of the above-mentioned processing in the upload unit may be performed, for example, using AI, or may be performed without using AI. For example, the upload unit can input the generated minutes to a generation AI and cause the generation AI to execute uploading to the cloud storage.

[0037] The collection unit can collect voice data by using a recording app on the smartphone. The collection unit collects voice data by using, for example, a recording app on the smartphone. For example, the collection unit starts the recording app on the smartphone and collects voice data of the meeting. The collection unit can also collect voice data of the meeting in real time by using the recording app on the smartphone. Furthermore, the collection unit can also upload the collected voice data to cloud storage by using the recording app on the smartphone. This makes it possible to easily collect voice data by using the recording app on the smartphone. A part or all of the above-mentioned processing in the collection unit may be performed by using, for example, AI, or may be performed without using AI. For example, the collection unit can input the voice data collected by the recording app on the smartphone to the generation AI and cause the generation AI to analyze the voice data.

[0038] The transmission unit can transmit the minutes by a chat tool. The transmission unit transmits, for example, the generated minutes by a chat tool. For example, the transmission unit transmits the minutes by using an API of the chat tool. The transmission unit can also automatically transmit the minutes by the chat tool. Furthermore, the transmission unit can monitor the transmission status of the minutes and confirm that the transmission has been completed. In this way, by transmitting the minutes by the chat tool, the participating members can quickly check the minutes. A part or all of the above-mentioned processing in the transmission unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the transmission unit can input the generated minutes into a generation AI and cause the generation AI to transmit the minutes by the chat tool.

[0039] The collection unit can dynamically adjust the quality of the voice data to be collected according to the progress of the conference. The collection unit dynamically adjusts the quality of the voice data to be collected according to, for example, the progress of the conference. For example, the collection unit collects low-quality voice data at the beginning of the conference, and collects high-quality voice data when an important topic is discussed. The collection unit can also collect standard-quality voice data when the conference is progressing smoothly. Furthermore, when the conference is in chaos, the collection unit can also strengthen noise canceling to collect high-quality voice data. This allows important parts to be collected with high quality by adjusting the quality of the voice data according to the progress of the conference. Some or all of the above-mentioned processing in the collection unit may be performed, for example, using AI, or may be performed without using AI. For example, the collection unit can input conference progress data to the generation AI and cause the generation AI to adjust the quality of the voice data.

[0040] The collection unit can limit the collection range based on the importance of the conference when collecting voice data. For example, the collection unit limits the collection range based on the importance of the conference when collecting voice data. For example, the collection unit collects all remarks in an important conference, and collects only major remarks in a normal conference. The collection unit can also collect detailed voice data for important topics, and collect only outlines for other topics. Furthermore, the collection unit can also collect remarks of important participants preferentially, and collect remarks of other participants as necessary. This allows important information to be collected efficiently by limiting the collection range based on the importance of the conference. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using AI, or may be performed without using AI. For example, the collection unit can input importance data of the conference to the generation AI, and cause the generation AI to limit the collection range.

[0041] The collection unit can select an appropriate collection method based on the location information of the participants in the conference when collecting voice data. For example, the collection unit selects an appropriate collection method based on the location information of the participants in the conference room when collecting voice data. For example, the collection unit optimizes the placement of microphones according to the positions of the participants in the conference room. In addition, in the case of an online conference, the collection unit can also adjust the collection method taking into account the quality of the voice input device of each participant. Furthermore, when the participants in the conference are moving, the collection unit can also update the location information in real time and select the optimal collection method. As a result, optimal voice data can be collected by selecting a collection method based on the location information of the participants in the conference. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using AI or may be performed without using AI. For example, the collection unit can input the location information data of the participants to the generation AI and cause the generation AI to select the collection method.

[0042] The collection unit can customize the collection method based on the theme of the meeting when collecting the voice data. For example, the collection unit customizes the collection method based on the theme of the meeting when collecting the voice data. For example, the collection unit selects a method for accurately collecting technical terms in a technical meeting. In addition, the collection unit can also select a method for preferentially collecting important decisions in a business meeting. Furthermore, the collection unit can also select a method for collecting lecture contents in detail in an educational meeting. In this way, appropriate voice data can be collected by customizing the collection method based on the theme of the meeting. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using AI or may be performed without using AI. For example, the collection unit can input the theme data of the meeting to the generation AI and cause the generation AI to execute customization of the collection method.

[0043] The analysis unit can dynamically change the analysis algorithm according to the progress of the conference when analyzing the voice data. For example, the analysis unit dynamically changes the analysis algorithm according to the progress of the conference when analyzing the voice data. For example, the analysis unit uses a simple analysis algorithm at the beginning of the conference, and switches to a detailed analysis algorithm when an important topic is discussed. The analysis unit can also use a standard analysis algorithm when the conference is progressing smoothly. Furthermore, the analysis unit can also use an analysis algorithm with enhanced noise canceling when the conference is in chaos. This allows important parts to be analyzed in detail by changing the analysis algorithm according to the progress of the conference. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input conference progress data to the generation AI and cause the generation AI to change the analysis algorithm.

[0044] The analysis unit can adjust the level of detail of the analysis based on the importance of the meeting when analyzing the voice data. For example, the analysis unit adjusts the level of detail of the analysis based on the importance of the meeting when analyzing the voice data. For example, the analysis unit performs a detailed analysis on an important meeting, and analyzes only an outline of a normal meeting. The analysis unit can also perform a detailed analysis on important topics, and perform a simple analysis on other topics. Furthermore, the analysis unit can also analyze the remarks of important participants preferentially, and analyze the remarks of other participants as necessary. As a result, important information can be analyzed efficiently by adjusting the level of detail of the analysis based on the importance of the meeting. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the importance data of the meeting to the generation AI, and cause the generation AI to adjust the level of detail of the analysis.

[0045] The analysis unit can perform analysis based on attribute information of the participants of the conference when analyzing the voice data. For example, the analysis unit performs analysis based on attribute information of the participants of the conference when analyzing the voice data. For example, the analysis unit adjusts the analysis accuracy of technical terms according to the expertise of the participants. The analysis unit can also adjust the analysis accuracy of important remarks according to the position of the participants. Furthermore, the analysis unit can adjust the priority of the analysis according to the frequency of remarks of the participants. In this way, an appropriate analysis result can be obtained by performing analysis based on the attribute information of the participants of the conference. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input attribute information data of the participants to the generation AI and cause the generation AI to execute the analysis.

[0046] The analysis unit can customize the analysis method based on the theme of the meeting when analyzing the voice data. For example, the analysis unit customizes the analysis method based on the theme of the meeting when analyzing the voice data. For example, the analysis unit selects a method for accurately analyzing technical terms in a technical meeting. Also, the analysis unit can select a method for preferentially analyzing important decisions in a business meeting. Furthermore, the analysis unit can select a method for analyzing lecture contents in detail in an educational meeting. In this way, by customizing the analysis method based on the theme of the meeting, appropriate analysis results can be obtained. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the theme data of the meeting to the generation AI and cause the generation AI to execute customization of the analysis method.

[0047] The generating unit can dynamically change the generating algorithm according to the progress of the meeting when generating the minutes. For example, the generating unit dynamically changes the generating algorithm according to the progress of the meeting when generating the minutes. For example, the generating unit uses a simple generating algorithm at the beginning of the meeting, and switches to a detailed generating algorithm when the important agenda is reached. The generating unit can also use a standard generating algorithm when the meeting is progressing smoothly. Furthermore, the generating unit can also use a generating algorithm with enhanced noise canceling when the meeting is in chaos. This allows important parts to be recorded in detail by changing the generating algorithm according to the progress of the meeting. Part or all of the above-mentioned processing in the generating unit may be performed, for example, using AI, or may be performed without using AI. For example, the generating unit can input progress data of the meeting to the generating AI and cause the generating AI to change the generating algorithm.

[0048] The generation unit can adjust the level of detail of the generation based on the importance of the meeting when generating the minutes. For example, the generation unit adjusts the level of detail of the generation based on the importance of the meeting when generating the minutes. For example, the generation unit generates detailed minutes for important meetings and generates only an outline for normal meetings. The generation unit can also generate detailed minutes for important topics and generate simple minutes for other topics. Furthermore, the generation unit can also reflect the remarks of important participants in the minutes with priority, and reflect the remarks of other participants as necessary. As a result, important information can be efficiently recorded by adjusting the level of detail of the generation based on the importance of the meeting. Part or all of the above-mentioned processing in the generation unit may be performed, for example, using AI, or may be performed without using AI. For example, the generation unit can input the importance data of the meeting to the generation AI and cause the generation AI to adjust the level of detail of the generation.

[0049] The generating unit can generate minutes based on attribute information of the participants of the meeting when generating the minutes. For example, the generating unit generates minutes based on attribute information of the participants of the meeting when generating the minutes. For example, the generating unit generates minutes that accurately reflect technical terms according to the expertise of the participants. The generating unit can also generate minutes that preferentially reflect important remarks according to the positions of the participants. Furthermore, the generating unit can adjust the priority of the minutes according to the frequency of remarks of the participants. As a result, appropriate minutes can be generated by generating the minutes based on the attribute information of the participants of the meeting. A part or all of the above-mentioned processing in the generating unit may be performed using, for example, AI, or may be performed without using AI. For example, the generating unit can input attribute information data of the participants to the generating AI and cause the generating AI to execute the generation.

[0050] The generation unit can customize the generation method based on the theme of the meeting when generating the minutes. For example, the generation unit customizes the generation method based on the theme of the meeting when generating the minutes. For example, the generation unit generates minutes that accurately reflect technical terms in a technical meeting. Also, the generation unit can generate minutes that preferentially reflect important decisions in a business meeting. Furthermore, the generation unit can generate minutes that reflect the lecture content in detail in an educational meeting. In this way, appropriate minutes can be generated by customizing the generation method based on the theme of the meeting. Part or all of the above-mentioned processing in the generation unit may be performed, for example, using AI, or may be performed without using AI. For example, the generation unit can input the theme data of the meeting to the generation AI and cause the generation AI to execute customization of the generation method.

[0051] The transmission unit can dynamically change the transmission method according to the progress of the meeting when transmitting the minutes. For example, the transmission unit dynamically changes the transmission method according to the progress of the meeting when transmitting the minutes. For example, if the meeting is proceeding smoothly, the transmission unit transmits the minutes by normal mail. Also, if the meeting is chaotic, the transmission unit can transmit the minutes in a format that emphasizes important parts. Furthermore, if the meeting lasts for a long time, the transmission unit can transmit the minutes in a divided form. In this way, by changing the transmission method according to the progress of the meeting, important parts can be emphasized and transmitted. A part or all of the above-mentioned processing in the transmission unit may be performed, for example, using AI, or may be performed without using AI. For example, the transmission unit can input the progress data of the meeting to the generation AI and cause the generation AI to execute the change of the transmission method.

[0052] The transmission unit can adjust the level of detail of the transmission based on the importance of the meeting when transmitting the minutes. For example, the transmission unit adjusts the level of detail of the transmission based on the importance of the meeting when transmitting the minutes. For example, the transmission unit transmits detailed minutes for important meetings and transmits only summaries for normal meetings. The transmission unit can also transmit detailed minutes for important topics and transmit simple minutes for other topics. Furthermore, the transmission unit can also preferentially reflect the remarks of important participants in the minutes and reflect the remarks of other participants as necessary. This allows important information to be transmitted efficiently by adjusting the level of detail of the transmission based on the importance of the meeting. Part or all of the above-mentioned processing in the transmission unit may be performed, for example, using AI or may be performed without using AI. For example, the transmission unit can input importance data of the meeting to the generation AI and cause the generation AI to adjust the level of detail of the transmission.

[0053] The transmission unit can transmit the minutes based on the attribute information of the participants of the conference when transmitting the minutes. For example, the transmission unit transmits the minutes based on the attribute information of the participants of the conference when transmitting the minutes. For example, the transmission unit transmits minutes in which important remarks are emphasized according to the positions of the participants. The transmission unit can also transmit minutes that accurately reflect technical terms according to the expertise of the participants. Furthermore, the transmission unit can adjust the priority of the minutes and transmit them according to the frequency of remarks of the participants. As a result, appropriate minutes can be transmitted by transmitting based on the attribute information of the participants of the conference. A part or all of the above-mentioned processing in the transmission unit may be performed using, for example, AI, or may be performed without using AI. For example, the transmission unit can input the attribute information data of the participants to the generation AI and cause the generation AI to execute the transmission.

[0054] The transmission unit can customize the transmission method based on the theme of the meeting when transmitting the minutes. For example, the transmission unit customizes the transmission method based on the theme of the meeting when transmitting the minutes. For example, in a technical meeting, the transmission unit transmits minutes that accurately reflect technical terms. In addition, in a business meeting, the transmission unit can also transmit minutes that emphasize important decisions. Furthermore, in an educational meeting, the transmission unit can transmit minutes that reflect the lecture content in detail. In this way, by customizing the transmission method based on the theme of the meeting, appropriate minutes can be transmitted. Part or all of the above-mentioned processing in the transmission unit may be performed, for example, using AI, or may be performed without using AI. For example, the transmission unit can input the theme data of the meeting to the generation AI and cause the generation AI to customize the transmission method.

[0055] The identification unit can dynamically change the identification algorithm according to the progress of the conference when identifying a speaker. For example, the identification unit dynamically changes the identification algorithm according to the progress of the conference when identifying a speaker. For example, the identification unit uses a simple identification algorithm at the beginning of the conference, and switches to a detailed identification algorithm when an important topic is discussed. The identification unit can also use a standard identification algorithm when the conference is progressing smoothly. Furthermore, the identification unit can also use an identification algorithm with enhanced noise canceling when the conference is chaotic. This allows important remarks to be accurately identified by changing the identification algorithm according to the progress of the conference. Part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI, or may be performed without using AI. For example, the identification unit can input conference progress data to the generation AI and cause the generation AI to change the identification algorithm.

[0056] The identification unit can perform identification based on the attribute information of the conference participants when identifying the speaker. For example, the identification unit performs identification based on the attribute information of the conference participants when identifying the speaker. For example, the identification unit adjusts the identification accuracy of technical terms according to the expertise of the participants. In addition, the identification unit can also adjust the identification accuracy of important remarks according to the position of the participants. Furthermore, the identification unit can adjust the priority of identification according to the frequency of remarks of the participants. In this way, by performing identification based on the attribute information of the conference participants, an appropriate identification result can be obtained. Part or all of the above-mentioned processing in the identification unit may be performed using, for example, AI, or may be performed without using AI. For example, the identification unit can input the attribute information data of the participants to the generation AI and cause the generation AI to perform the identification.

[0057] The extraction unit can dynamically change the extraction algorithm according to the progress of the meeting when extracting the main points of the discussion. For example, the extraction unit dynamically changes the extraction algorithm according to the progress of the meeting when extracting the main points of the discussion. For example, the extraction unit uses a simple extraction algorithm at the beginning of the meeting, and switches to a detailed extraction algorithm when the important topics are discussed. The extraction unit can also use a standard extraction algorithm when the meeting is progressing smoothly. Furthermore, the extraction unit can also use an extraction algorithm with enhanced noise canceling when the meeting is chaotic. This allows important parts to be extracted in detail by changing the extraction algorithm according to the progress of the meeting. Part or all of the above-mentioned processing in the extraction unit may be performed, for example, using AI, or may be performed without using AI. For example, the extraction unit can input the progress data of the meeting to the generation AI and cause the generation AI to execute the change of the extraction algorithm.

[0058] The extraction unit can perform extraction based on the attribute information of the participants of the meeting when extracting the main points of the discussion. For example, the extraction unit performs extraction based on the attribute information of the participants of the meeting when extracting the main points of the discussion. For example, the extraction unit adjusts the extraction accuracy of technical terms according to the expertise of the participants. The extraction unit can also adjust the extraction accuracy of important remarks according to the position of the participants. Furthermore, the extraction unit can adjust the priority of extraction according to the frequency of remarks of the participants. As a result, appropriate extraction results can be obtained by performing extraction based on the attribute information of the participants of the meeting. Part or all of the above-mentioned processing in the extraction unit may be performed using, for example, AI, or may be performed without using AI. For example, the extraction unit can input the attribute information data of the participants to the generation AI and cause the generation AI to execute the extraction.

[0059] The upload unit can dynamically change the upload method according to the progress of the meeting when uploading the minutes. The upload unit dynamically changes the upload method according to the progress of the meeting when uploading the minutes, for example. For example, if the meeting is proceeding smoothly, the upload unit uploads the minutes in a normal manner. Also, if the meeting is chaotic, the upload unit can upload the minutes in a format that emphasizes important parts. Furthermore, if the meeting lasts for a long time, the upload unit can upload the minutes in parts. In this way, by changing the upload method according to the progress of the meeting, it is possible to upload the minutes with emphasis on important parts. A part or all of the above-mentioned processing in the upload unit may be performed, for example, using AI, or may be performed without using AI. For example, the upload unit can input the progress data of the meeting to the generation AI and cause the generation AI to execute the change of the upload method.

[0060] The uploading unit can upload the minutes based on the attribute information of the participants of the meeting when uploading the minutes. For example, the uploading unit uploads the minutes based on the attribute information of the participants of the meeting when uploading the minutes. For example, the uploading unit uploads minutes that emphasize important remarks according to the positions of the participants. The uploading unit can also upload minutes that accurately reflect technical terms according to the expertise of the participants. Furthermore, the uploading unit can adjust the priority of the minutes according to the frequency of remarks of the participants and upload them. In this way, appropriate minutes can be uploaded by uploading based on the attribute information of the participants of the meeting. Part or all of the above-mentioned processing in the uploading unit may be performed using, for example, AI, or may be performed without using AI. For example, the uploading unit can input the attribute information data of the participants to the generation AI and cause the generation AI to execute the upload.

[0061] The recording app of the smartphone can dynamically change the recording settings according to the progress of the meeting when the recording app is used. The recording app of the smartphone dynamically changes the recording settings according to the progress of the meeting when the recording app is used. For example, the recording app of the smartphone uses a low-quality recording setting at the beginning of the meeting, and switches to a high-quality recording setting when an important topic is discussed. The recording app of the smartphone can also use a standard-quality recording setting when the meeting is progressing smoothly. Furthermore, the recording app of the smartphone can also use a recording setting with enhanced noise cancellation when the meeting is chaotic. This allows important parts to be recorded in high quality by changing the recording settings according to the progress of the meeting. Some or all of the above-mentioned processing in the recording app of the smartphone may be performed, for example, using AI or may be performed without using AI. For example, the recording app of the smartphone can input the progress data of the meeting to the generation AI and cause the generation AI to change the recording settings.

[0062] The recording app of the smartphone can set the recording based on the attribute information of the participants of the conference when the recording app is used. The recording app of the smartphone, for example, sets the recording based on the attribute information of the participants of the conference when the recording app is used. For example, the recording app of the smartphone sets the recording to accurately record technical terms according to the expertise of the participants. In addition, the recording app of the smartphone can set the recording to prioritize important remarks according to the position of the participants. Furthermore, the recording app of the smartphone can set the recording to adjust the recording priority according to the frequency of remarks of the participants. In this way, by setting the recording based on the attribute information of the participants of the conference, appropriate recording results can be obtained. Part or all of the above-mentioned processing in the recording app of the smartphone may be performed using, for example, AI, or may be performed without using AI. For example, the recording app of the smartphone can input the attribute information data of the participants to the generation AI and cause the generation AI to execute the recording settings.

[0063] The chat tool can dynamically change the transmission method according to the progress of the meeting when the chat tool is used. The chat tool dynamically changes the transmission method according to the progress of the meeting when the chat tool is used. For example, when the meeting is proceeding smoothly, the chat tool transmits the minutes by the chat tool in a normal manner. Also, when the meeting is chaotic, the chat tool can transmit the minutes by the chat tool in a format in which important parts are emphasized. Furthermore, when the meeting is long, the chat tool can transmit the minutes by the chat tool in parts. In this way, by changing the transmission method according to the progress of the meeting, important parts can be emphasized and transmitted. A part or all of the above-mentioned processing in the chat tool may be performed, for example, using AI, or may be performed without using AI. For example, the chat tool can input the progress data of the meeting to the generation AI and cause the generation AI to execute the change of the transmission method.

[0064] The chat tool can set transmission settings based on the attribute information of the participants of the meeting when the chat tool is used. The chat tool can set transmission settings based on the attribute information of the participants of the meeting when the chat tool is used. For example, the chat tool transmits minutes in which important remarks are emphasized according to the roles of the participants. The chat tool can also transmit minutes that accurately reflect technical terms according to the expertise of the participants. Furthermore, the chat tool can adjust the priority of the minutes according to the frequency of remarks of the participants and transmit them by the chat tool. This allows appropriate minutes to be transmitted by setting transmission settings based on the attribute information of the participants of the meeting. A part or all of the above-mentioned processing in the chat tool may be performed using, for example, AI, or may be performed without using AI. For example, the chat tool can input attribute information data of the participants to a generation AI and cause the generation AI to execute transmission settings.

[0065] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0066] The analysis unit may include a translation unit that translates the contents of the meeting in real time. The translation unit may, for example, translate the audio data of the meeting in real time and immediately provide the translation results to participants who speak different languages. The translation unit may also adjust the accuracy of the translation as the meeting progresses. Furthermore, the translation unit may also include a dictionary function for accurately translating technical terms and industry-specific terms. This makes it easier for participants who speak different languages ​​to understand the contents of the meeting, enabling smooth communication even in international meetings.

[0067] The analysis unit may include a summarization unit that summarizes the contents of the meeting. The summarization unit may, for example, analyze audio data of the meeting and extract important points and decisions to create a summary. The summarization unit may also adjust the level of detail of the summary according to the progress of the meeting. Furthermore, the summarization unit may customize the content of the summary according to the roles and expertise of the participants. This allows participants to quickly grasp the overall picture of the meeting, enabling efficient information sharing.

[0068] The analysis unit may include a visualization unit that visually displays the contents of the conference. The visualization unit may, for example, analyze audio data of the conference and display the flow of discussion and relationships in graphs and charts. The visualization unit may also dynamically update the display content according to the progress of the conference. Furthermore, the visualization unit may customize the display content according to the roles and expertise of the participants. This allows participants to intuitively understand the contents of the conference and enables effective communication through visual information.

[0069] The transmission unit can include an audio providing unit that provides the minutes in audio format. The audio providing unit, for example, converts the generated minutes into audio data and provides them to the participants in audio format. The audio providing unit can also adjust the speed and tone of the audio according to the preferences of the participants. Furthermore, the audio providing unit can also provide the minutes in audio format with important parts emphasized. This allows the participants to check the minutes while on the move or at work, improving convenience.

[0070] The collection unit may include a video collection unit that collects video data of the conference. The video collection unit, for example, collects video data of the conference and records the facial expressions and gestures of the speakers. The video collection unit may also adjust the range of video data collection according to the progress of the conference. Furthermore, the video collection unit may analyze the collected video data, extract important scenes, and reflect them in the minutes. This allows participants to more accurately grasp the atmosphere and nuances of the conference.

[0071] The process flow of the first embodiment will be briefly described below.

[0072] Step 1: The collection unit collects audio data of the meeting. The audio data of the meeting includes audio data collected using microphones installed in the meeting room and a recording function of the online meeting system. The collection unit can start collecting audio data simultaneously with the start of the meeting, collect the data continuously while the meeting is in progress, and stop collecting the data simultaneously with the end of the meeting. Step 2: The analysis unit uses the generation AI to analyze the voice data collected by the collection unit. The analysis unit converts the voice data into text data and automatically compiles the contents of the meeting as minutes. The generation AI analyzes the voice data using a text generation AI (e.g., LLM) and creates minutes by organizing the remarks made by each speaker, key points of the discussion, and decisions made. Step 3: The generation unit generates minutes based on the data analyzed by the analysis unit. The generation unit generates minutes based on text data generated from the voice data using the generation AI. The generation AI understands the content of the voice data, extracts important information, and creates minutes. Step 4: The sending unit sends the minutes generated by the generating unit. The sending unit can automatically send the generated minutes by email, upload them to cloud storage, or send them via a chat tool.

[0073] (Example 2) The automatic minutes creation system according to the embodiment of the present invention is a system that automatically creates minutes of meetings held daily and automatically sends them to participating members. This automatic minutes creation system is a mechanism that collects audio data of a meeting, analyzes it with a generation AI, automatically generates minutes, and automatically sends them to participating members. For example, when collecting audio data of a meeting, collection of audio data begins at the same time as the start of the meeting. Audio data can be collected using a microphone installed in a conference room or a recording function of an online conference system. Next, the generation AI analyzes the collected audio data. The generation AI converts the audio data into text data and automatically summarizes the contents of the meeting as minutes. For example, the minutes are created by organizing the remarks of each speaker, the main points of the discussion, and the decisions made. The generated minutes are automatically sent to participating members. For example, it can be set to be automatically sent by email after the meeting ends. This makes it easier for participating members to check the contents of the meeting, and prevents omissions and oversights. This mechanism makes it more efficient to create minutes of a meeting, and all participating members can share accurate information. For example, even if a person receives a phone call during a meeting and has to temporarily leave their seat, they can later check the minutes and understand the contents of the meeting. In addition, the automatic transmission of minutes ensures that all participants have the same information, making information sharing smoother. This allows the automatic minutes creation system to automatically create meeting minutes and quickly provide participants with accurate information.

[0074] The automatic minutes creation system according to the embodiment includes a collection unit, an analysis unit, a generation unit, and a transmission unit. The collection unit collects audio data of the conference. The audio data of the conference includes, for example, audio data collected using a microphone installed in a conference room or a recording function of an online conference system, but is not limited to such examples. The collection unit starts collecting audio data, for example, at the same time as the conference starts. The collection unit can also continuously collect audio data while the conference is in progress. Furthermore, the collection unit can stop collecting audio data at the same time as the conference ends. The analysis unit uses a generation AI to analyze the audio data collected by the collection unit. The analysis unit, for example, converts the audio data into text data and automatically summarizes the contents of the conference as minutes. The generation AI analyzes the audio data using a text generation AI (for example, LLM) and creates minutes by organizing the remarks of each speaker, the main points of the discussion, and decisions. The generation unit generates minutes based on the data analyzed by the analysis unit. The generation unit generates minutes based on text data generated from the audio data, for example, using a generation AI. The generation AI understands the contents of the voice data, extracts important information, and creates minutes. The transmission unit transmits the minutes generated by the generation unit. The transmission unit, for example, automatically transmits the generated minutes by email. The transmission unit can also upload the generated minutes to cloud storage. Furthermore, the transmission unit can also transmit the generated minutes by a chat tool. As a result, the automatic minutes creation system according to the embodiment can accurately record the contents of the meeting and share them with participating members by collecting and analyzing the voice data of the meeting, generating minutes, and transmitting them.

[0075] The collection unit collects audio data of the conference. The audio data of the conference includes, for example, a microphone installed in a conference room and audio data collected using a recording function of an online conference system, but is not limited to such examples. Specifically, a high-sensitivity microphone installed in a conference room can clearly collect the remarks of multiple participants. As a result, all remarks are accurately recorded regardless of the volume or position of the speaker's voice. In addition, when the recording function of an online conference system is used, audio data of a conference held via the Internet can be collected with high quality. For example, the collection unit starts collecting audio data at the same time as the start of the conference. As a result, all remarks are recorded from the beginning of the conference, and important information is not leaked. Furthermore, the collection unit can also continuously collect audio data while the conference is in progress. As a result, all remarks are consistently recorded without interruption in the middle of the conference. The collection unit can also stop collecting audio data at the same time as the end of the conference. As a result, unnecessary audio data is prevented from being collected after the end of the conference, making it easier to manage the data. The collection unit can transmit the collected audio data to the analysis unit in real time. As a result, the analysis unit can instantly analyze the collected audio data and quickly create minutes. In addition, the collection unit can use noise canceling technology to improve the quality of the audio data, eliminating background noise and other distracting sounds, allowing the spoken content to be recorded more clearly.

[0076] The analysis unit uses the generation AI to analyze the voice data collected by the collection unit. For example, the analysis unit converts the voice data into text data and automatically compiles the contents of the meeting as minutes. Specifically, the generation AI analyzes the voice data using voice recognition technology and converts the contents of each speaker's remarks into text data. The generation AI can identify the voice of the speaker from the voice data and record each speaker's remarks individually. This makes it possible to accurately grasp who said what. Furthermore, the generation AI uses a text generation AI (for example, LLM) to analyze the remarks and organize the main points of the discussion and decisions. The generation AI can use natural language processing technology to understand the meaning of the remarks and extract important information. For example, it can detect important keywords such as "decisions" and "action items" from the remarks and reflect them in the minutes. The analysis unit can analyze the collected voice data in real time and update the minutes as the meeting progresses. This allows the minutes to be completed immediately after the end of the meeting and shared with participants. Furthermore, the analysis unit can refer to past meeting data and automatically associate similar discussions and decisions. This allows for a deeper understanding of the content of the meeting and a grasp of its relevance to past discussions. The analysis unit sends the results of the analysis of the audio data to the generation unit to assist in the generation of minutes.

[0077] The generation unit generates minutes based on the data analyzed by the analysis unit. The generation unit generates minutes based on text data generated from voice data, for example, using a generation AI. The generation AI understands the contents of the voice data and extracts important information to create minutes. Specifically, the generation AI organizes the remarks of each speaker and clearly describes the main points of the discussion and the decisions. The generation unit automatically determines the structure of the minutes based on the information extracted by the generation AI. For example, the minutes begin with the date and time of the meeting and information about the participants, followed by a bullet list of the main points of the discussion and the decisions. The generation unit can flexibly change the format of the minutes. For example, the format of the minutes can be customized according to the type and purpose of the meeting, and the necessary information can be appropriately described. The generation unit can automatically save the generated minutes and edit them as necessary. For example, if there is an error in the generated minutes, it can be manually corrected. The generation unit transmits the generated minutes to the transmission unit and prepares to share them with the participants. In addition, the generation department can compare the generated minutes with past minutes to grasp the progress and unresolved issues, which allows for consistent management of meeting content and continuous improvement.

[0078] The transmission unit transmits the minutes generated by the generation unit. For example, the transmission unit automatically transmits the generated minutes by email. Specifically, the transmission unit can transmit the generated minutes as an attachment to all participants of the meeting by email. This allows the participants to quickly check the contents of the meeting and take necessary actions. The transmission unit can also upload the generated minutes to cloud storage. This allows the participants to access the minutes anytime and anywhere and refer to the contents of past meetings. Furthermore, the transmission unit can transmit the generated minutes by a chat tool. For example, a link to the minutes can be shared in a team chat group so that participants can easily access it. The transmission unit can automatically manage a list of destinations and transmit the minutes appropriately to participants and related parties of the meeting. This allows important information to be shared with all related parties without leaking out. Furthermore, the transmission unit can record the transmission history and check the transmission status of past minutes. This prevents the minutes from being sent out or sent in duplicate, realizing efficient information sharing. The transmission unit can collect feedback on the transmitted minutes and provide the feedback to the generation unit. This allows us to continually improve the quality of our minutes and provide more accurate and useful information.

[0079] The analysis unit may include an identification unit that identifies the speaker. The identification unit uses, for example, an algorithm that identifies the speaker from the voice data. For example, the identification unit uses a voice recognition technology to analyze the characteristics of the speaker's voice and identify the speaker. The identification unit can also learn the speaker's voice pattern and identify the speaker. Furthermore, the identification unit can organize the remarks of each speaker based on the characteristics of the speaker's voice. In this way, by identifying the speaker, the remarks of each speaker can be accurately recorded in the minutes. A part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI, or may be performed without using AI. For example, the identification unit can input the voice data to the generation AI and cause the generation AI to identify the speaker.

[0080] The analysis unit may include an extraction unit that extracts the main points of the discussion. The extraction unit uses, for example, an algorithm that extracts the main points of the discussion from the voice data. For example, the extraction unit analyzes the voice data using natural language processing technology and extracts important information. The extraction unit can also understand the content of the voice data and extract the main points of the discussion. Furthermore, the extraction unit can also create minutes based on the main points of the discussion. In this way, by extracting the main points of the discussion, important information can be efficiently recorded in the minutes. Some or all of the above-mentioned processing in the extraction unit may be performed, for example, using AI, or may be performed without using AI. For example, the extraction unit may input the voice data to a generation AI and cause the generation AI to extract the main points of the discussion.

[0081] The transmission unit may include an upload unit that uploads the minutes to the cloud storage. The upload unit uploads the generated minutes to the cloud storage, for example. For example, the upload unit uploads the minutes using the API of the cloud storage. The upload unit can also automatically upload the minutes to the cloud storage. Furthermore, the upload unit can monitor the upload status of the minutes and confirm that the upload is complete. This allows the participating members to access the minutes at any time by uploading the minutes to the cloud storage. A part or all of the above-mentioned processing in the upload unit may be performed, for example, using AI, or may be performed without using AI. For example, the upload unit can input the generated minutes to a generation AI and cause the generation AI to execute uploading to the cloud storage.

[0082] The collection unit can collect voice data by using a recording app on the smartphone. The collection unit collects voice data by using, for example, a recording app on the smartphone. For example, the collection unit starts the recording app on the smartphone and collects voice data of the meeting. The collection unit can also collect voice data of the meeting in real time by using the recording app on the smartphone. Furthermore, the collection unit can also upload the collected voice data to cloud storage by using the recording app on the smartphone. This makes it possible to easily collect voice data by using the recording app on the smartphone. A part or all of the above-mentioned processing in the collection unit may be performed by using, for example, AI, or may be performed without using AI. For example, the collection unit can input the voice data collected by the recording app on the smartphone to the generation AI and cause the generation AI to analyze the voice data.

[0083] The transmission unit can transmit the minutes by a chat tool. The transmission unit transmits, for example, the generated minutes by a chat tool. For example, the transmission unit transmits the minutes by using an API of the chat tool. The transmission unit can also automatically transmit the minutes by the chat tool. Furthermore, the transmission unit can monitor the transmission status of the minutes and confirm that the transmission has been completed. In this way, by transmitting the minutes by the chat tool, the participating members can quickly check the minutes. A part or all of the above-mentioned processing in the transmission unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the transmission unit can input the generated minutes into a generation AI and cause the generation AI to transmit the minutes by the chat tool.

[0084] The collection unit can estimate the user's emotion and adjust the timing of collecting voice data based on the estimated user's emotion. The collection unit, for example, estimates the user's emotion and adjusts the timing of collecting voice data based on the estimated user's emotion. For example, when the user is nervous, the collection unit starts collecting voice data immediately after the start of the meeting. Also, when the user is relaxed, the collection unit can collect voice data in stages in accordance with the progress of the meeting. Furthermore, when the user is in a hurry, the collection unit can preferentially collect only important parts of the meeting. In this way, by adjusting the timing of collecting voice data according to the user's emotion, voice data can be collected at a more appropriate timing. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the collection unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the collection unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the timing of collecting voice data.

[0085] The collection unit can dynamically adjust the quality of the voice data to be collected according to the progress of the conference. The collection unit dynamically adjusts the quality of the voice data to be collected according to, for example, the progress of the conference. For example, the collection unit collects low-quality voice data at the beginning of the conference, and collects high-quality voice data when an important topic is discussed. The collection unit can also collect standard-quality voice data when the conference is progressing smoothly. Furthermore, when the conference is in chaos, the collection unit can also strengthen noise canceling to collect high-quality voice data. This allows important parts to be collected with high quality by adjusting the quality of the voice data according to the progress of the conference. Some or all of the above-mentioned processing in the collection unit may be performed, for example, using AI, or may be performed without using AI. For example, the collection unit can input conference progress data to the generation AI and cause the generation AI to adjust the quality of the voice data.

[0086] The collection unit can limit the collection range based on the importance of the conference when collecting voice data. For example, the collection unit limits the collection range based on the importance of the conference when collecting voice data. For example, the collection unit collects all remarks in an important conference, and collects only major remarks in a normal conference. The collection unit can also collect detailed voice data for important topics, and collect only outlines for other topics. Furthermore, the collection unit can also collect remarks of important participants preferentially, and collect remarks of other participants as necessary. This allows important information to be collected efficiently by limiting the collection range based on the importance of the conference. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using AI, or may be performed without using AI. For example, the collection unit can input importance data of the conference to the generation AI, and cause the generation AI to limit the collection range.

[0087] The collection unit can estimate the user's emotion and determine the priority of the voice data to be collected based on the estimated user's emotion. The collection unit, for example, estimates the user's emotion and determines the priority of the voice data to be collected based on the estimated user's emotion. For example, when the user is nervous, the collection unit preferentially collects important utterances. In addition, when the user is relaxed, the collection unit can also collect all utterances equally. Furthermore, when the user is in a hurry, the collection unit can also preferentially collect utterances that cover the main points. In this way, important information can be preferentially collected by determining the priority of the voice data according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the collection unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the collection unit can input the user's emotion data to the generation AI and cause the generation AI to execute the determination of the priority of the voice data.

[0088] The collection unit can select an appropriate collection method based on the location information of the participants in the conference when collecting voice data. For example, the collection unit selects an appropriate collection method based on the location information of the participants in the conference room when collecting voice data. For example, the collection unit optimizes the placement of microphones according to the positions of the participants in the conference room. In addition, in the case of an online conference, the collection unit can also adjust the collection method taking into account the quality of the voice input device of each participant. Furthermore, when the participants in the conference are moving, the collection unit can also update the location information in real time and select the optimal collection method. As a result, optimal voice data can be collected by selecting a collection method based on the location information of the participants in the conference. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using AI or may be performed without using AI. For example, the collection unit can input the location information data of the participants to the generation AI and cause the generation AI to select the collection method.

[0089] The collection unit can customize the collection method based on the theme of the meeting when collecting the voice data. For example, the collection unit customizes the collection method based on the theme of the meeting when collecting the voice data. For example, the collection unit selects a method for accurately collecting technical terms in a technical meeting. In addition, the collection unit can also select a method for preferentially collecting important decisions in a business meeting. Furthermore, the collection unit can also select a method for collecting lecture contents in detail in an educational meeting. In this way, appropriate voice data can be collected by customizing the collection method based on the theme of the meeting. Part or all of the above-mentioned processing in the collection unit may be performed, for example, using AI or may be performed without using AI. For example, the collection unit can input the theme data of the meeting to the generation AI and cause the generation AI to execute customization of the collection method.

[0090] The analysis unit can estimate the user's emotion and adjust the accuracy of the analysis based on the estimated user's emotion. The analysis unit, for example, estimates the user's emotion and adjusts the accuracy of the analysis based on the estimated user's emotion. For example, when the user is nervous, the analysis unit increases the accuracy of the analysis to create detailed minutes. In addition, when the user is relaxed, the analysis unit can also perform analysis with standard accuracy. Furthermore, when the user is in a hurry, the analysis unit can also perform analysis with a focus on the main points. This allows more appropriate minutes to be generated by adjusting the accuracy of the analysis according to the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the analysis unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the accuracy of the analysis.

[0091] The analysis unit can dynamically change the analysis algorithm according to the progress of the conference when analyzing the voice data. For example, the analysis unit dynamically changes the analysis algorithm according to the progress of the conference when analyzing the voice data. For example, the analysis unit uses a simple analysis algorithm at the beginning of the conference, and switches to a detailed analysis algorithm when an important topic is discussed. The analysis unit can also use a standard analysis algorithm when the conference is progressing smoothly. Furthermore, the analysis unit can also use an analysis algorithm with enhanced noise canceling when the conference is in chaos. This allows important parts to be analyzed in detail by changing the analysis algorithm according to the progress of the conference. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input conference progress data to the generation AI and cause the generation AI to change the analysis algorithm.

[0092] The analysis unit can adjust the level of detail of the analysis based on the importance of the meeting when analyzing the voice data. For example, the analysis unit adjusts the level of detail of the analysis based on the importance of the meeting when analyzing the voice data. For example, the analysis unit performs a detailed analysis on an important meeting, and analyzes only an outline of a normal meeting. The analysis unit can also perform a detailed analysis on important topics, and perform a simple analysis on other topics. Furthermore, the analysis unit can also analyze the remarks of important participants preferentially, and analyze the remarks of other participants as necessary. As a result, important information can be analyzed efficiently by adjusting the level of detail of the analysis based on the importance of the meeting. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input the importance data of the meeting to the generation AI, and cause the generation AI to adjust the level of detail of the analysis.

[0093] The analysis unit can estimate the user's emotion and adjust the display method of the analysis result based on the estimated user's emotion. The analysis unit, for example, estimates the user's emotion and adjusts the display method of the analysis result based on the estimated user's emotion. For example, when the user is nervous, the analysis unit provides a simple and highly visible display method. Furthermore, when the user is relaxed, the analysis unit can also provide a display method including detailed information. Furthermore, when the user is in a hurry, the analysis unit can also provide a display method that focuses on the main points. This allows for highly visible display by adjusting the display method of the analysis result according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the analysis unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the display method of the analysis result.

[0094] The analysis unit can perform analysis based on attribute information of the participants of the conference when analyzing the voice data. For example, the analysis unit performs analysis based on attribute information of the participants of the conference when analyzing the voice data. For example, the analysis unit adjusts the analysis accuracy of technical terms according to the expertise of the participants. The analysis unit can also adjust the analysis accuracy of important remarks according to the position of the participants. Furthermore, the analysis unit can adjust the priority of the analysis according to the frequency of remarks of the participants. In this way, an appropriate analysis result can be obtained by performing analysis based on the attribute information of the participants of the conference. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can input attribute information data of the participants to the generation AI and cause the generation AI to execute the analysis.

[0095] The analysis unit can customize the analysis method based on the theme of the meeting when analyzing the voice data. For example, the analysis unit customizes the analysis method based on the theme of the meeting when analyzing the voice data. For example, the analysis unit selects a method for accurately analyzing technical terms in a technical meeting. Also, the analysis unit can select a method for preferentially analyzing important decisions in a business meeting. Furthermore, the analysis unit can select a method for analyzing lecture contents in detail in an educational meeting. In this way, by customizing the analysis method based on the theme of the meeting, appropriate analysis results can be obtained. Part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can input the theme data of the meeting to the generation AI and cause the generation AI to execute customization of the analysis method.

[0096] The generation unit can estimate the user's emotion and adjust the generation method of the minutes based on the estimated user's emotion. The generation unit, for example, estimates the user's emotion and adjusts the generation method of the minutes based on the estimated user's emotion. For example, the generation unit generates detailed minutes when the user is nervous. The generation unit can also generate standard minutes when the user is relaxed. Furthermore, the generation unit can also generate minutes that focus on the main points when the user is in a hurry. This allows more appropriate minutes to be generated by adjusting the generation method of the minutes according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the generation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the generation unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the generation method of the minutes.

[0097] The generating unit can dynamically change the generating algorithm according to the progress of the meeting when generating the minutes. For example, the generating unit dynamically changes the generating algorithm according to the progress of the meeting when generating the minutes. For example, the generating unit uses a simple generating algorithm at the beginning of the meeting, and switches to a detailed generating algorithm when the important agenda is reached. The generating unit can also use a standard generating algorithm when the meeting is progressing smoothly. Furthermore, the generating unit can also use a generating algorithm with enhanced noise canceling when the meeting is in chaos. This allows important parts to be recorded in detail by changing the generating algorithm according to the progress of the meeting. Part or all of the above-mentioned processing in the generating unit may be performed, for example, using AI, or may be performed without using AI. For example, the generating unit can input progress data of the meeting to the generating AI and cause the generating AI to change the generating algorithm.

[0098] The generation unit can adjust the level of detail of the generation based on the importance of the meeting when generating the minutes. For example, the generation unit adjusts the level of detail of the generation based on the importance of the meeting when generating the minutes. For example, the generation unit generates detailed minutes for important meetings and generates only an outline for normal meetings. The generation unit can also generate detailed minutes for important topics and generate simple minutes for other topics. Furthermore, the generation unit can also reflect the remarks of important participants in the minutes with priority, and reflect the remarks of other participants as necessary. As a result, important information can be efficiently recorded by adjusting the level of detail of the generation based on the importance of the meeting. Part or all of the above-mentioned processing in the generation unit may be performed, for example, using AI, or may be performed without using AI. For example, the generation unit can input the importance data of the meeting to the generation AI and cause the generation AI to adjust the level of detail of the generation.

[0099] The generation unit can estimate the user's emotion and adjust the display method of the minutes based on the estimated user's emotion. The generation unit, for example, estimates the user's emotion and adjusts the display method of the minutes based on the estimated user's emotion. For example, when the user is nervous, the generation unit provides a simple and highly visible display method. In addition, when the user is relaxed, the generation unit can also provide a display method including detailed information. Furthermore, when the user is in a hurry, the generation unit can also provide a display method that focuses on the main points. This allows for highly visible display by adjusting the display method of the minutes according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the generation unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the generation unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the display method of the minutes.

[0100] The generating unit can generate minutes based on attribute information of the participants of the meeting when generating the minutes. For example, the generating unit generates minutes based on attribute information of the participants of the meeting when generating the minutes. For example, the generating unit generates minutes that accurately reflect technical terms according to the expertise of the participants. The generating unit can also generate minutes that preferentially reflect important remarks according to the positions of the participants. Furthermore, the generating unit can adjust the priority of the minutes according to the frequency of remarks of the participants. As a result, appropriate minutes can be generated by generating the minutes based on the attribute information of the participants of the meeting. A part or all of the above-mentioned processing in the generating unit may be performed using, for example, AI, or may be performed without using AI. For example, the generating unit can input attribute information data of the participants to the generating AI and cause the generating AI to execute the generation.

[0101] The generation unit can customize the generation method based on the theme of the meeting when generating the minutes. For example, the generation unit customizes the generation method based on the theme of the meeting when generating the minutes. For example, the generation unit generates minutes that accurately reflect technical terms in a technical meeting. Also, the generation unit can generate minutes that preferentially reflect important decisions in a business meeting. Furthermore, the generation unit can generate minutes that reflect the lecture content in detail in an educational meeting. In this way, appropriate minutes can be generated by customizing the generation method based on the theme of the meeting. Part or all of the above-mentioned processing in the generation unit may be performed, for example, using AI, or may be performed without using AI. For example, the generation unit can input the theme data of the meeting to the generation AI and cause the generation AI to execute customization of the generation method.

[0102] The transmission unit can estimate the user's emotion and adjust the transmission timing based on the estimated user's emotion. The transmission unit, for example, estimates the user's emotion and adjusts the transmission timing based on the estimated user's emotion. For example, if the user is nervous, the transmission unit transmits the minutes immediately after the end of the meeting. Also, if the user is relaxed, the transmission unit can transmit the minutes after a certain time has passed after the end of the meeting. Furthermore, if the user is in a hurry, the transmission unit can transmit the minutes in real time during the meeting. In this way, the transmission timing can be adjusted according to the user's emotion, so that the minutes can be transmitted at an appropriate timing. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the transmission unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the transmission unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the transmission timing.

[0103] The transmission unit can dynamically change the transmission method according to the progress of the meeting when transmitting the minutes. For example, the transmission unit dynamically changes the transmission method according to the progress of the meeting when transmitting the minutes. For example, if the meeting is proceeding smoothly, the transmission unit transmits the minutes by normal mail. Also, if the meeting is chaotic, the transmission unit can transmit the minutes in a format that emphasizes important parts. Furthermore, if the meeting lasts for a long time, the transmission unit can transmit the minutes in a divided form. In this way, by changing the transmission method according to the progress of the meeting, important parts can be emphasized and transmitted. A part or all of the above-mentioned processing in the transmission unit may be performed, for example, using AI, or may be performed without using AI. For example, the transmission unit can input the progress data of the meeting to the generation AI and cause the generation AI to execute the change of the transmission method.

[0104] The transmission unit can adjust the level of detail of the transmission based on the importance of the meeting when transmitting the minutes. For example, the transmission unit adjusts the level of detail of the transmission based on the importance of the meeting when transmitting the minutes. For example, the transmission unit transmits detailed minutes for important meetings and transmits only summaries for normal meetings. The transmission unit can also transmit detailed minutes for important topics and transmit simple minutes for other topics. Furthermore, the transmission unit can also preferentially reflect the remarks of important participants in the minutes and reflect the remarks of other participants as necessary. This allows important information to be transmitted efficiently by adjusting the level of detail of the transmission based on the importance of the meeting. Part or all of the above-mentioned processing in the transmission unit may be performed, for example, using AI or may be performed without using AI. For example, the transmission unit can input importance data of the meeting to the generation AI and cause the generation AI to adjust the level of detail of the transmission.

[0105] The transmission unit can estimate the user's emotion and select a transmission method based on the estimated user's emotion. The transmission unit, for example, estimates the user's emotion and selects a transmission method based on the estimated user's emotion. For example, when the user is nervous, the transmission unit transmits the minutes by email. Also, when the user is relaxed, the transmission unit can transmit the minutes by a chat tool. Furthermore, when the user is in a hurry, the transmission unit can transmit the minutes by SMS. In this way, by selecting a transmission method according to the user's emotion, the minutes can be transmitted in an appropriate manner. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the transmission unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the transmission unit can input the user's emotion data to the generation AI and cause the generation AI to select a transmission method.

[0106] The transmission unit can transmit the minutes based on the attribute information of the participants of the conference when transmitting the minutes. For example, the transmission unit transmits the minutes based on the attribute information of the participants of the conference when transmitting the minutes. For example, the transmission unit transmits minutes in which important remarks are emphasized according to the positions of the participants. The transmission unit can also transmit minutes that accurately reflect technical terms according to the expertise of the participants. Furthermore, the transmission unit can adjust the priority of the minutes and transmit them according to the frequency of remarks of the participants. As a result, appropriate minutes can be transmitted by transmitting based on the attribute information of the participants of the conference. A part or all of the above-mentioned processing in the transmission unit may be performed using, for example, AI, or may be performed without using AI. For example, the transmission unit can input the attribute information data of the participants to the generation AI and cause the generation AI to execute the transmission.

[0107] The transmission unit can customize the transmission method based on the theme of the meeting when transmitting the minutes. For example, the transmission unit customizes the transmission method based on the theme of the meeting when transmitting the minutes. For example, in a technical meeting, the transmission unit transmits minutes that accurately reflect technical terms. In addition, in a business meeting, the transmission unit can also transmit minutes that emphasize important decisions. Furthermore, in an educational meeting, the transmission unit can transmit minutes that reflect the lecture content in detail. In this way, by customizing the transmission method based on the theme of the meeting, appropriate minutes can be transmitted. Part or all of the above-mentioned processing in the transmission unit may be performed, for example, using AI, or may be performed without using AI. For example, the transmission unit can input the theme data of the meeting to the generation AI and cause the generation AI to customize the transmission method.

[0108] The identification unit can estimate the user's emotion and adjust the speaker's identification accuracy based on the estimated user's emotion. The identification unit, for example, estimates the user's emotion and adjusts the speaker's identification accuracy based on the estimated user's emotion. For example, when the user is nervous, the identification unit increases the speaker's identification accuracy to create detailed minutes. In addition, when the user is relaxed, the identification unit can also identify the speaker with a standard identification accuracy. Furthermore, when the user is in a hurry, the identification unit can also identify the speaker with an identification accuracy that covers the main points. This allows more appropriate minutes to be generated by adjusting the speaker's identification accuracy according to the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the identification unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the speaker's identification accuracy.

[0109] The identification unit can dynamically change the identification algorithm according to the progress of the conference when identifying a speaker. For example, the identification unit dynamically changes the identification algorithm according to the progress of the conference when identifying a speaker. For example, the identification unit uses a simple identification algorithm at the beginning of the conference, and switches to a detailed identification algorithm when an important topic is discussed. The identification unit can also use a standard identification algorithm when the conference is progressing smoothly. Furthermore, the identification unit can also use an identification algorithm with enhanced noise canceling when the conference is chaotic. This allows important remarks to be accurately identified by changing the identification algorithm according to the progress of the conference. Part or all of the above-mentioned processing in the identification unit may be performed, for example, using AI, or may be performed without using AI. For example, the identification unit can input conference progress data to the generation AI and cause the generation AI to change the identification algorithm.

[0110] The identification unit can estimate the user's emotion and adjust the display method of the identification result based on the estimated user's emotion. The identification unit, for example, estimates the user's emotion and adjusts the display method of the identification result based on the estimated user's emotion. For example, the identification unit provides a simple and highly visible display method when the user is nervous. Furthermore, the identification unit can also provide a display method including detailed information when the user is relaxed. Furthermore, the identification unit can also provide a display method that focuses on the main points when the user is in a hurry. This allows for highly visible display by adjusting the display method of the identification result according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the identification unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the identification unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the display method of the identification result.

[0111] The identification unit can perform identification based on the attribute information of the conference participants when identifying the speaker. For example, the identification unit performs identification based on the attribute information of the conference participants when identifying the speaker. For example, the identification unit adjusts the identification accuracy of technical terms according to the expertise of the participants. In addition, the identification unit can also adjust the identification accuracy of important remarks according to the position of the participants. Furthermore, the identification unit can adjust the priority of identification according to the frequency of remarks of the participants. In this way, by performing identification based on the attribute information of the conference participants, an appropriate identification result can be obtained. Part or all of the above-mentioned processing in the identification unit may be performed using, for example, AI, or may be performed without using AI. For example, the identification unit can input the attribute information data of the participants to the generation AI and cause the generation AI to perform the identification.

[0112] The extraction unit can estimate the user's emotion and adjust the extraction accuracy of the main points of the discussion based on the estimated user's emotion. The extraction unit, for example, estimates the user's emotion and adjusts the extraction accuracy of the main points of the discussion based on the estimated user's emotion. For example, when the user is nervous, the extraction unit increases the extraction accuracy to create detailed minutes. In addition, when the user is relaxed, the extraction unit can also extract the main points of the discussion with standard accuracy. Furthermore, when the user is in a hurry, the extraction unit can also extract the main points of the discussion with an extraction accuracy that captures the main points. In this way, by adjusting the extraction accuracy of the main points of the discussion according to the user's emotion, more appropriate minutes can be generated. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the extraction unit may be performed, for example, using an AI, or may be performed without using an AI. For example, the extraction unit can input a user's emotional data into the generation AI and have the generation AI adjust the accuracy of extracting the main points of the discussion.

[0113] The extraction unit can dynamically change the extraction algorithm according to the progress of the meeting when extracting the main points of the discussion. For example, the extraction unit dynamically changes the extraction algorithm according to the progress of the meeting when extracting the main points of the discussion. For example, the extraction unit uses a simple extraction algorithm at the beginning of the meeting, and switches to a detailed extraction algorithm when the important topics are discussed. The extraction unit can also use a standard extraction algorithm when the meeting is progressing smoothly. Furthermore, the extraction unit can also use an extraction algorithm with enhanced noise canceling when the meeting is chaotic. This allows important parts to be extracted in detail by changing the extraction algorithm according to the progress of the meeting. Part or all of the above-mentioned processing in the extraction unit may be performed, for example, using AI, or may be performed without using AI. For example, the extraction unit can input the progress data of the meeting to the generation AI and cause the generation AI to execute the change of the extraction algorithm.

[0114] The extraction unit can estimate the user's emotion and adjust the display method of the extraction result based on the estimated user's emotion. The extraction unit, for example, estimates the user's emotion and adjusts the display method of the extraction result based on the estimated user's emotion. For example, when the user is nervous, the extraction unit provides a simple and highly visible display method. In addition, when the user is relaxed, the extraction unit can also provide a display method including detailed information. Furthermore, when the user is in a hurry, the extraction unit can also provide a display method that focuses on the main points. This allows for highly visible display by adjusting the display method of the extraction result according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the extraction unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the extraction unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the display method of the extraction result.

[0115] The extraction unit can perform extraction based on the attribute information of the participants of the meeting when extracting the main points of the discussion. For example, the extraction unit performs extraction based on the attribute information of the participants of the meeting when extracting the main points of the discussion. For example, the extraction unit adjusts the extraction accuracy of technical terms according to the expertise of the participants. The extraction unit can also adjust the extraction accuracy of important remarks according to the position of the participants. Furthermore, the extraction unit can adjust the priority of extraction according to the frequency of remarks of the participants. As a result, appropriate extraction results can be obtained by performing extraction based on the attribute information of the participants of the meeting. Part or all of the above-mentioned processing in the extraction unit may be performed using, for example, AI, or may be performed without using AI. For example, the extraction unit can input the attribute information data of the participants to the generation AI and cause the generation AI to execute the extraction.

[0116] The upload unit can estimate the user's emotion and adjust the upload timing based on the estimated user's emotion. The upload unit, for example, estimates the user's emotion and adjusts the upload timing based on the estimated user's emotion. For example, if the user is nervous, the upload unit uploads the minutes immediately after the end of the meeting. Also, if the user is relaxed, the upload unit can upload the minutes after a certain time has passed after the end of the meeting. Furthermore, if the user is in a hurry, the upload unit can upload the minutes in real time during the meeting. In this way, by adjusting the upload timing according to the user's emotion, the minutes can be uploaded at an appropriate timing. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the upload unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the upload unit can input the user's emotion data to the generation AI and cause the generation AI to adjust the upload timing.

[0117] The upload unit can dynamically change the upload method according to the progress of the meeting when uploading the minutes. The upload unit dynamically changes the upload method according to the progress of the meeting when uploading the minutes, for example. For example, if the meeting is proceeding smoothly, the upload unit uploads the minutes in a normal manner. Also, if the meeting is chaotic, the upload unit can upload the minutes in a format that emphasizes important parts. Furthermore, if the meeting lasts for a long time, the upload unit can upload the minutes in parts. In this way, by changing the upload method according to the progress of the meeting, it is possible to upload the minutes with emphasis on important parts. A part or all of the above-mentioned processing in the upload unit may be performed, for example, using AI, or may be performed without using AI. For example, the upload unit can input the progress data of the meeting to the generation AI and cause the generation AI to execute the change of the upload method.

[0118] The upload unit can estimate the user's emotion and select an upload method based on the estimated user's emotion. The upload unit, for example, estimates the user's emotion and selects an upload method based on the estimated user's emotion. For example, when the user is nervous, the upload unit uploads the minutes to a cloud storage. Also, when the user is relaxed, the upload unit can upload the minutes to an in-house server. Furthermore, when the user is in a hurry, the upload unit can upload the minutes by attaching them to an email. In this way, by selecting an upload method according to the user's emotion, the minutes can be uploaded in an appropriate manner. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the upload unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the upload unit can input the user's emotion data to the generation AI and cause the generation AI to select the upload method.

[0119] The uploading unit can upload the minutes based on the attribute information of the participants of the meeting when uploading the minutes. For example, the uploading unit uploads the minutes based on the attribute information of the participants of the meeting when uploading the minutes. For example, the uploading unit uploads minutes that emphasize important remarks according to the positions of the participants. The uploading unit can also upload minutes that accurately reflect technical terms according to the expertise of the participants. Furthermore, the uploading unit can adjust the priority of the minutes according to the frequency of remarks of the participants and upload them. In this way, appropriate minutes can be uploaded by uploading based on the attribute information of the participants of the meeting. Part or all of the above-mentioned processing in the uploading unit may be performed using, for example, AI, or may be performed without using AI. For example, the uploading unit can input the attribute information data of the participants to the generation AI and cause the generation AI to execute the upload.

[0120] The recording app of the smartphone can estimate the user's emotion and adjust the operation of the recording app based on the estimated user's emotion. The recording app of the smartphone, for example, estimates the user's emotion and adjusts the operation of the recording app based on the estimated user's emotion. For example, the recording app of the smartphone simplifies the operation of the recording app to make it easier to use when the user is nervous. The recording app of the smartphone can also provide detailed setting options when the user is relaxed. Furthermore, the recording app of the smartphone can also enable the user to start recording with one touch when the user is in a hurry. This allows an easy-to-use recording environment to be provided by adjusting the operation of the recording app according to the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the recording app of the smartphone may be performed using, for example, AI, or may be performed without using AI. For example, the recording app of the smartphone can input the user's emotion data to the generation AI and cause the generation AI to adjust the operation of the recording app.

[0121] The recording app of the smartphone can dynamically change the recording settings according to the progress of the meeting when the recording app is used. The recording app of the smartphone dynamically changes the recording settings according to the progress of the meeting when the recording app is used. For example, the recording app of the smartphone uses a low-quality recording setting at the beginning of the meeting, and switches to a high-quality recording setting when an important topic is discussed. The recording app of the smartphone can also use a standard-quality recording setting when the meeting is progressing smoothly. Furthermore, the recording app of the smartphone can also use a recording setting with enhanced noise cancellation when the meeting is chaotic. This allows important parts to be recorded in high quality by changing the recording settings according to the progress of the meeting. Some or all of the above-mentioned processing in the recording app of the smartphone may be performed, for example, using AI or may be performed without using AI. For example, the recording app of the smartphone can input the progress data of the meeting to the generation AI and cause the generation AI to change the recording settings.

[0122] The recording app of the smartphone can estimate the user's emotion and adjust the display method of the recording app based on the estimated user's emotion. The recording app of the smartphone, for example, estimates the user's emotion and adjusts the display method of the recording app based on the estimated user's emotion. For example, the recording app of the smartphone provides a simple and highly visible display method when the user is nervous. The recording app of the smartphone can also provide a display method including detailed information when the user is relaxed. Furthermore, the recording app of the smartphone can also provide a display method that focuses on the main points when the user is in a hurry. This allows for a highly visible display by adjusting the display method of the recording app according to the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the recording app of the smartphone may be performed using, for example, an AI, or may be performed without using an AI. For example, a smartphone recording app can input a user's emotional data into a generation AI and have the generation AI adjust the way the recording app is displayed.

[0123] The recording app of the smartphone can set the recording based on the attribute information of the participants of the conference when the recording app is used. The recording app of the smartphone, for example, sets the recording based on the attribute information of the participants of the conference when the recording app is used. For example, the recording app of the smartphone sets the recording to accurately record technical terms according to the expertise of the participants. In addition, the recording app of the smartphone can set the recording to prioritize important remarks according to the position of the participants. Furthermore, the recording app of the smartphone can set the recording to adjust the recording priority according to the frequency of remarks of the participants. In this way, by setting the recording based on the attribute information of the participants of the conference, appropriate recording results can be obtained. Part or all of the above-mentioned processing in the recording app of the smartphone may be performed using, for example, AI, or may be performed without using AI. For example, the recording app of the smartphone can input the attribute information data of the participants to the generation AI and cause the generation AI to execute the recording settings.

[0124] The chat tool can estimate the user's emotion and adjust the transmission timing of the chat tool based on the estimated user's emotion. The chat tool, for example, estimates the user's emotion and adjusts the transmission timing of the chat tool based on the estimated user's emotion. For example, if the user is nervous, the chat tool transmits the minutes of the meeting through the chat tool immediately after the end of the meeting. Also, if the user is relaxed, the chat tool can transmit the minutes of the meeting through the chat tool after a certain time has passed after the end of the meeting. Furthermore, if the user is in a hurry, the chat tool can transmit the minutes of the meeting through the chat tool in real time during the meeting. In this way, the minutes of the meeting can be transmitted at an appropriate timing by adjusting the transmission timing of the chat tool according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the chat tool may be performed using, for example, an AI, or may be performed without using an AI. For example, a chat tool can input a user's emotional data into a generation AI and have the generation AI adjust the timing of sending messages.

[0125] The chat tool can dynamically change the transmission method according to the progress of the meeting when the chat tool is used. The chat tool dynamically changes the transmission method according to the progress of the meeting when the chat tool is used. For example, when the meeting is proceeding smoothly, the chat tool transmits the minutes by the chat tool in a normal manner. Also, when the meeting is chaotic, the chat tool can transmit the minutes by the chat tool in a format in which important parts are emphasized. Furthermore, when the meeting is long, the chat tool can transmit the minutes by the chat tool in parts. In this way, by changing the transmission method according to the progress of the meeting, important parts can be emphasized and transmitted. A part or all of the above-mentioned processing in the chat tool may be performed, for example, using AI, or may be performed without using AI. For example, the chat tool can input the progress data of the meeting to the generation AI and cause the generation AI to execute the change of the transmission method.

[0126] The chat tool can estimate the user's emotion and adjust the display method of the chat tool based on the estimated user's emotion. The chat tool, for example, estimates the user's emotion and adjusts the display method of the chat tool based on the estimated user's emotion. For example, when the user is nervous, the chat tool provides a simple and highly visible display method. Furthermore, when the user is relaxed, the chat tool can also provide a display method including detailed information. Furthermore, when the user is in a hurry, the chat tool can also provide a display method that focuses on the main points. This allows for highly visible display by adjusting the display method of the chat tool according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a multimodal generation AI, but is not limited to such examples. A part or all of the above-mentioned processing in the chat tool may be performed using, for example, an AI, or may be performed without using an AI. For example, the chat tool can input the user's emotion data into the generation AI and cause the generation AI to adjust the display method.

[0127] The chat tool can set transmission settings based on the attribute information of the participants of the meeting when the chat tool is used. The chat tool can set transmission settings based on the attribute information of the participants of the meeting when the chat tool is used. For example, the chat tool transmits minutes in which important remarks are emphasized according to the roles of the participants. The chat tool can also transmit minutes that accurately reflect technical terms according to the expertise of the participants. Furthermore, the chat tool can adjust the priority of the minutes according to the frequency of remarks of the participants and transmit them by the chat tool. This allows appropriate minutes to be transmitted by setting transmission settings based on the attribute information of the participants of the meeting. A part or all of the above-mentioned processing in the chat tool may be performed using, for example, AI, or may be performed without using AI. For example, the chat tool can input attribute information data of the participants to a generation AI and cause the generation AI to execute transmission settings.

[0128] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0129] The analysis unit may include a translation unit that translates the contents of the meeting in real time. The translation unit may, for example, translate the audio data of the meeting in real time and immediately provide the translation results to participants who speak different languages. The translation unit may also adjust the accuracy of the translation as the meeting progresses. Furthermore, the translation unit may also include a dictionary function for accurately translating technical terms and industry-specific terms. This makes it easier for participants who speak different languages ​​to understand the contents of the meeting, enabling smooth communication even in international meetings.

[0130] The analysis unit may include a summarization unit that summarizes the contents of the meeting. The summarization unit may, for example, analyze audio data of the meeting and extract important points and decisions to create a summary. The summarization unit may also adjust the level of detail of the summary according to the progress of the meeting. Furthermore, the summarization unit may customize the content of the summary according to the roles and expertise of the participants. This allows participants to quickly grasp the overall picture of the meeting, enabling efficient information sharing.

[0131] The analysis unit may include a visualization unit that visually displays the contents of the conference. The visualization unit may, for example, analyze audio data of the conference and display the flow of discussion and relationships in graphs and charts. The visualization unit may also dynamically update the display content according to the progress of the conference. Furthermore, the visualization unit may customize the display content according to the roles and expertise of the participants. This allows participants to intuitively understand the contents of the conference and enables effective communication through visual information.

[0132] The transmission unit can include an audio providing unit that provides the minutes in audio format. The audio providing unit, for example, converts the generated minutes into audio data and provides them to the participants in audio format. The audio providing unit can also adjust the speed and tone of the audio according to the preferences of the participants. Furthermore, the audio providing unit can also provide the minutes in audio format with important parts emphasized. This allows the participants to check the minutes while on the move or at work, improving convenience.

[0133] The collection unit may include a video collection unit that collects video data of the conference. The video collection unit, for example, collects video data of the conference and records the facial expressions and gestures of the speakers. The video collection unit may also adjust the range of video data collection according to the progress of the conference. Furthermore, the video collection unit may analyze the collected video data, extract important scenes, and reflect them in the minutes. This allows participants to more accurately grasp the atmosphere and nuances of the conference.

[0134] The collection unit can estimate the user's emotions and filter the collected voice data based on the estimated user's emotions. For example, when the user is stressed, the collection unit strengthens noise canceling to collect clear voice data. When the user is relaxed, the collection unit can also collect voice data including natural environmental sounds. Furthermore, when the user is concentrating, the collection unit can also preferentially collect only important utterances. This makes it possible to collect optimal voice data according to the user's emotions.

[0135] The analysis unit can estimate the user's emotions and provide feedback of the analysis results based on the estimated user's emotions. For example, if the user feels anxious, the analysis unit can provide a simple summary of the analysis results. In addition, if the user feels excited, the analysis unit can provide detailed analysis results. Furthermore, if the user feels tired, the analysis unit can provide analysis results that focus on the main points. This makes it possible to provide appropriate feedback according to the user's emotions.

[0136] The generating unit can estimate the user's emotion and adjust the format of the minutes based on the estimated user's emotion. For example, the generating unit generates minutes in a bullet-point format when the user is nervous. The generating unit can also generate minutes including detailed explanations when the user is relaxed. Furthermore, the generating unit can generate minutes that summarize only the main points when the user is in a hurry. This makes it possible to provide minutes in an optimal format according to the user's emotion.

[0137] The transmission unit can estimate the user's emotion and select the media to be transmitted based on the estimated user's emotion. For example, the transmission unit transmits the minutes in text format when the user is nervous. The transmission unit can also transmit the minutes in video format when the user is relaxed. Furthermore, the transmission unit can transmit the minutes in audio format when the user is in a hurry. This allows the minutes to be provided in the optimal media according to the user's emotion.

[0138] The sending unit can estimate the user's emotion and adjust the frequency of transmission based on the estimated user's emotion. For example, the sending unit can reduce the frequency of sending minutes when the user is stressed. The sending unit can also send minutes periodically when the user is relaxed. Furthermore, the sending unit can send minutes every time there is an important update when the user is focused. In this way, the minutes can be provided at an optimal frequency according to the user's emotion.

[0139] The process flow of the second embodiment will be briefly described below.

[0140] Step 1: The collection unit collects audio data of the meeting. The audio data of the meeting includes audio data collected using microphones installed in the meeting room and a recording function of the online meeting system. The collection unit can start collecting audio data simultaneously with the start of the meeting, collect the data continuously while the meeting is in progress, and stop collecting the data simultaneously with the end of the meeting. Step 2: The analysis unit uses the generation AI to analyze the voice data collected by the collection unit. The analysis unit converts the voice data into text data and automatically compiles the contents of the meeting as minutes. The generation AI analyzes the voice data using a text generation AI (e.g., LLM) and creates minutes by organizing the remarks made by each speaker, key points of the discussion, and decisions made. Step 3: The generation unit generates minutes based on the data analyzed by the analysis unit. The generation unit generates minutes based on text data generated from the voice data using the generation AI. The generation AI understands the content of the voice data, extracts important information, and creates minutes. Step 4: The sending unit sends the minutes generated by the generating unit. The sending unit can automatically send the generated minutes by email, upload them to cloud storage, or send them via a chat tool.

[0141] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0142] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0143] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0144] Each of the multiple elements including the above-mentioned collection unit, analysis unit, generation unit, and transmission unit is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the collection unit collects voice data using the microphone 38B of the smart device 14 or the recording function of the online conference system. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and converts the voice data into text data using a generation AI to create minutes. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and generates minutes based on the analyzed data. The transmission unit is realized, for example, by the control unit 46A of the smart device 14, and transmits the generated minutes by email, cloud storage, or chat tool. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0145] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0146] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0147] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0148] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0149] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0150] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0151] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0152] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0153] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0154] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0155] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0156] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0157] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0158] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0159] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0160] Each of the multiple elements including the above-mentioned collection unit, analysis unit, generation unit, and transmission unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the collection unit collects voice data using the microphone 238 of the smart glasses 214 or the recording function of the online conference system. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and converts the voice data into text data using a generation AI to create minutes. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and generates minutes based on the analyzed data. The transmission unit is realized, for example, by the control unit 46A of the smart glasses 214, and transmits the generated minutes by email, cloud storage, or chat tool. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0161] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0162] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0163] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0164] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0165] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0166] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0167] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0168] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0169] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0170] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0171] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0172] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0173] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0174] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0175] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0176] Each of the multiple elements including the above-mentioned collection unit, analysis unit, generation unit, and transmission unit is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the collection unit collects voice data using the microphone 238 of the headset type terminal 314 or the recording function of the online conference system. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and converts the voice data into text data using a generation AI to create minutes. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and generates minutes based on the analyzed data. The transmission unit is realized, for example, by the control unit 46A of the headset type terminal 314, and transmits the generated minutes by email, cloud storage, or chat tool. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0177] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0178] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0179] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0180] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0181] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0182] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0183] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0184] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0185] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0186] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0187] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0188] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0189] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0190] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0191] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0192] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0193] Each of the multiple elements including the above-mentioned collection unit, analysis unit, generation unit, and transmission unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the collection unit collects voice data using the microphone 238 of the robot 414 or the recording function of the online conference system. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and converts the voice data into text data using a generation AI to create minutes. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and generates minutes based on the analyzed data. The transmission unit is realized, for example, by the control unit 46A of the robot 414, and transmits the generated minutes by email, cloud storage, or chat tool. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0194] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0195] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0196] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0197] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0198] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0199] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0200] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0201] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0202] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0203] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0204] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0205] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0206] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0207] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0208] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0209] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0210] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0211] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0212] (Appendix 1) A collection unit that collects audio data of the conference; an analysis unit that analyzes the voice data collected by the collection unit; A generation unit that generates minutes based on the data analyzed by the analysis unit; A transmission unit that transmits the minutes generated by the generation unit. A system characterized by: (Appendix 2) The analysis unit is Equipped with an identification unit that identifies the speaker 2. The system of claim 1, (Appendix 3) The analysis unit is Equipped with an extraction section that extracts the main points of the discussion 2. The system of claim 1, (Appendix 4) The transmission unit is Equipped with an upload section to upload minutes to cloud storage 2. The system of claim 1, (Appendix 5) The collecting unit includes: Collect audio data using a smartphone recording app 2. The system of claim 1, (Appendix 6) The transmission unit is Send meeting minutes via chat tool 2. The system of claim 1, (Appendix 7) The collecting unit includes: To estimate a user's emotion and adjust a timing of collecting voice data based on the estimated user's emotion. 2. The system of claim 1, (Appendix 8) The collecting unit includes: Dynamically adjust the quality of collected audio data according to the progress of the meeting 2. The system of claim 1, (Appendix 9) The collecting unit includes: When collecting voice data, limit the scope of collection based on the importance of the meeting 2. The system of claim 1, (Appendix 10) The collecting unit includes: Estimate user emotions and prioritize voice data to be collected based on the estimated user emotions. 2. The system of claim 1, (Appendix 11) The collecting unit includes: When collecting voice data, select the appropriate collection method based on the location information of meeting participants 2. The system of claim 1, (Appendix 12) The collecting unit includes: When collecting audio data, customize collection methods based on the topic of the meeting 2. The system of claim 1, (Appendix 13) The analysis unit is Estimate user emotions and adjust analysis accuracy based on the estimated user emotions 2. The system of claim 1, (Appendix 14) The analysis unit is When analyzing voice data, the analysis algorithm is dynamically changed according to the progress of the meeting. 2. The system of claim 1, (Appendix 15) The analysis unit is When analyzing audio data, adjust the level of analysis detail based on the importance of the meeting 2. The system of claim 1, (Appendix 16) The analysis unit is Infer user emotions and adjust the way analysis results are displayed based on the inferred user emotions 2. The system of claim 1, (Appendix 17) The analysis unit is When analyzing voice data, analysis is performed based on the attribute information of meeting participants. 2. The system of claim 1, (Appendix 18) The analysis unit is When analyzing audio data, customize the analysis method based on the topic of the meeting 2. The system of claim 1, (Appendix 19) The generation unit is Inferring user emotions and adjusting the generation method of meeting minutes based on the estimated user emotions 2. The system of claim 1, (Appendix 20) The generation unit is Dynamically change the generation algorithm when generating minutes depending on the progress of the meeting 2. The system of claim 1, (Appendix 21) The generation unit is When generating minutes, adjust the level of detail generated based on the importance of the meeting 2. The system of claim 1, (Appendix 22) The generation unit is Infer user sentiment and adjust the presentation of meeting minutes based on the inferred user sentiment 2. The system of claim 1, (Appendix 23) The generation unit is When generating minutes, generate them based on the attributes of the meeting participants 2. The system of claim 1, (Appendix 24) The generation unit is Customize the generation of minutes based on the topic of the meeting 2. The system of claim 1, (Appendix 25) The transmission unit is To estimate a user's emotion and adjust a transmission timing based on the estimated user's emotion. 2. The system of claim 1, (Appendix 26) The transmission unit is When sending minutes, dynamically change the sending method depending on the progress of the meeting 2. The system of claim 1, (Appendix 27) The transmission unit is When sending minutes, adjust the level of detail in your transmission based on the importance of the meeting 2. The system of claim 1, (Appendix 28) The transmission unit is A user's emotion is estimated, and a transmission method is selected based on the estimated user's emotion. 2. The system of claim 1, (Appendix 29) The transmission unit is When sending minutes, send them based on the attribute information of the meeting participants 2. The system of claim 1, (Appendix 30) The transmission unit is When sending minutes, customize the sending method based on the topic of the meeting 2. The system of claim 1, (Appendix 31) The identification unit is Estimating user emotion and adjusting speaker identification accuracy based on the estimated user emotion 3. The system of claim 2, (Appendix 32) The identification unit is When identifying speakers, dynamically change the identification algorithm depending on the progress of the meeting 3. The system of claim 2, (Appendix 33) The identification unit is Estimating a user's emotion and adjusting the display method of the classification result based on the estimated user's emotion 3. The system of claim 2, (Appendix 34) The identification unit is When identifying speakers, identification is performed based on the attribute information of meeting participants. 3. The system of claim 2, (Appendix 35) The extraction unit is Estimate the user's emotions and adjust the accuracy of extracting the main points of the discussion based on the estimated user's emotions. 4. The system of claim 3. (Appendix 36) The extraction unit is When extracting key points of discussion, the extraction algorithm is dynamically changed according to the progress of the meeting. 4. The system of claim 3. (Appendix 37) The extraction unit is Estimate the user's emotion and adjust the display method of the extracted results based on the estimated user's emotion. 4. The system of claim 3. (Appendix 38) The extraction unit is When extracting the main points of a discussion, extraction is performed based on the attribute information of meeting participants. 4. The system of claim 3. (Appendix 39) The upload unit is Infer user emotions and adjust upload timing based on the estimated user emotions 5. The system of claim 4. (Appendix 40) The upload unit is When uploading minutes, dynamically change the upload method depending on the progress of the meeting 5. The system of claim 4. (Appendix 41) The upload unit is The user's emotions are estimated, and an upload method is selected based on the estimated user's emotions. 5. The system of claim 4. (Appendix 42) The upload unit is When uploading minutes, upload them based on the attributes of the meeting participants 5. The system of claim 4. (Appendix 43) The recording app on the smartphone is Infer user emotions and adjust the behavior of the recording app based on the estimated user emotions 6. The system of claim 5, (Appendix 44) The recording app on the smartphone is When using a recording app, dynamically change recording settings as the meeting progresses 6. The system of claim 5, (Appendix 45) The recording app on the smartphone is Infer user emotions and adjust the display of the recording app based on the estimated user emotions 6. The system of claim 5, (Appendix 46) The recording app on the smartphone is When using a recording app, set recording settings based on meeting participant attributes 6. The system of claim 5, (Appendix 47) The chat tool is To estimate a user's emotion and adjust a timing of sending a message in a chat tool based on the estimated emotion of the user. 7. The system of claim 6, (Appendix 48) The chat tool is When using chat tools, dynamically change your sending method depending on the progress of the meeting 7. The system of claim 6, (Appendix 49) The chat tool is Infer user emotions and adjust chat tool display based on the estimated user emotions 7. The system of claim 6, (Appendix 50) The chat tool is When using chat tools, set sending settings based on the attribute information of meeting participants 7. The system of claim 6, [Explanation of symbols]

[0213] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A collection unit that collects audio data of the conference; an analysis unit that analyzes the voice data collected by the collection unit; A generation unit that generates minutes based on the data analyzed by the analysis unit; A transmission unit that transmits the minutes generated by the generation unit. A system characterized by:

2. The analysis unit is Equipped with an identification unit that identifies the speaker 2. The system of claim 1 .

3. The analysis unit is Equipped with an extraction section that extracts the main points of the discussion 2. The system of claim 1 .

4. The transmission unit is Equipped with an upload section to upload minutes to cloud storage 2. The system of claim 1 .

5. The collecting unit includes: Collect audio data using a smartphone recording app 2. The system of claim 1 .

6. The transmission unit is Send meeting minutes via chat tool 2. The system of claim 1 .

7. The collecting unit includes: To estimate a user's emotion and adjust a timing of collecting voice data based on the estimated user's emotion.

2. The system of claim 1 .

8. The collecting unit includes: Dynamically adjust the quality of collected audio data according to the progress of the meeting 2. The system of claim 1 .

9. The collecting unit includes: When collecting voice data, limit the scope of collection based on the importance of the meeting 2. The system of claim 1 .

Images