System

Abstract

To provide a system for converting a voice during a meeting into a text in real time, extracting important information, and automatically generating minutes.SOLUTION: A system includes a recognition section, an analysis section, a generation section, a determination section, and an assistance section. The recognition section converts a voice during a meeting into text data in real time. The analysis section analyzes the text data obtained through the conversion performed by the recognition section, and extracts important determination items or assignment items. The generation section automatically generates minutes on the basis of the information extracted by the analysis section. The determination section monitors the progress situation of the meeting in real time and determines the progress situation. The assistance section assists the progress of the meeting on the basis of a determination made by the determination section.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology does not adequately convert the audio of meetings into text in real time, extract important decisions and homework items, and automatically generate minutes, so there is room for improvement.

[0005] The system according to the embodiment aims to convert speech during a meeting into text in real time, extract important information, and automatically generate minutes of the meeting. [Means for solving the problem]

[0006] The system according to the embodiment includes a recognition unit, an analysis unit, a generation unit, a judgment unit, and an assistance unit. The recognition unit converts voice during a meeting into text data in real time. The analysis unit analyzes the text data converted by the recognition unit and extracts important decisions or homework items. The generation unit automatically generates minutes based on the information extracted by the analysis unit. The judgment unit monitors the progress of the meeting in real time and judges the progress. The assistance unit assists in the progress of the meeting based on the judgment of the judgment unit. Effect of the Invention

[0007] The system according to the embodiment can convert speech during a meeting into text in real time, extract important information, and automatically generate minutes of the meeting. [Brief description of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., voice and / or text). Display 40A displays visible information such as text and images according to instructions from processor 46. Speaker 40B outputs voice according to instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Also, the data processing device 12 may be a server device, or may be a terminal device (e.g., a mobile phone, a robot, a home appliance, etc.) owned by a user. Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) The AI ​​system according to the embodiment of the present invention is a system that recognizes voices during a meeting using natural language processing, and generates minutes and summaries. This system analyzes the contents of statements made during a meeting in real time, and automatically extracts important decisions and homework items. This allows the next action after the meeting to be taken quickly. Furthermore, during the meeting, the AI ​​judges the progress of the meeting and assists in leading the meeting to the goal. This improves the performance of the meeting. For example, the voice during the meeting is recognized in real time and converted into text data. At this time, the contents of the statement are analyzed using natural language processing technology. For example, if a speaker says, "I will prepare materials by the next meeting," the AI ​​analyzes this statement and extracts it as a homework item. Next, minutes are automatically generated based on the extracted decisions and homework items. The minutes include the date and time of the meeting, participants, agenda, statement content, decisions, homework items, etc. This eliminates the need to manually create minutes after the meeting. Furthermore, during the meeting, the AI ​​judges the progress of the meeting and assists in leading the meeting to the goal. For example, if a meeting is about to exceed the scheduled time, the AI ​​will assist by saying, "Time is running out, so let's move on to the next agenda item." This allows the meeting to proceed smoothly and efficiently. This system improves meeting performance and enables next actions to be taken quickly. For example, homework can be started immediately after the meeting, and necessary preparations can be made before the next meeting. In addition, smoother meeting progress allows all participants to proceed with discussions efficiently. This allows the AI ​​system to recognize voices during the meeting in real time and generate minutes and summaries, allowing next actions to be taken quickly after the meeting. In addition, the performance of the meeting is improved by determining the progress of the meeting and assisting in leading it to the goal.

[0029] The AI ​​system according to the embodiment includes a recognition unit, an analysis unit, a generation unit, a judgment unit, and an assistance unit. The recognition unit converts voices during a meeting into text data in real time. The recognition unit instantly converts statements during a meeting into text data, for example, using a voice recognition technology. For example, the recognition unit converts the contents of statements into text data in real time using voice recognition software. The recognition unit can also use voice recognition technology to identify the voice of a speaker and convert the contents of statements into text data individually. For example, the recognition unit can use voice recognition technology to generate different text data for each speaker. Furthermore, the recognition unit can also use voice recognition technology to filter background sounds during a meeting and recognize only the voice of the speaker. For example, the recognition unit uses noise canceling technology to remove noise in the meeting room and clearly recognize only the voice of the speaker. The analysis unit uses natural language processing technology to analyze the text data converted by the recognition unit and extract important decisions and homework items. The analysis unit uses, for example, morphological analysis technology to analyze the contents of statements and extract important keywords. For example, the analysis unit uses a morphological analysis technique to extract important keywords from the utterance content and classifies them according to importance. The analysis unit can also use a grammar analysis technique to analyze the grammatical structure of the utterance content and extract important information. For example, the analysis unit uses a grammar analysis technique to analyze the grammatical structure of the utterance content and extract important information. The analysis unit can also use a semantic analysis technique to analyze the meaning of the utterance content and extract important information. For example, the analysis unit uses a semantic analysis technique to analyze the meaning of the utterance content and extract important information. The generation unit automatically generates minutes based on the information extracted by the analysis unit. The generation unit automatically generates minutes using, for example, a template. For example, the generation unit automatically generates minutes including the date and time of the meeting, participants, agenda, utterance content, decisions, homework items, etc., using a template. The generation unit can also automatically generate minutes using a generation algorithm. For example, the generation unit automatically generates minutes based on the extracted information using a generation algorithm. Furthermore, when generating the minutes, the generation unit can display the content of each speaker's comments in different colors.For example, the generation unit displays the speech content of each speaker in a different color to visually distinguish them. The judgment unit monitors the progress of the conference in real time and judges the progress. The judgment unit monitors the progress in real time, for example, and identifies delays in progress and problems. For example, the judgment unit monitors the progress in real time and identifies delays in progress and problems. The judgment unit can also analyze the progress and propose the timing of moving to the next agenda. For example, the judgment unit analyzes the progress and proposes the timing of moving to the next agenda. The judgment unit can also analyze the progress and visualize the speech frequency and speech time of the participants. For example, the judgment unit analyzes the progress and visualizes the speech frequency and speech time of the participants. The assist unit assists the progress of the conference based on the judgment of the judgment unit. The assist unit, for example, sends a reminder to encourage moving to the next agenda. For example, the assist unit sends a reminder to encourage moving to the next agenda. The assist unit can also propose the next agenda and assist the progress of the conference. For example, the assist unit suggests the next agenda item and assists in the progress of the meeting. Furthermore, when assisting in the progress of the meeting, the assist unit can also select the optimal assist method by referring to past meeting data. For example, the assist unit refers to past meeting data and selects the optimal assist method. As a result, the AI ​​system according to the embodiment recognizes voices during the meeting in real time and generates minutes and summaries, thereby enabling the next action after the meeting to be taken quickly. Furthermore, the performance of the meeting is improved by determining the progress of the meeting and assisting in leading it to the goal.

[0030] The recognition unit converts the voice during the meeting into text data in real time. The recognition unit, for example, uses a voice recognition technology to instantly convert remarks during the meeting into text data. Specifically, the voice recognition software converts the voice signal into digital data, and analyzes the data to convert it into text. The voice recognition technology includes an acoustic model and a language model, where the acoustic model captures the characteristics of the voice signal, and the language model selects the most appropriate words based on the context. For example, the recognition unit uses the voice recognition software to convert the remarks into text data in real time. The recognition unit can also use the voice recognition technology to identify the voice of the speaker and convert the remarks into text data individually. For example, the recognition unit uses the voice recognition technology to generate different text data for each speaker. This is done by using a speaker identification technology to learn the voice characteristics of each speaker and identify the speaker. Furthermore, the recognition unit can also use the voice recognition technology to filter background sounds during the meeting and recognize only the speaker's voice. For example, the recognition unit can use a noise canceling technology to remove noise in the meeting room and clearly recognize only the speaker's voice. Noise canceling technology detects ambient noise in real time and generates an inverse phase sound to cancel out the noise. This allows the recognition unit to convert the voices in a meeting into text data with high accuracy and clearly record the content of each speaker's speech.

[0031] The analysis unit uses natural language processing technology to analyze the text data converted by the recognition unit and extracts important decisions and homework items. The analysis unit uses, for example, morphological analysis technology to analyze the content of the statement and extract important keywords. The morphological analysis technology divides the text into words and analyzes the parts of speech and meanings of each word. For example, the analysis unit uses morphological analysis technology to extract important keywords from the content of the statement and classifies them according to importance. The analysis unit can also use grammatical analysis technology to analyze the grammatical structure of the content of the statement and extract important information. The grammatical analysis technology analyzes the grammatical structure of the text and clarifies the relationship between the subject, predicate, object, and the like. For example, the analysis unit uses grammatical analysis technology to analyze the grammatical structure of the content of the statement and extract important information. The analysis unit can also use semantic analysis technology to analyze the meaning of the content of the statement and extract important information. The semantic analysis technology understands the meaning of the text and identifies important information based on the context. For example, the analysis unit uses semantic analysis technology to analyze the meaning of what is said and extract important information. This allows the analysis unit to analyze what is said during a meeting from multiple angles and accurately extract important decisions and homework items. Furthermore, the analysis unit can comprehensively evaluate the progress of the meeting and the content of the participants' comments based on the extracted information and propose the next action.

[0032] The generating unit automatically generates the minutes based on the information extracted by the analyzing unit. The generating unit automatically generates the minutes using, for example, a template. The template includes items such as the date and time of the meeting, participants, agenda, remarks, decisions, and homework items, and the minutes are automatically generated by embedding the extracted information in these items. For example, the generating unit automatically generates the minutes including the date and time of the meeting, participants, agenda, remarks, decisions, and homework items using a template. The generating unit can also automatically generate the minutes using a generating algorithm. The generating algorithm generates natural sentences based on the extracted information and creates the minutes. For example, the generating unit automatically generates the minutes based on the extracted information using the generating algorithm. Furthermore, the generating unit can display the remarks of each speaker in different colors when generating the minutes. For example, the generating unit displays the remarks of each speaker in different colors to visually distinguish them. In this way, the generation unit can visually display the contents of the meeting in an easy-to-understand manner, making it easy for participants to understand the minutes. The generation unit can also store the generated minutes on the cloud, making it possible for participants to access them at any time. In this way, the generation unit can accurately and quickly record the contents of the meeting, making it easy for participants to check the minutes later.

[0033] The judgment unit monitors the progress of the conference in real time and judges the progress. For example, the judgment unit monitors the progress in real time and identifies delays in progress and problems. Specifically, the judgment unit monitors the progress of the conference and records the progress time of each agenda and the speaking time of the speaker. For example, the judgment unit monitors the progress in real time and identifies delays in progress and problems. The judgment unit can also analyze the progress and propose the timing of moving to the next agenda. For example, the judgment unit analyzes the progress and proposes the timing of moving to the next agenda. Furthermore, the judgment unit can analyze the progress and visualize the speech frequency and speaking time of the participants. For example, the judgment unit analyzes the progress and visualizes the speech frequency and speaking time of the participants. In this way, the judgment unit can comprehensively evaluate the progress of the conference and move to the next agenda at an appropriate timing. The judgment unit can also adjust the speaking balance of the participants based on the progress of the conference so that everyone can speak equally. For example, if a specific participant speaks for a long time, the judgment unit encourages other participants to provide an opportunity to speak as well. This allows the decision-making unit to ensure that the meeting proceeds smoothly and creates an environment in which everyone can actively participate.

[0034] The assisting unit assists the progress of the conference based on the judgment of the judging unit. The assisting unit, for example, sends a reminder to prompt the transition to the next agenda item. Specifically, the assisting unit monitors the progress of the conference and sends a reminder at the timing of transition to the next agenda item. For example, the assisting unit sends a reminder to prompt the transition to the next agenda item. The assisting unit can also propose the next agenda item and assist the progress of the conference. For example, the assisting unit proposes the next agenda item and assists the progress of the conference. Furthermore, when assisting the progress of the conference, the assisting unit can also select an optimal assisting method by referring to past conference data. For example, the assisting unit refers to past conference data and selects an optimal assisting method. This allows the assisting unit to smoothly progress the conference and efficiently progress it. The assisting unit can also provide appropriate feedback to the participants to assist the progress of the conference. For example, the assisting unit provides feedback to the participants on the timing and content of remarks to improve the quality of the conference. This allows the assisting unit to effectively assist the progress of the conference and create an environment in which all participants can actively participate.

[0035] The recognition unit can convert the voice during the meeting into text data in real time. The recognition unit, for example, uses voice recognition technology to instantly convert remarks during the meeting into text data. For example, the recognition unit uses voice recognition software to convert the remarks into text data in real time. The recognition unit can also use voice recognition technology to identify the voice of the speaker and convert the remarks into text data individually. For example, the recognition unit uses voice recognition technology to generate different text data for each speaker. Furthermore, the recognition unit can also use voice recognition technology to filter background sounds during the meeting and recognize only the speaker's voice. For example, the recognition unit uses noise canceling technology to remove noise in the meeting room and clearly recognize only the speaker's voice. This allows the remarks to be instantly recorded by converting the voice during the meeting into text data in real time. Part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can use a generating AI to convert the voice during the meeting into text data in real time. Examples of the generation AI include, but are not limited to, a text generation AI (e.g., LLM) and a voice recognition AI.

[0036] The analysis unit can analyze the content of the statement using natural language processing technology and extract important decisions and homework items. The analysis unit can analyze the content of the statement using, for example, morphological analysis technology and extract important keywords. For example, the analysis unit can use morphological analysis technology to extract important keywords from the content of the statement and classify them according to importance. The analysis unit can also use grammatical analysis technology to analyze the grammatical structure of the content of the statement and extract important information. For example, the analysis unit can use grammatical analysis technology to analyze the grammatical structure of the content of the statement and extract important information. The analysis unit can also use semantic analysis technology to analyze the meaning of the content of the statement and extract important information. For example, the analysis unit can use semantic analysis technology to analyze the meaning of the content of the statement and extract important information. As a result, important information can be accurately extracted from the content of the statement by using natural language processing technology. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can use a generation AI to analyze the content of the statement. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and natural language processing AI.

[0037] The generating unit can automatically generate minutes based on the extracted information. The generating unit automatically generates minutes using, for example, a template. For example, the generating unit automatically generates minutes including the date and time of the meeting, participants, agenda, remarks, decisions, homework, etc., using a template. The generating unit can also automatically generate minutes using a generation algorithm. For example, the generating unit automatically generates minutes based on the extracted information using a generation algorithm. Furthermore, the generating unit can display the remarks of each speaker in different colors when generating the minutes. For example, the generating unit displays the remarks of each speaker in different colors to visually distinguish them. This allows the minutes to be automatically generated based on the extracted information, thereby reducing the effort required after the meeting. Some or all of the above-mentioned processing in the generating unit may be performed using, for example, AI, or may be performed without using AI. For example, the generating unit can use a generating AI to automatically generate the minutes. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and natural language processing AI.

[0038] The judgment unit can monitor the progress of the conference in real time and judge the progress. The judgment unit, for example, monitors the progress in real time and identifies delays and problems in the progress. For example, the judgment unit monitors the progress in real time and identifies delays and problems in the progress. The judgment unit can also analyze the progress and propose the timing of moving to the next agenda. For example, the judgment unit analyzes the progress and proposes the timing of moving to the next agenda. Furthermore, the judgment unit can analyze the progress and visualize the speech frequency and speech time of the participants. For example, the judgment unit analyzes the progress and visualizes the speech frequency and speech time of the participants. In this way, the progress of the conference is monitored in real time and the progress is judged, thereby making the progress of the conference smooth. A part or all of the above-mentioned processing in the judgment unit may be performed, for example, using AI, or may be performed without using AI. For example, the judgment unit can use a generation AI to monitor the progress of the conference in real time and judge the progress. The generation AI is, for example, a text generation AI (for example, LLM) or a real-time analysis AI, but is not limited to such examples.

[0039] The assist unit can assist the progress of the conference based on the judgment of the judgment unit. The assist unit, for example, sends a reminder to encourage the transition to the next agenda item. For example, the assist unit sends a reminder to encourage the transition to the next agenda item. The assist unit can also propose the next agenda item and assist the progress of the conference. For example, the assist unit proposes the next agenda item and assists the progress of the conference. Furthermore, when assisting the progress of the conference, the assist unit can also select an optimal assist method by referring to past conference data. For example, the assist unit refers to past conference data and selects an optimal assist method. As a result, the progress of the conference is assisted based on the judgment of the judgment unit, thereby improving the performance of the conference. A part or all of the above-mentioned processing in the assist unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the assist unit can use a generation AI to assist the progress of the conference. The generation AI is, for example, a text generation AI (for example, LLM) or a real-time analysis AI, but is not limited to such examples.

[0040] The recognition unit can filter background sounds during a meeting and recognize only the voice of the speaker. For example, the recognition unit removes noise in the meeting room and clearly recognizes only the voice of the speaker. For example, the recognition unit removes noise in the meeting room and clearly recognizes only the voice of the speaker. The recognition unit can also filter the sound of a projector or air conditioner during a meeting and prioritize the voice of the speaker. For example, the recognition unit filters the sound of a projector or air conditioner during a meeting and prioritizes the voice of the speaker. Furthermore, the recognition unit can also individually recognize the voices of multiple speakers during a meeting and eliminate background sounds. For example, the recognition unit individually recognizes the voices of multiple speakers during a meeting and eliminates background sounds. As a result, the voice of the speaker can be clearly recognized by filtering the background sounds. A part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can use a generation AI to filter background sounds during a meeting and recognize only the voice of the speaker. Examples of the generation AI include, but are not limited to, a voice recognition AI or a noise canceling AI.

[0041] The recognition unit can analyze the tone and speed of the speaker's voice, determine the importance of the speech, and improve the recognition accuracy. For example, when the speaker's voice becomes louder, the recognition unit increases the recognition accuracy as an important speech. For example, when the speaker's voice becomes louder, the recognition unit increases the recognition accuracy as an important speech. In addition, when the speaker's voice becomes lower, the recognition unit can determine that the importance is low and adjust the recognition accuracy. For example, when the speaker's voice becomes lower, the recognition unit can determine that the importance is low and adjust the recognition accuracy. Furthermore, when the speaker's speaking speed is fast, the recognition unit can determine that important information is included and improve the recognition accuracy. For example, when the speaker's speaking speed is fast, the recognition unit can determine that important information is included and improve the recognition accuracy. In this way, important speech can be accurately recognized by analyzing the tone and speed of the speaker's voice. Part or all of the above-mentioned processing in the recognition unit may be performed, for example, using AI, or may be performed without using AI. For example, the recognition unit can use a generation AI to analyze the tone and speed of the speaker's voice, determine the importance of the speech, and improve the recognition accuracy. Examples of the generation AI include, but are not limited to, speech recognition AI and natural language processing AI.

[0042] The recognition unit can simultaneously recognize multiple speakers in a conference and separate the text data for each speaker. The recognition unit, for example, recognizes the voices of multiple speakers simultaneously and separates them as individual text data. For example, the recognition unit recognizes the voices of multiple speakers simultaneously and separates them as individual text data. The recognition unit can also display text data for each speaker in a different color to visually distinguish them. For example, the recognition unit displays text data for each speaker in a different color to visually distinguish them. Furthermore, the recognition unit can automatically recognize the name of the speaker and tag the text data. For example, the recognition unit automatically recognizes the name of the speaker and tags the text data. This allows multiple speakers to be recognized simultaneously and the text data to be separated for each speaker, allowing the content of the speech to be clearly recorded. A part or all of the above-mentioned processing in the recognition unit may be performed, for example, using AI, or may be performed without using AI. For example, the recognition unit can use a generation AI to simultaneously recognize multiple speakers in a conference and separate text data for each speaker. Examples of the generation AI include, but are not limited to, speech recognition AI and natural language processing AI.

[0043] The recognition unit can translate the voice during the conference in real time to accommodate participants of different languages. The recognition unit, for example, translates remarks during the conference in real time to generate text data in different languages. For example, the recognition unit translates remarks during the conference in real time to generate text data in different languages. The recognition unit can also display the translated text data on the participant's device in real time. For example, the recognition unit displays the translated text data on the participant's device in real time. Furthermore, the recognition unit can automatically translate and share with everyone when a participant of a different language speaks. For example, the recognition unit automatically translates and shares with everyone when a participant of a different language speaks. This allows participants of different languages ​​to communicate smoothly by translating the voice during the conference in real time. Part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can use a generation AI to translate the voice during the conference in real time to accommodate participants of different languages. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and translation AI.

[0044] The analysis unit can analyze the content of the speech based on the context and refer to related past minutes and materials. The analysis unit, for example, analyzes the content of the speech based on the context and automatically refers to related past minutes. For example, the analysis unit analyzes the content of the speech based on the context and automatically refers to related past minutes. The analysis unit can also automatically search and refer to materials related to the content of the speech. For example, the analysis unit automatically searches and refers to materials related to the content of the speech. Furthermore, the analysis unit can also analyze the context of the content of the speech and provide related information. For example, the analysis unit analyzes the context of the content of the speech and provides related information. As a result, the content of the speech can be analyzed based on the context, and related information can be accurately provided. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can use a generation AI to analyze the content of the speech based on the context and refer to related past minutes and materials. The generation AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples.

[0045] The analysis unit can extract keywords from the remarks and classify them according to importance. The analysis unit, for example, extracts important keywords from the remarks and classifies them according to importance. For example, the analysis unit extracts important keywords from the remarks and classifies them according to importance. The analysis unit can also automatically tag and classify the extracted keywords. For example, the analysis unit automatically tags and classifies the extracted keywords. The analysis unit can also analyze the importance of the keywords and prioritize and classify them. For example, the analysis unit analyzes the importance of the keywords and prioritizes and classifies them. As a result, important information can be efficiently organized by extracting keywords from the remarks and classifying them according to importance. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the analysis unit can use a generation AI to extract keywords from the remarks and classify them according to importance. The generation AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples.

[0046] The analysis unit can analyze the content of the statement and automatically refer to related laws and regulations and industry standards. The analysis unit, for example, analyzes the content of the statement and automatically refers to related laws and regulations. For example, the analysis unit analyzes the content of the statement and automatically refers to related laws and regulations. The analysis unit can also automatically search and refer to industry standards related to the content of the statement. For example, the analysis unit automatically searches and refers to industry standards related to the content of the statement. Furthermore, the analysis unit can also analyze the context of the content of the statement and provide related information. For example, the analysis unit analyzes the context of the content of the statement and provides related information. As a result, appropriate information can be provided by analyzing the content of the statement and referring to related laws and regulations and industry standards. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can use a generation AI to analyze the content of the statement and refer to related laws and regulations and industry standards. The generation AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples.

[0047] The analysis unit can analyze the content of the remark and provide related market data and competitive information. The analysis unit can, for example, analyze the content of the remark and automatically provide related market data. For example, the analysis unit can analyze the content of the remark and automatically provide related market data. The analysis unit can also automatically search and provide competitive information related to the content of the remark. For example, the analysis unit can automatically search and provide competitive information related to the content of the remark. Furthermore, the analysis unit can also analyze the context of the content of the remark and provide related information. For example, the analysis unit can analyze the context of the content of the remark and provide related information. This improves the quality of the meeting by analyzing the content of the remark and providing related market data and competitive information. A part or all of the above-mentioned processing in the analysis unit can be performed, for example, using AI, or can be performed without using AI. For example, the analysis unit can use a generation AI to analyze the content of the remark and provide related market data and competitive information. The generation AI can be, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples.

[0048] The generating unit can display the remarks of each speaker in different colors when generating the minutes. The generating unit, for example, displays the remarks of each speaker in different colors to visually distinguish them. For example, the generating unit displays the remarks of each speaker in different colors to visually distinguish them. The generating unit can also automatically recognize the name of the speaker and display them in different colors. For example, the generating unit can automatically recognize the name of the speaker and display them in different colors. Furthermore, the generating unit can color-code the remarks of each speaker to improve the visibility of the minutes. For example, the generating unit color-codes the remarks of each speaker to improve the visibility of the minutes. As a result, the visibility of the minutes is improved by color-coding the remarks of each speaker. A part or all of the above-mentioned processing in the generating unit may be performed, for example, using AI, or may be performed without using AI. For example, the generating unit can use a generating AI to color-code the remarks of each speaker when generating the minutes. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and natural language processing AI.

[0049] The generating unit can highlight important decisions and homework items when generating the minutes. The generating unit, for example, highlights important decisions in bold or in a color. For example, the generating unit highlights important decisions in bold or in a color. The generating unit can also highlight homework items in a list format. For example, the generating unit highlights homework items in a list format. The generating unit can also automatically extract and highlight decisions and homework items. For example, the generating unit automatically extracts and highlights decisions and homework items. As a result, important information in the minutes can be seen at a glance by highlighting important decisions and homework items. A part or all of the above-mentioned processing in the generating unit may be performed, for example, using AI, or may be performed without using AI. For example, the generating unit can use a generating AI to highlight important decisions and homework items when generating the minutes. The generating AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples.

[0050] The generating unit can include the progress of the meeting and the timeline when generating the minutes. For example, the generating unit displays the progress of the meeting in a timeline format and includes it in the minutes. For example, the generating unit displays the progress of the meeting in a timeline format and includes it in the minutes. The generating unit can also display the progress of each agenda of the meeting in a timeline and include it in the minutes. For example, the generating unit displays the progress of each agenda of the meeting in a timeline and includes it in the minutes. Furthermore, the generating unit can display the start time and end time of the meeting in a timeline and include it in the minutes. For example, the generating unit displays the start time and end time of the meeting in a timeline and includes it in the minutes. As a result, by including the progress of the meeting and the timeline, the contents of the minutes become more detailed. A part or all of the above-mentioned processing in the generating unit may be performed using, for example, AI, or may be performed without using AI. For example, the generating unit can use a generating AI to include the progress of the meeting and the timeline when generating the minutes. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and natural language processing AI.

[0051] The generating unit can automatically attach related materials and links when generating the minutes. The generating unit, for example, automatically attaches materials referenced during the meeting to the minutes. For example, the generating unit automatically attaches materials referenced during the meeting to the minutes. The generating unit can also automatically attach links related to the contents of remarks to the minutes. For example, the generating unit automatically attaches links related to the contents of remarks to the minutes. The generating unit can also automatically attach presentation materials used during the meeting to the minutes. For example, the generating unit automatically attaches presentation materials used during the meeting to the minutes. As a result, the information in the minutes is enriched by automatically attaching related materials and links. A part or all of the above-mentioned processing in the generating unit may be performed, for example, using AI, or may be performed without using AI. For example, the generating unit can use a generating AI to automatically attach related materials and links when generating the minutes. The generating AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples.

[0052] The judgment unit can analyze the progress of the conference in real time and identify delays in progress and problems. The judgment unit, for example, analyzes the progress of the conference in real time and identifies delays in progress. For example, the judgment unit analyzes the progress of the conference in real time and identifies delays in progress. The judgment unit can also analyze the progress of the conference and automatically identify problems. For example, the judgment unit analyzes the progress of the conference and automatically identifies problems. Furthermore, the judgment unit can monitor the progress of the conference in real time and identify delays in progress and problems. For example, the judgment unit monitors the progress of the conference in real time and identifies delays in progress and problems. As a result, delays in progress and problems can be quickly identified by analyzing the progress of the conference in real time. A part or all of the above-mentioned processing in the judgment unit may be performed using, for example, AI, or may be performed without using AI. For example, the judgment unit can use a generation AI to analyze the progress of the conference in real time and identify delays in progress and problems. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and real-time analysis AI.

[0053] The judgment unit can analyze the progress of the meeting and propose the timing of transition to the next agenda. The judgment unit, for example, analyzes the progress of the meeting and proposes the timing of transition to the next agenda. For example, the judgment unit analyzes the progress of the meeting and proposes the timing of transition to the next agenda. The judgment unit can also monitor the progress of the meeting in real time and propose an appropriate transition timing. For example, the judgment unit monitors the progress of the meeting in real time and proposes an appropriate transition timing. The judgment unit can also analyze the progress of the meeting and propose the timing of ending the agenda. For example, the judgment unit analyzes the progress of the meeting and proposes the timing of ending the agenda. In this way, an appropriate transition timing can be proposed by analyzing the progress of the meeting. A part or all of the above-mentioned processing in the judgment unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the judgment unit can use a generation AI to analyze the progress of the meeting and propose the timing of transition to the next agenda. The generation AI is, for example, a text generation AI (for example, LLM) or a real-time analysis AI, but is not limited to such examples.

[0054] The judgment unit can analyze the progress of the conference and visualize the speech frequency and speech time of the participants. The judgment unit, for example, analyzes the progress of the conference and displays the speech frequency of the participants in a graph. For example, the judgment unit analyzes the progress of the conference and displays the speech frequency of the participants in a graph. The judgment unit can also analyze the progress of the conference and display the speech time of the participants in a timeline. For example, the judgment unit analyzes the progress of the conference and displays the speech time of the participants in a timeline. The judgment unit can also analyze the progress of the conference and visualize the speech frequency and speech time of each participant. For example, the judgment unit analyzes the progress of the conference and visualizes the speech frequency and speech time of each participant. This makes it possible to more effectively manage the progress of the conference by analyzing the progress of the conference and visualizing the speech frequency and speech time of the participants. A part or all of the above-mentioned processing in the judgment unit may be performed, for example, using AI, or may be performed without using AI. For example, the judgment unit can use a generation AI to analyze the progress of the conference and visualize the speech frequency and speech time of the participants. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and real-time analysis AI.

[0055] The judgment unit can analyze the progress of the conference and evaluate the interest and concentration of the participants. The judgment unit, for example, analyzes the progress of the conference and evaluates the interest of the participants. For example, the judgment unit analyzes the progress of the conference and evaluates the interest of the participants. The judgment unit can also analyze the progress of the conference and evaluate the concentration of the participants. For example, the judgment unit analyzes the progress of the conference and evaluates the concentration of the participants. The judgment unit can also analyze the progress of the conference and evaluate the interest and concentration of the participants. For example, the judgment unit analyzes the progress of the conference and evaluates the interest and concentration of the participants. In this way, the quality of the conference is improved by analyzing the progress of the conference and evaluating the interest and concentration of the participants. A part or all of the above-mentioned processing in the judgment unit may be performed, for example, using AI, or may be performed without using AI. For example, the judgment unit can use a generation AI to analyze the progress of the conference and evaluate the interest and concentration of the participants. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and real-time analysis AI.

[0056] When assisting the progress of a conference, the assisting unit can select an optimal assisting method by referring to past conference data. For example, the assisting unit selects an optimal assisting method by referring to past conference data. For example, the assisting unit selects an optimal assisting method by referring to past conference data. The assisting unit can also analyze past conference data and select an assisting method that was effective in a similar conference. For example, the assisting unit analyzes past conference data and selects an assisting method that was effective in a similar conference. Furthermore, the assisting unit can also propose an optimal assisting method based on past conference data. For example, the assisting unit proposes an optimal assisting method based on past conference data. In this way, the optimal assisting method can be selected by referring to past conference data. A part or all of the above-mentioned processing in the assisting unit may be performed, for example, using AI, or may be performed without using AI. For example, the assisting unit can use a generating AI to select an optimal assisting method by referring to past conference data when assisting the progress of a conference. The generating AI is, for example, a text generating AI (for example, LLM) or a data analysis AI, but is not limited to such examples.

[0057] The assist unit can customize the assist contents based on the roles and expertise of the participants when assisting the progress of the conference. The assist unit provides appropriate assist contents based on, for example, the roles of the participants. For example, the assist unit provides appropriate assist contents based on the roles of the participants. The assist unit can also customize the assist contents based on the expertise of the participants. For example, the assist unit customizes the assist contents based on the expertise of the participants. Furthermore, the assist unit can also provide optimal assist contents by considering the roles and expertise of the participants. For example, the assist unit provides optimal assist contents by considering the roles and expertise of the participants. As a result, more effective assistance can be achieved by customizing the assist contents based on the roles and expertise of the participants. A part or all of the above-mentioned processing in the assist unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the assist unit can use a generation AI to customize the assist contents based on the roles and expertise of the participants when assisting the progress of the conference. The generation AI is, for example, a text generation AI (for example, LLM) or a data analysis AI, but is not limited to such examples.

[0058] The assist unit can take into consideration the schedule and time constraints of the participants when assisting the progress of the conference. The assist unit, for example, takes into consideration the schedule of the participants and provides optimal assist timing. For example, the assist unit takes into consideration the schedule of the participants and provides optimal assist timing. The assist unit can also provide efficient assistance by taking into consideration the time constraints of the participants. For example, the assist unit provides efficient assistance by taking into consideration the time constraints of the participants. Furthermore, the assist unit can also propose an optimal assist method based on the schedule and time constraints of the participants. For example, the assist unit proposes an optimal assist method based on the schedule and time constraints of the participants. This enables efficient assistance by taking into consideration the schedule and time constraints of the participants. A part or all of the above-mentioned processing in the assist unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the assist unit can use a generation AI to take into consideration the schedule and time constraints of the participants when assisting the progress of the conference. The generation AI is, for example, a text generation AI (for example, LLM) or a data analysis AI, but is not limited to such examples.

[0059] The assist unit can provide related external data and information when assisting the progress of the conference. For example, the assist unit provides related external data when assisting the progress of the conference. For example, the assist unit provides related external data when assisting the progress of the conference. In addition, the assist unit can automatically provide related information when assisting the progress of the conference. For example, the assist unit automatically provides related information when assisting the progress of the conference. Furthermore, the assist unit can provide optimal assistance based on external data and information when assisting the progress of the conference. For example, the assist unit provides optimal assistance based on external data and information when assisting the progress of the conference. As a result, by providing related external data and information, the progress of the conference can be more effectively assisted. A part or all of the above-mentioned processing in the assist unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the assist unit can use a generation AI to provide related external data and information when assisting the progress of the conference. The generation AI is, for example, a text generation AI (for example, LLM) or a data analysis AI, but is not limited to such examples.

[0060] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0061] The AI ​​system can further include a translation unit that translates remarks made during a meeting in real time and accommodates participants who speak different languages. The translation unit, for example, translates remarks made during a meeting in real time and generates text data in different languages. The translation unit can also display the translated text data on the participants' devices in real time. Furthermore, the translation unit can automatically translate remarks made by participants who speak different languages ​​and share them with everyone. This allows participants who speak different languages ​​to communicate smoothly.

[0062] The recognition unit can analyze the tone and speed of a speaker's voice during a meeting, determine the importance of the speech, and improve the recognition accuracy. For example, if the speaker's voice gets louder, the recognition accuracy can be increased as the speech is important. Also, if the speaker's voice gets quieter, the recognition accuracy can be adjusted as the speech is less important. Furthermore, if the speaker speaks quickly, the recognition accuracy can be improved as the speech contains important information. In this way, important speech can be accurately recognized by analyzing the tone and speed of the speaker's voice.

[0063] When generating the minutes, the generation unit can display the remarks of each speaker in a different color. For example, the remarks of each speaker can be displayed in a different color to visually distinguish them. The name of the speaker can also be automatically recognized and displayed in a different color. Furthermore, the remarks of each speaker can be color-coded to improve the visibility of the minutes. Thus, by color-coding the remarks of each speaker, the visibility of the minutes can be improved.

[0064] The determination unit can analyze the progress of the conference and visualize the speech frequency and speech duration of each participant. For example, the progress of the conference can be analyzed and the speech frequency of each participant can be displayed in a graph. The progress of the conference can also be analyzed and the speech duration of each participant can be displayed in a timeline. Furthermore, the progress of the conference can also be analyzed and the speech frequency and speech duration of each participant can be visualized. In this way, by analyzing the progress of the conference and visualizing the speech frequency and speech duration of each participant, the progress of the conference can be managed more effectively.

[0065] The recognition unit can translate speech during a meeting in real time to accommodate participants who speak different languages. For example, statements made during a meeting can be translated in real time to generate text data in different languages. The translated text data can also be displayed in real time on the participants' devices. Furthermore, when a participant who speaks a different language speaks, the text can be automatically translated and shared with everyone. This allows participants who speak different languages ​​to communicate smoothly by translating speech during a meeting in real time.

[0066] The analysis unit can analyze the content of a statement based on the context and refer to related past minutes and materials. For example, the analysis unit can analyze the content of a statement based on the context and automatically refer to related past minutes. It can also automatically search for and refer to materials related to the content of a statement. It can also analyze the context of the content of a statement and provide related information. This makes it possible to accurately provide related information by analyzing the content of a statement based on the context.

[0067] The process flow of the first embodiment will be briefly described below.

[0068] Step 1: The recognition unit converts the voices being spoken during the meeting into text data in real time. The recognition unit uses voice recognition technology to instantly convert what is being said during the meeting into text data. For example, it is possible to use voice recognition software to convert what is being said into text data in real time, and to identify the speaker's voice and convert it into text data individually. Noise canceling technology is also used to remove noise in the meeting room, clearly recognizing only the speaker's voice. Step 2: The analysis unit uses natural language processing technology to analyze the text data converted by the recognition unit and extract important decisions and homework items. For example, it uses morphological analysis technology to analyze the content of the speech and extracts important keywords, and uses grammatical analysis technology and semantic analysis technology to analyze the grammatical structure and meaning of the speech and extract important information. Step 3: The generator automatically generates minutes based on the information extracted by the analyzer. Using templates and generation algorithms, the generator automatically generates minutes including the meeting date and time, participants, agenda, remarks, decisions, homework items, etc. Furthermore, it can also display the remarks of each speaker in a different color. Step 4: The judgment unit monitors the progress of the meeting in real time and judges its progress. For example, it identifies delays in progress and problems and suggests the timing to move on to the next agenda item. It can also visualize the frequency and duration of participants' comments. Step 5: The assisting unit assists the progress of the meeting based on the judgment of the judging unit. For example, it can send reminders to encourage moving on to the next agenda item and suggest the next agenda item. It can also select the optimal assistance method by referring to past meeting data.

[0069] (Example 2) The AI ​​system according to the embodiment of the present invention is a system that recognizes voices during a meeting using natural language processing, and generates minutes and summaries. This system analyzes the contents of statements made during a meeting in real time, and automatically extracts important decisions and homework items. This allows the next action after the meeting to be taken quickly. Furthermore, during the meeting, the AI ​​judges the progress of the meeting and assists in leading the meeting to the goal. This improves the performance of the meeting. For example, the voice during the meeting is recognized in real time and converted into text data. At this time, the contents of the statement are analyzed using natural language processing technology. For example, if a speaker says, "I will prepare materials by the next meeting," the AI ​​analyzes this statement and extracts it as a homework item. Next, minutes are automatically generated based on the extracted decisions and homework items. The minutes include the date and time of the meeting, participants, agenda, statement content, decisions, homework items, etc. This eliminates the need to manually create minutes after the meeting. Furthermore, during the meeting, the AI ​​judges the progress of the meeting and assists in leading the meeting to the goal. For example, if a meeting is about to exceed the scheduled time, the AI ​​will assist by saying, "Time is running out, so let's move on to the next agenda item." This allows the meeting to proceed smoothly and efficiently. This system improves meeting performance and enables next actions to be taken quickly. For example, homework can be started immediately after the meeting, and necessary preparations can be made before the next meeting. In addition, smoother meeting progress allows all participants to proceed with discussions efficiently. This allows the AI ​​system to recognize voices during the meeting in real time and generate minutes and summaries, allowing next actions to be taken quickly after the meeting. In addition, the performance of the meeting is improved by determining the progress of the meeting and assisting in leading it to the goal.

[0070] The AI ​​system according to the embodiment includes a recognition unit, an analysis unit, a generation unit, a judgment unit, and an assistance unit. The recognition unit converts voices during a meeting into text data in real time. The recognition unit instantly converts statements during a meeting into text data, for example, using a voice recognition technology. For example, the recognition unit converts the contents of statements into text data in real time using voice recognition software. The recognition unit can also use voice recognition technology to identify the voice of a speaker and convert the contents of statements into text data individually. For example, the recognition unit can use voice recognition technology to generate different text data for each speaker. Furthermore, the recognition unit can also use voice recognition technology to filter background sounds during a meeting and recognize only the voice of the speaker. For example, the recognition unit uses noise canceling technology to remove noise in the meeting room and clearly recognize only the voice of the speaker. The analysis unit uses natural language processing technology to analyze the text data converted by the recognition unit and extract important decisions and homework items. The analysis unit uses, for example, morphological analysis technology to analyze the contents of statements and extract important keywords. For example, the analysis unit uses a morphological analysis technique to extract important keywords from the utterance content and classifies them according to importance. The analysis unit can also use a grammar analysis technique to analyze the grammatical structure of the utterance content and extract important information. For example, the analysis unit uses a grammar analysis technique to analyze the grammatical structure of the utterance content and extract important information. The analysis unit can also use a semantic analysis technique to analyze the meaning of the utterance content and extract important information. For example, the analysis unit uses a semantic analysis technique to analyze the meaning of the utterance content and extract important information. The generation unit automatically generates minutes based on the information extracted by the analysis unit. The generation unit automatically generates minutes using, for example, a template. For example, the generation unit automatically generates minutes including the date and time of the meeting, participants, agenda, utterance content, decisions, homework items, etc., using a template. The generation unit can also automatically generate minutes using a generation algorithm. For example, the generation unit automatically generates minutes based on the extracted information using a generation algorithm. Furthermore, when generating the minutes, the generation unit can display the content of each speaker's comments in different colors.For example, the generation unit displays the speech content of each speaker in a different color to visually distinguish them. The judgment unit monitors the progress of the conference in real time and judges the progress. The judgment unit monitors the progress in real time, for example, and identifies delays in progress and problems. For example, the judgment unit monitors the progress in real time and identifies delays in progress and problems. The judgment unit can also analyze the progress and propose the timing of moving to the next agenda. For example, the judgment unit analyzes the progress and proposes the timing of moving to the next agenda. The judgment unit can also analyze the progress and visualize the speech frequency and speech time of the participants. For example, the judgment unit analyzes the progress and visualizes the speech frequency and speech time of the participants. The assist unit assists the progress of the conference based on the judgment of the judgment unit. The assist unit, for example, sends a reminder to encourage moving to the next agenda. For example, the assist unit sends a reminder to encourage moving to the next agenda. The assist unit can also propose the next agenda and assist the progress of the conference. For example, the assist unit suggests the next agenda item and assists in the progress of the meeting. Furthermore, when assisting in the progress of the meeting, the assist unit can also select the optimal assist method by referring to past meeting data. For example, the assist unit refers to past meeting data and selects the optimal assist method. As a result, the AI ​​system according to the embodiment recognizes voices during the meeting in real time and generates minutes and summaries, thereby enabling the next action after the meeting to be taken quickly. Furthermore, the performance of the meeting is improved by determining the progress of the meeting and assisting in leading it to the goal.

[0071] The recognition unit converts the voice during the meeting into text data in real time. The recognition unit, for example, uses a voice recognition technology to instantly convert remarks during the meeting into text data. Specifically, the voice recognition software converts the voice signal into digital data, and analyzes the data to convert it into text. The voice recognition technology includes an acoustic model and a language model, where the acoustic model captures the characteristics of the voice signal, and the language model selects the most appropriate words based on the context. For example, the recognition unit uses the voice recognition software to convert the remarks into text data in real time. The recognition unit can also use the voice recognition technology to identify the voice of the speaker and convert the remarks into text data individually. For example, the recognition unit uses the voice recognition technology to generate different text data for each speaker. This is done by using a speaker identification technology to learn the voice characteristics of each speaker and identify the speaker. Furthermore, the recognition unit can also use the voice recognition technology to filter background sounds during the meeting and recognize only the speaker's voice. For example, the recognition unit can use a noise canceling technology to remove noise in the meeting room and clearly recognize only the speaker's voice. Noise canceling technology detects ambient noise in real time and generates an inverse phase sound to cancel out the noise. This allows the recognition unit to convert the voices in a meeting into text data with high accuracy and clearly record the content of each speaker's speech.

[0072] The analysis unit uses natural language processing technology to analyze the text data converted by the recognition unit and extracts important decisions and homework items. The analysis unit uses, for example, morphological analysis technology to analyze the content of the statement and extract important keywords. The morphological analysis technology divides the text into words and analyzes the parts of speech and meanings of each word. For example, the analysis unit uses morphological analysis technology to extract important keywords from the content of the statement and classifies them according to importance. The analysis unit can also use grammatical analysis technology to analyze the grammatical structure of the content of the statement and extract important information. The grammatical analysis technology analyzes the grammatical structure of the text and clarifies the relationship between the subject, predicate, object, and the like. For example, the analysis unit uses grammatical analysis technology to analyze the grammatical structure of the content of the statement and extract important information. The analysis unit can also use semantic analysis technology to analyze the meaning of the content of the statement and extract important information. The semantic analysis technology understands the meaning of the text and identifies important information based on the context. For example, the analysis unit uses semantic analysis technology to analyze the meaning of what is said and extract important information. This allows the analysis unit to analyze what is said during a meeting from multiple angles and accurately extract important decisions and homework items. Furthermore, the analysis unit can comprehensively evaluate the progress of the meeting and the content of the participants' comments based on the extracted information and propose the next action.

[0073] The generating unit automatically generates the minutes based on the information extracted by the analyzing unit. The generating unit automatically generates the minutes using, for example, a template. The template includes items such as the date and time of the meeting, participants, agenda, remarks, decisions, and homework items, and the minutes are automatically generated by embedding the extracted information in these items. For example, the generating unit automatically generates the minutes including the date and time of the meeting, participants, agenda, remarks, decisions, and homework items using a template. The generating unit can also automatically generate the minutes using a generating algorithm. The generating algorithm generates natural sentences based on the extracted information and creates the minutes. For example, the generating unit automatically generates the minutes based on the extracted information using the generating algorithm. Furthermore, the generating unit can display the remarks of each speaker in different colors when generating the minutes. For example, the generating unit displays the remarks of each speaker in different colors to visually distinguish them. In this way, the generation unit can visually display the contents of the meeting in an easy-to-understand manner, making it easy for participants to understand the minutes. The generation unit can also store the generated minutes on the cloud, making it possible for participants to access them at any time. In this way, the generation unit can accurately and quickly record the contents of the meeting, making it easy for participants to check the minutes later.

[0074] The judgment unit monitors the progress of the conference in real time and judges the progress. For example, the judgment unit monitors the progress in real time and identifies delays in progress and problems. Specifically, the judgment unit monitors the progress of the conference and records the progress time of each agenda and the speaking time of the speaker. For example, the judgment unit monitors the progress in real time and identifies delays in progress and problems. The judgment unit can also analyze the progress and propose the timing of moving to the next agenda. For example, the judgment unit analyzes the progress and proposes the timing of moving to the next agenda. Furthermore, the judgment unit can analyze the progress and visualize the speech frequency and speaking time of the participants. For example, the judgment unit analyzes the progress and visualizes the speech frequency and speaking time of the participants. In this way, the judgment unit can comprehensively evaluate the progress of the conference and move to the next agenda at an appropriate timing. The judgment unit can also adjust the speaking balance of the participants based on the progress of the conference so that everyone can speak equally. For example, if a specific participant speaks for a long time, the judgment unit encourages other participants to provide an opportunity to speak as well. This allows the decision-making unit to ensure that the meeting proceeds smoothly and creates an environment in which everyone can actively participate.

[0075] The assisting unit assists the progress of the conference based on the judgment of the judging unit. The assisting unit, for example, sends a reminder to prompt the transition to the next agenda item. Specifically, the assisting unit monitors the progress of the conference and sends a reminder at the timing of transition to the next agenda item. For example, the assisting unit sends a reminder to prompt the transition to the next agenda item. The assisting unit can also propose the next agenda item and assist the progress of the conference. For example, the assisting unit proposes the next agenda item and assists the progress of the conference. Furthermore, when assisting the progress of the conference, the assisting unit can also select an optimal assisting method by referring to past conference data. For example, the assisting unit refers to past conference data and selects an optimal assisting method. This allows the assisting unit to smoothly progress the conference and efficiently progress it. The assisting unit can also provide appropriate feedback to the participants to assist the progress of the conference. For example, the assisting unit provides feedback to the participants on the timing and content of remarks to improve the quality of the conference. This allows the assisting unit to effectively assist the progress of the conference and create an environment in which all participants can actively participate.

[0076] The recognition unit can convert the voice during the meeting into text data in real time. The recognition unit, for example, uses voice recognition technology to instantly convert remarks during the meeting into text data. For example, the recognition unit uses voice recognition software to convert the remarks into text data in real time. The recognition unit can also use voice recognition technology to identify the voice of the speaker and convert the remarks into text data individually. For example, the recognition unit uses voice recognition technology to generate different text data for each speaker. Furthermore, the recognition unit can also use voice recognition technology to filter background sounds during the meeting and recognize only the speaker's voice. For example, the recognition unit uses noise canceling technology to remove noise in the meeting room and clearly recognize only the speaker's voice. This allows the remarks to be instantly recorded by converting the voice during the meeting into text data in real time. Part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can use a generating AI to convert the voice during the meeting into text data in real time. Examples of the generation AI include, but are not limited to, a text generation AI (e.g., LLM) and a voice recognition AI.

[0077] The analysis unit can analyze the content of the statement using natural language processing technology and extract important decisions and homework items. The analysis unit can analyze the content of the statement using, for example, morphological analysis technology and extract important keywords. For example, the analysis unit can use morphological analysis technology to extract important keywords from the content of the statement and classify them according to importance. The analysis unit can also use grammatical analysis technology to analyze the grammatical structure of the content of the statement and extract important information. For example, the analysis unit can use grammatical analysis technology to analyze the grammatical structure of the content of the statement and extract important information. The analysis unit can also use semantic analysis technology to analyze the meaning of the content of the statement and extract important information. For example, the analysis unit can use semantic analysis technology to analyze the meaning of the content of the statement and extract important information. As a result, important information can be accurately extracted from the content of the statement by using natural language processing technology. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can use a generation AI to analyze the content of the statement. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and natural language processing AI.

[0078] The generating unit can automatically generate minutes based on the extracted information. The generating unit automatically generates minutes using, for example, a template. For example, the generating unit automatically generates minutes including the date and time of the meeting, participants, agenda, remarks, decisions, homework, etc., using a template. The generating unit can also automatically generate minutes using a generation algorithm. For example, the generating unit automatically generates minutes based on the extracted information using a generation algorithm. Furthermore, the generating unit can display the remarks of each speaker in different colors when generating the minutes. For example, the generating unit displays the remarks of each speaker in different colors to visually distinguish them. This allows the minutes to be automatically generated based on the extracted information, thereby reducing the effort required after the meeting. Some or all of the above-mentioned processing in the generating unit may be performed using, for example, AI, or may be performed without using AI. For example, the generating unit can use a generating AI to automatically generate the minutes. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and natural language processing AI.

[0079] The judgment unit can monitor the progress of the conference in real time and judge the progress. The judgment unit, for example, monitors the progress in real time and identifies delays and problems in the progress. For example, the judgment unit monitors the progress in real time and identifies delays and problems in the progress. The judgment unit can also analyze the progress and propose the timing of moving to the next agenda. For example, the judgment unit analyzes the progress and proposes the timing of moving to the next agenda. Furthermore, the judgment unit can analyze the progress and visualize the speech frequency and speech time of the participants. For example, the judgment unit analyzes the progress and visualizes the speech frequency and speech time of the participants. In this way, the progress of the conference is monitored in real time and the progress is judged, thereby making the progress of the conference smooth. A part or all of the above-mentioned processing in the judgment unit may be performed, for example, using AI, or may be performed without using AI. For example, the judgment unit can use a generation AI to monitor the progress of the conference in real time and judge the progress. The generation AI is, for example, a text generation AI (for example, LLM) or a real-time analysis AI, but is not limited to such examples.

[0080] The assist unit can assist the progress of the conference based on the judgment of the judgment unit. The assist unit, for example, sends a reminder to encourage the transition to the next agenda item. For example, the assist unit sends a reminder to encourage the transition to the next agenda item. The assist unit can also propose the next agenda item and assist the progress of the conference. For example, the assist unit proposes the next agenda item and assists the progress of the conference. Furthermore, when assisting the progress of the conference, the assist unit can also select an optimal assist method by referring to past conference data. For example, the assist unit refers to past conference data and selects an optimal assist method. As a result, the progress of the conference is assisted based on the judgment of the judgment unit, thereby improving the performance of the conference. A part or all of the above-mentioned processing in the assist unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the assist unit can use a generation AI to assist the progress of the conference. The generation AI is, for example, a text generation AI (for example, LLM) or a real-time analysis AI, but is not limited to such examples.

[0081] The recognition unit can estimate the user's emotion and improve the accuracy of the voice recognition based on the estimated user's emotion. For example, when the user is nervous, the recognition unit increases the sensitivity of the voice recognition to accurately capture the speech. For example, when the user is nervous, the recognition unit increases the sensitivity of the voice recognition to accurately capture the speech. In addition, when the user is relaxed, the recognition unit can return the sensitivity of the voice recognition to normal to capture a natural conversation. For example, when the user is relaxed, the recognition unit returns the sensitivity of the voice recognition to normal to capture a natural conversation. Furthermore, when the user is in a hurry, the recognition unit can also increase the speed of the voice recognition to quickly convert the voice into text data. For example, when the user is in a hurry, the recognition unit increases the speed of the voice recognition to quickly convert the voice into text data. In this way, the speech can be accurately captured by adjusting the accuracy of the voice recognition according to the user's emotion. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a voice recognition AI, but is not limited to such examples. Some or all of the above-mentioned processes in the recognition unit may be performed using AI or may be performed without using AI. For example, the recognition unit may use a generative AI to estimate the user's emotion and improve the accuracy of the voice recognition based on the estimated user's emotion.

[0082] The recognition unit can filter background sounds during a meeting and recognize only the voice of the speaker. For example, the recognition unit removes noise in the meeting room and clearly recognizes only the voice of the speaker. For example, the recognition unit removes noise in the meeting room and clearly recognizes only the voice of the speaker. The recognition unit can also filter the sound of a projector or air conditioner during a meeting and prioritize the voice of the speaker. For example, the recognition unit filters the sound of a projector or air conditioner during a meeting and prioritizes the voice of the speaker. Furthermore, the recognition unit can also individually recognize the voices of multiple speakers during a meeting and eliminate background sounds. For example, the recognition unit individually recognizes the voices of multiple speakers during a meeting and eliminates background sounds. As a result, the voice of the speaker can be clearly recognized by filtering the background sounds. A part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can use a generation AI to filter background sounds during a meeting and recognize only the voice of the speaker. Examples of the generation AI include, but are not limited to, a voice recognition AI or a noise canceling AI.

[0083] The recognition unit can analyze the tone and speed of the speaker's voice, determine the importance of the speech, and improve the recognition accuracy. For example, when the speaker's voice becomes louder, the recognition unit increases the recognition accuracy as an important speech. For example, when the speaker's voice becomes louder, the recognition unit increases the recognition accuracy as an important speech. In addition, when the speaker's voice becomes lower, the recognition unit can determine that the importance is low and adjust the recognition accuracy. For example, when the speaker's voice becomes lower, the recognition unit can determine that the importance is low and adjust the recognition accuracy. Furthermore, when the speaker's speaking speed is fast, the recognition unit can determine that important information is included and improve the recognition accuracy. For example, when the speaker's speaking speed is fast, the recognition unit can determine that important information is included and improve the recognition accuracy. In this way, important speech can be accurately recognized by analyzing the tone and speed of the speaker's voice. Part or all of the above-mentioned processing in the recognition unit may be performed, for example, using AI, or may be performed without using AI. For example, the recognition unit can use a generation AI to analyze the tone and speed of the speaker's voice, determine the importance of the speech, and improve the recognition accuracy. Examples of the generation AI include, but are not limited to, speech recognition AI and natural language processing AI.

[0084] The recognition unit can estimate the user's emotion and set a priority order for the voice to be recognized based on the estimated user's emotion. For example, when the user is nervous, the recognition unit preferentially recognizes important utterances. For example, when the user is nervous, the recognition unit preferentially recognizes important utterances. In addition, when the user is relaxed, the recognition unit can also recognize all utterances equally. For example, when the user is relaxed, the recognition unit recognizes all utterances equally. In addition, when the user is in a hurry, the recognition unit can also preferentially recognize utterances that focus on the main points. For example, when the user is in a hurry, the recognition unit preferentially recognizes utterances that focus on the main points. In this way, by determining the priority order of the voice according to the user's emotion, important utterances can be preferentially recognized. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a voice recognition AI, but is not limited to such examples. A part or all of the above-mentioned processing in the recognition unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the recognition unit can use generative AI to estimate a user's emotions and prioritize speech recognition based on the estimated user's emotions.

[0085] The recognition unit can simultaneously recognize multiple speakers in a conference and separate the text data for each speaker. The recognition unit, for example, recognizes the voices of multiple speakers simultaneously and separates them as individual text data. For example, the recognition unit recognizes the voices of multiple speakers simultaneously and separates them as individual text data. The recognition unit can also display text data for each speaker in a different color to visually distinguish them. For example, the recognition unit displays text data for each speaker in a different color to visually distinguish them. Furthermore, the recognition unit can automatically recognize the name of the speaker and tag the text data. For example, the recognition unit automatically recognizes the name of the speaker and tags the text data. This allows multiple speakers to be recognized simultaneously and the text data to be separated for each speaker, allowing the content of the speech to be clearly recorded. A part or all of the above-mentioned processing in the recognition unit may be performed, for example, using AI, or may be performed without using AI. For example, the recognition unit can use a generation AI to simultaneously recognize multiple speakers in a conference and separate text data for each speaker. Examples of the generation AI include, but are not limited to, speech recognition AI and natural language processing AI.

[0086] The recognition unit can translate the voice during the conference in real time to accommodate participants of different languages. The recognition unit, for example, translates remarks during the conference in real time to generate text data in different languages. For example, the recognition unit translates remarks during the conference in real time to generate text data in different languages. The recognition unit can also display the translated text data on the participant's device in real time. For example, the recognition unit displays the translated text data on the participant's device in real time. Furthermore, the recognition unit can automatically translate and share with everyone when a participant of a different language speaks. For example, the recognition unit automatically translates and shares with everyone when a participant of a different language speaks. This allows participants of different languages ​​to communicate smoothly by translating the voice during the conference in real time. Part or all of the above-mentioned processing in the recognition unit may be performed using, for example, AI, or may be performed without using AI. For example, the recognition unit can use a generation AI to translate the voice during the conference in real time to accommodate participants of different languages. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and translation AI.

[0087] The analysis unit can estimate the user's emotion and change the analysis method of the remark contents based on the estimated user's emotion. For example, when the user is nervous, the analysis unit prioritizes analysis of important keywords. For example, when the user is nervous, the analysis unit prioritizes analysis of important keywords. In addition, when the user is relaxed, the analysis unit can analyze all remark contents evenly. For example, when the user is relaxed, the analysis unit analyzes all remark contents evenly. In addition, when the user is in a hurry, the analysis unit can also analyze remark contents that focus on the main points preferentially. For example, when the user is in a hurry, the analysis unit prioritizes analysis of remark contents that focus on the main points. In this way, important information can be accurately analyzed by adjusting the analysis method of the remark contents according to the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the analysis unit can use the generation AI to estimate the user's emotions and change the analysis method of the speech content based on the estimated user's emotions.

[0088] The analysis unit can analyze the content of the speech based on the context and refer to related past minutes and materials. The analysis unit, for example, analyzes the content of the speech based on the context and automatically refers to related past minutes. For example, the analysis unit analyzes the content of the speech based on the context and automatically refers to related past minutes. The analysis unit can also automatically search and refer to materials related to the content of the speech. For example, the analysis unit automatically searches and refers to materials related to the content of the speech. Furthermore, the analysis unit can also analyze the context of the content of the speech and provide related information. For example, the analysis unit analyzes the context of the content of the speech and provides related information. As a result, the content of the speech can be analyzed based on the context, and related information can be accurately provided. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, AI, or may be performed without using AI. For example, the analysis unit can use a generation AI to analyze the content of the speech based on the context and refer to related past minutes and materials. The generation AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples.

[0089] The analysis unit can extract keywords from the remarks and classify them according to importance. The analysis unit, for example, extracts important keywords from the remarks and classifies them according to importance. For example, the analysis unit extracts important keywords from the remarks and classifies them according to importance. The analysis unit can also automatically tag and classify the extracted keywords. For example, the analysis unit automatically tags and classifies the extracted keywords. The analysis unit can also analyze the importance of the keywords and prioritize and classify them. For example, the analysis unit analyzes the importance of the keywords and prioritizes and classifies them. As a result, important information can be efficiently organized by extracting keywords from the remarks and classifying them according to importance. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the analysis unit can use a generation AI to extract keywords from the remarks and classify them according to importance. The generation AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples.

[0090] The analysis unit can estimate the user's emotion and change the display method of the analysis result based on the estimated emotion of the user. For example, when the user is nervous, the analysis unit provides a simple and highly visible display method. For example, when the user is nervous, the analysis unit provides a simple and highly visible display method. In addition, when the user is relaxed, the analysis unit can also provide a display method including detailed information. For example, when the user is relaxed, the analysis unit provides a display method including detailed information. In addition, when the user is in a hurry, the analysis unit can also provide a display method that focuses on the main points. For example, when the user is in a hurry, the analysis unit provides a display method that focuses on the main points. This makes it possible to provide information with high visibility by adjusting the display method of the analysis result according to the emotion of the user. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples. A part or all of the above-mentioned processing in the analysis unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the analysis unit can use the generation AI to estimate a user's emotions and change the way the analysis results are displayed based on the estimated user's emotions.

[0091] The analysis unit can analyze the content of the statement and automatically refer to related laws and regulations and industry standards. The analysis unit, for example, analyzes the content of the statement and automatically refers to related laws and regulations. For example, the analysis unit analyzes the content of the statement and automatically refers to related laws and regulations. The analysis unit can also automatically search and refer to industry standards related to the content of the statement. For example, the analysis unit automatically searches and refers to industry standards related to the content of the statement. Furthermore, the analysis unit can also analyze the context of the content of the statement and provide related information. For example, the analysis unit analyzes the context of the content of the statement and provides related information. As a result, appropriate information can be provided by analyzing the content of the statement and referring to related laws and regulations and industry standards. A part or all of the above-mentioned processing in the analysis unit may be performed, for example, using AI, or may be performed without using AI. For example, the analysis unit can use a generation AI to analyze the content of the statement and refer to related laws and regulations and industry standards. The generation AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples.

[0092] The analysis unit can analyze the content of the remark and provide related market data and competitive information. The analysis unit can, for example, analyze the content of the remark and automatically provide related market data. For example, the analysis unit can analyze the content of the remark and automatically provide related market data. The analysis unit can also automatically search and provide competitive information related to the content of the remark. For example, the analysis unit can automatically search and provide competitive information related to the content of the remark. Furthermore, the analysis unit can also analyze the context of the content of the remark and provide related information. For example, the analysis unit can analyze the context of the content of the remark and provide related information. This improves the quality of the meeting by analyzing the content of the remark and providing related market data and competitive information. A part or all of the above-mentioned processing in the analysis unit can be performed, for example, using AI, or can be performed without using AI. For example, the analysis unit can use a generation AI to analyze the content of the remark and provide related market data and competitive information. The generation AI can be, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples.

[0093] The generation unit can estimate the user's emotion and change the expression method of the minutes based on the estimated user's emotion. For example, when the user is nervous, the generation unit generates minutes that are simple and highly visible. For example, when the user is nervous, the generation unit generates minutes that are simple and highly visible. In addition, the generation unit can also generate minutes that include detailed information when the user is relaxed. For example, when the user is relaxed, the generation unit generates minutes that include detailed information. In addition, the generation unit can also generate minutes that focus on the main points when the user is in a hurry. For example, when the user is in a hurry, the generation unit generates minutes that focus on the main points. In this way, by adjusting the expression method of the minutes according to the user's emotion, it is possible to generate minutes with high visibility. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples. Some or all of the above-mentioned processes in the generating unit may be performed using, for example, AI, or may be performed without using AI. For example, the generating unit may use a generating AI to estimate the user's emotion and change the way the minutes are expressed based on the estimated user's emotion.

[0094] The generating unit can display the remarks of each speaker in different colors when generating the minutes. The generating unit, for example, displays the remarks of each speaker in different colors to visually distinguish them. For example, the generating unit displays the remarks of each speaker in different colors to visually distinguish them. The generating unit can also automatically recognize the name of the speaker and display them in different colors. For example, the generating unit can automatically recognize the name of the speaker and display them in different colors. Furthermore, the generating unit can color-code the remarks of each speaker to improve the visibility of the minutes. For example, the generating unit color-codes the remarks of each speaker to improve the visibility of the minutes. As a result, the visibility of the minutes is improved by color-coding the remarks of each speaker. A part or all of the above-mentioned processing in the generating unit may be performed, for example, using AI, or may be performed without using AI. For example, the generating unit can use a generating AI to color-code the remarks of each speaker when generating the minutes. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and natural language processing AI.

[0095] The generating unit can highlight important decisions and homework items when generating the minutes. The generating unit, for example, highlights important decisions in bold or in a color. For example, the generating unit highlights important decisions in bold or in a color. The generating unit can also highlight homework items in a list format. For example, the generating unit highlights homework items in a list format. The generating unit can also automatically extract and highlight decisions and homework items. For example, the generating unit automatically extracts and highlights decisions and homework items. As a result, important information in the minutes can be seen at a glance by highlighting important decisions and homework items. A part or all of the above-mentioned processing in the generating unit may be performed, for example, using AI, or may be performed without using AI. For example, the generating unit can use a generating AI to highlight important decisions and homework items when generating the minutes. The generating AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples.

[0096] The generation unit can estimate the user's emotion and change the length of the minutes based on the estimated user's emotion. For example, when the user is in a hurry, the generation unit generates minutes that are short and to the point. For example, when the user is in a hurry, the generation unit generates minutes that are short and to the point. In addition, the generation unit can also generate longer minutes including detailed explanations when the user is relaxed. For example, when the user is relaxed, the generation unit generates longer minutes including detailed explanations. In addition, the generation unit can also generate minutes with visually stimulating effects when the user is excited. For example, when the user is excited, the generation unit generates minutes with visually stimulating effects. In this way, minutes with an appropriate amount of information can be generated by adjusting the length of the minutes according to the user's emotion. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples. Some or all of the above-mentioned processes in the generating unit may be performed using AI or may be performed without using AI. For example, the generating unit may use a generating AI to estimate the user's emotion and change the length of the minutes based on the estimated user's emotion.

[0097] The generating unit can include the progress of the meeting and the timeline when generating the minutes. For example, the generating unit displays the progress of the meeting in a timeline format and includes it in the minutes. For example, the generating unit displays the progress of the meeting in a timeline format and includes it in the minutes. The generating unit can also display the progress of each agenda of the meeting in a timeline and include it in the minutes. For example, the generating unit displays the progress of each agenda of the meeting in a timeline and includes it in the minutes. Furthermore, the generating unit can display the start time and end time of the meeting in a timeline and include it in the minutes. For example, the generating unit displays the start time and end time of the meeting in a timeline and includes it in the minutes. As a result, by including the progress of the meeting and the timeline, the contents of the minutes become more detailed. A part or all of the above-mentioned processing in the generating unit may be performed using, for example, AI, or may be performed without using AI. For example, the generating unit can use a generating AI to include the progress of the meeting and the timeline when generating the minutes. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and natural language processing AI.

[0098] The generating unit can automatically attach related materials and links when generating the minutes. The generating unit, for example, automatically attaches materials referenced during the meeting to the minutes. For example, the generating unit automatically attaches materials referenced during the meeting to the minutes. The generating unit can also automatically attach links related to the contents of remarks to the minutes. For example, the generating unit automatically attaches links related to the contents of remarks to the minutes. The generating unit can also automatically attach presentation materials used during the meeting to the minutes. For example, the generating unit automatically attaches presentation materials used during the meeting to the minutes. As a result, the information in the minutes is enriched by automatically attaching related materials and links. A part or all of the above-mentioned processing in the generating unit may be performed, for example, using AI, or may be performed without using AI. For example, the generating unit can use a generating AI to automatically attach related materials and links when generating the minutes. The generating AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples.

[0099] The judgment unit can estimate the user's emotion and change the judgment criteria for the progress of the conference based on the estimated emotion of the user. For example, when the user is nervous, the judgment unit tightens the judgment criteria for the progress of the conference. For example, when the user is nervous, the judgment unit tightens the judgment criteria for the progress of the conference. In addition, the judgment unit can also loosen the judgment criteria for the progress of the conference when the user is relaxed. For example, when the user is relaxed, the judgment unit loosens the judgment criteria for the progress of the conference. In addition, the judgment unit can also speed up the judgment criteria for the progress of the conference when the user is in a hurry. For example, when the user is in a hurry, the judgment unit speeds up the judgment criteria for the progress of the conference. This allows appropriate progress management by adjusting the judgment criteria for the progress of the conference according to the emotion of the user. The estimation of the emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples. A part or all of the above-mentioned processing in the determination unit may be performed using, for example, AI, or may be performed without using AI. For example, the determination unit may use a generation AI to estimate the user's emotion and change the judgment criteria for the progress of the conference based on the estimated user's emotion.

[0100] The judgment unit can analyze the progress of the conference in real time and identify delays in progress and problems. The judgment unit, for example, analyzes the progress of the conference in real time and identifies delays in progress. For example, the judgment unit analyzes the progress of the conference in real time and identifies delays in progress. The judgment unit can also analyze the progress of the conference and automatically identify problems. For example, the judgment unit analyzes the progress of the conference and automatically identifies problems. Furthermore, the judgment unit can monitor the progress of the conference in real time and identify delays in progress and problems. For example, the judgment unit monitors the progress of the conference in real time and identifies delays in progress and problems. As a result, delays in progress and problems can be quickly identified by analyzing the progress of the conference in real time. A part or all of the above-mentioned processing in the judgment unit may be performed using, for example, AI, or may be performed without using AI. For example, the judgment unit can use a generation AI to analyze the progress of the conference in real time and identify delays in progress and problems. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and real-time analysis AI.

[0101] The judgment unit can analyze the progress of the meeting and propose the timing of transition to the next agenda. The judgment unit, for example, analyzes the progress of the meeting and proposes the timing of transition to the next agenda. For example, the judgment unit analyzes the progress of the meeting and proposes the timing of transition to the next agenda. The judgment unit can also monitor the progress of the meeting in real time and propose an appropriate transition timing. For example, the judgment unit monitors the progress of the meeting in real time and proposes an appropriate transition timing. The judgment unit can also analyze the progress of the meeting and propose the timing of ending the agenda. For example, the judgment unit analyzes the progress of the meeting and proposes the timing of ending the agenda. In this way, an appropriate transition timing can be proposed by analyzing the progress of the meeting. A part or all of the above-mentioned processing in the judgment unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the judgment unit can use a generation AI to analyze the progress of the meeting and propose the timing of transition to the next agenda. The generation AI is, for example, a text generation AI (for example, LLM) or a real-time analysis AI, but is not limited to such examples.

[0102] The judgment unit can estimate the user's emotion and change the display method of the progress status based on the estimated emotion of the user. For example, when the user is nervous, the judgment unit provides a simple and highly visible display method. For example, when the user is nervous, the judgment unit provides a simple and highly visible display method. In addition, when the user is relaxed, the judgment unit can also provide a display method including detailed information. For example, when the user is relaxed, the judgment unit provides a display method including detailed information. In addition, when the user is in a hurry, the judgment unit can also provide a display method that focuses on the main points. For example, when the user is in a hurry, the judgment unit provides a display method that focuses on the main points. This makes it possible to provide information with high visibility by adjusting the display method of the progress status according to the emotion of the user. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a natural language processing AI, but is not limited to such examples. A part or all of the above-mentioned processing in the judgment unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the judgment unit can use the generation AI to estimate the user's emotions and change the way the progress is displayed based on the estimated user's emotions.

[0103] The judgment unit can analyze the progress of the conference and visualize the speech frequency and speech time of the participants. The judgment unit, for example, analyzes the progress of the conference and displays the speech frequency of the participants in a graph. For example, the judgment unit analyzes the progress of the conference and displays the speech frequency of the participants in a graph. The judgment unit can also analyze the progress of the conference and display the speech time of the participants in a timeline. For example, the judgment unit analyzes the progress of the conference and displays the speech time of the participants in a timeline. The judgment unit can also analyze the progress of the conference and visualize the speech frequency and speech time of each participant. For example, the judgment unit analyzes the progress of the conference and visualizes the speech frequency and speech time of each participant. This makes it possible to more effectively manage the progress of the conference by analyzing the progress of the conference and visualizing the speech frequency and speech time of the participants. A part or all of the above-mentioned processing in the judgment unit may be performed, for example, using AI, or may be performed without using AI. For example, the judgment unit can use a generation AI to analyze the progress of the conference and visualize the speech frequency and speech time of the participants. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and real-time analysis AI.

[0104] The judgment unit can analyze the progress of the conference and evaluate the interest and concentration of the participants. The judgment unit, for example, analyzes the progress of the conference and evaluates the interest of the participants. For example, the judgment unit analyzes the progress of the conference and evaluates the interest of the participants. The judgment unit can also analyze the progress of the conference and evaluate the concentration of the participants. For example, the judgment unit analyzes the progress of the conference and evaluates the concentration of the participants. The judgment unit can also analyze the progress of the conference and evaluate the interest and concentration of the participants. For example, the judgment unit analyzes the progress of the conference and evaluates the interest and concentration of the participants. In this way, the quality of the conference is improved by analyzing the progress of the conference and evaluating the interest and concentration of the participants. A part or all of the above-mentioned processing in the judgment unit may be performed, for example, using AI, or may be performed without using AI. For example, the judgment unit can use a generation AI to analyze the progress of the conference and evaluate the interest and concentration of the participants. Examples of generation AI include, but are not limited to, text generation AI (e.g., LLM) and real-time analysis AI.

[0105] The assist unit can estimate the user's emotion and change the method of assistance based on the estimated user's emotion. For example, when the user is nervous, the assist unit provides assistance in a calm voice. For example, when the user is nervous, the assist unit provides assistance in a calm voice. Furthermore, when the user is relaxed, the assist unit can also provide assistance in a bright voice. For example, when the user is relaxed, the assist unit provides assistance in a bright voice. Furthermore, when the user is in a hurry, the assist unit can also provide quick and concise assistance. For example, when the user is in a hurry, the assist unit provides quick and concise assistance. This allows for more effective assistance by adjusting the method of assistance according to the user's emotion. The estimation of emotion is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a voice recognition AI, but is not limited to such examples. A part or all of the above-mentioned processing in the assist unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the assist unit can use the generation AI to estimate the user's emotions and change the method of assistance based on the estimated user's emotions.

[0106] When assisting the progress of a conference, the assisting unit can select an optimal assisting method by referring to past conference data. For example, the assisting unit selects an optimal assisting method by referring to past conference data. For example, the assisting unit selects an optimal assisting method by referring to past conference data. The assisting unit can also analyze past conference data and select an assisting method that was effective in a similar conference. For example, the assisting unit analyzes past conference data and selects an assisting method that was effective in a similar conference. Furthermore, the assisting unit can also propose an optimal assisting method based on past conference data. For example, the assisting unit proposes an optimal assisting method based on past conference data. In this way, the optimal assisting method can be selected by referring to past conference data. A part or all of the above-mentioned processing in the assisting unit may be performed, for example, using AI, or may be performed without using AI. For example, the assisting unit can use a generating AI to select an optimal assisting method by referring to past conference data when assisting the progress of a conference. The generating AI is, for example, a text generating AI (for example, LLM) or a data analysis AI, but is not limited to such examples.

[0107] The assist unit can customize the assist contents based on the roles and expertise of the participants when assisting the progress of the conference. The assist unit provides appropriate assist contents based on, for example, the roles of the participants. For example, the assist unit provides appropriate assist contents based on the roles of the participants. The assist unit can also customize the assist contents based on the expertise of the participants. For example, the assist unit customizes the assist contents based on the expertise of the participants. Furthermore, the assist unit can also provide optimal assist contents by considering the roles and expertise of the participants. For example, the assist unit provides optimal assist contents by considering the roles and expertise of the participants. As a result, more effective assistance can be achieved by customizing the assist contents based on the roles and expertise of the participants. A part or all of the above-mentioned processing in the assist unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the assist unit can use a generation AI to customize the assist contents based on the roles and expertise of the participants when assisting the progress of the conference. The generation AI is, for example, a text generation AI (for example, LLM) or a data analysis AI, but is not limited to such examples.

[0108] The assist unit can estimate the user's emotion and set the priority order of the assist based on the estimated emotion of the user. For example, when the user is nervous, the assist unit prioritizes important assist. For example, when the user is nervous, the assist unit prioritizes important assist. In addition, when the user is relaxed, the assist unit can perform all assists equally. For example, when the user is relaxed, the assist unit performs all assists equally. In addition, when the user is in a hurry, the assist unit can prioritize assists that focus on the main points. For example, when the user is in a hurry, the assist unit prioritizes assists that focus on the main points. In this way, by determining the priority order of the assists according to the emotion of the user, important assists can be prioritized. The emotion estimation is realized using an emotion estimation function using, for example, an emotion engine or a generation AI. The generation AI is, for example, a text generation AI (for example, LLM) or a voice recognition AI, but is not limited to such examples. A part or all of the above-mentioned processing in the assist unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the assist unit can use the generation AI to estimate the user's emotions and set priorities for assistance based on the estimated user's emotions.

[0109] The assist unit can take into consideration the schedule and time constraints of the participants when assisting the progress of the conference. The assist unit, for example, takes into consideration the schedule of the participants and provides optimal assist timing. For example, the assist unit takes into consideration the schedule of the participants and provides optimal assist timing. The assist unit can also provide efficient assistance by taking into consideration the time constraints of the participants. For example, the assist unit provides efficient assistance by taking into consideration the time constraints of the participants. Furthermore, the assist unit can also propose an optimal assist method based on the schedule and time constraints of the participants. For example, the assist unit proposes an optimal assist method based on the schedule and time constraints of the participants. This enables efficient assistance by taking into consideration the schedule and time constraints of the participants. A part or all of the above-mentioned processing in the assist unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the assist unit can use a generation AI to take into consideration the schedule and time constraints of the participants when assisting the progress of the conference. The generation AI is, for example, a text generation AI (for example, LLM) or a data analysis AI, but is not limited to such examples.

[0110] The assist unit can provide related external data and information when assisting the progress of the conference. For example, the assist unit provides related external data when assisting the progress of the conference. For example, the assist unit provides related external data when assisting the progress of the conference. In addition, the assist unit can automatically provide related information when assisting the progress of the conference. For example, the assist unit automatically provides related information when assisting the progress of the conference. Furthermore, the assist unit can provide optimal assistance based on external data and information when assisting the progress of the conference. For example, the assist unit provides optimal assistance based on external data and information when assisting the progress of the conference. As a result, by providing related external data and information, the progress of the conference can be more effectively assisted. A part or all of the above-mentioned processing in the assist unit may be performed using, for example, an AI, or may be performed without using an AI. For example, the assist unit can use a generation AI to provide related external data and information when assisting the progress of the conference. The generation AI is, for example, a text generation AI (for example, LLM) or a data analysis AI, but is not limited to such examples.

[0111] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0112] The AI ​​system can further include a translation unit that translates remarks made during a meeting in real time and accommodates participants who speak different languages. The translation unit, for example, translates remarks made during a meeting in real time and generates text data in different languages. The translation unit can also display the translated text data on the participants' devices in real time. Furthermore, the translation unit can automatically translate remarks made by participants who speak different languages ​​and share them with everyone. This allows participants who speak different languages ​​to communicate smoothly.

[0113] The recognition unit can analyze the tone and speed of a speaker's voice during a meeting, determine the importance of the speech, and improve the recognition accuracy. For example, if the speaker's voice gets louder, the recognition accuracy can be increased as the speech is important. Also, if the speaker's voice gets quieter, the recognition accuracy can be adjusted as the speech is less important. Furthermore, if the speaker speaks quickly, the recognition accuracy can be improved as the speech contains important information. In this way, important speech can be accurately recognized by analyzing the tone and speed of the speaker's voice.

[0114] The analysis unit can estimate the user's emotions and change the analysis method of the utterance contents based on the estimated user's emotions. For example, if the user is nervous, important keywords are given priority in analysis. Also, if the user is relaxed, all utterance contents can be analyzed equally. Furthermore, if the user is in a hurry, utterance contents that focus on the main points can be given priority in analysis. In this way, important information can be accurately analyzed by adjusting the analysis method of the utterance contents according to the user's emotions.

[0115] When generating the minutes, the generation unit can display the remarks of each speaker in a different color. For example, the remarks of each speaker can be displayed in a different color to visually distinguish them. The name of the speaker can also be automatically recognized and displayed in a different color. Furthermore, the remarks of each speaker can be color-coded to improve the visibility of the minutes. Thus, by color-coding the remarks of each speaker, the visibility of the minutes can be improved.

[0116] The determination unit can analyze the progress of the conference and visualize the speech frequency and speech duration of each participant. For example, the progress of the conference can be analyzed and the speech frequency of each participant can be displayed in a graph. The progress of the conference can also be analyzed and the speech duration of each participant can be displayed in a timeline. Furthermore, the progress of the conference can also be analyzed and the speech frequency and speech duration of each participant can be visualized. In this way, by analyzing the progress of the conference and visualizing the speech frequency and speech duration of each participant, the progress of the conference can be managed more effectively.

[0117] The assist unit can estimate the user's emotions and change the method of assistance based on the estimated user's emotions. For example, if the user is nervous, assistance can be provided in a calm voice. If the user is relaxed, assistance can be provided in a cheerful voice. Furthermore, if the user is in a hurry, quick and concise assistance can be provided. This allows for more effective assistance by adjusting the method of assistance according to the user's emotions.

[0118] The recognition unit can translate speech during a meeting in real time to accommodate participants who speak different languages. For example, statements made during a meeting can be translated in real time to generate text data in different languages. The translated text data can also be displayed in real time on the participants' devices. Furthermore, when a participant who speaks a different language speaks, the text can be automatically translated and shared with everyone. This allows participants who speak different languages ​​to communicate smoothly by translating speech during a meeting in real time.

[0119] The analysis unit can analyze the content of a statement based on the context and refer to related past minutes and materials. For example, the analysis unit can analyze the content of a statement based on the context and automatically refer to related past minutes. It can also automatically search for and refer to materials related to the content of a statement. It can also analyze the context of the content of a statement and provide related information. This makes it possible to accurately provide related information by analyzing the content of a statement based on the context.

[0120] The generation unit can estimate the user's emotions and change the way the minutes are expressed based on the estimated user's emotions. For example, if the user is nervous, simple minutes with high visibility can be generated. If the user is relaxed, minutes including detailed information can also be generated. Furthermore, if the user is in a hurry, minutes that focus on the main points can also be generated. In this way, by adjusting the way the minutes are expressed according to the user's emotions, minutes with high visibility can be generated.

[0121] The judgment unit can estimate the user's emotions and change the judgment criteria for the progress of the conference based on the estimated user's emotions. For example, if the user is nervous, the judgment criteria for the progress can be set to be stricter. Also, if the user is relaxed, the judgment criteria for the progress can be set to be looser. Furthermore, if the user is in a hurry, the judgment criteria for the progress can be set to be quicker. In this way, the judgment criteria for the progress of the conference can be adjusted according to the user's emotions, enabling appropriate progress management.

[0122] The process flow of the second embodiment will be briefly described below.

[0123] Step 1: The recognition unit converts the voices being spoken during the meeting into text data in real time. The recognition unit uses voice recognition technology to instantly convert what is being said during the meeting into text data. For example, it is possible to use voice recognition software to convert what is being said into text data in real time, and to identify the speaker's voice and convert it into text data individually. Noise canceling technology is also used to remove noise in the meeting room, clearly recognizing only the speaker's voice. Step 2: The analysis unit uses natural language processing technology to analyze the text data converted by the recognition unit and extract important decisions and homework items. For example, it uses morphological analysis technology to analyze the content of the speech and extracts important keywords, and uses grammatical analysis technology and semantic analysis technology to analyze the grammatical structure and meaning of the speech and extract important information. Step 3: The generator automatically generates minutes based on the information extracted by the analyzer. Using templates and generation algorithms, the generator automatically generates minutes including the meeting date and time, participants, agenda, remarks, decisions, homework items, etc. Furthermore, it can also display the remarks of each speaker in a different color. Step 4: The judgment unit monitors the progress of the meeting in real time and judges its progress. For example, it identifies delays in progress and problems and suggests the timing to move on to the next agenda item. It can also visualize the frequency and duration of participants' comments. Step 5: The assisting unit assists the progress of the meeting based on the judgment of the judging unit. For example, it can send reminders to encourage moving on to the next agenda item and suggest the next agenda item. It can also select the optimal assistance method by referring to past meeting data.

[0124] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0125] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) are input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may also be an AI agent. In addition, when the processes of each of the above-mentioned parts are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples.In addition, processing performed by AI including the generation AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI including the generation AI.

[0126] Furthermore, the above-described processing by the data processing system 10 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information required for processing from the data processing device 12 or an external device.

[0127] Each of the multiple elements including the above-mentioned recognition unit, analysis unit, generation unit, judgment unit, and assist unit is realized, for example, by at least one of the smart device 14 and the data processing device 12. For example, the recognition unit acquires voice during a meeting using the microphone 38B of the smart device 14, and converts it into text data in real time by the control unit 46A. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the text data using natural language processing technology to extract important decisions and homework items. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and automatically generates minutes based on the extracted information. The judgment unit is realized, for example, by the control unit 46A of the smart device 14, and monitors the progress of the meeting in real time and judges the progress. The assist unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and assists the progress of the meeting based on the judgment of the judgment unit. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0128] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0129] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0130] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0131] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0132] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0133] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0134] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0135] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0136] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0137] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0138] In the smart glasses 214, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart glasses 214 have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0139] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0140] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0141] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0142] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device.

[0143] Each of the multiple elements including the above-mentioned recognition unit, analysis unit, generation unit, judgment unit, and assist unit is realized, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the recognition unit acquires voice during a meeting using the microphone 238 of the smart glasses 214, and converts the voice into text data in real time by the control unit 46A. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the text data using natural language processing technology to extract important decisions and homework items. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and automatically generates minutes based on the extracted information. The judgment unit is realized, for example, by the control unit 46A of the smart glasses 214, and monitors the progress of the meeting in real time and judges the progress. The assist unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and assists the progress of the meeting based on the judgment of the judgment unit. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0144] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0145] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0146] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0147] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0148] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0149] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0150] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0151] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0152] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0153] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0154] In the headset type terminal 314, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50, and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0155] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0156] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0157] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0158] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device.

[0159] Each of the multiple elements including the above-mentioned recognition unit, analysis unit, generation unit, judgment unit, and assist unit is realized, for example, by at least one of the headset type terminal 314 and the data processing device 12. For example, the recognition unit acquires voice during the conference using the microphone 238 of the headset type terminal 314, and converts it into text data in real time by the control unit 46A. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the text data using natural language processing technology to extract important decisions and homework items. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and automatically generates minutes based on the extracted information. The judgment unit is realized, for example, by the control unit 46A of the headset type terminal 314, and monitors the progress of the conference in real time and judges the progress. The assist unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and assists the progress of the conference based on the judgment of the judgment unit. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0160] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0161] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0162] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0163] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0164] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0165] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of a typical healthy person).

[0166] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0167] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and legs. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, and legs. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0168] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0169] The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0170] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the emotion of the user using the emotion identification model 59 and perform identification processing using the user's emotion. In the emotion estimation function (emotion identification function) using the emotion identification model 59, various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, are performed, but are not limited to such examples. In addition, the estimation and prediction of the emotion also include, for example, analysis of the emotion.

[0171] In the robot 414, the specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads out the specific program 60 from the storage 50, and executes the specific program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific program 60 executed on the RAM 48. The robot 414 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0172] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (such as a prediction result) using the data generation model 58 by communicating with the server device having the data generation model 58. In addition, the data processing device 12 may be a server device, or may be a terminal device (such as a mobile phone, a robot, or a home appliance) owned by a user.

[0173] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0174] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image (e.g., still image data or video data) is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, and image data. The data generation model 58 includes, for example, a text generation AI, an image generation AI, and a multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing while using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction, and in this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. In the data processing device 12 and the like, the data generation model 58 includes a plurality of types, and the data generation model 58 includes an AI other than the generation AI. The AI ​​other than the generation AI is, for example, a linear regression, a logistic regression, a decision tree, a random forest, a support vector machine (SVM), a k-means clustering, a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), or a naive Bayes, and can perform various processes, but is not limited to such examples. The AI ​​may be an AI agent. In addition, when the processes of each part described above are performed by an AI, the processes are performed in part or in whole by the AI, but are not limited to such examples. In addition, the processes performed by the AI ​​including the generation AI may be replaced with rule-based processes, and the rule-based processes may be replaced with processes performed by the AI ​​including the generation AI.

[0175] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0176] Each of the multiple elements including the above-mentioned recognition unit, analysis unit, generation unit, judgment unit, and assist unit is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the recognition unit acquires voice during the conference using the microphone 238 of the robot 414, and converts it into text data in real time by the control unit 46A. The analysis unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and analyzes the text data using natural language processing technology to extract important decisions and homework items. The generation unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and automatically generates minutes based on the extracted information. The judgment unit is realized, for example, by the control unit 46A of the robot 414, and monitors the progress of the conference in real time and judges the progress. The assist unit is realized, for example, by the specific processing unit 290 of the data processing device 12, and assists the progress of the conference based on the judgment of the judgment unit. The correspondence between each unit and the device or control unit is not limited to the above-mentioned example, and various changes are possible.

[0177] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[0178] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0179] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0180] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0181] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[0182] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[0183] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[0184] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0185] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0186] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0187] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0188] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0189] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0190] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0191] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0192] In the above example, the first to fourth embodiments are described separately, but some or all of these embodiments may be combined. The smart device 14, the smart glasses 214, the headset terminal 314, and the robot 414 are only examples, and they may be combined with each other, or may be other devices. In the above example, the first and second embodiments are described separately, but they may be combined.

[0193] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replaced with respect to the description and illustrations shown above, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the description and illustrations shown above omit explanations of technical common sense that do not require particular explanation in enabling the implementation of the technology of the present disclosure.

[0194] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0195] (Appendix 1) A recognition unit that converts speech during a meeting into text data in real time; an analysis unit that analyzes the text data converted by the recognition unit and extracts important decision items or homework items; A generation unit that automatically generates minutes based on the information extracted by the analysis unit; a judgment unit that monitors the progress of the meeting in real time and judges the progress; and an assist unit that assists in the progress of the conference based on the judgment of the judgment unit. A system characterized by: (Appendix 2) The recognition unit is Converts audio during meetings into text data in real time 2. The system of claim 1, (Appendix 3) The analysis unit is Analyze the content of statements using natural language processing technology and extract important decisions and homework items 2. The system of claim 1, (Appendix 4) The generation unit is Automatically generate minutes based on extracted information 2. The system of claim 1, (Appendix 5) The determination unit is Monitor meeting progress in real time and gauge progress 2. The system of claim 1, (Appendix 6) The assist unit is Assisting the progress of the conference based on the judgment of the judgment unit 2. The system of claim 1, (Appendix 7) The recognition unit is To estimate a user's emotion and improve the accuracy of speech recognition based on the estimated user's emotion. 2. The system of claim 1, (Appendix 8) The recognition unit is Filter out background noise during meetings and hear only the speaker 2. The system of claim 1, (Appendix 9) The recognition unit is Analyze the tone and speed of the speaker's voice to determine the importance of what is said and improve recognition accuracy 2. The system of claim 1, (Appendix 10) The recognition unit is Estimate user emotions and prioritize speech recognition based on the estimated user emotions. 2. The system of claim 1, (Appendix 11) The recognition unit is Simultaneously recognize multiple speakers during a meeting and separate the text data for each speaker 2. The system of claim 1, (Appendix 12) The recognition unit is Translate audio in real time during meetings to accommodate participants speaking different languages 2. The system of claim 1, (Appendix 13) The analysis unit is Infer the user's emotions and change the way the speech is analyzed based on the estimated user emotions. 2. The system of claim 1, (Appendix 14) The analysis unit is Analyze what is being said in context and refer to relevant past minutes and documents 2. The system of claim 1, (Appendix 15) The analysis unit is Extract keywords from comments and classify them according to importance 2. The system of claim 1, (Appendix 16) The analysis unit is Infer user emotions and change the way analysis results are displayed based on the estimated user emotions 2. The system of claim 1, (Appendix 17) The analysis unit is Analyzes what is being said and automatically references relevant laws, regulations and industry standards 2. The system of claim 1, (Appendix 18) The analysis unit is Analyze statements and provide relevant market data and competitive intelligence 2. The system of claim 1, (Appendix 19) The generation unit is Infer user emotions and change the presentation of minutes based on the estimated user emotions 2. The system of claim 1, (Appendix 20) The generation unit is When generating minutes, display the content of each speaker in different colors 2. The system of claim 1, (Appendix 21) The generation unit is Highlight important decisions and homework items when generating minutes 2. The system of claim 1, (Appendix 22) The generation unit is Infer user sentiment and change the length of the minutes based on the estimated user sentiment 2. The system of claim 1, (Appendix 23) The generation unit is Include meeting progress and timeline when generating minutes 2. The system of claim 1, (Appendix 24) The generation unit is Automatically attach relevant materials and links when generating minutes 2. The system of claim 1, (Appendix 25) The determination unit is To estimate a user's emotion and change a criterion for judging the progress of a conference based on the estimated user's emotion. 2. The system of claim 1, (Appendix 26) The determination unit is Analyze meeting progress in real time to identify delays and issues 2. The system of claim 1, (Appendix 27) The determination unit is Analyze meeting progress and suggest when to move on to the next topic 2. The system of claim 1, (Appendix 28) The determination unit is Infer the user's emotions and change how the progress is displayed based on the inferred user emotions 2. The system of claim 1, (Appendix 29) The determination unit is Analyze the progress of a meeting and visualize the frequency and duration of participants' comments 2. The system of claim 1, (Appendix 30) The determination unit is Analyze meeting progress and evaluate participants' interest and concentration 2. The system of claim 1, (Appendix 31) The assist unit is Estimate the user's emotions and change the way of assistance based on the estimated user's emotions 2. The system of claim 1, (Appendix 32) The assist unit is When assisting with the progress of a meeting, the system selects the optimal assistance method by referring to past meeting data. 2. The system of claim 1, (Appendix 33) The assist unit is Provide tailored meeting assistance based on participants' roles and expertise 2. The system of claim 1, (Appendix 34) The assist unit is Estimate the user's emotions and set the priority of assistance based on the estimated user's emotions 2. The system of claim 1, (Appendix 35) The assist unit is Consider participants' schedules and time constraints when assisting with meeting facilitators 2. The system of claim 1, (Appendix 36) The assist unit is Providing relevant external data and information to assist in running meetings 2. The system of claim 1, [Explanation of symbols]

[0196] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. A recognition unit that converts speech during a meeting into text data in real time; an analysis unit that analyzes the text data converted by the recognition unit and extracts important decision items or homework items; A generation unit that automatically generates minutes based on the information extracted by the analysis unit; a judgment unit that monitors the progress of the meeting in real time and judges the progress; and an assist unit that assists in the progress of the conference based on the judgment of the judgment unit. A system characterized by:

2. The recognition unit is Converts audio during meetings into text data in real time 2. The system of claim 1 .

3. The analysis unit is Analyze the content of statements using natural language processing technology and extract important decisions and homework items 2. The system of claim 1 .

4. The determination unit is Monitor meeting progress in real time and gauge progress 2. The system of claim 1 .

5. The recognition unit is To estimate a user's emotion and improve the accuracy of speech recognition based on the estimated user's emotion.

2. The system of claim 1 .

6. The recognition unit is Filter out background noise during meetings and hear only the speaker 2. The system of claim 1 .

7. The recognition unit is Analyze the tone and speed of the speaker's voice to determine the importance of what is said and improve recognition accuracy 2. The system of claim 1 .

Images