system

Abstract

The system according to the embodiment aims to reduce the business workload by extracting the names of speakers and the content of their comments from video or audio recordings of a conference, and then organizing and transmitting the extracted information. [Solution] A system according to an embodiment includes an acquisition unit, an analysis unit, a compilation unit, a transmission unit, and a link issuing unit. The acquisition unit acquires video or audio data of a conference. The analysis unit analyzes the data acquired by the acquisition unit and extracts the names of speakers and the content of their comments. The compilation unit organizes the content of the comments extracted by the analysis unit and assigns a timestamp. The transmission unit transmits the data compiled by the compilation unit to the conference organizer. The link issuing unit issues a link to the data compiled by the compilation unit.

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, the process of extracting the names of speakers and the content of their remarks from video or audio recordings of meetings, organizing them, and sending them out was time-consuming, resulting in an increase in work workload.

[0005] The system according to the embodiment aims to reduce the business workload by extracting the names of speakers and the content of their comments from video or audio recordings of a conference, and then organizing and transmitting the extracted information. [Means for solving the problem]

[0006] The system according to the embodiment includes an acquisition unit, an analysis unit, a compilation unit, a transmission unit, and a link issuing unit. The acquisition unit acquires video or audio data of a conference. The analysis unit analyzes the data acquired by the acquisition unit and extracts the names of speakers and the content of their comments. The compilation unit organizes the content of the comments extracted by the analysis unit and assigns a timestamp. The transmission unit transmits the data compiled by the compilation unit to the conference organizer. The link issuing unit issues a link to the data compiled by the compilation unit. [Effects of the Invention]

[0007] The system according to the embodiment can reduce the workload by extracting the names of speakers and the content of their comments from video or audio recordings of a conference, organizing them, and transmitting them. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more nonvolatile storage devices that store various programs, various parameters, etc. Examples of nonvolatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A meeting recording system according to an embodiment of the present invention acquires video or audio recordings of web conferences, concisely transcribes the names of speakers and their comments after the conference, and sends them to the conference organizer or issues a link to the minutes file to the communication tool team. This meeting recording system reduces work workload by acquiring video or audio recordings of conferences, extracting and organizing the speaker names and their comments, and adding a timestamp before sending or issuing a link. For example, the meeting recording system configures a web conferencing tool such as Zoom or Teams to record a conference. After the conference ends, the video or audio recording is analyzed to extract the speaker names and their comments. This analysis is performed using a generation AI. The generation AI uses speech recognition technology to convert the audio data into text and organize it by speaker. The organized text data is timestamped and sent to the conference organizer, or a link to the minutes file is issued to the communication tool team. Access to this link is restricted to ensure security. This system significantly reduces the time and effort required to create meeting minutes. This enables the meeting recording system to efficiently acquire, analyze, organize, transmit, and issue links to video or audio recordings of meetings.

[0029] A conference recording system according to an embodiment includes an acquisition unit, an analysis unit, a compilation unit, a transmission unit, and a link issuing unit. The acquisition unit acquires video or audio recording data of a conference. For example, the acquisition unit configures a web conferencing tool such as Zoom or Teams to record the conference. The acquisition unit can also store the video or audio recording data of the conference in cloud storage. For example, the acquisition unit can also store the video or audio recording data of the conference in local storage. The analysis unit analyzes the data acquired by the acquisition unit and extracts the names of speakers and the content of their comments. For example, the analysis unit converts the audio data into text using speech recognition technology. For example, the analysis unit organizes the data by speaker using generation AI. For example, the analysis unit extracts the names of speakers using speech recognition technology. The organization unit organizes the content of the comments extracted by the analysis unit and assigns timestamps to the extracted text data. For example, the organization unit assigns timestamps to the organized text data. For example, the organization unit organizes the content of the comments in chronological order. For example, the organization unit organizes the content of the comments by speaker. The sending unit sends the data organized by the organizing unit to the conference organizer. The sending unit, for example, sends the organized data by email. The sending unit, for example, uploads the organized data to cloud storage and sends a link to the conference organizer. The sending unit, for example, sends the organized data by a chat tool. The link issuing unit issues a link to the data organized by the organizing unit. The link issuing unit, for example, generates a link to the organized data and imposes access restrictions. The link issuing unit, for example, sends the link to the organized data by email. The link issuing unit, for example, sends the link to the organized data by a chat tool. This enables the conference recording system according to the embodiment to efficiently acquire, analyze, organize, send, and issue links to video or audio recording data of a conference.

[0030] The acquisition unit can set up a web conferencing tool, such as Zoom or Teams, to record or record a meeting. For example, the acquisition unit sets up a web conferencing tool, such as Zoom, to record a meeting. For example, the acquisition unit sets up a web conferencing tool, such as Teams, to record a meeting. For example, the acquisition unit selects a recording or audio recording option on a setting screen of the web conferencing tool. This enables recording or audio recording settings in the web conferencing tool. Some or all of the above-described processing in the acquisition unit may be performed using, or without, a generation AI. For example, the acquisition unit inputs the setting screen of the web conferencing tool into the generation AI and causes the generation AI to set up recording or audio recording.

[0031] The analysis unit can convert the voice data into text using voice recognition technology and organize the text by speaker. The analysis unit, for example, converts the voice data into text using voice recognition technology. The analysis unit, for example, analyzes the voice data using deep learning. The analysis unit, for example, analyzes the voice data using HMM (Hidden Markov Model). The analysis unit, for example, extracts the name of the speaker using voice recognition technology. The analysis unit, for example, converts the speech content into text using voice recognition technology. The analysis unit, for example, applies an algorithm for organizing the speech by speaker. This makes it possible to convert the voice data into text and organize it by speaker. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input the voice data into a generation AI and have the generation AI convert it into text data.

[0032] The organizing unit can assign a timestamp to the organized text data. For example, the organizing unit assigns a timestamp to the organized text data. For example, the organizing unit assigns a timestamp to each utterance in the text data. For example, the organizing unit organizes the utterance content in chronological order. For example, the organizing unit organizes the utterance content by speaker. This makes it possible to assign a timestamp to the organized text data. Some or all of the above-described processing in the organizing unit may be performed using, or without, the generation AI. For example, the organizing unit may input the text data to the generation AI and cause the generation AI to assign a timestamp.

[0033] The sending unit can send the organized data to the conference organizer. For example, the sending unit sends the organized data by email. For example, the sending unit uploads the organized data to cloud storage and sends a link to the conference organizer. For example, the sending unit sends the organized data by a chat tool. This makes it possible to send the organized data to the conference organizer. Some or all of the above-described processing in the sending unit may be performed using, for example, a generation AI, or may be performed without using the generation AI. For example, the sending unit can input the organized data into the generation AI and have the generation AI execute the transmission.

[0034] The link issuing unit can issue a link to the organized data and impose access restrictions. The link issuing unit, for example, generates a link to the organized data and imposes access restrictions. The link issuing unit, for example, sends a link to the organized data by email. The link issuing unit, for example, sends a link to the organized data using a chat tool. This makes it possible to issue a link to the organized data and impose access restrictions. Some or all of the above-mentioned processing in the link issuing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the link issuing unit may cause the generation AI to generate the link and set the access restrictions.

[0035] The acquisition unit can automatically optimize the video or audio recording settings according to the type and importance of the meeting. For example, if the meeting is important, the acquisition unit records or records in high resolution. For example, if the meeting is a regular meeting, the acquisition unit records or records in standard resolution. For example, if the meeting is informal, the acquisition unit records or records in low resolution. This makes it possible to optimize the video or audio recording settings according to the type and importance of the meeting. Some or all of the above-mentioned processing in the acquisition unit may be performed using, or without, the generation AI. For example, the acquisition unit may input the type and importance of the meeting into the generation AI and cause the generation AI to optimize the video or audio recording settings.

[0036] The acquisition unit can acquire a list of participants in a conference in advance and perform filtering to improve the accuracy of speaker identification. The acquisition unit, for example, acquires a list of participants in a conference and learns the voice characteristics of the speakers in advance. The acquisition unit, for example, performs filtering to improve the accuracy of speaker identification based on the participant list. The acquisition unit, for example, acquires information necessary for speaker identification based on the participant list in advance. This makes it possible to improve the accuracy of speaker identification. Some or all of the above-mentioned processing in the acquisition unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the acquisition unit can input the participant list to the generation AI and cause the generation AI to perform filtering to improve the accuracy of speaker identification.

[0037] The acquisition unit can adjust the video recording or audio recording settings based on the geographical location information of the meeting. For example, if the meeting is held in a different time zone, the acquisition unit adjusts the start time of the video recording or audio recording. For example, if the meeting is held in a different country, the acquisition unit adjusts the video recording or audio recording settings in accordance with local laws. For example, if the meeting is held remotely, the acquisition unit adjusts the video recording or audio recording settings according to the quality of the Internet connection. This makes it possible to adjust the video recording or audio recording settings taking into account the geographical location information of the meeting. Some or all of the above-described processing in the acquisition unit may be performed using, or without, the generation AI. For example, the acquisition unit can input the geographical location information to the generation AI and cause the generation AI to adjust the video recording or audio recording settings.

[0038] The acquisition unit can automatically adjust the quality of the video or audio recording based on the theme and content of the meeting. For example, in the case of a meeting on an important topic, the acquisition unit records or records in high resolution. For example, in the case of a meeting on technical content, the acquisition unit records or records with an emphasis on audio clarity. For example, in the case of an informal meeting, the acquisition unit records or records in standard resolution. This makes it possible to adjust the quality of the video or audio recording based on the theme and content of the meeting. Some or all of the above-mentioned processing in the acquisition unit may be performed using, or without, a generation AI. For example, the acquisition unit may input the theme and content of the meeting into the generation AI and have the generation AI adjust the quality of the video or audio recording.

[0039] The analysis unit can remove noise from the audio data and perform filtering to make the speaker's voice clearer. The analysis unit, for example, removes background noise to make the speaker's voice clearer. The analysis unit, for example, removes echoes to make the speaker's voice clearer. The analysis unit, for example, filters ambient noise to emphasize the speaker's voice. This makes it possible to remove noise from the audio data and make the speaker's voice clearer. Some or all of the above-described processing in the analysis unit may be performed using, or without, the generation AI. For example, the analysis unit can input audio data to the generation AI and have the generation AI perform noise removal.

[0040] The analysis unit can apply an algorithm to learn the characteristics of the speaker's voice and improve the identification accuracy. The analysis unit, for example, learns the characteristics of the speaker's voice and improves the identification accuracy. The analysis unit, for example, analyzes the tone and pitch of the speaker's voice and improves the identification accuracy. The analysis unit, for example, learns the speaker's voice pattern and improves the identification accuracy. This makes it possible to learn the characteristics of the speaker's voice and improve the identification accuracy. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input data on the speaker's voice characteristics into the generation AI and cause the generation AI to improve the identification accuracy.

[0041] When analyzing the audio data, the analysis unit can improve the analysis accuracy based on the background sound of the conference. For example, the analysis unit filters the background sound of the conference to make the speaker's voice clearer. For example, the analysis unit analyzes the background sound of the conference to emphasize the speaker's voice. For example, the analysis unit removes the background sound of the conference to make the speaker's voice clearer. This makes it possible to improve the analysis accuracy by taking the background sound of the conference into consideration. Some or all of the above-mentioned processing in the analysis unit may be performed using, or without, the generation AI. For example, the analysis unit may input background sound data into the generation AI and have the generation AI improve the analysis accuracy.

[0042] The analysis unit can analyze the tone and speed of the speaker's voice and extract emotional nuances from the speech content. The analysis unit, for example, analyzes the tone of the speaker's voice and extracts emotional nuances. The analysis unit, for example, analyzes the speed of the speaker's voice and extracts emotional nuances. The analysis unit, for example, analyzes the strength and weakness of the speaker's voice and extracts emotional nuances. This makes it possible to analyze the tone and speed of the speaker's voice and extract emotional nuances. Some or all of the above-described processing in the analysis unit may be performed using, or without, the generation AI. For example, the analysis unit can input the speaker's voice data into the generation AI and cause the generation AI to extract emotional nuances.

[0043] The organizing unit can assign tags to the organized text data according to the importance of the comments. For example, the organizing unit assigns an "important" tag to important comments. For example, the organizing unit assigns a "decision" tag to decisions. For example, the organizing unit assigns an "action" tag to action items. By assigning tags according to the importance of comments, important comments can be easily identified. Some or all of the above-mentioned processing in the organizing unit may be performed using, or without, a generation AI. For example, the organizing unit can input text data to a generation AI and have the generation AI assign tags.

[0044] The organizing unit can analyze the context of the text data and group related statements. For example, the organizing unit groups statements about the same topic. For example, the organizing unit groups statements by the same speaker. For example, the organizing unit groups statements made in the same time period. In this way, by grouping related statements, it becomes possible to organize based on context. Some or all of the above-mentioned processing in the organizing unit may be performed using, or without, the generation AI. For example, the organizing unit may input text data into the generation AI and have the generation AI perform the grouping.

[0045] The organizing unit can color-code the organized text data according to the theme of the meeting. For example, the organizing unit displays important themes in red. For example, the organizing unit displays decisions in blue. For example, the organizing unit displays action items in green. This improves visibility by color-coding according to the theme of the meeting. Some or all of the above-described processing in the organizing unit may be performed using, or without, a generation AI. For example, the organizing unit can input text data into the generation AI and have the generation AI perform the color coding.

[0046] When organizing text data, the organizing unit can adjust the organizing method based on the speaker's job title and field of expertise. The organizing unit organizes the text data, for example, taking into account importance according to the speaker's job title. The organizing unit groups related statements, for example, according to the speaker's field of expertise. The organizing unit adjusts the display order of the text data, for example, taking into account the speaker's job title and field of expertise. This enables more appropriate organization by adjusting the organizing method taking into account the speaker's job title and field of expertise. Some or all of the above-described processing in the organizing unit may be performed using, or without, a generation AI. For example, the organizing unit can input data on the speaker's job title and field of expertise into the generation AI and have the generation AI adjust the organizing method.

[0047] The sending unit can automatically select a destination according to the importance of the data when sending. For example, the sending unit sends important data to the conference organizer. For example, the sending unit sends decisions to the relevant team. For example, the sending unit sends action items to the person in charge. This makes it possible to automatically select a destination according to the importance of the data. Some or all of the above-mentioned processing in the sending unit may be performed using, or without, the generation AI. For example, the sending unit can input the importance of the data into the generation AI and have the generation AI select a destination.

[0048] The transmitting unit can check the consistency of the data and detect errors before transmission. The transmitting unit, for example, checks the consistency of the data and detects errors. The transmitting unit, for example, checks the format of the data and detects errors. The transmitting unit, for example, checks the content of the data and detects errors. This makes it possible to check the consistency of the data and detect errors. Some or all of the above-mentioned processing in the transmitting unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the transmitting unit can input data to the generation AI and have the generation AI perform consistency checks and error checks.

[0049] The transmitting unit can encrypt data during transmission to enhance security. For example, the transmitting unit applies AES encryption when transmitting important data. For example, the transmitting unit applies RSA encryption when transmitting data including confidential information. For example, the transmitting unit applies SSL / TLS encryption to all transmitted data. This makes it possible to encrypt data and enhance security. Some or all of the above-mentioned processing in the transmitting unit may be performed using, or without, the generation AI. For example, the transmitting unit can input data to the generation AI and have the generation AI perform encryption.

[0050] The transmitting unit can select an appropriate transmission method based on the device information of the destination. For example, if the destination is a smartphone, the transmitting unit transmits in a mobile-friendly format. For example, if the destination is a desktop, the transmitting unit transmits in a format including detailed information. For example, if the destination is a tablet, the transmitting unit transmits in a format with high visibility. This makes it possible to select the optimal transmission method taking into account the device information of the destination. Some or all of the above-mentioned processing in the transmitting unit may be performed using, or without, the generation AI. For example, the transmitting unit can input the device information of the destination into the generation AI and have the generation AI select the transmission method.

[0051] The link issuing unit can automatically set access permissions when issuing a link, ensuring security. For example, the link issuing unit issues a link that can only be accessed by specific users for important data. For example, the link issuing unit issues a password-protected link for confidential information. For example, the link issuing unit records an access log for all links. This makes it possible to automatically set access permissions and ensure security. Some or all of the above-mentioned processing in the link issuing unit may be performed using, or without, a generation AI. For example, the link issuing unit can have the generation AI set the access permissions for the link.

[0052] The link issuing unit can set an expiration date for a link and automatically disable the link after it expires. For example, the link issuing unit sets a 24-hour expiration date for important data. For example, the link issuing unit sets a one-week expiration date for confidential information. For example, the link issuing unit sets a one-month expiration date for all links. This makes it possible to set an expiration date for a link and automatically disable the link after it expires. Some or all of the above-described processing in the link issuing unit may be performed using, or without, a generation AI. For example, the link issuing unit can have a generation AI set the expiration date for the link.

[0053] The link issuing unit can record an access log when issuing a link so that it can be checked later. The link issuing unit, for example, records information about users who accessed the link. The link issuing unit, for example, records the date and time when the link was accessed. The link issuing unit, for example, records the number of times the link was accessed. This makes it possible to record an access log so that it can be checked later. Some or all of the above-mentioned processing in the link issuing unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the link issuing unit can cause the generation AI to record the access log.

[0054] The link issuing unit can automatically attach related materials and files when issuing a link. For example, the link issuing unit attaches a meeting minutes file to the link. For example, the link issuing unit attaches presentation materials for the meeting to the link. For example, the link issuing unit attaches a recording file of the meeting to the link. This makes it possible to automatically attach related materials and files. Some or all of the above-mentioned processing in the link issuing unit may be performed using, or without, a generation AI. For example, the link issuing unit may cause a generation AI to attach materials and files.

[0055] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0056] The acquisition unit can detect the type of device used by the meeting participants and automatically adjust the optimal recording or audio recording settings. For example, if a participant is using a smartphone, a low-bandwidth recording setting can be used. If a participant is using a high-resolution camera, a high-resolution recording setting can be used. Furthermore, if a participant is using multiple devices, the optimal device can be selected for recording or audio recording. This enables optimal recording or audio recording settings to be set according to the participant's device.

[0057] The analysis unit can generate a summary of meeting minutes in real time based on what is said during the meeting. For example, if an important statement is made, that statement can be instantly summarized and displayed while the meeting is in progress. It can also ask the speaker for confirmation before the statement is reflected in the minutes. Furthermore, if the statement is related to a specific keyword, it can be automatically tagged based on that keyword. This makes it possible to summarize and check meeting minutes in real time during the meeting.

[0058] The organizer can automatically classify and visually display the content of comments according to the progress of the meeting. For example, the content of comments can be classified by agenda and displayed in different colors. Also, different icons can be used to visually distinguish between speakers. Furthermore, if the content of a comment is related to a specific action item, the action item can be highlighted. This allows for visual organization according to the progress of the meeting.

[0059] The transmission unit can automatically send a survey to all participants after the end of a meeting to collect evaluations of the meeting. For example, it can send a survey requesting evaluations of the content and progress of the meeting. It can also collect feedback from participants and identify areas for improvement for the next meeting. Furthermore, it can automatically compile the survey results and report them to the meeting organizer. This allows for efficient evaluation of meetings and collection of feedback.

[0060] The link publisher can provide a search function for meeting recordings or audio recordings based on specific keywords. For example, by entering a specific keyword used during a meeting, you can quickly search for speeches containing that keyword. Search results can also be displayed with timestamps, allowing direct access to the relevant section. Furthermore, search results can be exported for future reference. This allows for efficient search and reference of meeting recordings or audio recordings.

[0061] The processing flow of the first embodiment will be briefly explained below.

[0062] Step 1: The acquisition unit acquires video or audio recording data of the meeting. The acquisition unit may, for example, configure a web conferencing tool such as Zoom or Teams to record the meeting. The acquisition unit may also save the video or audio recording data of the meeting in cloud storage or local storage. Step 2: The analysis unit analyzes the data acquired by the acquisition unit and extracts the speaker's name and the content of the speech. The analysis unit converts the voice data into text using speech recognition technology and organizes it by speaker using generation AI. Furthermore, the speech recognition technology is used to extract the speaker's name. Step 3: The organizer organizes the utterances extracted by the analyzer and assigns timestamps. The organizer assigns timestamps to the organized text data and organizes the utterances in chronological order. The utterances are also organized by speaker. Step 4: The sending unit sends the data organized by the organizing unit to the conference organizer. The sending unit can send the organized data by email, upload it to cloud storage, and send a link to the conference organizer. It can also send it via a chat tool. Step 5: The link issuing unit issues a link to the data organized by the organizing unit. The link issuing unit generates a link to the organized data and can restrict access. It is also possible to send the link via email or chat tool.

[0063] (Example 2) A meeting recording system according to an embodiment of the present invention acquires video or audio recordings of web conferences, concisely transcribes the names of speakers and their comments after the conference, and sends them to the conference organizer or issues a link to the minutes file to the communication tool team. This meeting recording system reduces work workload by acquiring video or audio recordings of conferences, extracting and organizing the speaker names and their comments, and adding a timestamp before sending or issuing a link. For example, the meeting recording system configures a web conferencing tool such as Zoom or Teams to record a conference. After the conference ends, the video or audio recording is analyzed to extract the speaker names and their comments. This analysis is performed using a generation AI. The generation AI uses speech recognition technology to convert the audio data into text and organize it by speaker. The organized text data is timestamped and sent to the conference organizer, or a link to the minutes file is issued to the communication tool team. Access to this link is restricted to ensure security. This system significantly reduces the time and effort required to create meeting minutes. This enables the meeting recording system to efficiently acquire, analyze, organize, transmit, and issue links to video or audio recordings of meetings.

[0064] A conference recording system according to an embodiment includes an acquisition unit, an analysis unit, a compilation unit, a transmission unit, and a link issuing unit. The acquisition unit acquires video or audio recording data of a conference. For example, the acquisition unit configures a web conferencing tool such as Zoom or Teams to record the conference. The acquisition unit can also store the video or audio recording data of the conference in cloud storage. For example, the acquisition unit can also store the video or audio recording data of the conference in local storage. The analysis unit analyzes the data acquired by the acquisition unit and extracts the names of speakers and the content of their comments. For example, the analysis unit converts the audio data into text using speech recognition technology. For example, the analysis unit organizes the data by speaker using generation AI. For example, the analysis unit extracts the names of speakers using speech recognition technology. The organization unit organizes the content of the comments extracted by the analysis unit and assigns timestamps to the extracted text data. For example, the organization unit assigns timestamps to the organized text data. For example, the organization unit organizes the content of the comments in chronological order. For example, the organization unit organizes the content of the comments by speaker. The sending unit sends the data organized by the organizing unit to the conference organizer. The sending unit, for example, sends the organized data by email. The sending unit, for example, uploads the organized data to cloud storage and sends a link to the conference organizer. The sending unit, for example, sends the organized data by a chat tool. The link issuing unit issues a link to the data organized by the organizing unit. The link issuing unit, for example, generates a link to the organized data and imposes access restrictions. The link issuing unit, for example, sends the link to the organized data by email. The link issuing unit, for example, sends the link to the organized data by a chat tool. This enables the conference recording system according to the embodiment to efficiently acquire, analyze, organize, send, and issue links to video or audio recording data of a conference.

[0065] The acquisition unit can set up a web conferencing tool, such as Zoom or Teams, to record or record a meeting. For example, the acquisition unit sets up a web conferencing tool, such as Zoom, to record a meeting. For example, the acquisition unit sets up a web conferencing tool, such as Teams, to record a meeting. For example, the acquisition unit selects a recording or audio recording option on a setting screen of the web conferencing tool. This enables recording or audio recording settings in the web conferencing tool. Some or all of the above-described processing in the acquisition unit may be performed using, or without, a generation AI. For example, the acquisition unit inputs the setting screen of the web conferencing tool into the generation AI and causes the generation AI to set up recording or audio recording.

[0066] The analysis unit can convert the voice data into text using voice recognition technology and organize the text by speaker. The analysis unit, for example, converts the voice data into text using voice recognition technology. The analysis unit, for example, analyzes the voice data using deep learning. The analysis unit, for example, analyzes the voice data using HMM (Hidden Markov Model). The analysis unit, for example, extracts the name of the speaker using voice recognition technology. The analysis unit, for example, converts the speech content into text using voice recognition technology. The analysis unit, for example, applies an algorithm for organizing the speech by speaker. This makes it possible to convert the voice data into text and organize it by speaker. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input the voice data into a generation AI and have the generation AI convert it into text data.

[0067] The organizing unit can assign a timestamp to the organized text data. For example, the organizing unit assigns a timestamp to the organized text data. For example, the organizing unit assigns a timestamp to each utterance in the text data. For example, the organizing unit organizes the utterance content in chronological order. For example, the organizing unit organizes the utterance content by speaker. This makes it possible to assign a timestamp to the organized text data. Some or all of the above-described processing in the organizing unit may be performed using, or without, the generation AI. For example, the organizing unit may input the text data to the generation AI and cause the generation AI to assign a timestamp.

[0068] The sending unit can send the organized data to the conference organizer. For example, the sending unit sends the organized data by email. For example, the sending unit uploads the organized data to cloud storage and sends a link to the conference organizer. For example, the sending unit sends the organized data by a chat tool. This makes it possible to send the organized data to the conference organizer. Some or all of the above-described processing in the sending unit may be performed using, for example, a generation AI, or may be performed without using the generation AI. For example, the sending unit can input the organized data into the generation AI and have the generation AI execute the transmission.

[0069] The link issuing unit can issue a link to the organized data and impose access restrictions. The link issuing unit, for example, generates a link to the organized data and imposes access restrictions. The link issuing unit, for example, sends a link to the organized data by email. The link issuing unit, for example, sends a link to the organized data using a chat tool. This makes it possible to issue a link to the organized data and impose access restrictions. Some or all of the above-mentioned processing in the link issuing unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the link issuing unit may cause the generation AI to generate the link and set the access restrictions.

[0070] The acquisition unit can estimate the user's emotions and adjust the start timing of video or audio recording based on the estimated user emotions. For example, if the user is nervous, the acquisition unit starts video or audio recording immediately after the start of the meeting. For example, if the user is relaxed, the acquisition unit records only important parts of the meeting. For example, if the user is in a hurry, the acquisition unit records the entire meeting. This makes it possible to adjust the start timing of video or audio recording based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-mentioned processing in the acquisition unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the acquisition unit can input the user's emotion data into the generation AI and have the generation AI adjust the start timing of video or audio recording.

[0071] The acquisition unit can automatically optimize the video or audio recording settings according to the type and importance of the meeting. For example, if the meeting is important, the acquisition unit records or records in high resolution. For example, if the meeting is a regular meeting, the acquisition unit records or records in standard resolution. For example, if the meeting is informal, the acquisition unit records or records in low resolution. This makes it possible to optimize the video or audio recording settings according to the type and importance of the meeting. Some or all of the above-mentioned processing in the acquisition unit may be performed using, or without, the generation AI. For example, the acquisition unit may input the type and importance of the meeting into the generation AI and cause the generation AI to optimize the video or audio recording settings.

[0072] The acquisition unit can acquire a list of participants in a conference in advance and perform filtering to improve the accuracy of speaker identification. The acquisition unit, for example, acquires a list of participants in a conference and learns the voice characteristics of the speakers in advance. The acquisition unit, for example, performs filtering to improve the accuracy of speaker identification based on the participant list. The acquisition unit, for example, acquires information necessary for speaker identification based on the participant list in advance. This makes it possible to improve the accuracy of speaker identification. Some or all of the above-mentioned processing in the acquisition unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the acquisition unit can input the participant list to the generation AI and cause the generation AI to perform filtering to improve the accuracy of speaker identification.

[0073] The acquisition unit can estimate the user's emotions and determine the priority of video or audio recording based on the estimated user emotions. For example, if the user is nervous, the acquisition unit prioritizes video or audio recording of important statements. For example, if the user is relaxed, the acquisition unit records video or audio recording while taking into account the overall balance. For example, if the user is in a hurry, the acquisition unit records video or audio recording of the entire meeting. This makes it possible to determine the priority of video or audio recording based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-described processing in the acquisition unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the acquisition unit can input the user's emotion data into the generation AI and have the generation AI determine the priority of video or audio recording.

[0074] The acquisition unit can adjust the video recording or audio recording settings based on the geographical location information of the meeting. For example, if the meeting is held in a different time zone, the acquisition unit adjusts the start time of the video recording or audio recording. For example, if the meeting is held in a different country, the acquisition unit adjusts the video recording or audio recording settings in accordance with local laws. For example, if the meeting is held remotely, the acquisition unit adjusts the video recording or audio recording settings according to the quality of the Internet connection. This makes it possible to adjust the video recording or audio recording settings taking into account the geographical location information of the meeting. Some or all of the above-described processing in the acquisition unit may be performed using, or without, the generation AI. For example, the acquisition unit can input the geographical location information to the generation AI and cause the generation AI to adjust the video recording or audio recording settings.

[0075] The acquisition unit can automatically adjust the quality of the video or audio recording based on the theme and content of the meeting. For example, in the case of a meeting on an important topic, the acquisition unit records or records in high resolution. For example, in the case of a meeting on technical content, the acquisition unit records or records with an emphasis on audio clarity. For example, in the case of an informal meeting, the acquisition unit records or records in standard resolution. This makes it possible to adjust the quality of the video or audio recording based on the theme and content of the meeting. Some or all of the above-mentioned processing in the acquisition unit may be performed using, or without, a generation AI. For example, the acquisition unit may input the theme and content of the meeting into the generation AI and have the generation AI adjust the quality of the video or audio recording.

[0076] The analysis unit can estimate the user's emotions and adjust the analysis method of the voice data based on the estimated user emotions. For example, if the user is nervous, the analysis unit removes noise from the voice data and analyzes clearer voice. For example, if the user is relaxed, the analysis unit analyzes natural voice. For example, if the user is in a hurry, the analysis unit analyzes the voice data quickly. This makes it possible to adjust the analysis method of the voice data based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the analysis unit can input the user's emotion data into the generation AI and have the generation AI adjust the analysis method of the voice data.

[0077] The analysis unit can remove noise from the audio data and perform filtering to make the speaker's voice clearer. The analysis unit, for example, removes background noise to make the speaker's voice clearer. The analysis unit, for example, removes echoes to make the speaker's voice clearer. The analysis unit, for example, filters ambient noise to emphasize the speaker's voice. This makes it possible to remove noise from the audio data and make the speaker's voice clearer. Some or all of the above-described processing in the analysis unit may be performed using, or without, the generation AI. For example, the analysis unit can input audio data to the generation AI and have the generation AI perform noise removal.

[0078] The analysis unit can apply an algorithm to learn the characteristics of the speaker's voice and improve the identification accuracy. The analysis unit, for example, learns the characteristics of the speaker's voice and improves the identification accuracy. The analysis unit, for example, analyzes the tone and pitch of the speaker's voice and improves the identification accuracy. The analysis unit, for example, learns the speaker's voice pattern and improves the identification accuracy. This makes it possible to learn the characteristics of the speaker's voice and improve the identification accuracy. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, a generation AI, or may be performed without using a generation AI. For example, the analysis unit can input data on the speaker's voice characteristics into the generation AI and cause the generation AI to improve the identification accuracy.

[0079] The analysis unit can estimate the user's emotions and adjust the display method of the analysis results based on the estimated user emotions. For example, if the user is nervous, the analysis unit provides a simple, highly visible display method. For example, if the user is relaxed, the analysis unit provides a display method including detailed information. For example, if the user is in a hurry, the analysis unit provides a display method that focuses on the main points. This makes it possible to adjust the display method of the analysis results based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-mentioned processing in the analysis unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the analysis unit can input the user's emotion data into the generation AI and have the generation AI adjust the display method of the analysis results.

[0080] When analyzing the audio data, the analysis unit can improve the analysis accuracy based on the background sound of the conference. For example, the analysis unit filters the background sound of the conference to make the speaker's voice clearer. For example, the analysis unit analyzes the background sound of the conference to emphasize the speaker's voice. For example, the analysis unit removes the background sound of the conference to make the speaker's voice clearer. This makes it possible to improve the analysis accuracy by taking the background sound of the conference into consideration. Some or all of the above-mentioned processing in the analysis unit may be performed using, or without, the generation AI. For example, the analysis unit may input background sound data into the generation AI and have the generation AI improve the analysis accuracy.

[0081] The analysis unit can analyze the tone and speed of the speaker's voice and extract emotional nuances from the speech content. The analysis unit, for example, analyzes the tone of the speaker's voice and extracts emotional nuances. The analysis unit, for example, analyzes the speed of the speaker's voice and extracts emotional nuances. The analysis unit, for example, analyzes the strength and weakness of the speaker's voice and extracts emotional nuances. This makes it possible to analyze the tone and speed of the speaker's voice and extract emotional nuances. Some or all of the above-described processing in the analysis unit may be performed using, or without, the generation AI. For example, the analysis unit can input the speaker's voice data into the generation AI and cause the generation AI to extract emotional nuances.

[0082] The organizing unit can estimate the user's emotions and adjust the way the text data is organized based on the estimated user's emotions. For example, if the user is nervous, the organizing unit provides simple, highly visible text data. For example, if the user is relaxed, the organizing unit provides text data containing detailed information. For example, if the user is in a hurry, the organizing unit provides text data that focuses on the main points. This makes it possible to adjust the way the text data is organized based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the organizing unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the organizing unit can input the user's emotion data into the generation AI and cause the generation AI to adjust the way the text data is organized.

[0083] The organizing unit can assign tags to the organized text data according to the importance of the comments. For example, the organizing unit assigns an "important" tag to important comments. For example, the organizing unit assigns a "decision" tag to decisions. For example, the organizing unit assigns an "action" tag to action items. By assigning tags according to the importance of comments, important comments can be easily identified. Some or all of the above-mentioned processing in the organizing unit may be performed using, or without, a generation AI. For example, the organizing unit can input text data to a generation AI and have the generation AI assign tags.

[0084] The organizing unit can analyze the context of the text data and group related statements. For example, the organizing unit groups statements about the same topic. For example, the organizing unit groups statements by the same speaker. For example, the organizing unit groups statements made in the same time period. In this way, by grouping related statements, it becomes possible to organize based on context. Some or all of the above-mentioned processing in the organizing unit may be performed using, or without, the generation AI. For example, the organizing unit may input text data into the generation AI and have the generation AI perform the grouping.

[0085] The organizing unit can estimate the user's emotions and adjust the display order of text data based on the estimated user's emotions. For example, if the user is nervous, the organizing unit prioritizes displaying important comments. For example, if the user is relaxed, the organizing unit displays the text while taking into consideration the overall balance. For example, if the user is in a hurry, the organizing unit prioritizes displaying comments that highlight the main points. This makes it possible to adjust the display order of text data based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the organizing unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the organizing unit can input the user's emotion data into the generation AI and have the generation AI adjust the display order.

[0086] The organizing unit can color-code the organized text data according to the theme of the meeting. For example, the organizing unit displays important themes in red. For example, the organizing unit displays decisions in blue. For example, the organizing unit displays action items in green. This improves visibility by color-coding according to the theme of the meeting. Some or all of the above-described processing in the organizing unit may be performed using, or without, a generation AI. For example, the organizing unit can input text data into the generation AI and have the generation AI perform the color coding.

[0087] When organizing text data, the organizing unit can adjust the organizing method based on the speaker's job title and field of expertise. The organizing unit organizes the text data, for example, taking into account importance according to the speaker's job title. The organizing unit groups related statements, for example, according to the speaker's field of expertise. The organizing unit adjusts the display order of the text data, for example, taking into account the speaker's job title and field of expertise. This enables more appropriate organization by adjusting the organizing method taking into account the speaker's job title and field of expertise. Some or all of the above-described processing in the organizing unit may be performed using, or without, a generation AI. For example, the organizing unit can input data on the speaker's job title and field of expertise into the generation AI and have the generation AI adjust the organizing method.

[0088] The sending unit can estimate the user's emotions and adjust the sending method based on the estimated user's emotions. For example, if the user is nervous, the sending unit sends a simple email. For example, if the user is relaxed, the sending unit sends an email containing detailed information. For example, if the user is in a hurry, the sending unit sends an email that focuses on the main points. This makes it possible to adjust the sending method based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, with an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the sending unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the sending unit can input the user's emotion data into the generation AI and have the generation AI adjust the sending method.

[0089] The sending unit can automatically select a destination according to the importance of the data when sending. For example, the sending unit sends important data to the conference organizer. For example, the sending unit sends decisions to the relevant team. For example, the sending unit sends action items to the person in charge. This makes it possible to automatically select a destination according to the importance of the data. Some or all of the above-mentioned processing in the sending unit may be performed using, or without, the generation AI. For example, the sending unit can input the importance of the data into the generation AI and have the generation AI select a destination.

[0090] The transmitting unit can check the consistency of the data and detect errors before transmission. The transmitting unit, for example, checks the consistency of the data and detects errors. The transmitting unit, for example, checks the format of the data and detects errors. The transmitting unit, for example, checks the content of the data and detects errors. This makes it possible to check the consistency of the data and detect errors. Some or all of the above-mentioned processing in the transmitting unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the transmitting unit can input data to the generation AI and have the generation AI perform consistency checks and error checks.

[0091] The transmission unit can estimate the user's emotions and adjust the transmission timing based on the estimated user emotions. For example, if the user is nervous, the transmission unit transmits immediately after the end of a meeting. For example, if the user is relaxed, the transmission unit transmits after summarizing the main points of the meeting. For example, if the user is in a hurry, the transmission unit transmits in real time during the meeting. This makes it possible to adjust the transmission timing based on the user's emotions. Emotion estimation is realized using an emotion estimation function, for example, using an emotion engine or a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above-mentioned processing in the transmission unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the transmission unit can input the user's emotion data into the generation AI and have the generation AI adjust the transmission timing.

[0092] The transmitting unit can encrypt data during transmission to enhance security. For example, the transmitting unit applies AES encryption when transmitting important data. For example, the transmitting unit applies RSA encryption when transmitting data including confidential information. For example, the transmitting unit applies SSL / TLS encryption to all transmitted data. This makes it possible to encrypt data and enhance security. Some or all of the above-mentioned processing in the transmitting unit may be performed using, or without, the generation AI. For example, the transmitting unit can input data to the generation AI and have the generation AI perform encryption.

[0093] The transmitting unit can select an appropriate transmission method based on the device information of the destination. For example, if the destination is a smartphone, the transmitting unit transmits in a mobile-friendly format. For example, if the destination is a desktop, the transmitting unit transmits in a format including detailed information. For example, if the destination is a tablet, the transmitting unit transmits in a format with high visibility. This makes it possible to select the optimal transmission method taking into account the device information of the destination. Some or all of the above-mentioned processing in the transmitting unit may be performed using, or without, the generation AI. For example, the transmitting unit can input the device information of the destination into the generation AI and have the generation AI select the transmission method.

[0094] The link issuing unit can estimate the user's emotions and adjust the link issuing method based on the estimated user's emotions. For example, if the user is nervous, the link issuing unit issues a simple link. For example, if the user is relaxed, the link issuing unit issues a link containing detailed information. For example, if the user is in a hurry, the link issuing unit issues a link that can be accessed quickly. This makes it possible to adjust the link issuing method based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-mentioned processing in the link issuing unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the link issuing unit can input the user's emotion data into the generation AI and have the generation AI adjust the link issuing method.

[0095] The link issuing unit can automatically set access permissions when issuing a link, ensuring security. For example, the link issuing unit issues a link that can only be accessed by specific users for important data. For example, the link issuing unit issues a password-protected link for confidential information. For example, the link issuing unit records an access log for all links. This makes it possible to automatically set access permissions and ensure security. Some or all of the above-mentioned processing in the link issuing unit may be performed using, or without, a generation AI. For example, the link issuing unit can have the generation AI set the access permissions for the link.

[0096] The link issuing unit can set an expiration date for a link and automatically disable the link after it expires. For example, the link issuing unit sets a 24-hour expiration date for important data. For example, the link issuing unit sets a one-week expiration date for confidential information. For example, the link issuing unit sets a one-month expiration date for all links. This makes it possible to set an expiration date for a link and automatically disable the link after it expires. Some or all of the above-described processing in the link issuing unit may be performed using, or without, a generation AI. For example, the link issuing unit can have a generation AI set the expiration date for the link.

[0097] The link issuing unit can estimate the user's emotions and adjust the link display method based on the estimated user emotions. For example, if the user is nervous, the link issuing unit displays a simple, highly visible link. For example, if the user is relaxed, the link issuing unit displays a link containing detailed information. For example, if the user is in a hurry, the link issuing unit displays a link that can be accessed quickly. This makes it possible to adjust the link display method based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, for example, an emotion engine or a generation AI. The generation AI may be, but is not limited to, a text generation AI (e.g., LLM) or a multimodal generation AI. Some or all of the above-mentioned processing in the link issuing unit may be performed using, for example, the generation AI, or may be performed without using the generation AI. For example, the link issuing unit can input the user's emotion data into the generation AI and have the generation AI adjust the link display method.

[0098] The link issuing unit can record an access log when issuing a link so that it can be checked later. The link issuing unit, for example, records information about users who accessed the link. The link issuing unit, for example, records the date and time when the link was accessed. The link issuing unit, for example, records the number of times the link was accessed. This makes it possible to record an access log so that it can be checked later. Some or all of the above-mentioned processing in the link issuing unit may be performed, for example, using a generation AI, or may be performed without using a generation AI. For example, the link issuing unit can cause the generation AI to record the access log.

[0099] The link issuing unit can automatically attach related materials and files when issuing a link. For example, the link issuing unit attaches a meeting minutes file to the link. For example, the link issuing unit attaches presentation materials for the meeting to the link. For example, the link issuing unit attaches a recording file of the meeting to the link. This makes it possible to automatically attach related materials and files. Some or all of the above-mentioned processing in the link issuing unit may be performed using, or without, a generation AI. For example, the link issuing unit may cause a generation AI to attach materials and files. === Hard Collateral 1-1 === Each of the multiple elements, including the acquisition unit, analysis unit, organization unit, transmission unit, and link issuing unit, is implemented, for example, by at least one of the smart device 14 and the data processing device 12. For example, the acquisition unit acquires video or audio data of the conference using the camera 42 or microphone 38B of the smart device 14. The analysis unit, implemented, for example, by the specific processing unit 290 of the data processing device 12, converts audio data into text using voice recognition technology. The organization unit, implemented, for example, by the specific processing unit 290 of the data processing device 12, organizes the content of comments in chronological order and assigns timestamps. The transmission unit, implemented, for example, by the control unit 46A of the smart device 14, transmits the organized data to the conference organizer. The link issuing unit, implemented, for example, by the specific processing unit 290 of the data processing device 12, issues a link to the organized data and imposes access restrictions. === Hard Collateral 1-2 === Each of the multiple elements, including the acquisition unit, analysis unit, organization unit, transmission unit, and link issuing unit, described above, is implemented, for example, by at least one of the smart glasses 214 and the data processing device 12. For example, the acquisition unit acquires video or audio data of the conference using the camera 42 or microphone 238 of the smart glasses 214. The analysis unit, implemented, for example, by the specific processing unit 290 of the data processing device 12, converts audio data into text using voice recognition technology. The organization unit, implemented, for example, by the specific processing unit 290 of the data processing device 12, organizes the speech content in chronological order and assigns timestamps. The transmission unit, implemented, for example, by the control unit 46A of the smart glasses 214, transmits the organized data to the conference organizer. The link issuing unit, implemented, for example, by the specific processing unit 290 of the data processing device 12, issues a link to the organized data and imposes access restrictions. === Hard Collateral 1-3 === Each of the multiple elements, including the acquisition unit, analysis unit, organization unit, transmission unit, and link issuing unit, described above, is realized, for example, by at least one of the headset-type terminal 314 and the data processing device 12. For example, the acquisition unit acquires video or audio data of the conference using the camera 42 or microphone 238 of the headset-type terminal 314. The analysis unit, realized, for example, by the specific processing unit 290 of the data processing device 12, converts audio data into text using voice recognition technology. The organization unit, realized, for example, by the specific processing unit 290 of the data processing device 12, organizes the content of comments in chronological order and assigns timestamps. The transmission unit, realized, for example, by the control unit 46A of the headset-type terminal 314, transmits the organized data to the conference organizer. The link issuing unit, realized, for example, by the specific processing unit 290 of the data processing device 12, issues a link to the organized data and imposes access restrictions. === Hard Collateral 1-4 === Each of the multiple elements, including the acquisition unit, analysis unit, organization unit, transmission unit, and link issuing unit, described above, is realized, for example, by at least one of the robot 414 and the data processing device 12. For example, the acquisition unit acquires video or audio data of the conference using the camera 42 or microphone 238 of the robot 414. The analysis unit, realized, for example, by the specific processing unit 290 of the data processing device 12, converts audio data into text using speech recognition technology. The organization unit, realized, for example, by the specific processing unit 290 of the data processing device 12, organizes the content of comments in chronological order and assigns timestamps. The transmission unit, realized, for example, by the control unit 46A of the robot 414, transmits the organized data to the conference organizer. The link issuing unit, realized, for example, by the specific processing unit 290 of the data processing device 12, issues a link to the organized data and imposes access restrictions.

[0100] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0101] The acquisition unit can detect the type of device used by the meeting participants and automatically adjust the optimal recording or audio recording settings. For example, if a participant is using a smartphone, a low-bandwidth recording setting can be used. If a participant is using a high-resolution camera, a high-resolution recording setting can be used. Furthermore, if a participant is using multiple devices, the optimal device can be selected for recording or audio recording. This enables optimal recording or audio recording settings to be set according to the participant's device.

[0102] The analysis unit can generate a summary of meeting minutes in real time based on what is said during the meeting. For example, if an important statement is made, that statement can be instantly summarized and displayed while the meeting is in progress. It can also ask the speaker for confirmation before the statement is reflected in the minutes. Furthermore, if the statement is related to a specific keyword, it can be automatically tagged based on that keyword. This makes it possible to summarize and check meeting minutes in real time during the meeting.

[0103] The organizer can automatically classify and visually display the content of comments according to the progress of the meeting. For example, the content of comments can be classified by agenda and displayed in different colors. Also, different icons can be used to visually distinguish between speakers. Furthermore, if the content of a comment is related to a specific action item, the action item can be highlighted. This allows for visual organization according to the progress of the meeting.

[0104] The transmission unit can automatically send a survey to all participants after the end of a meeting to collect evaluations of the meeting. For example, it can send a survey requesting evaluations of the content and progress of the meeting. It can also collect feedback from participants and identify areas for improvement for the next meeting. Furthermore, it can automatically compile the survey results and report them to the meeting organizer. This allows for efficient evaluation of meetings and collection of feedback.

[0105] The link publisher can provide a search function for meeting recordings or audio recordings based on specific keywords. For example, by entering a specific keyword used during a meeting, you can quickly search for speeches containing that keyword. Search results can also be displayed with timestamps, allowing direct access to the relevant section. Furthermore, search results can be exported for future reference. This allows for efficient search and reference of meeting recordings or audio recordings.

[0106] The acquisition unit can estimate the user's emotions and adjust the quality of the video or audio recording based on the estimated user's emotions. For example, if the user is nervous, the recording can be performed with emphasis on audio clarity. If the user is relaxed, the video resolution can be increased. Also, if the user is in a hurry, the video or audio recording time can be shortened. This provides optimal video or audio recording quality according to the user's emotions.

[0107] The analysis unit can estimate the user's emotions and evaluate the importance of the content of the comments based on the estimated user emotions. For example, if the user is excited, the content of the comments can be determined to be important and analyzed with priority. If the user is calm, the content of the comments can be analyzed with normal importance. Furthermore, if the user is feeling anxious, the content of the comments can be analyzed carefully. This makes it possible to evaluate the importance of the content of the comments based on the user's emotions.

[0108] The organizer can estimate the user's emotions and adjust the format of the text data based on the estimated user's emotions. For example, if the user is nervous, a simple, highly visible format can be provided. If the user is relaxed, a format including detailed information can be provided. If the user is in a hurry, a format that focuses on the main points can be provided. In this way, the optimal text data format according to the user's emotions can be provided.

[0109] The transmission unit can estimate the user's emotions and customize the content to be sent based on the estimated user's emotions. For example, if the user is nervous, it can send concise and to-the-point content. If the user is relaxed, it can send content that includes detailed information. If the user is in a hurry, it can send content that can be quickly confirmed. In this way, optimal content to be sent according to the user's emotions can be provided.

[0110] The link issuing unit can estimate the user's emotions and set the expiration date of the link based on the estimated user's emotions. For example, if the user is nervous, a short expiration date can be set. If the user is relaxed, a long expiration date can be set. Also, if the user is in a hurry, an instantly accessible link can be issued. In this way, the optimal expiration date of the link can be set according to the user's emotions.

[0111] The processing flow of the second embodiment will be briefly explained below.

[0112] Step 1: The acquisition unit acquires video or audio recording data of the meeting. The acquisition unit may, for example, configure a web conferencing tool such as Zoom or Teams to record the meeting. The acquisition unit may also save the video or audio recording data of the meeting in cloud storage or local storage. Step 2: The analysis unit analyzes the data acquired by the acquisition unit and extracts the speaker's name and the content of the speech. The analysis unit converts the voice data into text using speech recognition technology and organizes it by speaker using generation AI. Furthermore, the speech recognition technology is used to extract the speaker's name. Step 3: The organizer organizes the utterances extracted by the analyzer and assigns timestamps. The organizer assigns timestamps to the organized text data and organizes the utterances in chronological order. The utterances are also organized by speaker. Step 4: The sending unit sends the data organized by the organizing unit to the conference organizer. The sending unit can send the organized data by email, upload it to cloud storage, and send a link to the conference organizer. It can also send it via a chat tool. Step 5: The link issuing unit issues a link to the data organized by the organizing unit. The link issuing unit generates a link to the organized data and can restrict access. It is also possible to send the link via email or chat tool.

[0113] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0114] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> Examples of the generative AI include a neural network (NN) and a neural network (NN). The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats of voice data, text data, image data, etc. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and may perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-mentioned parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. The processing performed by an AI including the generative AI may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI including the generative AI.

[0115] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0116] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0117] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0118] 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0119] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0120] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0121] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0122] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0123] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0124] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0125] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0126] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0127] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0128] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0129] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0130] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0131] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0132] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0133] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0134] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0135] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0136] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0137] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0138] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0139] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0140] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0141] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0142] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0143] In the headset type terminal 314, the identification process is performed by the processor 46. A identification program 60 is stored in the storage 50. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as a control unit 46A in accordance with the identification program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the identification processing unit 290 using these models.

[0144] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0145] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0146] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0147] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0148] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0149] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0150] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0151] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0152] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0153] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0154] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0155] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0156] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0157] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0158] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0159] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0160] In the robot 414, the processor 46 performs the identification process. The storage 50 stores the identification program 60. The processor 46 reads the identification program 60 from the storage 50 and executes the read identification program 60 on the RAM 48. The identification process is realized by the processor 46 operating as the control unit 46A in accordance with the identification program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform the same process as the identification processing unit 290 using these models.

[0161] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0162] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0163] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt including an instruction, as well as inference data such as audio data indicating speech, text data indicating text, and image data indicating an image (e.g., still image data or video data). The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in one or more data formats, such as audio data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation models 58 include AIs other than the generation AI. Examples of AIs other than the generation AI include, but are not limited to, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), and naive Bayes. These AIs can perform various types of processing, but are not limited to these examples. The AI ​​may also be an AI agent. When the processing of each of the above-described parts is performed by an AI, the processing may be performed in part or entirely by the AI, but is not limited to these examples. Processing performed by an AI, including the generation AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by an AI, including the generation AI.

[0164] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0165] The correspondence between each part and the device or control part is not limited to the example described above, and various modifications are possible.

[0166] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0167] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0168] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0169] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0170] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0171] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0172] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0173] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0174] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0175] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0176] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0177] The hardware resource for executing a specific process can be any of the following processors: A CPU is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A dedicated electrical circuit, such as a field-programmable gate array (FPGA), a programmable logic device (PLD), or an application-specific integrated circuit (ASIC), is a processor with a circuit configuration specifically designed to execute a specific process. Each processor has built-in or connected memory, and uses the memory to execute the specific process.

[0178] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0179] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0180] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0181] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0182] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0183] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[0184] [Explanation of symbols]

[0185] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. an acquisition unit that acquires video or audio recording data of a meeting; an analysis unit that analyzes the data acquired by the acquisition unit and extracts the name of the speaker and the content of the comment; a compilation unit that compiles the comment contents extracted by the analysis unit and assigns timestamps to them; a sending unit that sends the data organized by the organizing unit to a conference organizer; a link issuing unit that issues a link to the data organized by the organizing unit; A system characterized by:

2. The acquisition unit Set up recording or audio recording of meetings in Zoom or Teams web conferencing tools The system of claim 1 .

3. The analysis unit Using voice recognition technology, audio data is converted into text and organized by speaker. The system of claim 1 .

4. The organizing unit Adding timestamps to organized text data The system of claim 1 .

5. The transmission unit Send the organized data to the conference organizer The system of claim 1 .

6. The link issuing unit Issue links to organized data and restrict access The system of claim 1 .

7. The acquisition unit Estimate the user's emotions and adjust the start timing of video recording based on the estimated user emotions. The system of claim 1 .

8. The acquisition unit Automatically adjust recording settings based on the type and importance of the meeting The system of claim 1 .

Images