Processing device, processing method, and processing program

Abstract

To make it possible to support smooth progress of a conference.SOLUTION: A server device 10 includes: a conference information acquisition unit 132 that acquires a schedule of a conference and a list of agenda items; a prompt creating unit 133 that creates a prompt for instructing to create a summary of the conference on the basis of input text data and sets the created prompt in generated AI (Artificial Intelligence); a recognition unit 134 that converts voice data of each user participated in the conference into text data and inputs the converted text data to the generated AI together with time information; a determination unit 136 that determines a progress status of the conference on the basis of the list of agenda items, a summary of the conference output from the generated AI, and elapsed time of the conference; and an advising unit 137 that causes each of user terminals used by each of the users to output advice information for advising to move to a next agenda item when the progress of the conference is delayed with respect to the list of agenda items.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a processing device, a processing method, and a processing program. [Background technology]

[0002] In recent years, web conferencing services that connect via applications or browsers have become widespread. In these services, the equipment that provides the conferencing service is installed on the network, and users participate in the conference using applications or browsers running on their terminals. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-230532 Summary of the Invention [Problem to be solved by the invention]

[0004] To participate in a web conference, a user must install software that runs on a terminal device, for example, and then start the terminal device, launch the installed software, and then participate in the web conference using the registered conference ID.

[0005] It is desirable to discuss all of the agenda items in a meeting within the allotted time. However, if the meeting is running behind schedule, it may not be possible to discuss all of the agenda items within the allotted time.

[0006] The present invention has been made in view of the above, and has an object to provide a processing device, a processing method, and a processing program that can support the smooth progress of a conference. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the objectives, the processing device of the present invention is characterized by having an acquisition unit that acquires a schedule of a conference in which a user will participate and a list of agenda items for the conference; a creation unit that creates a prompt that instructs the user to create a summary of the conference based on input text data and sets the created prompt in a generative model; a recognition unit that converts the voice data of each user participating in the conference into text data and inputs the converted text data together with time information into the generative model; a determination unit that determines the progress of the conference based on the list of agenda items for the conference, the summary of the conference output from the generative model, and the elapsed time of the conference; and an advice unit that outputs advice information to each user terminal used by each user advising the user to move on to the next agenda item if the progress of the conference is lagging behind the list of agenda items for the conference. [Effects of the Invention]

[0008] According to the present invention, it is possible to support the smooth progress of a conference. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a processing system according to the first embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the configuration of the server device illustrated in FIG. [Figure 3] FIG. 3 is a diagram illustrating the flow of processing in the processing system shown in FIG. [Figure 4] FIG. 4 is a diagram illustrating the flow of processing in the processing system shown in FIG. [Figure 5] FIG. 5 is a diagram illustrating the flow of processing in the processing system shown in FIG. [Figure 6] FIG. 6 is a diagram showing an example of a screen of a user terminal. [Figure 7] FIG. 7 is an example of a sequence diagram showing a processing procedure of the processing method according to the first embodiment. [Figure 8]FIG. 8 is a diagram illustrating a processing flow in the first modification of the first embodiment. [Figure 9] FIG. 9 is an example of a sequence diagram illustrating a processing procedure of a processing method according to the first modification of the first embodiment. [Figure 10] FIG. 10 is an example of a sequence diagram illustrating a processing procedure of a processing method according to the second modification of the first embodiment. [Figure 11] FIG. 11 is a diagram illustrating an example of the configuration of a processing system according to the second embodiment. [Figure 12] FIG. 12 is a diagram illustrating the flow of processing in the processing system according to the second embodiment. [Figure 13] FIG. 13 is an example of a sequence diagram illustrating a processing procedure of the processing method according to the second embodiment. [Figure 14] FIG. 14 is a diagram illustrating an example of a computer that implements a server device by executing a program. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings. Note that the present invention is not limited to this embodiment. In addition, in the description of the drawings, the same parts are designated by the same reference numerals.

[0011] [Embodiment 1] [Processing System] The following describes the configuration of a processing system according to embodiment 1. In a conference (for example, a web conference) between multiple users, the processing system according to embodiment 1 advises users to move on to the next agenda item when the progress of the conference is delayed based on a summary created by a generative AI (artificial intelligence) (generative model).

[0012] 1 is a diagram illustrating an example of the configuration of a processing system according to embodiment 1. As illustrated in FIG. 1, the processing system 1 includes a server device 10, user terminals 20a to 20d, and a generation AI server 30. The server device 10 communicates with the user terminals 20a to 20d and the generation AI server 30.

[0013] The user terminals 20a to 20d are terminals used by the users Ua to Ud. The user terminals 20a to 20d are, for example, personal computers (PCs), notebook PCs, tablet terminals, smartphones, etc. The user terminals 20a to 20d are terminal devices that can input and output voice data and text data and communicate with the server device 10. The users Ua to Ud participate in, for example, a web conference held between the users Ua to Ud via an application or browser running on the user terminals 20a to 20d. In the following description, the user terminals 20a to 20d will be referred to as user terminals 20 when no particular distinction is made. The number of user terminals 20 may be two or more and is not limited to four.

[0014] The generation AI server 30 is equipped with a generation AI 31 (generation model), which is a natural language processing model. The generation AI 31 performs natural language processing on input text data in accordance with set prompts, creates a summary, and outputs it. The generation AI 31 is, for example, a large-scale natural language processing model.

[0015] The server device 10 communicates with the generation AI server 40 and the generation AI server 50. The server device 10 converts the voice data of the users Ua to Ud during the conference into text data and transmits it to the generation AI server 30.

[0016] The server device 10 determines the progress of the conference based on the list of conference agenda items, the summary output from the generation AI server 30, and the elapsed time of the conference. If the progress of the conference is behind schedule based on the list of conference agenda items, the server device 10 advises the users Ua to Ud to move on to the next agenda item, thereby supporting the smooth progress of the conference held between the users Ua to Ud.

[0017] [Server device] Next, a description will be given of the server device 10. Fig. 2 is a diagram showing an example of the configuration of the server device 10 shown in Fig. 1. As shown in Fig. 2, the server device 10 includes a communication unit 11, a storage unit 12, and a control unit 13.

[0018] The communication unit 11 is a communication interface that transmits and receives various information to and from other devices connected via a network, etc. The communication unit 11 is realized by a NIC (Network Interface Card) or the like, and performs communication between the control unit 13 (described later) and other devices (e.g., user terminals 20a to 20d, generation AI server 30) via telecommunication lines such as a LAN (Local Area Network) or the Internet.

[0019] The storage unit 12 is realized by semiconductor memory elements such as RAM (Random Access Memory) and flash memory, and stores processing programs that operate the server device 10, data used during execution of the processing programs, etc. The storage unit 12 has user information 121, schedule information 122, meeting information 123, text data 124, summary data 125, and advice list 126.

[0020] The user information 121 is information including the ID, name, department, position, and project in charge of each of the users Ua to Ud who use the user terminals 20a to 20d.

[0021] The schedule information 122 is a conference schedule for each of the users Ua to Ud. The conference schedule includes, for example, the conference room (in the case of a Web conference, the Web conference ID, etc.), date and time, agenda, members participating in the conference, and information on related conferences and projects.

[0022] The conference information 123 includes the date and time of the conference, the users participating in the conference, an outline of the conference, a list of conference topics, etc. The list of conference topics is a list of topics to be discussed in the conference, and is input by the users participating in the conference before the conference, for example. The conference information 123 may also include the level of confidentiality of each conference.

[0023] The text data 124 is text data converted from the voice data by a recognition unit 134 (described later). The voice data is voice data of each user Ua to Ud who participated in the conference, and is transmitted from the user terminals 20a to 20d.

[0024] The summary data 125 is a summary of the conference generated by the generation AI 31. The summary data 125 associates the ID of each conference, the summary, and time information indicating the period of the conference that the summary covers.

[0025] The advice list 126 is a list of each piece of advice that is transmitted to the user terminals 20a to 20d by the server device 10. In the advice list 126, various pieces of advice are associated with each other according to the progress of the conference.

[0026] For example, in the advice list 126, when the progress of the meeting is delayed, the text "The meeting is delayed" is associated. In the advice list 126, when the progress of the meeting is delayed and there is an agenda item that has not been discussed in the meeting, the text "The meeting is delayed. Let's move on to the next agenda item" is associated.

[0027] Furthermore, in the advice list 126, when the progress of the meeting is delayed and there are multiple (e.g., two) topics that have not been discussed in the meeting, the text "The meeting is delayed. Two topics have not been discussed. Let's move on to the next topic" is associated. The advice list 126 may include text indicating the remaining time in the meeting and the number of topics that have not been discussed, or text indicating the remaining time in the meeting and the titles of the topics that have not been discussed.

[0028] The control unit 13 controls the entire server device 10. The control unit 13 is, for example, an electronic circuit such as a CPU (Central Processing Unit), or an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 13 also has an internal memory for storing programs that define various processing procedures and control data, and executes each process using the internal memory. The control unit 13 also functions as various processing units when various programs are run. The control unit 13 has a schedule acquisition unit 131 (acquisition unit), a conference information acquisition unit 132 (acquisition unit), a prompt creation unit 133 (creation unit), a recognition unit 134, a summary acquisition unit 135, a determination unit 136, and an advice unit 137.

[0029] The schedule acquisition unit 131 acquires the schedules of the users Ua to Ud and stores them in the schedule information 122. The schedule acquisition unit 131 acquires the schedules of the users Ua to Ud, for example, by referring to the conference schedules of the users Ua to Ud registered in the company network.

[0030] The conference information acquisition unit 132 acquires a list of agenda items for conferences held by users Ua to Ud. For example, the conference information acquisition unit 132 causes the user terminal 20a to output a voice or text saying, "Please tell me the agenda items for today's conference," and acquires a list of agenda items for conferences held by users Ua to Ud by the user Ua's voice or text input in response to that. The conference information acquisition unit 132 acquires a list of agenda items for conferences before or at the start of a conference.

[0031] The prompt creation unit 133 creates a prompt that commands the creation of a summary of the meeting based on the input text data, and sets the created prompt in the generation AI 31. The prompt creation unit 133 commands the creation of a summary at a predetermined timing, for example, in the prompt. The predetermined timing may be, for example, every five minutes, when silence continues for a predetermined period of time, when a frequently used word changes, when the agenda changes, or the like, and is set as appropriate. The prompt creation unit 133 may add information about the meeting for which a summary is to be created to the prompt, thereby facilitating the creation of the summary.

[0032] The recognition unit 134 acquires voice data of each user Ua to Ud who spoke in the conference via communication with the user terminals 20a to 20d. The recognition unit 134 performs voice recognition on the acquired voice data and converts the voice data into text data. The recognition unit 134 performs voice recognition processing using a trained voice recognition model (machine learning model) that outputs text data of the spoken voice from the voice data. The recognition unit 134 also associates time information and user information indicating the speaker with the text data and inputs the text data to the generation AI 31.

[0033] The summary acquisition unit 135 acquires the summary created by the generation AI 31. The summary acquisition unit 135 outputs the acquired summary to the determination unit 136 and stores it in the storage unit 12 as summary data 125.

[0034] The determination unit 136 determines the progress of the conference based on the list of agenda items of the conference, the conference summary output from the generation AI 31, and the elapsed time of the conference.

[0035] The determination unit 136 determines which of the agenda items of the meeting have been discussed based on the list of agenda items and the summary of the meeting output from the generation AI 31. Then, the determination unit 136 calculates the remaining time of the meeting based on the elapsed time of the meeting, and determines whether there is sufficient time left in the meeting compared to the number and content of agenda items that have not been discussed. If there is sufficient time left in the meeting compared to the number and content of agenda items that have not been discussed, the determination unit 136 determines that the meeting is not behind schedule. If the determination unit 136 determines that there is not enough time left in the meeting compared to the number and content of agenda items that have not been discussed, the determination unit 136 determines that the meeting is not behind schedule.

[0036] For example, consider a 60-minute meeting with a list of three agenda items. If the determination unit 136 determines that there are 20 minutes left in the meeting and that only one of the agenda items has been discussed based on the meeting summary output from the generation AI 31, it determines that only one of the agenda items has been discussed and that the meeting is behind schedule. Furthermore, if the determination unit 136 determines that there are 20 minutes left in the meeting and that the third agenda item is being discussed based on the meeting summary output from the generation AI 31, it determines that the meeting is not behind schedule. The determination unit 136 outputs the determination result to the advice unit 137.

[0037] The advice unit 137 outputs advice information advising each of the users Ua to Ud to move on to the next agenda item when the progress of the conference is behind schedule compared to the list of agenda items based on the determination result by the determination unit 136. The advice unit 137 refers to the advice list 126 and creates the advice information in accordance with predetermined rules.

[0038] [Processing flow] Next, a description will be given of the flow of processing in the processing system 1. Figures 3 to 5 are diagrams illustrating the flow of processing in the processing system 1 shown in Figure 1. Figure 6 is a diagram showing an example of a screen of the user terminal 20a.

[0039] 3, the server device 10 communicates with, for example, the user terminal 20a used by the user Ua to acquire the conference schedule and conference information (step S1). The server device 10 inputs the conference information into a prompt that commands the creation of a conference summary (step S2). The server device 10 sets the prompt in the generation AI 31 (steps S3 and S4).

[0040] 4, when the conference begins, the server device 10 acquires voice data of the users Ua to Ud during the conference through communication with the user terminals 20a to 20d (steps S5-1 to S5-4). Then, the server device 10 performs voice recognition on the acquired voice data of the users Ua to Ud and converts the voice data into text data (step S6). The server device 10 associates time information and user information indicating the speaker with the conference text data and inputs them to the generation AI 31 (step S7).

[0041] The generation AI 31 creates a summary of the meeting based on the input text data (step S8). The generation AI 31 may associate the user's identification information with the portion of the summary that corresponds to the user's remarks. The generation AI 31 then transmits the created summary to the server device 10 (step S9).

[0042] 5, the server device 10 determines the progress of the conference based on the list of agenda items of the conference, the conference summary output from the generation AI 31, and the elapsed time of the conference (step S10). If the server device 10 determines that the progress of the conference is delayed, it transmits advice information to each of the user terminals 20a to 20d (steps S11-1 to S11-4).

[0043] 6 is a diagram showing an example of a screen of the user terminal 20a. For example, as shown in screen M1 in FIG. 6, the server device 10 causes the user terminal 20a to display advice C1 in a chat field, which states, "The meeting is running behind schedule. Let's move on to the next topic."

[0044] By visually checking this advice C1, the users Ua to Ud can recognize that the progress of the conference is delayed and can take measures such as moving on to the next agenda item, thereby ensuring that the conference is completed within the scheduled time. Note that the server device 10 may not only display the advice C1 on the user terminals 20a to 20d, but also output the advice C1 as voice.

[0045] [Processing method] Next, a description will be given of a processing method executed by the processing system 1. Fig. 7 is an example of a sequence diagram showing the processing procedure of the processing method according to the first embodiment.

[0046] As shown in FIG. 7, for example, when a Web conference is started on the user terminals 20a to 20d via an application or a browser (steps S21-1 to S21-4), conference room information in which users Ua to Ud can participate is output to the user terminals 20a to 20d through communication between the server device 10 and the user terminals 20a to 20d (steps S22-1 to 22-4).

[0047] The server device 10 acquires the conference schedule, such as the date and time, agenda, members participating in the conference, and information on related conferences and projects (step S23). The users Ua to Ud operate the user terminals 20a to 20d to select a conference room to participate in. Then, for example, the user Ua inputs a list of conference agendas as conference information into the user terminal 20a (step S24). The user terminal 20a transmits the input conference information to the server device 10 (step S25). It is sufficient for any one of the users Ua to Ud to input the conference information. Alternatively, the conference information may be registered in advance in the conference schedule, in which case steps S24 and S25 are omitted.

[0048] The server device 10 generates a prompt instructing the user to create a summary of the conference based on the input text data (step S26). The server device 10 may add information about the conference for which a summary is to be created to the prompt. The server device 10 sets the generated prompt in the generation AI 31 (steps S27 and S28).

[0049] When the server device 10 receives voice data of users Ua to Ud from the user terminals 20a to 20d (steps S29-1 to S29-4, S30-1 to S30-4), it performs voice recognition on the voice data and converts the voice data into text data (step S31). The server device 10 associates time information and user information indicating the speaker with the converted text data and inputs the text data to the generation AI 31 of the generation AI server 30 (step S32).

[0050] In the generation AI server 30, the generation AI 31 creates a summary of the meeting based on the input text data (step S33). The generation AI server 30 transmits the summary created by the generation AI 31 to the server device 10 (step S34).

[0051] The server device 10 determines the progress of the conference based on the list of agenda items, the conference summary output from the generation AI 31, and the elapsed time of the conference. Then, the server device 10 determines whether the progress of the conference is delayed (step S35).

[0052] If the progress of the conference is not delayed (step S35: No), the server device 10 receives the voices of the user terminals 20a to 20d (steps S30-1 to S30-4) and performs voice recognition (step S31).

[0053] If the progress of the conference is behind schedule (step S35: Yes), the server device 10 creates advice information (step S36) and transmits the created advice information to the user terminals 20a to 20d (steps S37-1 to S37-4). The advice information is output from each of the user terminals 20a to 20d (steps S38-1 to S38-4).

[0054] [Effects of the First Embodiment] In this way, in the first embodiment, the server device 10 sets a prompt to the generation AI 31 to instruct it to create a summary of the conference, and inputs text data converted from the voice data of each user who participated in the conference along with time information, thereby causing the server device 10 to create a summary of the conference.

[0055] The server device 10 then determines the progress of the conference based on the list of conference agenda items, the summary output from the generation AI server 30, and the elapsed time of the conference. If the progress of the conference is behind schedule based on the list of conference agenda items, the server device 10 advises the users Ua to Ud to move on to the next agenda item.

[0056] This advice allows the users Ua to Ud to recognize that the progress of the conference is behind schedule, and allows them to proceed with the conference so that the agenda is discussed within the set time. In this way, in the first embodiment, the users Ua to Ud can receive support for the progress of the conference from the server device 10. Therefore, according to the first embodiment, it is possible to support the smooth progress of the conference held between the users Ua to Ud.

[0057] [First Modification of First Embodiment] In the first embodiment, the progress of the conference is determined by the server device 10, but the progress of the conference may be determined by the generation AI 31. The determination unit 136 uses the generation AI 31 to determine the progress of the conference.

[0058] Fig. 8 is a diagram illustrating a processing flow in Modification 1 of Embodiment 1. Fig. 9 is an example of a sequence diagram illustrating a processing procedure of a processing method according to Modification 1 of Embodiment 1.

[0059] Steps S41-1 to S45 in FIG. 9 are the same processes as steps S21-1 to S25 in FIG.

[0060] The server device 10 creates a prompt instructing the generation AI 31 to create a summary of the meeting and to determine the progress of the meeting based on the list of agenda items, the summary of the meeting created by the generation AI 31, and the elapsed time of the meeting, and sets the prompt to the generation AI 31 (steps S46 to S48 in FIG. 9). The generation AI 31 is set with a prompt instructing the generation AI 31 to create a summary of the meeting and to determine the progress of the meeting based on the list of agenda items, the summary of the meeting created by the generation AI 31, and the elapsed time of the meeting. The server device 10 may add information about the meeting for which a summary is to be created to the prompt.

[0061] When the conference starts, the server device 10 performs speech recognition on the voice data of users Ua to Ud input from the user terminals 20a to 20d (steps S49-1 to S49-4, S50-1 to S50-4 in FIG. 9), converts the voice data into text data (step S51 in FIG. 9), associates time information and user information indicating the speaker with the conference text data, and inputs the associated data to the generation AI 31 (step S7 in FIG. 8, step S52 in FIG. 9).

[0062] The generation AI31 creates a summary of the meeting (step S8 in FIG. 8, step S53 in FIG. 9). Then, the generation AI31 determines the progress of the meeting, specifically, whether the meeting is running behind schedule, based on the list of agenda items for the meeting, the summary of the meeting created by the generation AI31, and the elapsed time of the meeting (step S8A in FIG. 8, step S54 in FIG. 9). In addition to determining whether the meeting is running behind schedule, the generation AI31 may also determine which agenda items to be discussed at the meeting have not been completed, and whether there is sufficient time remaining in the meeting compared to the number and content of agenda items that have not been discussed.

[0063] If the progress of the conference is not delayed (step S54 in FIG. 9: No), the generation AI 31 returns to the process of creating a summary based on the input conference text data (step S53 in FIG. 9).

[0064] On the other hand, if the progress of the conference is delayed (step S54 in FIG. 9: Yes), the generation AI 31 transmits the determination result to the server device 10 and also transmits an instruction to output advice (step S9A in FIG. 8, step S55 in FIG. 9). Note that the generation AI 31 may transmit the determination result regarding the progress of the conference to the server device 10, and the server device 10 may determine whether the progress of the conference is delayed and determine whether or not advice output is required.

[0065] In response to the advice output instruction, the server device 10 creates advice information advising that the progress of the conference is behind schedule and that the participants should move on to the next agenda item (step S10A in FIG. 8, step S56 in FIG. 9), and transmits the created advice information to the user terminals 20a to 20d (steps S57-1 to S57-4 in FIG. 9). The advice information is output from each of the user terminals 20a to 20d (steps S11-1 to S11-4 in FIG. 8, steps S58-1 to S58-4 in FIG. 9).

[0066] As in the first modification of the first embodiment, by having the generation AI 31 determine the progress of the conference, it is possible to omit setting a determination algorithm for the server device 10.

[0067] [Modification 2 of Embodiment 1] Furthermore, in the first modification of the first embodiment, the server device 10 creates the advice information, but the advice information may also be created by the generation AI 31. The determination unit 136 determines the progress of the conference using the generation AI 31. The advice unit 137 uses the generation AI 31 to output the advice information to each of the user terminals 20a to 20d used by each of the users Ua to Ud.

[0068] 10 is an example of a sequence diagram showing the processing procedure of the processing method according to Modification 2 of Embodiment 1. Steps S61-1 to S65 in FIG. 10 are the same as steps S21-1 to S25 in FIG.

[0069] The server device 10 creates a prompt for the generation AI 31 to instruct the generation AI 31 to create a summary of the meeting, determine the progress of the meeting based on the list of meeting agenda items, the summary of the meeting created by the generation AI 31, and the elapsed time of the meeting, and create advice information advising the user to move on to the next agenda item if the progress of the meeting is behind schedule compared to the list of agenda items, and sets the prompt to the generation AI 31 (steps S66 to S68). The generation AI 31 is set with a prompt for instructing the generation AI 31 to create a summary of the meeting, determine the progress of the meeting based on the list of agenda items, the summary of the meeting created by the generation AI 31, and the elapsed time of the meeting, and create advice information advising the user to move on to the next agenda item if the progress of the meeting is behind schedule compared to the list of agenda items. The server device 10 may add information about the meeting for which a summary is to be created to the prompt.

[0070] Steps S69-1 to S74 in FIG. 10 are the same processes as steps S49-1 to S54 shown in FIG.

[0071] If the progress of the conference is not delayed (step S74: No), the generation AI 31 returns to the process of creating a summary based on the input conference text data (step S73).

[0072] On the other hand, if the progress of the conference is behind schedule (step S74: Yes), the generation AI 31 creates advice information to be output to the user terminals 20a to 20d (step S75) and transmits it to the server device 10 (step S76).

[0073] As in variant example 2 of embodiment 1, by having the generation AI 31 create advice information, the server device 10 can provide users Ua to Ud with advice that is appropriate in accordance with the context before and after the text data converted from the conference speech.

[0074] [Embodiment 2] Next, a description will be given of embodiment 2. In embodiment 2, one of a plurality of generation AIs is selected as the generation AI that creates a summary of the meeting, thereby enabling a more appropriate summary to be created.

[0075] Fig. 11 is a diagram showing an example of the configuration of a processing system according to embodiment 2. As shown in Fig. 11, in a processing system 201 according to embodiment 2, a generation AI server 40 is installed within an in-house network 200 in which user terminals 20a to 20d and a server device 210 are installed. In addition, the server device 210 is capable of communicating with a generation AI server 50, which is an external server.

[0076] The generation AI server 40 is equipped with Tsuzumi (registered trademark) 41 (first generation model), which is a generation AI. Tsuzumi 41 is a natural language processing model fine-tuned to a specific field. Examples of specific fields include finance, medicine, semiconductors, IT (Information Technology), academia, factories (plants), law, and office services. Tsuzumi 41 was built with an emphasis on low power consumption, and has a faster processing speed than ChatGPT 51 (described below). Note that the specific fields are not limited to those mentioned above.

[0077] The generation AI server 50 is equipped with ChatGPT (registered trademark) 51 (second generation model), which is a generation AI. ChatGPT 51 is a large-scale natural language processing model that is slower than Tsuzumi 41 but has higher accuracy.

[0078] Tsuzumi41 and ChatGPT51 perform natural language processing on the input text data according to the set prompts, and create and output a summary of the meeting based on the input text data. Note that the above generation AI is just an example, and a server equipped with multiple other generation AIs may also be provided.

[0079] In a conference between users Ua to Ud, the server device 210 obtains a summary of the conference by converting the voice data of each user participating in the conference into text data and inputting the converted text data into Tsuzumi 41 or ChatGPT 51. Then, if the progress of the conference is delayed based on the summary created by Tsuzumi 41 or ChatGPT 51, the server device 210 advises users Ua to Ud to move on to the next agenda item.

[0080] Specifically, the server device 210 selects Tsuzumi41 or ChatGPT51 based on the conference information about the conference the user is attending, the content of the conference, or predetermined rules. The server device 210 then sets a prompt to the selected generation AI to instruct it to create a summary of the conference based on the input text data. The server device 210 may add information about the conference to be summarized to the prompt to facilitate the creation of the summary.

[0081] This allows the server device 10 to obtain the most appropriate summary according to the situation from the selected generation AI, and to provide the user with more appropriate advice on the progress of the conference.

[0082] [Server device] The server device 210 will be described. The server device 210 has a control unit 213 instead of the control unit 13 of the server device 10 shown in Fig. 2. The control unit 213 has a selection unit 2133. Fig. 12 is a diagram illustrating the flow of processing in the processing system 201 according to the second embodiment.

[0083] The selection unit 2133 selects one of a plurality of generation AIs, which are natural language processing models, based on information about the meeting, the content of the meeting, or predetermined rules. The information about the meeting and the content of the meeting are included, for example, in the meeting schedule or the meeting information input from the user terminal 20a (step S81 in FIG. 12). For example, the selection unit 2133 selects Tsuzumi41 or ChatGPT51 as the generation AI suitable for the meeting based on the schedule and the meeting information (step S82 in FIG. 12). Furthermore, after the meeting starts, the selection unit 2133 may determine the content of the meeting from the summary created by the generation AI and switch the generation AI to create the summary.

[0084] The selection unit 2133 determines, for example, the accuracy of the meeting, the degree of response speed, the status of the meeting, the industry related to the meeting, whether the meeting is in a specific field or an accuracy-oriented meeting, or the level of confidentiality of the meeting based on information about the meeting and the content of the meeting. Furthermore, the selection unit 2133 may determine, in accordance with predetermined rules, the accuracy of the meeting, the degree of response speed, the status of the meeting, the industry related to the meeting, whether the meeting is in a specific field or an accuracy-oriented meeting, or the level of confidentiality of the meeting. The selection unit 2133 may change the content of the determination depending on the industry, field, members, and situation, without being limited to the above.

[0085] The selection unit 2133 may also use a generation AI (Tsuzumi41 or ChatGPT51) to determine the accuracy of the meeting, the degree of response speed, the status of the meeting, the industry related to the meeting, whether the meeting is in a specific field or an accuracy-oriented meeting, or the level of confidentiality of the meeting. In this case, the selection unit 2133 sets a prompt to the generation AI that instructs it to determine the accuracy of the meeting, the degree of response speed, the status of the meeting, the industry related to the meeting, whether the meeting is in a specific field or an accuracy-oriented meeting, or the level of confidentiality of the meeting, based on the input information about the meeting and / or the content of the meeting.

[0086] Based on the determined content, the selection unit 2133 selects Tsuzumi41 or ChatGPT51. For example, if the conference is in a specific field (for example, finance), the response speed is set to a relatively fast level, and speed is emphasized, the selection unit 2133 selects Tsuzumi41.

[0087] Furthermore, if the confidentiality level of the conference is above standard, the selection unit 2133 selects Tsuzumi 41 within the in-house network 200. The generation AI server 40 may have multiple Tsuzumi 41s corresponding to the fields of finance, medicine, semiconductors, IT, academia, factories, law, or office services. In this case, the selection unit 2133 selects Tsuzumi 41 corresponding to the determined field of the conference and creates a summary of the conference.

[0088] Furthermore, for example, the selection unit 2133 selects ChatGPT51 when accuracy is important. Furthermore, the selection unit 2133 selects ChatGPT51 when the confidentiality level of the conference is below standard. The selection unit 2133 may determine the content of the conference after the conference starts and switch the generation AI that will create a summary of the conference. Furthermore, the selection unit 2133 may input, for example, information about the conference and the content of the conference into the generation AI and have the generation AI determine the generation AI that will create the minutes. The generation AI that performs this determination may be generation AI31, Tsuzumi41, ChatGPT51, or another generation AI.

[0089] The prompt creation unit 133 inputs the information about the conference and / or the content determined based on the content of the conference into the prompt together with the conference information (step S82 in FIG. 12). The prompt creation unit 133 sets the created prompt to the generation AI (e.g., Tsuzumi41) selected by the selection unit 2133 (step S83 in FIG. 12).

[0090] In this way, the server device 210 selects a generation AI suitable for the conference based on the judgment results, adjusts the prompt when instructing the generation AI, and then instructs the selected generation AI to create a summary.

[0091] Then, the selected generation AI (for example, Tsuzumi41) creates a summary of the conference based on the input conference text data in accordance with the set prompts (step S84).

[0092] [Processing method] Next, a description will be given of the processing procedure of the processing method according to embodiment 2. Fig. 13 is an example of a sequence diagram showing the processing procedure of the processing method according to embodiment 2. Steps S91-1 to S95 in Fig. 13 are the same processes as steps S21-1 to S25 in Fig. 7.

[0093] The server device 210 selects Tsuzumi41 or ChatGPT51 as the generation AI for creating a summary based on the information about the meeting, the content of the meeting, or a predetermined rule (step S96). Fig. 13 shows the case where Tsuzumi41 is selected.

[0094] Based on the input text data, server device 210 creates a prompt instructing the user to create a summary of the conference (step S97). Server device 210 sets the created prompt to Tsuzumi 41, selected in step S96 (steps S98 and S99).

[0095] When server device 210 receives voice data of users Ua to Ud from user terminals 20a to 20d (steps S100-1 to S100-4, S101-1 to S101-4), it converts the voice data into text data (step S102). Server device 210 associates the time information and user information indicating the speaker with the converted text data and inputs the text data to Tsuzumi 41 selected in step S96 (step S103). Tsuzumi 41 creates a summary of the conference based on the input text data (step S104) and transmits the created summary to server device 210 (step S105).

[0096] Steps S106 to S109-4 in FIG. 13 are the same processes as steps S35 to S38-4 shown in FIG.

[0097] [Effects of the second embodiment] In the second embodiment, one of a plurality of generation AIs is selected as the generation AI for creating a summary of the meeting based on information about the meeting, the content of the meeting, or predetermined rules, thereby making it possible to create a summary that is more suitable for the meeting. Therefore, according to the second embodiment, it is possible to further optimize the judgment of the progress of the meeting and the advice given to the user.

[0098] Note that Modifications 1 and 2 of Embodiment 1 may be applied to Embodiment 2. For example, a case where Modification 1 of Embodiment 1 is applied will be described. In this case, server device 210 selects Tsuzumi41 or ChatGPT51 based on information about the meeting, the content of the meeting, or a predetermined rule. Then, server device 210 sets a prompt to the selected generation AI instructing it to create a summary of the meeting and to determine the progress of the meeting based on the list of meeting agendas, the summary of the meeting created by the generation AI, and the elapsed time of the meeting. In this way, server device 210 may determine the progress of the meeting using the selected generation AI.

[0099] A case where Modification 2 of Embodiment 1 is applied will be described. In this case, server device 210 selects Tsuzumi41 or ChatGPT51 based on information about the meeting, the content of the meeting, or a predetermined rule. Then, server device 210 sets prompts to the selected generation AI instructing it to create a summary of the meeting, determine the progress of the meeting based on the list of meeting agenda items, the meeting summary created by generation AI 31, and the elapsed time of the meeting, and create advice information advising the user to move on to the next agenda item if the progress of the meeting is behind schedule compared to the list of agenda items. In this way, server device 210 may use the selected generation AI to determine the progress of the meeting and create advice information for each user Ua to Ud.

[0100] In this embodiment, Tsuzumi41 and ChatGPT51 have been described as examples of the generation AIs used, but other generation AIs may be used, and the number of generation AIs is not limited to two, and any one of three or more generation AIs may be selected. Furthermore, the conference is not limited to a web conference, but may also be a face-to-face conference. In this case, it is sufficient if the conference room has a communication device that can transmit voice data of the conference participants to the server device 10, 210 and output advice from the server device 10, 210.

[0101] [System configuration of the embodiment] The server devices 10 and 210 are conceptual functional entities and do not necessarily have to be physically configured as shown in the drawings. In other words, the specific form of distribution and integration of the functions of the server devices 10 and 210 is not limited to that shown in the drawings, and all or part of the server devices 10 and 210 can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc.

[0102] Furthermore, all or any part of the processes performed by the server devices 10 and 210 may be realized by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a program analyzed and executed by the CPU and the GPU (Graphics Processing Unit). Furthermore, each process performed by the server devices 10 and 210 may be realized as hardware using wired logic.

[0103] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically using a known method. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters described above and illustrated can be changed as appropriate unless otherwise specified.

[0104] [program] 14 is a diagram showing an example of a computer in which a program is executed to realize the server device 10, 210. The computer 1000 has, for example, a memory 1010 and a CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0105] The memory 1010 includes a ROM (Read Only Memory) 1011 and a RAM (Random Access Memory) 1012. The ROM 1011 stores a boot program such as a BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0106] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, the programs that define the processes of the server devices 10 and 210 are implemented as program modules 1093 in which code executable by the computer 1000 is written. The program modules 1093 are stored, for example, in the hard disk drive 1090. For example, the program modules 1093 for executing processes similar to those of the functional configurations of the server devices 10 and 210 are stored in the hard disk drive 1090. The hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0107] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in memory 1010 or hard disk drive 1090. Then, CPU 1020 reads program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as necessary and executes them.

[0108] The program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090, but may also be stored in, for example, a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0109] Although the present invention has been described above as an embodiment, the present invention is not limited to the descriptions and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention. [Explanation of symbols]

[0110] 1,201 processing systems 10,210 server devices 11 Communications Department 12 Storage section 13,213 Control Unit 20, 20a, 20b, 20c, 20d User terminal 30, 40, 50 Generation AI Server 31 Generation AI 121 User Information 122 Schedule Information 123 Meeting Information 124 Text Data 125 Summary Data 126 Advice List 131 Schedule Acquisition Department 132 Meeting Information Acquisition Unit 133 Prompt Creation Department 134 Recognition part 135 Summary Acquisition Section 136 Judgment section 137 Advice Department 200 Internal network 2133 Selection Section

Claims

1. an acquisition unit that acquires a schedule of a meeting that the user will attend and a list of agenda items for the meeting; a generation unit that generates a prompt that commands the generation of a summary of the meeting based on input text data and sets the generated prompt in a generative model; a recognition unit that converts the voice data of each user who has participated in the conference into text data and inputs the converted text data together with time information into the generative model; a determination unit that determines the progress of the meeting based on the list of agenda items of the meeting, a summary of the meeting output from the generative model, and the elapsed time of the meeting; an advice unit that outputs, to each user terminal used by each user, advice information advising the user to move on to the next agenda item when the progress of the conference is behind schedule with respect to the list of agenda items of the conference; A processing device comprising:

2. the generative model is configured with prompts to generate a summary of the meeting based on the input text data, and to determine a progress status of the meeting based on a list of agenda items for the meeting, the summary of the meeting, and the elapsed time of the meeting; The processing device according to claim 1 , wherein the determining unit determines the progress of the conference using the generative model.

3. The generative model is configured with prompts that instruct the following: to create a summary of the meeting based on the input text data; to determine the progress of the meeting based on the list of agenda items of the meeting, the summary of the meeting, and the elapsed time of the meeting; and to create advice information that advises moving on to the next agenda item if the progress of the meeting is delayed based on the list of agenda items of the meeting; the determination unit determines a progress status of the conference using the generative model; The processing device according to claim 1 , wherein the advice unit uses the generative model to output the advice information to each user terminal used by each user.

4. 2. The processing device according to claim 1, further comprising a selection unit that selects one of a plurality of generative models as the generative model based on information about the conference, the content of the conference, or a predetermined rule.

5. The processing device according to claim 4, wherein the selection unit determines the accuracy of the meeting, the degree of response speed, the status of the meeting, the industry related to the meeting, whether the meeting is in a specific field or a meeting that emphasizes accuracy, or the level of confidentiality of the meeting based on information about the meeting, the content of the meeting, or the predetermined rules, and selects, as the generation model, a first generation model that is a natural language processing model fine-tuned to a specific field, or a second generation model that is a large-scale natural language processing model, based on the determined content.

6. A processing method executed by a processing device, obtaining a schedule of meetings in which the user will participate and a list of agenda items for said meetings; creating a prompt that commands the creation of a summary of the meeting based on the input text data, and setting the created prompt in a generative model; converting the voice data of each user participating in the conference into text data, and inputting the converted text data together with time information into the generative model; determining the progress of the meeting based on the list of agenda items of the meeting, the summary of the meeting output from the generative model, and the elapsed time of the meeting; outputting, to each user terminal used by each user, advice information advising the user to move on to the next agenda item when the progress of the conference is behind schedule with respect to the list of agenda items of the conference; A processing method comprising:

7. obtaining a schedule of meetings in which the user will participate and a list of agenda items for said meetings; creating a prompt that instructs the user to create a summary of the meeting based on the input text data, and setting the created prompt in a generative model; converting the voice data of each user participating in the conference into text data, and inputting the converted text data together with time information into the generative model; determining the progress of the meeting based on the list of agenda items of the meeting, the summary of the meeting output from the generative model, and the elapsed time of the meeting; outputting, to each user terminal used by each user, advice information advising the user to move on to the next agenda item when the progress of the conference is behind schedule with respect to the list of agenda items of the conference; A processing program that causes a computer to execute the above.

Images