Information processing apparatus, information processing method, and program

Abstract

To enable a user to easily correct minutes of a conference by using a machine learning model.SOLUTION: The information processing apparatus 1 includes the acquisition unit 131 that acquires character string data indicating remark contents in a conference, the determination unit 134 that determines, based on the character string data, a first input sentence for causing a machine learning model to generate minutes of the conference including a plurality of items, the display control unit 136 that causes an information terminal to display the minutes generated by inputting the first input sentence to the machine learning model, and the reception unit 133 that receives, at the information terminal, a correction instruction in which each of the plurality of items included in the minutes is associated with a correction content for the item. The determination unit 134 determines, based on the character string data, the minutes, and the correction instruction, a second input sentence for causing the machine learning model to correct the minutes, and the display control unit 136 causes the information terminal to display the minutes corrected by inputting the second input sentence to the machine learning model.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program for providing minutes of a meeting using a machine learning model. [Background technology]

[0002] Patent Document 1 describes a system that acquires keywords by performing speech recognition on speech data and inputs the acquired keywords into a machine learning model to generate text for minutes of a meeting. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-34722 Summary of the Invention [Problem to be solved by the invention]

[0004] In the past, when a user wanted to partially correct meeting minutes generated by a machine learning model, it was not easy for the user to create input sentences to have the machine learning model correct that part of the minutes. If the appropriate input sentences were not input to the machine learning model, the minutes might be corrected in a way that was different from the user's intention, or parts of the minutes that the user did not want to be corrected might be corrected.

[0005] Therefore, the present invention has been made in consideration of these points, and aims to enable users to easily revise meeting minutes using machine learning models. [Means for solving the problem]

[0006] An information processing device of a first aspect of the present invention includes an acquisition unit that acquires string data indicating the content of statements made in a meeting; a determination unit that determines, based on the string data, a first input sentence for causing a machine learning model to generate minutes of the meeting including a plurality of items; a display control unit that causes an information terminal to display the minutes generated by inputting the first input sentence into the machine learning model; and a reception unit that receives, at the information terminal, a correction instruction that associates each of the plurality of items included in the minutes with correction content for that item, wherein the determination unit determines, based on the string data, the minutes, and the correction instruction, a second input sentence for causing the machine learning model to correct the minutes, and the display control unit causes the information terminal to display the minutes after being corrected by inputting the second input sentence into the machine learning model.

[0007] The determination unit may determine the second input sentence so as to modify the content of an item among the plurality of items that is associated with the content of the modification, and not to modify the content of an item that is not associated with the content of the modification.

[0008] The determination unit may determine the second input sentence so as to modify the contents of all of the plurality of items, on the condition that the modification content is associated with a specific item among the plurality of items, and may determine the second input sentence so as to modify the contents of items among the plurality of items that are associated with the modification content and not modify the contents of items that are not associated with the modification content, on the condition that the modification content is associated with an item among the plurality of items other than the specific item.

[0009] The receiving unit may receive specification of the plurality of items from a user at the information terminal before the determination unit determines the first input sentence, and the determination unit may determine the first input sentence for causing the machine learning model to generate the minutes including the plurality of items specified by the user.

[0010] The accepting unit may accept a selection of the plurality of items from among a plurality of candidates for the plurality of items displayed on the information terminal before the determining unit determines the first input sentence.

[0011] The determination unit may determine the plurality of items according to the content of the utterance, and may determine the first input sentence for causing the machine learning model to generate the minutes including the determined plurality of items.

[0012] The determination unit may determine the plurality of items according to the type or purpose of the meeting, and determine the first input sentence for causing the machine learning model to generate the minutes including the determined plurality of items.

[0013] The display control unit may display multiple options for the correction content on the information terminal in association with each of the multiple items, and the reception unit may receive a selection of the correction content from the multiple options for each of the multiple items.

[0014] The display control unit may vary the plurality of options depending on each of the plurality of items.

[0015] The display control unit may vary the plurality of options depending on the content of the statement.

[0016] The determination unit may determine the first input sentence by applying the utterance content indicated by the character string data to a template stored in advance in a storage unit.

[0017] The determination unit may determine the second input sentence by applying the remark content indicated by the string data, the minutes, and the correction instructions for each of the plurality of items to a template stored in advance in a memory unit.

[0018] An information processing method of a second aspect of the present invention includes the steps of: acquiring string data indicating the content of remarks made in a meeting, based on the string data; determining a first input sentence for causing a machine learning model to generate minutes of the meeting including a plurality of items, based on the string data; displaying the minutes generated by inputting the first input sentence into the machine learning model on an information terminal; accepting, at the information terminal, a correction instruction associating each of the plurality of items included in the minutes with correction content for that item; determining a second input sentence for causing the machine learning model to correct the minutes, based on the string data, the minutes, and the correction instruction; and displaying, on the information terminal, the minutes after being corrected by inputting the second input sentence into the machine learning model.

[0019] A third aspect of the program of the present invention causes a processor to execute the following steps: acquiring character string data indicating what was said in a meeting; determining a first input sentence based on the character string data to cause a machine learning model to generate minutes of the meeting including a plurality of items; displaying the minutes generated by inputting the first input sentence into the machine learning model on an information terminal; accepting, at the information terminal, a correction instruction that associates each of the plurality of items included in the minutes with a correction for that item; determining a second input sentence based on the character string data, the minutes, and the correction instruction to cause the machine learning model to correct the minutes; and displaying, on the information terminal, the minutes after being corrected by inputting the second input sentence into the machine learning model. [Effects of the Invention]

[0020] The present invention has the effect of allowing a user to easily revise meeting minutes using a machine learning model. [Brief explanation of the drawings]

[0021] [Figure 1]1 is a schematic diagram of an information processing system according to an embodiment. [Figure 2] 1 is a block diagram of an information processing system according to an embodiment. [Figure 3] 10A and 10B are schematic diagrams for explaining a process in which an acquisition unit acquires character string data; [Figure 4] 10 is a schematic diagram for explaining a process in which a determination unit determines a first input sentence. FIG. [Figure 5] FIG. 2 is a schematic diagram showing an exemplary first input sentence. [Figure 6] FIG. 10 is a schematic diagram of an information terminal displaying a minutes display screen. [Figure 7] 10 is a schematic diagram for explaining a process in which a determination unit determines a second input sentence. FIG. [Figure 8] FIG. 10 is a schematic diagram showing an exemplary second input sentence. [Figure 9] 10 is a schematic diagram of an information terminal displaying a minutes display screen A. FIG. [Figure 10] FIG. 1 is a flowchart illustrating an exemplary information processing method executed by an information processing system according to an embodiment. [Figure 11] FIG. 10 is a schematic diagram of an information terminal displaying a minutes display screen according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0022] [Outline of Information Processing System S] 1 is a schematic diagram of an information processing system S according to this embodiment. The information processing system S includes an information processing device 1 and a plurality of information terminals 2. The information processing system S may also include other devices such as servers and terminals.

[0023] The information processing device 1 is a computer that processes information about a conference in which multiple users participate. The conference is held face-to-face by multiple users in real space, or remotely via a network using multiple information terminals 2. The information processing device 1 acquires the voices uttered by each of the multiple users in the conference, and displays minutes of the conference on the information terminal 2, generated based on the acquired voices. The information processing device 1 also accepts an instruction to amend the minutes from the information terminal 2, and amends the minutes based on the accepted instruction to amend the minutes.

[0024] The information terminal 2 is a computer used by each of multiple users. A user is a person who participates in a conference or a person who views the minutes of a conference. The information terminal 2 is, for example, a smartphone, a tablet terminal, or a personal computer. The information terminal 2 may also be a sound collection device such as a microphone that captures the audio of the conference.

[0025] The information terminal 2 has an operation unit such as a touch panel or keyboard for receiving operations, a display unit such as a liquid crystal display for displaying information, and an audio acquisition unit such as a speaker for acquiring audio. The information terminal 2 is associated in advance with a user by setting identification information (Identifier: ID) for identifying the user who uses the information terminal 2. The information terminal 2 is capable of communicating with the information processing device 1 via a network.

[0026] An outline of the processing executed by the information processing system S according to this embodiment will be described below. The information processing device 1 acquires voice data representing the voices of a conference from the information terminal 2 ((1) in FIG. 1). Based on the voice data, the information processing device 1 acquires character string data representing the content of remarks made in the conference.

[0027] The information processing device 1 uses the character string data to determine a first input sentence for causing the machine learning model to generate minutes containing multiple items ((2) in FIG. 1). The multiple items contained in the minutes are items such as parts, chapters, and sections in the chapter structure for dividing the minutes. The information processing device 1 generates the minutes by inputting the first input sentence into the machine learning model ((3) in FIG. 1). The information processing device 1 displays the minutes on the information terminal 2 ((4) in FIG. 1).

[0028] The information processing device 1 accepts, at the information terminal 2, a correction instruction associating each of the multiple items included in the minutes with the correction content for that item ((5) in FIG. 1). The correction instruction indicates, for each of the multiple items included in the minutes, the correction content selected from multiple options displayed on the information terminal 2 or the correction content input by the user on the information terminal 2.

[0029] The information processing device 1 determines a second input sentence for correcting the minutes into the machine learning model using character string data indicating the content of remarks made in the meeting, the minutes displayed on the information terminal 2, and the correction instruction received from the information terminal 2 ((6) in FIG. 1). The information processing device 1 corrects the minutes by inputting the second input sentence into the machine learning model ((7) in FIG. 1). The information processing device 1 displays the corrected minutes on the information terminal 2 ((8) in FIG. 1).

[0030] In this way, the information processing system S causes the machine learning model to generate minutes containing multiple items, accepts correction instructions from the user for each of the multiple items, and causes the machine learning model to correct the minutes based on the correction instructions. In this way, the information processing system S enables the user to easily correct the minutes of a meeting for each item using the machine learning model.

[0031] [Configuration of Information Processing System S] FIG. 2 is a block diagram of an information processing system S according to this embodiment. In FIG. 2, arrows indicate main data flows, and data flows other than those shown in FIG. 2 may also exist. In FIG. 2, each block indicates a functional configuration rather than a hardware (device) configuration. Therefore, the blocks shown in FIG. 2 may be implemented in a single device, or may be implemented separately in multiple devices. Data may be exchanged between blocks via any means, such as a data bus, a network, or a portable storage medium.

[0032] The information processing device 1 includes a communication unit 11, a storage unit 12, and a control unit 13. The information processing device 1 may be configured by connecting two or more physically separate devices via wired or wireless connections. The information processing device 1 may also be configured by a cloud, which is a collection of computer resources.

[0033] The communication unit 11 has a communication controller for transmitting and receiving data to and from the information terminal 2 via the network. The communication unit 11 notifies the control unit 13 of data received from the information terminal 2 via the network. The communication unit 11 also transmits data output from the control unit 13 to the information terminal 2 via the network.

[0034] The storage unit 12 is a storage medium including a ROM (Read Only Memory), a RAM (Random Access Memory), a hard disk drive, an SSD (Solid State Drive), etc. The storage unit 12 stores in advance programs to be executed by the control unit 13. The storage unit 12 may be provided outside the information processing device 1, in which case data may be exchanged between the storage unit 12 and the control unit 13 via a network.

[0035] The control unit 13 has an acquisition unit 131, a voice recognition unit 132, a reception unit 133, a determination unit 134, a minutes generation unit 135, and a display control unit 136. The control unit 13 is a processor such as a CPU (Central Processing Unit), and functions as the acquisition unit 131, the voice recognition unit 132, the reception unit 133, the determination unit 134, the minutes generation unit 135, and the display control unit 136 by executing a program stored in the storage unit 12.

[0036] The information terminal 2 may execute at least a part of the processing executed by the information processing device 1 according to this embodiment. In this case, the processor of the information terminal 2 functions as at least a part of the acquisition unit 131, the voice recognition unit 132, the reception unit 133, the determination unit 134, the minutes generation unit 135, and the display control unit 136.

[0037] The following describes in detail the processing executed by the information processing system S. In the information processing device 1, the acquisition unit 131 acquires character string data indicating the content of statements made in a conference in which multiple users participate.

[0038] 3 is a schematic diagram for explaining the process of acquiring character string data by the acquisition unit 131. The information terminal 2 acquires, by the voice acquisition unit, voices uttered by each of the multiple users participating in the conference. The information terminal 2 transmits voice data indicating the acquired voices to the information processing device 1. The information terminal 2 transmits the voice data sequentially while the conference is taking place, or transmits the voice data collectively after the conference has ended.

[0039] When multiple users hold a face-to-face conference, any one of the multiple information terminals 2 may transmit voice data to the information processing device 1. When multiple users hold a remote conference via a network, each of the multiple information terminals 2 may transmit voice data to the information processing device 1.

[0040] The acquisition unit 131 acquires the voice data transmitted by the information terminal 2. The voice recognition unit 132 executes a known voice recognition process on the voice data acquired by the acquisition unit 131, thereby generating character string data indicating the content of what was said in the conference.

[0041] When the acquisition unit 131 acquires a plurality of pieces of voice data from a plurality of information terminals 2, the voice recognition unit 132 may generate character string data from one piece of voice data generated by combining the plurality of pieces of voice data. Alternatively, the voice recognition unit 132 may generate character string data from each of the plurality of pieces of voice data, and combine the generated plurality of character string data to generate one piece of character string data.

[0042] The voice recognition unit 132 may cause an external device different from the information processing device 1 to perform the voice recognition process without performing the voice recognition process itself. In this case, the voice recognition unit 132 transmits voice data to the external device, receives character string data from the external device that has been generated by the external device performing the voice recognition process on the transmitted voice data, and stores the received character string data in the storage unit 12.

[0043] The acquisition unit 131 acquires character string data indicating the content of speeches made in a conference, which is generated by the speech recognition unit 132. The acquisition unit 131 may also acquire character string data indicating the content of speeches made in a conference, which is stored in the storage unit 12.

[0044] The receiving unit 133 receives designation of multiple items to be included in the minutes from the user at the information terminal 2 before the determining unit 134, which will be described later, determines the first input sentence (input sentence for generation). The multiple items to be included in the minutes are items such as parts, chapters, and sections in a chapter structure for dividing the minutes by content. The multiple items to be included in the minutes may also be first and second half items for dividing the minutes by the chronology of the meeting.

[0045] The receiving unit 133, for example, displays a plurality of candidates for a plurality of items included in the minutes on the information terminal 2. Each of the plurality of candidates is a set of a plurality of items used to divide the minutes, such as, for example, "summary, conclusion, next agenda item," "this period's schedule, this period's results, next period's schedule," etc., and is defined in advance in the storage unit 12. The receiving unit 133 receives a selection of one of the plurality of candidates from the user on the information terminal 2, and accepts the selected candidate as the plurality of items specified by the user. When the plurality of items included in the minutes are determined in advance, the receiving unit 133 does not need to receive a specification of the plurality of items from the user.

[0046] The determining unit 134 determines a first input sentence (input sentence for generation) for causing the machine learning model to generate minutes of a meeting including multiple items, based on the character string data acquired by the acquiring unit 131. The first input sentence includes, for example, a prompt (command) that is input to the machine learning model to cause the machine learning model to execute a predetermined process.

[0047] 4 is a schematic diagram illustrating the process of determining a first input sentence by the determination unit 134. The determination unit 134 determines a plurality of items to be included in the minutes. The determination unit 134 may determine a plurality of items specified by the user and accepted by the acceptance unit 133 as a plurality of items to be included in the minutes. This allows the information processing system S to provide minutes that include a plurality of items desired by the user. Furthermore, the determination unit 134 may determine a plurality of items predefined in the storage unit 12 as a plurality of items to be included in the minutes.

[0048] The determination unit 134 may determine a plurality of items to be included in the minutes according to the content of remarks made in the meeting indicated by the character string data. In this case, the determination unit 134 determines a plurality of items that are pre-associated with specific words included in the content of remarks, for example, in information associating words with items pre-stored in the storage unit 12. This allows the information processing system S to provide minutes that include a plurality of items that are suitable for the content of remarks made in the meeting.

[0049] The determination unit 134 may determine multiple items to be included in the minutes according to the type or purpose of the meeting. In this case, the determination unit 134, for example, identifies the type or purpose of the meeting (information sharing, decision-making, etc.) that is specified in advance by the user on the information terminal 2. The determination unit 134 determines multiple items that are pre-associated with the identified type or purpose of the meeting in information that associates the type or purpose of the meeting with items and that is pre-stored in the storage unit 12. This allows the information processing system S to provide minutes that include multiple items that are suitable for the type or purpose of the meeting.

[0050] The determination unit 134 acquires a first template for determining a first input sentence, which is stored in advance in the storage unit 12. The first template is, for example, a template including a fixed portion that does not change in the first input sentence and a variable portion that changes in the first input sentence. The fixed portion includes, for example, a character string that does not change according to the content of a speech made in a conference and a specification by a user (such as a tag used to instruct a machine learning model).

[0051] The determination unit 134 determines the first input sentence by, for example, applying (inserting) the determined items and the utterance content indicated by the character string data to the variable part of the first template. Furthermore, the determination unit 134 may insert information about the participants (speakers) of the conference designated in advance by the user on the information terminal 2 into the variable part of the first template.

[0052] FIG. 5 is a schematic diagram showing an exemplary first input sentence. The first input sentence shown in FIG. 5 includes portions corresponding to the fixed and variable portions included in the first template used to determine the first input sentence. As shown in FIG. 5, the variable portion of the first template contains multiple items determined by the determination unit 134 (such as "Participants" and "Summary" in FIG. 5) and the content of the remarks indicated by the character string data (the character string "Transcription" in FIG. 5). This allows the information processing system S to reduce the effort required for the user to create an input sentence for the machine learning model to generate minutes including multiple items.

[0053] The minutes generation unit 135 acquires a machine learning model for generating minutes, which is stored in advance in the storage unit 12. The machine learning model in this embodiment is, for example, a large-scale language model. The large-scale language model is generated in advance by machine learning a large amount of character string data using a known deep learning process such as a DNN (Deep Neural Network). When a character string is input, the large-scale language model outputs a character string corresponding to the input character string.

[0054] The minutes generation unit 135 inputs the first input sentence determined by the determination unit 134 into a machine learning model, and generates the content output by the machine learning model as minutes of the meeting. The minutes generated by the minutes generation unit 135 include multiple items indicated by the first input sentence, and the content of the minutes is divided into the multiple items.

[0055] The minutes generation unit 135 may generate the minutes using a machine learning model executed in an external device different from the information processing device 1. In this case, the minutes generation unit 135 transmits a first input sentence to the external device, and receives from the external device the minutes generated by the external device inputting the transmitted first input sentence into the machine learning model.

[0056] The display control unit 136 causes the minutes generated by the minutes generation unit 135, i.e., the minutes generated by inputting the first input sentence into the machine learning model, to be displayed on the information terminal 2. The display control unit 136 causes the minutes to be displayed on the information terminal 2 used by a user who performed a predetermined operation to display the minutes, among multiple users who participated in the conference. The display control unit 136 may also cause the minutes to be displayed on the information terminal 2 used by a user who did not participate in the conference.

[0057] The display control unit 136 transmits display information for displaying the minutes to the information terminal 2. The information terminal 2 displays a minutes display screen including the minutes on the display unit in accordance with the display information transmitted by the information processing device 1.

[0058] 6 is a schematic diagram of an information terminal 2 displaying a minutes display screen A. The minutes display screen A includes a minutes area A1 and a correction instruction area A2. The minutes area A1 includes a minutes A3 generated by inputting the first input sentence into a machine learning model.

[0059] The correction instruction area A2 includes a correction instruction field A4 associated with each of the multiple items included in the minutes A3 and accepting correction instructions for the item. The display control unit 136, for example, causes the correction instruction area A2 to display a correction instruction field A4 indicating multiple options for correction content associated with each of the multiple items. The options include, for each of the multiple items, changing the content of the item, shortening or lengthening the content of the item, simplifying or detailing the content of the item, deleting all or part of the item, etc. The display control unit 136 may also cause the correction instruction area A2 to display a correction instruction field A4 associated with each of the multiple items and allowing the user to input a character string indicating the correction content.

[0060] The display control unit 136 may vary the multiple options shown in the correction instruction field A4 depending on each of the multiple items included in the minutes A3. In this case, the display control unit 136 determines multiple options that are pre-associated with each of the multiple items included in the minutes A3, for example, in information that associates minutes items with options pre-stored in the storage unit 12. This allows the information processing system S to present the user with options for correction content that are appropriate for each item in the minutes, making it easier for the user to give correction instructions.

[0061] The display control unit 136 may vary the multiple options displayed in the correction instruction field A4 depending on the content of the utterance in the conference indicated by the character string data. In this case, the display control unit 136 determines multiple options that are pre-associated with specific words included in the content of the utterance, for example, in information that associates words with options pre-stored in the storage unit 12. This allows the information processing system S to present the user with options for correction that are suitable for the content of the utterance in the conference, making it easier for the user to give correction instructions.

[0062] The receiving unit 133 receives, at the information terminal 2, a correction instruction associating each of the multiple items included in the minutes with the correction content for that item. For example, the receiving unit 133 receives from the user a selection or input of correction content in the correction instruction field A4 for each of the multiple items included in the minutes A3 displayed in the minutes area A1. The receiving unit 133 stores, in the storage unit 12, the correction instruction associating each of the multiple items with the correction content selected or input for that item.

[0063] The determination unit 134 determines a second input sentence (input sentence for correction) for causing the machine learning model to correct the minutes that are the subject of the correction instruction specified by the user, based on the correction instruction received by the reception unit 133. The second input sentence includes, for example, a prompt (command) that is input to the machine learning model to cause the machine learning model to execute a predetermined process.

[0064] 7 is a schematic diagram illustrating a process in which the determination unit 134 determines a second input sentence. The determination unit 134 acquires a second template for determining a second input sentence, which is stored in advance in the storage unit 12. The second template is, for example, a template including a fixed portion that does not change in the second input sentence and a variable portion that changes in the second input sentence. The fixed portion includes, for example, a character string that does not change according to the content of a speech made in a conference and a specification by a user (such as a tag used to instruct a machine learning model).

[0065] The determination unit 134 determines the second input sentence, for example, by applying (inserting) into the variable part of the second template the content of the statement indicated by the string data, the minutes that are the subject of the correction instructions, and correction instructions for each of the multiple items included in the minutes.

[0066] Fig. 8 is a schematic diagram showing an exemplary second input sentence. The second input sentence shown in Fig. 8 includes portions corresponding to the fixed and variable portions included in the second template used to determine the second input sentence. As shown in Fig. 8, the variable portion of the second template contains multiple items included in the minutes that are the subject of the correction instruction (such as "Participants" and "Summary" in Fig. 8), the remarks indicated by the character string data (the character string "Transcription" in Fig. 8), the minutes that are the subject of the correction instruction (the character string "Previously Generated Minutes" in Fig. 8), and the correction instruction (the character string "User Correction Instruction" in Fig. 8).

[0067] The correction instruction included in the second input sentence is information that associates each of the multiple items included in the minutes that are the target of the correction instruction with the correction content indicated by the correction instruction for that item. This allows the information processing system S to reduce the effort required by the user to create an input sentence to have the machine learning model correct the minutes, which include multiple items, item by item.

[0068] The determination unit 134 may determine the second input sentence so as to modify the contents of the items associated with the modification content among the multiple items and not modify the contents of the items not associated with the modification content. In this case, the determination unit 134 determines the second input sentence including a command indicating that the contents of the items not associated with the modification content should not be modified. This enables the information processing system S to prevent a situation in which a part of the minutes that is not intended by the user is modified.

[0069] Furthermore, the determination unit 134 may reflect the correction content associated with a specific item among the multiple items in all the items. In this case, the determination unit 134 determines a second input sentence including an instruction to correct the contents of all the multiple items, on the condition that the correction content is associated with a specific item (e.g., "participant") among the multiple items.

[0070] On the other hand, the determination unit 134 determines a second input sentence including an instruction to modify the contents of items among the multiple items that are associated with the modification content, and not to modify the contents of items that are not associated with the modification content, on the condition that the modification content is associated with an item other than a specific item among the multiple items (for example, other than "participants"). This enables the information processing system S to modify the entire minutes for a specific item, such as participants, among the multiple items included in the minutes, and maintain the consistency of the entire minutes.

[0071] The minutes generation unit 135 acquires the machine learning model used to generate the minutes to be corrected, which is stored in advance in the storage unit 12. The minutes generation unit 135 inputs the second input sentence determined by the determination unit 134 into the machine learning model, and generates the content output by the machine learning model as corrected minutes. The corrected minutes generated by the minutes generation unit 135 reflect, for each of the multiple items, the correction content associated with that item in the correction instruction.

[0072] The minutes generation unit 135 may correct the minutes using a machine learning model executed in an external device different from the information processing device 1. In this case, the minutes generation unit 135 transmits a second input sentence to the external device, and receives from the external device the corrected minutes that have been generated by the external device inputting the transmitted second input sentence into the machine learning model.

[0073] The display control unit 136 displays the minutes corrected by the minutes generation unit 135, i.e., the corrected minutes generated by inputting the second input sentence into the machine learning model, on the information terminal 2. The display control unit 136 displays the corrected minutes on the information terminal 2 used by a user who performed a predetermined operation to display the minutes, among the multiple users who participated in the conference. The display control unit 136 may also display the corrected minutes on the information terminal 2 used by a user who did not participate in the conference.

[0074] The display control unit 136 transmits display information for displaying the revised minutes to the information terminal 2. The information terminal 2 displays a minutes display screen including the revised minutes on the display unit in accordance with the display information transmitted by the information processing device 1.

[0075] 9 is a schematic diagram of an information terminal 2 displaying a minutes display screen A. As in FIG. 6, the minutes display screen A includes a minutes area A1 and a correction instruction area A2. The minutes area A1 includes a corrected minutes A3 generated by inputting the second input sentence into the machine learning model.

[0076] The correction instruction area A2 includes a correction instruction field A4 for accepting correction instructions for each of the items included in the minutes after the correction, in association with the items. The accepting unit 133 further accepts, at the information terminal 2, correction instructions for each of the items included in the minutes after the correction.

[0077] This allows the information processing system S to enable the user to easily revise the minutes of a meeting item by item using the machine learning model, and to provide the revised minutes to the user.

[0078] [Information processing method flow] 10 is a flowchart illustrating an exemplary information processing method executed by the information processing system S according to this embodiment. The acquisition unit 131 acquires character string data indicating the content of statements made in a conference attended by multiple users (S11). The acquisition unit 131 acquires character string data generated by, for example, having the speech recognition unit 132 perform a known speech recognition process on speech data of the conference transmitted by the information terminal 2.

[0079] The determination unit 134 determines a plurality of items to be included in the minutes (S12). The determination unit 134 acquires a first template for determining a first input sentence, which is stored in advance in the storage unit 12. The determination unit 134 applies (inserts) the determined plurality of items and the utterance content indicated by the character string data to the variable part of the first template, thereby determining a first input sentence (input sentence for generation) for causing the machine learning model to generate minutes of the meeting including a plurality of items (S13).

[0080] The minutes generation unit 135 acquires a machine learning model for generating minutes that is stored in advance in the storage unit 12. The minutes generation unit 135 inputs the first input sentence determined by the determination unit 134 to the machine learning model, and generates the contents output by the machine learning model as minutes of the meeting (S14).

[0081] The display control unit 136 causes the information terminal 2 to display the minutes generated by the minutes generation unit 135, i.e., the minutes generated by inputting the first input sentence into the machine learning model (S15). The receiving unit 133 receives, at the information terminal 2, a correction instruction associating each of a plurality of items included in the minutes with correction content for that item (S16).

[0082] The determination unit 134 acquires a second template for determining a second input sentence, which is stored in advance in the storage unit 12. The determination unit 134 determines a second input sentence for causing the machine learning model to correct the minutes that are the subject of the correction instructions specified by the user, by applying (inserting) the remark content indicated by the character string data, the minutes that are the subject of the correction instructions, and the correction instructions for each of the multiple items included in the minutes to the variable part of the second template (S17).

[0083] The minutes generation unit 135 acquires the machine learning model used to generate the minutes to be corrected, which is stored in advance in the storage unit 12. The minutes generation unit 135 inputs the second input sentence determined by the determination unit 134 to the machine learning model, and generates the content output by the machine learning model as the corrected minutes (S18).

[0084] The display control unit 136 causes the information terminal 2 to display the minutes corrected by the minutes generation unit 135, i.e., the corrected minutes generated by inputting the second input sentence into the machine learning model (S19).

[0085] [Effects of the embodiment] According to the information processing system S of this embodiment, the information processing device 1 causes a machine learning model to generate minutes including multiple items, accepts correction instructions from a user for each of the multiple items, and causes the machine learning model to correct the minutes based on the correction instructions. This allows the information processing system S to enable a user to easily correct the minutes of a meeting for each item using the machine learning model.

[0086] <First Modification> In the above-described embodiment, the information processing device 1 accepts a correction instruction for each of the multiple items included in the minutes, whereas in this modified example, the information processing device 1 accepts a correction instruction for a part of the minutes. Below, the differences from the above-described embodiment will be mainly described.

[0087] The determining unit 134 determines a first input sentence (input sentence for generation) for causing the machine learning model to generate minutes of the meeting, based on the character string data acquired by the acquiring unit 131. In this modification, the minutes of the meeting do not need to include multiple items, and may be a sentence or a list of items, etc.

[0088] The determination unit 134 determines the first input sentence by, for example, applying (inserting) the utterance content indicated by the character string data to the variable part of the first template. The minutes generation unit 135 inputs the first input sentence determined by the determination unit 134 to a machine learning model, and generates the content output by the machine learning model as minutes of the meeting.

[0089] The display control unit 136 causes the minutes generated by the minutes generation unit 135, i.e., the minutes generated by inputting the first input sentence into the machine learning model, to be displayed on the information terminal 2. The display control unit 136 transmits display information for displaying the minutes to the information terminal 2. The information terminal 2 displays a minutes display screen including the minutes on the display unit in accordance with the display information transmitted by the information processing device 1.

[0090] 11 is a schematic diagram of an information terminal 2 displaying a minutes display screen A according to this modification. The minutes display screen A includes a minutes area A1. The minutes area A1 includes the minutes generated by inputting the first input sentence into the machine learning model.

[0091] The receiving unit 133 receives, from the user, a designation of a target portion A5 in the minutes of meeting that is to be subject to a correction instruction in the minutes of meeting area A1. For example, the receiving unit 133 receives a range selected by the user in the minutes of meeting as the target portion A5.

[0092] For example, in response to the designation of the target portion A5, the display control unit 136 displays a correction instruction field A4 showing multiple options for correction content in association with the target portion A5. Furthermore, the display control unit 136 may also display a correction instruction field A4 in association with the target portion A5, into which the user can input a character string indicating the correction content.

[0093] The receiving unit 133 receives a correction instruction that associates a target portion designated by the user with correction content for the target portion at the information terminal 2. For example, the receiving unit 133 receives from the user a selection or input of correction content for the target portion A5 in the correction instruction field A4. The receiving unit 133 stores in the storage unit 12 the correction instruction that associates the target portion with the correction content selected or input for the target portion.

[0094] The determination unit 134 determines a second input sentence (input sentence for correction) for causing the machine learning model to correct the minutes that are the subject of the correction instruction specified by the user, based on the correction instruction received by the reception unit 133.

[0095] The determination unit 134 determines the second input sentence, for example, by applying (inserting) into the variable part of the second template the content of the statement indicated by the string data, the minutes that are the subject of the correction instructions, and the correction instructions associated with the target part that is part of the minutes.

[0096] The correction instruction included in the second input sentence is information that associates the target portion, which is a part of the minutes that is the target of the correction instruction, with the correction content indicated by the correction instruction for the target portion. This allows the information processing system S to reduce the effort required by the user to create an input sentence to have the machine learning model correct the part of the minutes.

[0097] The minutes generation unit 135 inputs the second input sentence determined by the determination unit 134 into a machine learning model, and generates the content output by the machine learning model as revised minutes. The display control unit 136 displays the minutes revised by the minutes generation unit 135, i.e., the revised minutes generated by inputting the second input sentence into the machine learning model, on the information terminal 2. The revised minutes are displayed, for example, on a minutes display screen A illustrated in FIG. 11.

[0098] The information processing system S according to this modification allows a user to easily correct a portion of a meeting minutes by using a machine learning model, and can provide the corrected minutes to the user.

[0099] <Second Modification> In the above-described embodiment, the information processing device 1 generates the revised minutes after receiving the revision instruction, whereas in this modified example, the information processing device 1 generates candidates for the revised minutes before receiving the revision instruction. Below, the differences from the above-described embodiment will be mainly described.

[0100] After the minutes generation unit 135 generates the minutes using the first input sentence and before the display control unit 136 displays the minutes on the information terminal 2, the determination unit 134 generates second input sentences (input sentences for correction) for causing the machine learning model to correct the minutes. The determination unit 134 determines the multiple second input sentences, for example, using the correction contents indicated by each of multiple options that will be displayed later on the information terminal 2 (correction instruction field A4).

[0101] The minutes generation unit 135 acquires the machine learning model used to generate the minutes to be corrected, which is stored in advance in the storage unit 12. The minutes generation unit 135 inputs each of the multiple second input sentences determined by the determination unit 134 to the machine learning model, and generates the content output by the machine learning model as a candidate for the minutes after correction.

[0102] The display control unit 136 causes the information terminal 2 to display a minutes display screen A including the minutes generated by the minutes generation unit 135, i.e., the minutes generated by inputting the first input sentence into the machine learning model.

[0103] In response to the user selecting one of multiple options in the correction instruction field A4 of the minutes display screen A, the display control unit 136 causes the information terminal 2 to display a candidate revised minutes generated using the second input sentence corresponding to the selected option.

[0104] The information processing system S of this modified example generates candidates for the revised minutes in advance before receiving a correction instruction for the minutes from the user, thereby reducing the time it takes from receiving a correction instruction to providing the revised minutes to the user.

[0105] Furthermore, this invention will make it possible to contribute to Goal 9 of the United Nations' Sustainable Development Goals (SDGs), which is "Build resilient infrastructure, promote inclusive and sustainable industrialization, and promote innovation and resilience."

[0106] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. For example, all or part of the device can be configured by functionally or physically distributing or integrating any unit. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of the new embodiments resulting from the combination also have the effects of the original embodiments. [Explanation of symbols]

[0107] S Information Processing System 1. Information processing equipment 11 Communications Department 12 Storage section 13 Control Unit 131 Acquisition Department 132 Voice Recognition Unit 133 Reception Department 134 Decision Section 135 Minutes Generation Department 136 Display control unit 2. Information terminal

Claims

1. an acquisition unit that acquires character string data indicating the content of statements made in a conference; a determination unit that determines, based on the character string data, a first input sentence for causing a machine learning model to generate minutes of the meeting including a plurality of items; a display control unit that displays, on an information terminal, the minutes of the meeting generated by inputting the first input sentence into the machine learning model; a receiving unit in the information terminal that receives a correction instruction that associates each of the plurality of items included in the minutes with a correction content for the item; and the determination unit determines a second input sentence for causing the machine learning model to modify the minutes based on the character string data, the minutes, and the modification instruction; the display control unit causes the information terminal to display the minutes after the correction by inputting the second input sentence into the machine learning model. Information processing device.

2. the determination unit determines the second input sentence so as to modify content of an item associated with the content of the modification among the plurality of items and not to modify content of an item not associated with the content of the modification. The information processing device according to claim 1 .

3. the determination unit determines the second input sentence so as to modify the contents of all of the plurality of items on the condition that the modification content is associated with a specific item among the plurality of items; the determination unit determines the second input sentence such that, on condition that the correction content is associated with an item other than the specific item among the plurality of items, a content of an item associated with the correction content among the plurality of items is corrected and a content of an item not associated with the correction content is not corrected. The information processing device according to claim 2 .

4. the accepting unit accepts designation of the plurality of items from a user at the information terminal before the determining unit determines the first input sentence; the determination unit determines the first input sentence for causing the machine learning model to generate the minutes including the plurality of items specified by the user. The information processing device according to claim 1 .

5. the accepting unit accepts selection of the plurality of items from a plurality of candidates for the plurality of items displayed on the information terminal before the determining unit determines the first input sentence. The information processing device according to claim 4 .

6. the determination unit determines the plurality of items according to the content of the utterance, and determines the first input sentence for causing the machine learning model to generate the minutes including the determined plurality of items. The information processing device according to claim 1 .

7. the determination unit determines the plurality of items according to a type or a purpose of the meeting, and determines the first input sentence for causing the machine learning model to generate the minutes including the determined plurality of items. The information processing device according to claim 1 .

8. the display control unit causes the information terminal to display a plurality of options for the correction content in association with each of the plurality of items; the accepting unit accepts a selection of the correction content from the plurality of options for each of the plurality of items. The information processing device according to claim 1 .

9. the display control unit varies the plurality of options depending on each of the plurality of items. The information processing device according to claim 8 .

10. the display control unit varies the plurality of options depending on the content of the statement; The information processing device according to claim 8 .

11. the determination unit determines the first input sentence by applying the utterance content indicated by the character string data to a template stored in advance in a storage unit. The information processing device according to claim 1 .

12. the determination unit determines the second input sentence by applying the content of the statement indicated by the character string data, the minutes, and the correction instructions for each of the plurality of items to a template stored in advance in a storage unit. The information processing device according to claim 1 .

13. The processor executes acquiring character string data indicating the content of speeches made in a conference; determining, based on the character string data, a first input sentence for causing a machine learning model to generate minutes of the meeting including a plurality of items; displaying, on an information terminal, the minutes of the meeting generated by inputting the first input sentence into the machine learning model; receiving, at the information terminal, a correction instruction associating each of the plurality of items included in the minutes with correction content for the item; determining a second input sentence for causing the machine learning model to correct the minutes based on the character string data, the minutes, and the correction instruction; displaying, on an information terminal, the minutes of the meeting after being corrected by inputting the second input sentence into the machine learning model; An information processing method comprising:

14. The processor acquiring character string data indicating the content of speeches made in a conference; determining, based on the character string data, a first input sentence for causing a machine learning model to generate minutes of the meeting including a plurality of items; displaying, on an information terminal, the minutes of the meeting generated by inputting the first input sentence into the machine learning model; receiving, at the information terminal, a correction instruction associating each of the plurality of items included in the minutes with correction content for the item; determining a second input sentence for causing the machine learning model to correct the minutes based on the character string data, the minutes, and the correction instruction; displaying, on an information terminal, the minutes of the meeting after being corrected by inputting the second input sentence into the machine learning model; A program that executes.

Images