Information processing device, information processing method and program

Abstract

To address such a problem that it is not possible with conventional techniques to operate two or more devices using a unified and simple interface.SOLUTION: An information processing device 5 includes an information receiving part 521 for receiving input information including an instruction sentence input into a dialogue application by a user using a terminal device 6; a determination part 531 for determining one access destination according to the input information from two or more access destinations including one or more external devices 4; an instruction acquisition part 532 for acquiring instruction information according to the instruction sentence included in the input information and the access destination, and an instruction output part 541 for passing the instruction information to the access destination. The information processing device thus allows the user to easily operate two or more devices by inputting input information including natural language sentences.SELECTED DRAWING: Figure 23

Description

[Technical field]

[0001] The present invention relates to an information processing device or the like that performs processing according to an instruction sentence input by a user to a dialogue application. [Background technology]

[0002] In the prior art, there is an information presentation system that makes it possible to obtain necessary information by utilizing a past dialogue history without using direct keywords (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-77268 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, it is not possible to operate two or more devices using a unified and simple interface. [Means for solving the problem]

[0005] The information processing device of the first invention is an information processing device including an information receiving unit that receives input information including an instruction statement input by a user to an interactive application using a terminal device, a determination unit that determines one access destination from two or more access destinations including one or more external devices in accordance with the input information, a command acquisition unit that acquires command information in accordance with the instruction statement and the access destination included in the input information, and a command output unit that passes the command information to the access destination.

[0006] With this configuration, a user can easily operate two or more devices by inputting input information including a natural language sentence.

[0007] In addition, the information processing device of the second invention is an information processing device that, compared to the first invention, further includes a result receiving unit that receives result information corresponding to command information from the access destination, and a result sending unit that sends the result information to the interactive application.

[0008] With this configuration, a user can easily operate two or more devices by inputting input information including a natural language sentence, and can receive the results of the operation.

[0009] In addition, the information processing device of the third invention is an information processing device that, compared to the first or second invention, includes a template storage unit in which AI template information for creating a query sentence to be sent to the generation AI device, and an instruction acquisition unit that includes a template acquisition means for acquiring the AI ​​template information in the template storage unit, a sentence creation means for creating a query sentence using an instruction sentence and the AI ​​template information, a sentence sending means for sending the query sentence created by the sentence creation means to the generation AI device, an answer receiving means for receiving an answer to a question corresponding to the query sentence from the generation AI device in response to the transmission of the query sentence, and an instruction acquisition means for acquiring command information using the answer.

[0010] With this configuration, a query can be given to the generation AI, and the obtained answer can be used to easily obtain command information to be given to the accessed device.

[0011] In addition, in the information processing device of the fourth invention, compared to the third invention, the instruction acquisition unit further includes a type acquisition means for acquiring an instruction type that specifies the type of instruction statement, the template storage unit stores two or more AI template information corresponding to an access destination and an instruction type, and the template acquisition means is an information processing device that acquires AI template information corresponding to one access destination determined by the determination unit and the instruction type acquired by the type acquisition means.

[0012] With this configuration, a query can be obtained from the generation AI using AI template information corresponding to the instruction text.

[0013] In addition, in the information processing device of the fifth invention, compared to the third or fourth invention, the command acquisition unit further includes a search means for searching the access destination using the answer received by the answer receiving means and receiving search results including one or more search information from the access destination, a screen acquisition means for acquiring screen information constituting a screen for selecting one or more search information from the two or more search information when the search results include two or more search information, a screen output means for passing the screen information to the dialogue application, and a selection receiving means for receiving a selection of one or more search information corresponding to the screen information, and the command acquisition means is an information processing device that acquires command information using the one or more search information corresponding to the selection received by the selection receiving means.

[0014] With this configuration, before performing an operation using command information on the access destination, a screen is output to the user for selecting search information using two or more search information obtained from the access destination, and appropriate command information can be obtained using the search information selected by the user using that screen.

[0015] Furthermore, an information processing device of the sixth invention is an information processing device in which, compared to any one of the first to fifth inventions, when the command output unit passes command information to an access destination, the command acquisition unit acquires an attribute name of an attribute value that is a necessary attribute value that should be included in a record corresponding to the command information but does not exist, passes an input request to the dialogue application using the attribute name to prompt input of the attribute value, accepts input in response to the input request, acquires command information that also uses the input, and the command output unit passes the command information that also uses the input to the access destination.

[0016] With this configuration, by prompting the user to input required attribute values ​​in the records managed at the access destination, it is possible to enable the appropriate records to be managed at the access destination device.

[0017] Furthermore, an information processing device of the seventh invention is an information processing device in which, relative to any one of the third to sixth inventions, the input information corresponds to a user identifier, the command acquisition unit further includes an attribute value acquisition means for acquiring one or more user attribute values ​​corresponding to the user identifier, and the sentence creation means creates a query sentence including different parameters depending on the one or more user attribute values ​​acquired by the attribute value acquisition means.

[0018] With this configuration, an appropriate answer can be obtained from the generation AI using a learning model based on the user attribute values.

[0019] An information processing device according to an eighth aspect of the present invention is the information processing device according to any one of the first to seventh aspects of the present invention, wherein the determination unit determines one access destination by using a directive.

[0020] With this configuration, the access destination can be automatically determined from the received instruction text.

[0021] In addition, an information processing device of the ninth invention is an information processing device according to any one of the first to eighth inventions, wherein the information receiving unit receives input information after an online conference, and the command acquisition unit acquires command information corresponding to an instruction sentence from conversation information in the online conference.

[0022] With this configuration, after the online conference, command information can be constructed using information obtained from conversation information of the online conference.

[0023] In addition, the information processing device of the tenth invention is an information processing device in which the command acquisition unit is equipped with a template acquisition means for acquiring AI template information from a template storage unit, a sentence creation means for creating a query sentence using an instruction sentence and the AI ​​template information, a sentence sending means for sending the query sentence created by the sentence creation means to a generation AI device that has learned conversation information, an answer receiving means for receiving an answer to a question corresponding to the query sentence from the generation AI device in response to the transmission of the query sentence, and an command acquisition means for acquiring command information using the answer.

[0024] With this configuration, after an online conference, command information can be constructed using information obtained from a generation AI that has learned conversation information from the online conference.

[0025] Furthermore, the information processing device of the eleventh invention is different from the information processing device of the ninth invention in that the command acquisition unit is information included in the conversation information, and the information processing device includes: a candidate acquisition means for acquiring one or more candidate information constituting the command information; a screen acquisition means for acquiring screen information constituting a screen for selecting one or more candidate information from the two or more candidate information when the candidate acquisition means acquires two or more candidate information; a screen output means for passing the screen information to the dialogue application; a selection receiving means for accepting a selection of the one or more candidate information corresponding to the screen information; and a command acquisition means for acquiring command information using the one or more candidate information corresponding to the selection accepted by the selection receiving means.

[0026] With this configuration, after the online conference, command information can be constructed using candidate information selected by the user from two or more pieces of candidate information acquired from conversation information of the online conference. Effect of the Invention

[0027] According to the information processing device of the present invention, a user can easily operate two or more devices by inputting input information including a natural language sentence. [Brief description of the drawings]

[0028] [Figure 1] Conceptual diagram of information system A in embodiment 1. [Diagram 2] Block diagram of Information System A [Diagram 3] Block diagram of the information processing device 1 [Figure 4] A flowchart illustrating an example of the operation of the information processing device 1. [Diagram 5] A flowchart for explaining an example of the operation of the homogeneous information processing device 1. [Figure 6] A flowchart for explaining an example of a form acquisition process at the time of entry [Figure 7] A flowchart for explaining an example of a form acquisition process during a conversation [Figure 8] A flowchart illustrating an example of the question set acquisition process [Figure 9] A flowchart illustrating an example of the analysis process [Figure 10] A flowchart illustrating an example of the audio division process. [Figure 11] A flowchart for explaining an example of the exit form acquisition process [Figure 12] A flowchart illustrating an example of the processing information acquisition process [Figure 13] A flowchart for explaining an example of the operation of the terminal device 2 [Figure 14] The user management table [Figure 15] A diagram showing the admission form management table [Figure 16] A diagram showing the form management table for the conversation [Figure 17] A diagram showing the question management table [Figure 18] The accumulation management table [Figure 19] A diagram showing an example of the form [Figure 20] A diagram showing an example of the output [Figure 21] Conceptual diagram of information system B in embodiment 2 [Figure 22] Block diagram of Information System B [Diagram 23] Block diagram of the information processing device 5 [Figure 24] A flowchart for explaining an example of the operation of the information processing device 5. [Diagram 25] A flowchart illustrating an example of the access destination determination process. [Figure 26] A flowchart illustrating an example of the command acquisition process. [Figure 27] A flowchart for explaining an example of the template acquisition process [Figure 28] A flowchart for explaining an example of the process of constructing the query sentence. [Figure 29] A flowchart for explaining an example of the command configuration process. [Diagram 30] A flowchart for explaining an example of the operation of the terminal device 6 [Diagram 31] A diagram showing the template management table [Diagram 32] A diagram showing an example of the screen [Diagram 33] Overview of the computer system according to the above embodiment. [Diagram 34] Block diagram of the computer system DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Hereinafter, an embodiment of an information processing device and the like will be described with reference to the drawings. Note that, in the embodiments, components with the same reference numerals perform similar operations, and therefore repeated description may be omitted.

[0030] (Embodiment 1) In this embodiment, we will describe an information processing device that acquires reception information received at one or more of three or more points in time: before, during, or after an online conference (hereinafter referred to as "conference" as appropriate), an online conference, a conference, or a conference (usually immediately after the conference), acquires processed information based on the reception information, and stores the processed information.

[0031] In the present embodiment, an information processing device that searches for an external device using entry input information obtained when a user enters a conference and outputs the searched information will be described.

[0032] In addition, in this embodiment, form information for inputting entry information is managed in correspondence with one or more conditions, and an information processing device is described that outputs form information that pairs with a condition that the user matches.

[0033] In addition, in this embodiment, form information for inputting conversation-time input information to be obtained during a conference is managed in correspondence with one or more conditions, and an information processing device is described that outputs form information that pairs with a condition that the user matches. The information processing device will be described.

[0034] In addition, in this embodiment, an information processing device that outputs a set of question information for assisting a user in a conversation will be described.

[0035] In the present embodiment, an information processing device will be described that manages question information in association with one or more conditions and outputs question information paired with a condition that the user meets.

[0036] In addition, in this embodiment, an information processing apparatus will be described that employs processed information that is a result of analyzing conversation information including voice information in a conference, for information that is not input by a user.

[0037] Furthermore, in this embodiment, an information processing device that accumulates processing information in an external device will be described.

[0038] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is not important. Information X and information Y may be linked, may exist in the same buffer, information X may be included in information Y, or information Y may be included in information X, etc.

[0039] 1 is a conceptual diagram of an information system A according to the present embodiment. The information system A typically includes an information processing device 1, one or more terminal devices 2, one or more conference support devices 3, and one or more external devices 4.

[0040] The information processing device 1 is a device that acquires and accumulates information received regarding a conference. The information processing device 1 may have a function of supporting the conference. Such a function is, for example, a web conference function provided by Zoom (registered trademark) or a web conference function provided by Teams (registered trademark). The information processing device 1 is, for example, a cloud server or an ASP server, but the type does not matter. If the information processing device 1 is a terminal, it may be considered that the terminal device 2 is not necessary in the information system A, or that the information processing device 1 also serves as the terminal device 2. If the information processing device 1 is a terminal, the information processing device 1 is, for example, a so-called personal computer, a smartphone, or a tablet terminal, but the type does not matter.

[0041] The terminal device 2 is a terminal used by a user. A user is usually a participant in a conference. Video and audio of the other party in the conference are output to the terminal device 2. User information, an entry form, a conversation form, a set of questions, and the like provided by the information processing device 1 are output to the terminal device 2. The terminal device 2 is, for example, a so-called personal computer, a smartphone, or a tablet terminal, but the type is not important.

[0042] The conference support device 3 is a device that operates a conversation app that supports conversation between two or more users. The conversation app is an application that supports conversation. The conversation is, for example, a conversation in an online conference, but may also be a conversation by telephone. The conversation app is, for example, a web conference app or a telephone app. The conversation app is, for example, Zoom (registered trademark), Teams (registered trademark), or Zoom (registered trademark) Phone. The conference support device 3 is usually a server, for example, a cloud server or an ASP server, but the type does not matter.

[0043] The external device 4 is a device different from the information processing device 1. The external device 4 is, for example, a device that manages data. The data is, for example, personnel information, customer information, and sales information, but the type does not matter. The external device 4 is, for example, a CRM (customer management system) server. The external device 4 is, for example, a Salesforce (registered trademark) server or a Cybozu (registered trademark) server. The external device 4 is usually a server. It is preferable that the external device 4 has functions unrelated to online conferences, such as a personnel information management function, a customer information management function, and a sales information management function. The external device 4 is, for example, a cloud server or an ASP server, but may also be a database device or the like, and the type does not matter.

[0044] It is preferable that the information processing device 1 and one or more terminal devices 2, the information processing device 1 and one or more external devices 4, and the one or more terminal devices 2 and one or more external devices 4 are capable of communicating with each other via a network such as the Internet.

[0045] Fig. 2 is a block diagram of an information system A according to the present embodiment, and Fig. 3 is a block diagram of an information processing device 1.

[0046] The information processing device 1 includes a storage unit 11, a reception unit 12, a processing unit 13, and an output unit 14. The storage unit 11 includes a conversation storage unit 111, an entry form management unit 112, a conversation form management unit 113, and a question management unit 114. The reception unit 12 includes an information reception unit 121. The processing unit 13 includes an entry form acquisition unit 131, a search unit 132, a conversation form acquisition unit 133, a question acquisition unit 134, an information acquisition unit 135, and an accumulation unit 136. The output unit 14 includes a user information output unit 141, an entry form output unit 142, a conversation form output unit 143, and a question output unit 144.

[0047] The terminal device 2 includes a terminal storage unit 21, a terminal reception unit 22, a terminal processing unit 23, a terminal transmission unit 24, a terminal reception unit 25, and a terminal output unit .

[0048] Various types of information are stored in the storage unit 11 constituting the information processing device 1. The various types of information are, for example, conversation information, various types of form information, a set of questions, one or more pieces of user management information, conference information, and one or more pieces of storage control information. The various types of form information are, for example, entry form information, conversation form information, or exit form information.

[0049] User management information is information about users participating in a conference. User management information has a user identifier and one or more user attribute values. A user identifier is information that identifies a user. Here, the user identifier is, for example, a user ID, an email address, or a phone number. A user attribute value is a user's attribute value. The user attribute value is, for example, the user's affiliation (e.g., company name, school name), the user's job title, name, address, age, and gender.

[0050] Conference information is information about a conference. Conference information has, for example, one or more conversation attribute values. Conversation attribute values ​​are attribute values ​​related to a conversation. Conversation attribute values ​​are, for example, title information, information about one or more organizers, information about one or more participants, date and time, and time period.

[0051] The storage control information is information used to store the processed information. The storage control information includes, for example, form-specific information, storage destination information, storage method information, API, and attribute correspondence information. The form-specific information is information that specifies a form into which information from which the processed information is obtained is input. The form-specific information is, for example, one or more form identifiers. The form identifier is information that identifies the form information. The storage destination information is information that specifies a destination for storing the processed information. The storage destination information includes, for example, a device identifier. The storage destination information includes, for example, a table identifier. The device identifier is, for example, information that identifies the external device 4 or the information processing device 1. The device identifier is, for example, an IP address or a device ID. The table identifier is, for example, a table name or a table ID. The storage method information is information that specifies a method for storing the processed information. The storage method information is, for example, "APPEND", which always appends a record, or "UPDATE OR APPEND", which updates if a record with the same key exists, and appends a record if a record with the same key does not exist. The API is information indicating an API for storing processing information, such as a function name or a method name. The attribute correspondence information is information indicating the correspondence between one or more pieces of element information constituting the processing information and the attributes of the table in which the processing information is stored.

[0052] The conversation storage unit 111 stores one or more pieces of conversation information. The conversation information includes audio information. It is preferable that the conversation information includes video. The conversation information may include only audio information.

[0053] Conversation information is information acquired during a conversation between multiple users. Audio information is information recorded of audio during a conversation between multiple users. Video is video recorded during a conversation between multiple users. The conversation may be, for example, a conversation in a meeting, but the content of the conversation is not important. It is preferable that the conversation is a conversation in a business meeting, but the purpose and content of the conversation are not important. Note that a conversation may also be referred to as a meeting in some cases.

[0054] It is preferable that one or more conversation attribute values ​​are associated with the conversation information. The conversation attribute values ​​are attribute values ​​related to the conversation. For example, the conversation attribute values ​​are title information, organizer information, one or more participant information, date and time, and time period.

[0055] The title information is information that specifies the title of the conference. The title information is usually a character string. The conference title may also be called the title of the conversation.

[0056] Organizer information is information related to the organizer of a conference. Organizer information has, for example, an organizer identifier and one or more organizer attribute values. Organizer identifier is information that identifies the organizer. The organizer identifier is, for example, the name of the organizer and the organizer ID. The organizer identifier is, for example, the user identifier of the organizer. The organizer attribute value is, for example, the organizer's affiliation and the organizer's job title. The organizer's affiliation is, for example, the organizer's company name and the name of the department to which the organizer belongs. Normally, the organizer is also a participant.

[0057] Participant information is information about participants in a conference. Participant information has, for example, a participant identifier and one or more participant attribute values. Participant identifier is information that identifies a participant. The participant identifier is, for example, the participant's name and the participant's ID. The participant identifier is, for example, the participant's user identifier. The participant attribute value is, for example, the participant's affiliation and the participant's job title. The participant's affiliation is, for example, the participant's company name and the name of the department to which the participant belongs. A participant is a participant in a conversation. A participant may also be called a user.

[0058] The date and time is the date and time when the conference is held, for example, the start time or the start time and end time. The time period is information that specifies the time period of the conversation.

[0059] One or more pieces of entry form information are stored in the entry form management unit 112. It is preferable that each of the one or more pieces of entry form information corresponds to an entry condition.

[0060] The entry form information is information for configuring a form for inputting entry input information. The entry form information is information for configuring a form that is output to the terminal device 2 before the conference, such as when entering a conference. A form is usually a screen on which a user can input information. A form has one or more types of user interface, such as one or more fields, one or more check boxes, one or more buttons, etc. Furthermore, the data format and realization means of form information such as entry form information are not important. The form information is realized, for example, by HTML, XML, a data collection, a program, or a combination thereof.

[0061] The entry condition is a condition for outputting the paired entry form information. The entry condition is, for example, a user condition and a conversation attribute value condition. The user condition is a condition related to a user. The user condition is, for example, one or more user identifiers (that the participants of the conference include one or more specific users) and one or more group identifiers of the groups to which the user belongs (that the participants of the conference include users who belong to one or more specific groups). The conversation attribute value condition is a condition related to one or more conversation attribute values. The conversation attribute value condition is, for example, a title condition, a date and time condition, and a time period condition. The title condition is a condition related to title information. The title condition is, for example, that the title information includes a specific term. The date and time condition is information related to date and time. The date and time condition is, for example, that it is a specific day of the week, or a specific date. The time period condition is information related to a time period. The time period condition is, for example, that it is a specific time period (for example, between 10:00 p.m. and 7:00 a.m.).

[0062] One or more pieces of conversation-time form information are stored in the conversation-time form management unit 113. It is preferable that each of the one or more pieces of conversation-time form information corresponds to a conversation-time condition.

[0063] The conversation-time form information is information for configuring a form for inputting conversation-time input information. The conversation-time form information is information for configuring a form that is output to the terminal device 2 for the entire time during the conference or for a predetermined period of time or longer during the conference.

[0064] The conversation-time condition is a condition for outputting the conversation-time form information that is paired with the conversation-time condition, such as a user condition or a conversation attribute value condition.

[0065] The form information at the time of leaving the storage unit 11 is information for constructing a form for inputting the form information at the time of leaving. The form information at the time of leaving is information for constructing a form that is output after the end of the conference. It is preferable that the form information at the time of leaving corresponds to the conditions at the time of leaving. Note that the end of the conference usually means the time when the participant leaves the conference, but it may also be a considerable amount of time after the participant leaves the conference.

[0066] The exit condition is a condition for outputting the corresponding exit form information, for example, a user condition or a conversation attribute value condition.

[0067] One or more question sets are stored in the question management unit 114. Each of the one or more question sets preferably corresponds to a question condition.

[0068] A question set is a set of one or more pieces of question information. Question information is information indicating the content of a question that one user should ask other users in an online conference. Question information is usually a character string, but it may also be a still image or a video, and the data type does not matter. A check box may be associated with the question information.

[0069] The reception unit 12 receives various instructions and information. The various instructions and information are, for example, reception information, conversation information, and various instructions described below. The various instructions are, for example, an entry instruction, a start instruction, and an exit instruction. The reception unit 12 receives various instructions and information from, for example, the terminal device 2, but may also receive various instructions and information from the conference support device 3.

[0070] The entry instruction is an instruction to enter a conference. The entry instruction is usually associated with a user identifier. The reception unit 12 receives the entry instruction from the terminal device 2, for example, but may also receive the entry instruction from the conference support device 3.

[0071] The start instruction is an instruction to start a conference. The start instruction may be information that can specify that a conference will start or that the conference has started. The information that can specify that the conference has started is, for example, video or audio information of the conference. The reception unit 12 receives the start instruction from, for example, the terminal device 2, but may also receive the start instruction from the conference support device 3.

[0072] The exit instruction is an instruction to leave the conference. The exit instruction is usually associated with a user identifier. The reception unit 12 receives the exit instruction from, for example, the terminal device 2, but may also receive the exit instruction from the conference support device 3.

[0073] Here, acceptance typically means receiving information from the terminal device 2 or the conference support device 3, but it may also be a concept that includes acceptance of information input from an input device such as a keyboard, mouse, or touch panel, or acceptance of information read from a recording medium such as an optical disk, magnetic disk, or semiconductor memory.

[0074] The information receiving unit 121 acquires one or more pieces of reception information. The acquisition here is usually reception from the terminal device 2, but the means and route of acquisition are not important. The acquisition may be considered the same concept as reception.

[0075] Reception information is information acquired in association with a conference. For example, reception information is information received at one or more of three or more points in time: before an online conference, during an online conference, or after an online conference. Reception information is, for example, input information. Input information is information input by a user. For example, input information is information input at the time of entry, information input during a conversation, information input after a conversation, or answer information. Reception information is, for example, conversation information. Conversation information includes audio information of a conversation between users in a conference. It is preferable that the conversation information includes video.

[0076] The entry input information is information that is input when entering a conference. The entry input information is usually information that is input before entering a conference, but may be information that is input immediately after entering a conference. The entry input information is usually information that is input immediately before or immediately after entering a conference. However, the entry input information may be information that is input well before entering. Entering a conference means participating in an online conference. The input information here is usually information that is input by the user.

[0077] Conversation input information is information entered during a conference. Conversation input information is information entered by a user during a conference or information obtained from conversation information received during a conference.

[0078] The post-conversation input information is information input by the user after the conference. Here, "after the conference" preferably means immediately after the conference ends, but may also mean after a considerable amount of time has passed since the conference.

[0079] Answer information is information based on an answer to a question. Answer information is usually information that specifies an answer to a question, but may also be information acquired from an answer to a question. The answer to a question may be information entered by a user, or may be information acquired from conversation information. Information acquired from conversation information is information acquired from a character string acquired by performing character recognition processing on audio information of a conversation. Information acquired from audio information is, for example, a character string acquired by performing character recognition processing on audio information that is an answer to audio information indicating a question of a user, or information acquired from the character string.

[0080] The processing unit 13 performs various types of processing. The various types of processing are, for example, processing performed by an entry form acquisition unit 131, a search unit 132, a conversation form acquisition unit 133, a question acquisition unit 134, an information acquisition unit 135, and a storage unit 136.

[0081] The processing unit 13 performs, for example, the following (1) online conference support processing, (2) conversation information acquisition processing, and (3) analysis processing. (1) Online conference support processing

[0082] The processing unit 13, for example, supports two or more users in holding an online conference. The online conference support process is the same process as Zoom and Teams and is a well-known technology, so detailed description will be omitted. (2) Conversation information acquisition process:

[0083] The processing unit 13 acquires conversation information. For example, the processing unit 13 acquires conversation information in an online conference that the processing unit 13 supports.

[0084] Furthermore, the processing unit 13 receives conversation information, for example, from one or more conference supporting devices 3. The method by which the processing unit 13 acquires the conversation information may be a push type or a pull type. The push type means that the processing unit 13 receives conversation information transmitted from the conference supporting device 3. The pull type means that the processing unit 13 goes to check the conference supporting device 3 and acquires the conversation information from the conference supporting device 3.

[0085] The processing unit 13 preferably acquires conversation information associated with one or more user identifiers. The user identifier here is information for identifying a user who is a party to the conversation. The user identifier here is, for example, an email address, a telephone number, or an ID of a user of the conference.

[0086] Moreover, it is preferable that the processing unit 13 stores a part or all of the acquired conversation information. The processing unit 13 stores, for example, the audio information contained in the conversation information acquired by the processing unit 13. The destination where the processing unit 13 stores the information is, for example, the conversation storage unit 111, but it may also be the external device 4, and is not important. Note that the conversation information is, for example, an mp3 file or an mp4 file, but the data type is not important. (3) Analysis and processing

[0087] The processing unit 13 acquires an analysis result of the voice information included in the acquired conversation information. The processing unit 13 may analyze the voice information itself and acquire the analysis result, or may acquire the analysis result acquired by an external device. The external device may be the conference support device 3 or another device not shown. The analysis result is, for example, a tag. For example, the processing unit 13 passes the voice information included in the conversation information acquired by the processing unit 13 to an external device not shown, and receives the analysis result from the external device.

[0088] The analysis result has, for example, one or more types of information among partial character strings that are speech recognition results for two or more partial speeches obtained by dividing the speech information, one or more tags, and one or more class identifiers.

[0089] The processing unit 13 acquires a conversation character string, which is a result of performing a speech recognition process on the speech information, for example. The processing unit 13 may perform a speech recognition process on the speech information, or may pass the speech information to a speech recognition server (not shown) and receive the conversation character string from the speech recognition server. In other words, the location of the device or program performing the speech recognition process does not matter. Next, the processing unit 13 determines a user who has spoken for the partial character string, for example, based on the speech information. For example, the processing unit 13 acquires one or more speech features from a partial speech obtained by dividing the speech information, and determines a user who has spoken the partial speech using the one or more speech features. Note that determining a user means, for example, acquiring a user identifier and distinguishing the user from other users. The processing unit 13 acquires a user identifier, for example, by pairing it with the partial speech. Next, the processing unit 13 acquires, for example, one or more tags for each of one or more partial character strings. For example, the processing unit 13 acquires a tag that is paired with a term included in each partial character string from a correspondence table. For example, the processing unit 13 provides each partial character string to a module that determines a class, executes the module, and acquires a class. Such a class can be a tag. The module that determines the class by giving a character string is a well-known module, and therefore a detailed description thereof will be omitted.

[0090] The entry form acquisition unit 131 acquires entry form information. It is preferable that the entry form acquisition unit 131 acquires the entry form information from the entry form management unit 112 when a user enters the room. It is preferable that the entry form acquisition unit 131 acquires the entry form information in response to, for example, receiving an instruction to enter the conference. However, there is no restriction on the trigger for the entry form acquisition unit 131 to acquire the entry form information. It is sufficient that the entry form acquisition unit 131 acquires the entry form information before the conference.

[0091] It is preferable that the entry form acquisition unit 131 acquires entry form information paired with an entry condition that is matched by a user participating in a conference. The entry condition that is matched by a user is an entry condition that is matched by a user who is a participant of the conference. The entry condition that is matched by a user is, for example, an entry condition where one or more pieces of information among one or more user attribute values ​​of the user and one or more conversation attribute values ​​of the conference match.

[0092] The entry form acquisition unit 131 may acquire entry form information common to all users.

[0093] The search unit 132 acquires user information from the external device 4 using the entry input information accepted by the information accepting unit 121. The user information is information about the user who inputs the entry input information. The user information is, for example, the user's name, the user's affiliation, and the user's job title. The user's affiliation is, for example, the user's company name and the user's department.

[0094] The search unit 132 searches for user information from the external device 4 using a user identifier included in the entry input information accepted by the information accepting unit 121 as a key. The user identifier is, for example, an email address, a telephone number, or a user ID.

[0095] The search unit 132 normally acquires user information from the external device 4, triggered by the information receiving unit 121 receiving the entry input information. However, any trigger may be used for the search unit 132 to acquire user information. Note that the user information acquired by the search unit 132 is output to the terminal device 2 of another user different from the one user corresponding to the user information, so that the other user can view the information of the one user who is the other party in the conference and proceed with the conference.

[0096] The conversation-time form acquisition unit 133 acquires conversation-time form information for inputting conversation-time input information from the conversation-time form management unit 113. The conversation-time form information may include a set of questions. In such a case, it can be said that the conversation-time form acquisition unit 133 acquires a set of questions. The conversation-time form acquisition unit 133 may acquire a set of questions that are paired with a question condition matched by one user participating in an online conference.

[0097] It is preferable that the conversation-time form acquisition unit 133 acquires conversation-time form information paired with conversation-time conditions that are met by users participating in the conference. The conversation-time form acquisition unit 133 may acquire conversation-time form information common to all users.

[0098] It is preferable that the conversation time form acquisition unit 133 acquires the conversation time form information from the conversation time form management unit 113 at the start of the conference. However, there are no restrictions on the trigger or timing at which the conversation time form acquisition unit 133 acquires the conversation time form information. Note that, by outputting the conversation time form to the terminal device 2, the user can input and acquire information to be kept as a record during the conference.

[0099] The question acquisition unit 134 acquires a question set from the question management unit 114. A question set is a set of one or more pieces of question information. Question information is information that identifies a question. A question is something that one user should ask other users in a conference. It is preferable that the question acquisition unit 134 acquires, from the question management unit 114, a question set that pairs with a question condition that is matched by one user.

[0100] It is preferable for the question acquiring unit 134 to acquire the set of questions from the question management unit 114 at the start of the conference. However, there are no restrictions on the trigger or timing for the question acquiring unit 134 to acquire the set of questions. Note that a user who has viewed the set of questions can proceed with the conference while viewing the questions contained in the set of questions, allowing the conference to proceed smoothly.

[0101] The information acquiring unit 135 acquires processing information that is information based on the reception information received by the information receiving unit 121 .

[0102] The processed information is information stored in the storage unit 11 or the external device 4. The processed information may be information input at the time of entry, information input during conversation, information input after conversation, or answer information. The processed information is, for example, setting information that changes the operation of the information processing device 1 or the external device 4. The processed information does not include audio information or video uttered by the user during the conference.

[0103] The information acquisition unit 135 acquires one or more pieces of processed information based on input information entered by the user, and it is preferable to acquire, by using conversation information, one or more pieces of processed information that are not based on input information entered by the user, among the two or more pieces of processed information that are planned to be acquired in advance.

[0104] The storage unit 136 stores one or more pieces of processing information acquired by the information acquisition unit 135. The storage unit 136 may store two or more pieces of processing information acquired by the information acquisition unit 135. The storage destinations for the two or more pieces of processing information may be different. The storage unit 136 stores the processing information acquired by the information acquisition unit 135 in the external device 4, for example.

[0105] The output unit 14 outputs various types of information, such as user information, entry form information, conversation form information, and a set of questions.

[0106] Here, output usually means transmission to terminal device 2, but it may also be a concept including display on a display, projection using a projector, printing on a printer, transmission to external device 4, storage on a recording medium, and handing over the processing results to other processing devices or other programs.

[0107] The user information output unit 141 outputs the user information acquired by the search unit 132 to the terminal devices 2 of one or more users participating in the online conference. The terminal devices 2 of one or more users are usually terminal devices 2 of the other parties in the conference of the user corresponding to the user information.

[0108] The entry form output unit 142 transmits the entry form information acquired by the entry form acquisition unit 131 to the user's terminal device 2. It is preferable that the entry form output unit 142 transmits the entry form information to the terminal device 2 of the user who is entering the conference room when the user enters the conference room.

[0109] The conversation-time form output unit 143 transmits the conversation-time form information acquired by the conversation-time form acquisition unit 133 to the terminal device 2. It is preferable that the conversation-time form output unit 143 transmits the conversation-time form information to the terminal device 2 at the start of a conference, but it may also transmit the conversation-time form information to the terminal device 2 when entering a conference or during the conference.

[0110] The question output unit 144 transmits the set of questions acquired by the question acquisition unit 134 to one user's terminal device 2. It is preferable for the question output unit 144 to transmit the set of questions to the terminal device 2 at the start of a conference, but the question output unit 144 may transmit the set of questions to the terminal device 2 when entering the conference or during the conference.

[0111] Various types of information are stored in the terminal storage unit 21 constituting the terminal device 2. The various types of information are, for example, a user identifier. Note that the user identifier in the terminal storage unit 21 may be information input by a user and temporarily stored.

[0112] The terminal reception unit 22 receives various information, instructions, etc. Examples of the various information, instructions, etc. include input information, user voice information, and user video.

[0113] The means for inputting various information and instructions may be anything, such as a touch panel, a keyboard, a mouse, a menu screen, a camera, a microphone, etc.

[0114] The terminal processing unit 23 performs various types of processing. For example, the various types of processing are processing for converting received information, instructions, etc. into information or instructions with a structure to be transmitted. For example, the various types of processing are processing for converting received information into information with a structure to be output.

[0115] The terminal transmission unit 24 transmits various types of information, instructions, and the like to the information processing device 1. The various types of information, instructions, and the like are, for example, input information, user voice information, and user video. The terminal transmission unit 24 transmits the user voice information and user video to the conference support device 3, for example. The user voice information and user video are information acquired in an online conference.

[0116] The terminal receiving unit 25 receives various types of information, such as user information, various types of form information, or a set of questions, as well as audio information and video.

[0117] The terminal output unit 26 outputs various types of information, such as various forms, audio information, and video.

[0118] The storage unit 11, conversation storage unit 111, entry form management unit 112, conversation form management unit 113, question management unit 114, and terminal storage unit 21 are preferably non-volatile recording media, but can also be realized with volatile recording media.

[0119] There is no restriction on the process by which information is stored in the storage unit 11, etc. For example, information may be stored in the storage unit 11, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 11, etc., or information inputted via an input device may be stored in the storage unit 11, etc.

[0120] It is preferable that the reception unit 12 and the information reception unit 121 are realized by wireless or wired communication means, but they may also be realized by a means for receiving broadcasts, a device driver for an input means such as a touch panel or keyboard, or control software for a menu screen.

[0121] The processing unit 13, the entry form acquisition unit 131, the search unit 132, the conversation form acquisition unit 133, the question acquisition unit 134, the information acquisition unit 135, and the storage unit 136 can usually be realized by a processor, a memory, etc. The processing procedure of the processing unit 13, etc. is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may also be realized by hardware (dedicated circuit). The processor may be a CPU, MPU, GPU, etc., and the type is not important.

[0122] The output unit 14, the user information output unit 141, the entry form output unit 142, the conversation form output unit 143, and the question output unit 144 are usually realized by wireless or wired communication means, but may be realized by broadcasting means. The output unit 14, etc. may be realized by driver software for an output device, or driver software for an output device and an output device, etc.

[0123] The terminal transmitting unit 24 is usually realized by a wireless or wired communication means, but may also be realized by a broadcasting means.

[0124] The terminal receiving unit 25 is usually realized by a wireless or wired communication means, but may be realized by a means for receiving a broadcast.

[0125] The terminal output unit 26 may be considered to include, or may not include, an output device such as a display, a speaker, etc. The terminal output unit 26 may be realized by driver software for an output device, or a combination of driver software for an output device and an output device, etc.

[0126] Next, an example of the operation of the information processing device 1 will be described with reference to the flowcharts of FIGS.

[0127] (Step S401) The reception unit 12 judges whether or not an entry instruction has been received. If an entry instruction has been received, the process proceeds to step S402, and if an entry instruction has not been received, the process proceeds to step S405. Note that an entry instruction is usually associated with a user identifier.

[0128] (Step S402) The entry form acquisition unit 131 performs entry form acquisition processing. An example of the entry form acquisition processing will be described with reference to the flowchart of FIG.

[0129] (Step S403) The processing unit 13 judges whether or not the entry form information has been acquired in step S402. If the entry form information has been acquired, the process proceeds to step S404, and if the entry form information has not been acquired, the process returns to step S401.

[0130] (Step S404) The entry form output unit 142 outputs one or more pieces of entry form information acquired in step S402. Return to step S401. Note that the output here is, for example, a transmission to the terminal device 2 of the user identified by the user identifier associated with the entry instruction.

[0131] (Step S405) The reception unit 12 judges whether or not a start instruction has been received. If a start instruction has been received, the process proceeds to step S406, and if a start instruction has not been received, the process proceeds to step S418. Note that the start instruction is associated with, for example, a conference identifier for identifying the conference, or two or more user identifiers.

[0132] (Step S406) The conversation-time form acquisition unit 133 assigns 1 to a counter i.

[0133] (Step S407) The conversation time form acquisition unit 133 determines whether or not there is an i-th user participating in the conference. If there is an i-th user, the process proceeds to step S408, and if there is no i-th user, the process returns to step S401. Note that it is possible to acquire the user identifiers of the two or more users participating in the conference. The user identifiers of the two or more users participating in the conference are stored in the storage unit 11, for example.

[0134] (Step S408) The conversation-time form acquisition unit 133 performs a conversation-time form acquisition process for presenting the form to the i-th user. An example of the conversation-time form acquisition process will be described with reference to the flowchart of FIG.

[0135] (Step S409) The processing unit 13 judges whether or not the conversation-time form information has been acquired in step S408. If the conversation-time form information has been acquired, the process proceeds to step S410, and if the conversation-time form information has not been acquired, the process proceeds to step S411.

[0136] (Step S410) The conversation-time form output unit 143 outputs one or more pieces of conversation-time form information acquired in step S409. Note that the output here is, for example, a transmission to the terminal device 2 of the user identified by the user identifier associated with the start instruction.

[0137] (Step S411) The processing unit 13 judges whether or not to output the user information of the other users. If the user information of the other users is to be output, the process proceeds to step S412, and if not, the process proceeds to step S414. Note that the output here means, for example, transmitting the user information of the other users to the terminal device 2 of the i-th user.

[0138] (Step S412) The processing unit 13 acquires one or more pieces of user information to be output.

[0139] (Step S413) The user information output unit 141 outputs one or more pieces of user information acquired in step S412. Note that the output here is, for example, a transmission to the terminal device 2 of the i-th user.

[0140] (Step S414) The question acquiring unit 134 performs a process of acquiring a set of questions to be output to the i-th user. An example of the process of acquiring a set of questions will be described with reference to the flowchart of FIG.

[0141] (Step S415) The processing unit 13 judges whether or not the set of questions has been acquired in step S414. If the set of questions has been acquired, the process proceeds to step S416, and if the set of questions has not been acquired, the process returns to step S407.

[0142] (Step S416) The question output unit 144 outputs the set of questions acquired in step S414. Note that the output here is, for example, transmission to the terminal device 2 of the i-th user.

[0143] (Step S417) The conversation-time form acquisition unit 133 increments the counter i by 1. The process returns to step S407.

[0144] (Step S418) The processing unit 13 judges whether or not conversation information has been acquired. If conversation information has been acquired, the process proceeds to step S419, and if conversation information has not been acquired, the process proceeds to step S421.

[0145] (Step S419) The processing unit 13 accumulates the conversation information acquired in step S418 in the conversation storage unit 111.

[0146] (Step S420) The processing unit 13 performs an analysis process on the conversation information acquired in step S418. An example of the analysis process will be described with reference to the flowchart of FIG.

[0147] (Step S421) The reception unit 12 judges whether or not an instruction to leave the room has been received. If an instruction to leave the room has been received, the process proceeds to step S422. If an instruction to leave the room has not been received, the process proceeds to step S425. Note that an instruction to leave the room is usually associated with a user identifier.

[0148] (Step S422) The processing unit 13 performs a room exit form acquisition process. An example of the room exit form acquisition process will be described with reference to the flowchart of FIG.

[0149] (Step S423) The processing unit 13 judges whether or not the exit form information has been acquired in step S422. If the exit form information has been acquired, the process proceeds to step S424, and if the exit form information has not been acquired, the process returns to step S401.

[0150] (Step S424) The output unit 14 outputs one or more pieces of exit form information acquired in step S422. Return to step S401. Note that the output here is, for example, a transmission to the terminal device 2 of the user identified by the user identifier associated with the exit instruction.

[0151] (Step S425) The information receiving unit 121 judges whether or not the entry input information has been received. If the entry input information has been received, the process proceeds to step S426. If the entry input information has not been received, the process proceeds to step S432. The entry input information is usually associated with a user identifier.

[0152] (Step S426) The accumulation unit 136 temporarily accumulates in the storage unit 11 the entry input information accepted in step S425 in association with the user identifier.

[0153] (Step S427) The search unit 132 acquires a user identifier included in the entry input information. The user identifier is, for example, an email address.

[0154] (Step S428) The search unit 132 searches the external device 4 using the user identifier acquired in step S427 as a key, and acquires user information.

[0155] (Step S429) The processing unit 13 associates the information including the user information acquired in step S428 with the user identifier and accumulates it in the storage unit 11. Note that such accumulated information is also user information. The accumulated information includes, in addition to the searched user information, for example, information included in the entry input information.

[0156] (Step S430) The processing unit 13 judges whether or not to output the user information accumulated in step S429 to the terminal device 2 of one or more users other than the relevant user. If it is to be output, the process proceeds to step S431, and if it is not to be output, the process returns to step S401.

[0157] (Step S431) The user information output unit 141 outputs the user information accumulated in step S429 to one or more user terminal devices 2. Return to step S401. Note that the output here is, for example, transmission.

[0158] (Step S432) The information receiving unit 121 judges whether or not the conference-time input information has been received. If the conference-time input information has been received, the process proceeds to step S433, and if the conference-time input information has not been received, the process proceeds to step S434. Note that the conference-time input information is usually associated with a user identifier.

[0159] (Step S433) The accumulation unit 136 associates the conference-time input information accepted in step S432 with the user identifier and temporarily accumulates it in the storage unit 11. The process returns to step S401.

[0160] (Step S434) The information receiving unit 121 judges whether or not the information input at the time of leaving has been received. If the information input at the time of leaving has been received, the process proceeds to step S435. If the information input at the time of leaving has not been received, the process proceeds to step S436. The information input at the time of leaving is usually associated with a user identifier.

[0161] (Step S435) The accumulation unit 136 associates the room leaving input information accepted in step S434 with the user identifier and temporarily accumulates it in the storage unit 11. The process returns to step S401.

[0162] (Step S436) The information acquisition unit 135 judges whether it is time to acquire the processed information. If it is time to acquire the processed information, the process proceeds to step S437, and if it is not time to acquire the processed information, the process returns to step S401. The timing to acquire the processed information is, for example, when an instruction to end the conference is accepted, or when an instruction to leave the room is accepted.

[0163] (Step S437) The information acquiring unit 135 assigns 1 to the counter i.

[0164] (Step S438) The information acquisition unit 135 judges whether or not the i-th user who is a participant of the conference exists. If the i-th user exists, the process proceeds to step S439, and if the i-th user does not exist, the process returns to step S401.

[0165] For example, the information acquisition unit 135 refers to the storage unit 11 and determines whether or not the i-th user identifier is present among two or more user identifiers of the participants of the conference.

[0166] (Step S439) The information acquiring unit 135 assigns 1 to a counter j.

[0167] (Step S440) The information acquisition unit 135 judges whether or not to acquire the j-th processing information corresponding to the user identifier of the i-th user. If the j-th processing information is to be acquired, the process proceeds to step S441. If the j-th processing information is not to be acquired, the process proceeds to step S445.

[0168] (Step S441) The information acquisition unit 135 performs processing information acquisition processing. An example of the processing information acquisition processing will be described with reference to the flowchart of FIG.

[0169] (Step S442) The accumulation unit 136 acquires a storage destination identifier corresponding to the j-th processing information from the storage unit 11. Here, the accumulation unit 136 acquires, for example, an API. The API specifies the storage destination.

[0170] (Step S443) The storage unit 136 stores the processed information acquired in step S441 in the storage destination (eg, the external device 4) indicated by the storage destination identifier acquired in step S442 in association with the user identifier of the i-th user.

[0171] (Step S444) The information acquiring unit 135 increments the counter j by 1. The process returns to step S440.

[0172] (Step S445) The information acquiring unit 135 increments the counter i by 1. The process returns to step S438.

[0173] In the flowcharts of FIG. 4 and FIG. 5, the process ends when the power is turned off or an interrupt is issued to end the process.

[0174] Next, an example of the entry form acquisition process in step S402 will be described with reference to the flowchart in FIG.

[0175] (Step S601) The entry form acquiring unit 131 acquires a user identifier associated with the accepted entry instruction.

[0176] (Step S602) The entry form acquisition unit 131 acquires from the storage unit 11 one or more user attribute values ​​that are paired with the user identifier acquired in step S601.

[0177] (Step S603) The entry form acquisition unit 131 acquires, from the storage unit 11, one or more conversation attribute values ​​corresponding to the conference.

[0178] (Step S604) The entry form acquisition unit 131 assigns 1 to a counter i.

[0179] (Step S605) The entry form acquisition unit 131 judges whether or not the i-th entry condition exists in the entry form management unit 112. If the i-th entry condition exists, the process proceeds to step S606, and if not, the process returns to the upper process.

[0180] (Step S606) The entry form acquisition unit 131 acquires the i-th entry condition from the entry form management unit 112.

[0181] (Step S607) The entry form acquisition unit 131 acquires information to be used for judging the i-th entry condition. The entry form acquisition unit 131 judges whether the i-th entry condition is met. If the i-th entry condition is met, the process proceeds to step S608; if the i-th entry condition is not met, the process proceeds to step S609. The information used for judging the i-th entry condition is usually one or more pieces of information among one or more user attribute values ​​and one or more conversation attribute values.

[0182] (Step S608) The entry form acquisition unit 131 acquires, from the entry form management unit 112, entry form information paired with the i-th entry condition.

[0183] (Step S609) The entry form acquisition unit 131 increments the counter i by 1. The process returns to step S605.

[0184] Next, an example of the conversation-time form acquisition process in step S408 will be described with reference to the flowchart in FIG.

[0185] (Step S701) The conversation time form acquisition unit 133 acquires the user identifier of the i-th user.

[0186] (Step S702) The conversation time form acquisition unit 133 acquires from the storage unit 11 one or more user attribute values ​​that are paired with the user identifier acquired in step S701.

[0187] (Step S703) The conversation time form acquisition unit 133 acquires, from the storage unit 11, one or more conversation attribute values ​​corresponding to the conference.

[0188] (Step S704) The conversation-time form acquisition unit 133 assigns 1 to a counter i.

[0189] (Step S705) The conversation-time form acquisition unit 133 judges whether or not the i-th meeting-time condition exists in the conversation-time form management unit 113. If the i-th meeting-time condition exists, the process proceeds to step S706, and if not, the process returns to the upper level process.

[0190] (Step S 706 ) The conversation-time form acquisition unit 133 acquires the i-th meeting-time condition from the conversation-time form management unit 113 .

[0191] (Step S707) Conversation-time form acquisition unit 133 acquires information used to determine the i-th meeting-time condition. Conversation-time form acquisition unit 133 determines whether or not the i-th meeting-time condition is satisfied. If the i-th meeting-time condition is satisfied, the process proceeds to step S708, and if the i-th meeting-time condition is not satisfied, the process proceeds to step S709. Note that the information used to determine the i-th meeting-time condition is usually one or more pieces of information selected from one or more user attribute values ​​and one or more conversation attribute values.

[0192] (Step S708) The conversation-time form acquisition unit 133 acquires, from the conversation-time form management unit 113, meeting-time form information paired with the i-th meeting-time condition.

[0193] (Step S709) The conversation-time form acquisition unit 133 increments the counter i by 1. The process returns to step S705.

[0194] Next, an example of the question set acquisition process in step S414 will be described with reference to the flowchart in FIG.

[0195] (Step S801) The question acquiring unit 134 acquires the user identifier of the i-th user.

[0196] (Step S802) The question acquiring unit 134 acquires from the storage unit 11 one or more user attribute values ​​that are paired with the user identifier acquired in step S801.

[0197] (Step S803) The question acquiring unit 134 acquires from the storage unit 11 one or more conversation attribute values ​​corresponding to the conference.

[0198] (Step S804) The question acquiring unit 134 assigns 1 to a counter i.

[0199] (Step S805) The question acquiring unit 134 judges whether or not the i-th question condition exists in the question managing unit 114. If the i-th question condition exists, the process proceeds to step S806, and if not, the process returns to the upper process.

[0200] (Step S806) The question acquiring unit 134 acquires the i-th question condition from the question managing unit 114.

[0201] (Step S807) The question acquisition unit 134 acquires information to be used for judging the i-th question condition. The question acquisition unit 134 judges whether the i-th question condition is satisfied. If the i-th question condition is satisfied, the process proceeds to step S808, and if the i-th question condition is not satisfied, the process proceeds to step S809. Note that the information to be used for judging the i-th question condition is usually one or more pieces of information selected from one or more user attribute values ​​and one or more conversation attribute values.

[0202] (Step S808) The question acquiring unit 134 acquires, from the question managing unit 114, a set of questions paired with the i-th question condition.

[0203] (Step S809) The question acquiring unit 134 increments the counter i by 1. The process returns to step S805.

[0204] Next, an example of the analysis process in step S420 will be described with reference to the flowchart in FIG.

[0205] (Step S901) The processing unit 13 acquires voice information from the acquired conversation information.

[0206] (Step S902) The processing unit 13 performs a voice division process on the voice information acquired in step S901. An example of the voice division process will be described with reference to the flowchart of Fig. 10. Note that the voice division process is a process of dividing the voice information into two or more partial voices.

[0207] (Step S903) The processing unit 13 assigns 1 to a counter i.

[0208] (Step S904) The processing unit 13 judges whether or not the i-th partial character string exists. If the i-th partial character string exists, the process proceeds to step S905, and if the i-th partial character string does not exist, the process returns to the upper process.

[0209] (Step S905) The processing unit 13 provides the i-th partial character string to a module that determines a class (here, a question identifier), executes the module, and acquires the question identifier. Note that there are cases where the question identifier cannot be acquired at this stage. Note that the question identifier is usually information that identifies question information, but it may also be information that identifies a field of a form.

[0210] (Step S906) The processing unit 13 judges whether or not the question identifier was acquired in step S905. If the question identifier was acquired, the process proceeds to step S907, and if the question identifier was not acquired, the process proceeds to step S911.

[0211] (Step S907) The processing unit 13 acquires the (i+1)th partial character string.

[0212] (Step S908) The processing unit 13 acquires answer information corresponding to the partial character string acquired in step S907. The processing unit 13 provides the partial character string to a generative AI such as ChatGPT (registered trademark), and acquires answer information corresponding to the partial character string from the generative AI. Note that there are cases where the answer information cannot be acquired here. Note that the answer information here is usually an answer to a question, but may be information entered into a field identified by a question identifier.

[0213] (Step S909) The processing unit 13 judges whether or not the answer information has been acquired in step S908. If the answer information has been acquired, the process proceeds to step S910, and if the answer information has not been acquired, the process proceeds to step S911.

[0214] (Step S910) The processing unit 13 stores the answer information acquired in step S908 in association with the question identifier.

[0215] (Step S911) The processing unit 13 increments the counter i by 1. The process returns to step S94.

[0216] Next, an example of the audio division process in step S902 will be described with reference to the flowchart in FIG.

[0217] (Step S1001) The processing unit 13 assigns 1 to a counter i.

[0218] (Step S1002) The processing unit 13 judges whether or not the i-th unit section exists in the acquired voice information. If the i-th unit section exists, the processing unit 13 proceeds to step S1003. If the i-th unit section does not exist, the processing unit 13 returns to the upper processing.

[0219] The unit section is a section for detecting a break in the speech information and is a section for acquiring one or more speech features. The i-th unit section is, for example, a section of a predetermined length, such as from one silent section to the next silent section.

[0220] (Step S1003) The processing unit 13 acquires one or more speech features from the speech information of the i-th unit section. Note that the speech features may be, for example, Mel spectrum, MFCC, speech rate, or the like, but are not limited thereto.

[0221] (Step S1004) The processing unit 13 uses the speech feature acquired in step S1003 to determine whether the i-th unit section is a division. If it is a division, the process proceeds to step S1005. If it is not a division, the process proceeds to step S1009. Note that a division here is usually when the user of the speech is changed.

[0222] (Step S1005) The processing unit 13 adds a delimiter tag indicating a delimiter to the location of the i-th unit section in the audio information.

[0223] The location of the i-th unit interval is, for example, immediately before the i-th unit interval, immediately after the i-th unit interval, or within the i-th unit interval.

[0224] The processing unit 13 may also perform processing other than adding a delimiter tag, such as processing to clearly indicate a delimiter. The processing to clearly indicate a delimiter is, for example, processing to accumulate location information that identifies the location of the delimiter in a buffer (not shown).

[0225] (Step S1006) The processing unit 13 acquires a partial voice, which is a series of voice information from the unit section next to the unit section of the previous segment to the i-th unit section.

[0226] (Step S1007) The processing unit 13 acquires a partial character string that is a speech recognition result for the partial speech acquired in step S1006, and associates the partial character string with the partial speech.

[0227] The processing unit 13 performs speech recognition processing on the partial speech acquired in step S1005, for example, to acquire a partial character string. The processing unit 13 transmits the partial speech acquired in step S1005 to a speech recognition server (not shown), and receives the partial character string from the speech recognition server.

[0228] (Step S1008) The processing unit 13 acquires the user identifier of the user who has spoken the partial voice, and associates the user identifier with the partial character string and the partial voice. The user identifier is, for example, a telephone number. The method for acquiring the user identifier is not important.

[0229] The processing unit 13 acquires, for example, a user identifier associated with the partial voice. The processing unit 13 acquires the user identifier by using, for example, the voice feature acquired in step S1003.

[0230] (Step S1009) The processing unit 13 increments the counter i by 1. The process returns to step S1002.

[0231] Next, an example of the exit form acquisition process in step S422 will be described with reference to the flowchart in FIG.

[0232] (Step S1101) The processing unit 13 acquires the user identifier of the i-th user.

[0233] (Step S1102) The processing unit 13 acquires from the storage unit 11 one or more user attribute values ​​that are paired with the user identifier acquired in step S1101.

[0234] (Step S1103) The processing unit 13 acquires from the storage unit 11 one or more conversation attribute values ​​corresponding to the conference.

[0235] (Step S1104) The processing unit 13 assigns 1 to the counter i.

[0236] (Step S1105) The processing unit 13 judges whether or not the i-th exit condition exists in the storage unit 11. If the i-th exit condition exists, the process proceeds to step S1106, and if not, the process returns to the upper process.

[0237] (Step S1106) The processing unit 13 acquires the i-th room exit condition from the storage unit 11.

[0238] (Step S1107) Processing unit 13 acquires information used to determine whether the i-th condition for leaving the room is satisfied. Processing unit 13 determines whether the i-th condition for leaving the room is satisfied. If the i-th condition for leaving the room is satisfied, processing proceeds to step S1108, and if the i-th condition for leaving the room is not satisfied, processing proceeds to step S1109. The information used to determine whether the i-th condition for leaving the room is normally one or more pieces of information among one or more user attribute values ​​and one or more conversation attribute values.

[0239] (Step S1108) The processing unit 13 acquires from the storage unit 11 the room exit form information that is paired with the i-th room exit condition.

[0240] (Step S1109) The processing unit 13 increments the counter i by 1. The process returns to step S1105.

[0241] An example of the processing information acquisition process in step S441 will be described with reference to the flowchart in FIG.

[0242] (Step S1201) The information acquiring unit 135 assigns 1 to a counter i.

[0243] (Step S1202) The information acquisition unit 135 judges whether or not there is the i-th element information to be acquired in order to configure the processing information. If the i-th element information exists, the process proceeds to step S1203, and if the i-th element information does not exist, the process returns to the upper process.

[0244] (Step S1203) The information acquisition unit 135 determines whether the i-th element information constituting the processing information and acquired from the input information has been stored. If the i-th element information acquired from the input information has been stored, the process proceeds to step S1204, and if not, the process proceeds to step S1205.

[0245] (Step S1204) The information acquiring unit 135 acquires the i-th element information acquired from the input information and stored as the final i-th element information. Then, the process proceeds to step S1207.

[0246] (Step S1205) The information acquisition unit 135 determines whether the i-th element information constituting the processed information and acquired from the conversation information has been stored. If the i-th element information acquired from the conversation information has been stored, the process proceeds to step S1206, and if not, the process proceeds to step S1207.

[0247] (Step S1206) The information acquiring unit 135 acquires the i-th element information acquired from the conversation information as the final i-th element information.

[0248] (Step S1207) The information acquiring unit 135 increments the counter i by 1. The process returns to step S1202.

[0249] In the flowchart of FIG. 12, it is assumed that processing information has been configured when returning to the upper level process.

[0250] Next, an example of the operation of the terminal device 2 will be described with reference to the flowchart of FIG.

[0251] (Step S1301) The terminal reception unit 22 judges whether or not an entry instruction has been received. If an entry instruction has been received, the process proceeds to step S1302, and if an entry instruction has not been received, the process proceeds to step S1303.

[0252] (Step S1302) The terminal processing unit 23 configures an entry instruction to be transmitted, which is an entry instruction associated with the user identifier. The terminal transmitting unit 24 transmits the entry instruction associated with the user identifier to the conference supporting device 3, or the information processing device 1, or the information processing device 1 and the conference supporting device 3. Return to step S1301.

[0253] (Step S1303) The terminal receiving unit 25 judges whether or not audio information, etc., in the online conference has been received. If audio information, etc., has been received, the process proceeds to step S1304, and if audio information, etc., has not been received, the process proceeds to step S1305. Note that audio information, etc., is usually audio information and video.

[0254] (Step S1304) The terminal processing unit 23 composes voice information etc. to be output. The terminal output unit 26 outputs the voice information etc. Return to step S1301.

[0255] (Step S1305) The terminal receiving unit 25 judges whether or not form information has been received from the information processing device 1. If form information has been received, the process proceeds to step S1305, and if not, the process proceeds to step S1306.

[0256] (Step S1306) The terminal processing unit 23 uses the received form information to compose a form to be output. The terminal output unit 26 outputs the form. Return to step S1301.

[0257] (Step S1307) Terminal receiving unit 25 judges whether or not a set of questions has been received. If a set of questions has been received, the process proceeds to step S1308, and if not, the process proceeds to step S1309.

[0258] (Step S1308) Device processing unit 23 uses the received question set to construct a question set to be output. Terminal output unit 26 outputs the question set. Return to step S1301.

[0259] (Step S1309) The terminal reception unit 22 judges whether or not an instruction to leave the room has been received. If an instruction to leave the room has been received, the process proceeds to step S1310, and if an instruction to leave the room has not been received, the process proceeds to step S1311.

[0260] (Step S1310) The terminal processing unit 23 configures a room leaving instruction to be transmitted, which is an instruction to leave associated with the user identifier. The terminal transmitting unit 24 transmits the room leaving instruction associated with the user identifier to the conference supporting device 3, or the information processing device 1, or the information processing device 1 and the conference supporting device 3. Return to step S1301.

[0261] (Step S1311) The terminal reception unit 22 judges whether or not input information for the output form has been received. If the input information has been received, the process proceeds to step S1312, and if the input information has not been received, the process proceeds to step S1313.

[0262] (Step S1312) The terminal processing unit 23 configures input information to be transmitted, which is input information associated with the user identifier. The terminal transmitting unit 24 transmits the input information associated with the user identifier to the information processing device 1. Return to step S1301.

[0263] (Step S1313) The terminal receiving unit 25 judges whether or not the user information of the other user of the online conference has been received. If the user information has been received, the process proceeds to step S1314, and if the user information has not been received, the process returns to step S1301. Note that the audio information and the like usually refers to audio information and video.

[0264] (Step S1314) The device processing unit 23 composes user information to be output. The device output unit 26 outputs the user information. The process returns to step S1301.

[0265] A specific example of the operation of the information system A in this embodiment will be described below.

[0266] The storage unit 11 of the information processing device 1 stores a user management table shown in FIG. 14. The user management table is a table that manages user management information. The user management table manages one or more records having an "ID", "user ID", "email address", "phone number", and "group identifier". "ID" is information that identifies a record. "Email address" or "phone number" are examples of a user identifier. "Group identifier" is the identifier of a group to which the user belongs.

[0267] The entry form management unit 112 stores an entry form management table having the structure shown in Fig. 15. The entry form management table is a table that manages entry form information. The entry form management table manages one or more records that have an "ID", "entry conditions", "form identifier", and "entry form information". The form identifier is information that identifies form information.

[0268] The record with "ID=1" in the entry form management table (FIG. 15) is an online conference whose title information includes "business negotiation," and indicates that entry form information identified by "F01" is sent to the terminal device 2 of a user who belongs to a group identified by a group identifier "G1," which is a user attribute value, upon entry to the conference. Also, the entry form information identified by "F01" indicates that the entry form information defines a form for inputting four pieces of information, "<first name><last name><telephone number><department name>" of the other party in the conference.

[0269] The entry condition for the record with "ID=2" in the entry form management table (Figure 15) is "group identifier=G2", indicating that the entry form information identified by "F02" is sent to the terminal device 2 of the user belonging to "group identifier=G2".

[0270] The entry condition for the record with "ID=3" in the entry form management table (Figure 15) is "User identifier = 'U301, U302'", which indicates that the entry form information identified by "F03" is sent to the terminal device 2 of the user identified by the "user identifier of U301 or U302."

[0271] The conversation-time form management unit 113 stores a conversation-time form management table having the structure shown in FIG. 16. The conversation-time form management table (FIG. 16) is a table that manages conversation-time form information. The conversation-time form management table manages one or more records having an "ID," "conference-time conditions," "form identifier," and "conference-time form information." The conference-time form information is written in HTML here. A file indicated by the conference-time form information is stored, for example, in the storage unit 11.

[0272] The question management unit 114 stores a question management table having a structure shown in FIG. 17. The question management table is a table for managing a set of questions. The question management table (FIG. 17) manages one or more records having an "ID", "question condition", "form identifier", "question set", and "input UI". Here, the "question set" is a character string constituting the question set. The "input UI" is an identifier of a user interface for inputting answer information to question information contained in the question set. The "input UI" is, for example, a check box or a field. A check box is a UI that the user can check, and is, for example, a UI into which "1" or "0" is input. A field is a UI into which the user can input any character string. When the "input UI" is "-" (NULL), only the question set is output to the terminal device 2, and answer information corresponding to the question set is not input.

[0273] Furthermore, the storage unit 11 stores an accumulation management table having a structure shown in FIG. 18. The accumulation management table (FIG. 18) is a table for managing information required when accumulating the processing information in the external device 4 or the information processing device 1. The accumulation management table (FIG. 18) is a table for managing one or more pieces of accumulation control information. The accumulation management table (FIG. 18) has one or more records having "ID", "form identifier", "device identifier", "accumulation method", "API", and "attribute correspondence information". The "attribute correspondence information" has "processing information", "table identifier", and "attribute". The "form identifier" is an identifier of form information into which information that is the source of the processing information is input. The "device identifier" is information for identifying the device in which the processing information is accumulated. The "API" is information of the API used when the processing information is accumulated. The "attribute correspondence information" is information indicating the correspondence between one or more pieces of element information contained in the processing information and the attributes of the table in which the processing information is accumulated. The "processing information" is information for identifying one or more pieces of element information that are contained in the processing information. The "processing information" is information for identifying one or more pieces of element information that are stored. The "table identifier" is an identifier of the table in which the processing information is accumulated, and is the table name in this case. "Attribute" is the attribute name of the storage destination table.

[0274] The record with "ID=1" in the storage management table (Fig. 18) indicates that the reception information output for the form identified by the form identifier "F01" is stored in external device A using the API "up_app()." The record with "ID=1" also indicates that the element information "first name" from the reception information is stored as the attribute value of the attribute "first name" in the table "responsible person", the element information "last name" from the reception information is stored as the attribute value of the attribute "last name" in the table "responsible person", the element information "phone number" from the reception information is stored as the attribute value of the attribute "phone" in the table "responsible person", and the element information "email address" from the reception information is stored as the attribute value of the attribute "address" in the table "responsible person".

[0275] The record of "ID=2" in the storage management table (FIG. 18) indicates that the reception information outputted for the form identified by the form identifier "F11" is stored in the external device B using the API "app()." The record of "ID=2" also indicates that the element information "business negotiation name" of the reception information is stored as the attribute value of the attribute "business negotiation" of the table "T1," the element information "client" of the reception information is stored as the attribute value of the attribute "client" of the table "T1" and the attribute value of the attribute "company name" of the table "T2," the element information "type" of the reception information is stored as the attribute value of the attribute "type" of the table "T1," the element information "lead source" of the reception information is stored as the attribute value of the attribute "lead source" of the table "T1," and the element information "number of salespeople" of the reception information is stored as the attribute value of the attribute "number of salespeople" of the table "T1" and the attribute value of the attribute "number of staff" of the table "T2."

[0276] In the above situation, the following specific example will be described.

[0277] Now, assume that user X with user identifier "U001" and user Y with user identifier "U102" are holding a web conference. The title information of the web conference is "Business Negotiation with ABC Company", and the organizer identifier of the web conference is "U001". The title information, organizer identifier "U001", and participant user identifier "U102" are stored in storage unit 11 of information processing device 1 as a pair with conference identifier "M01". Also assume that the transmission condition stored in storage unit 11 for transmitting user information is "transmit user information of a person other than the organizer to terminal device 2 of the organizer identified by the organizer identifier".

[0278] It is assumed that user X then inputs an instruction to enter the web conference into his / her terminal device 2. Next, the terminal receiving unit 22 of the terminal device 2 receives the instruction, and the terminal transmitting unit 24 transmits the instruction to enter the web conference having the user identifier "U001" to the conference support device 3.

[0279] Also, assume that user Y inputs an instruction to enter the web conference into his / her terminal device 2. Next, the terminal receiving unit 22 of the terminal device 2 receives the entry instruction, and the terminal transmitting unit 24 transmits the entry instruction having the user identifier "U102" to the conference support device 3.

[0280] Next, the conference supporting device 3 receives an entry instruction having a user identifier "U001" and an entry instruction having a user identifier "U102" from each user's terminal device 2. Then, the conference supporting device 3 performs processing to start an online conference of the two users identified by the user identifiers "U001" and "U102".

[0281] Furthermore, the reception unit 12 of the information processing device 1 acquires an entry instruction having the conference identifier "M01" and the user identifiers "U001" and "U102" from the conference supporting device 3. Furthermore, the reception unit 12 acquires an entry instruction having the conference identifier "M01" and the user identifier "U102" from the conference supporting device 3.

[0282] Next, the entry form acquisition unit 131 acquires the group identifier "G1" corresponding to the entry instruction having the user identifier "U001" from the user management table (FIG. 14). Next, the entry form acquisition unit 131 acquires the title information "Business negotiation with ABC company" of the Web conference paired with the conference identifier "M01" of the entry instruction from the storage unit 11. Then, the entry form acquisition unit 131 determines that the title information "Business negotiation with ABC company" and the group identifier "G1" match the "ID=1" entry condition. Next, the entry form acquisition unit 131 acquires the form identifier "F01" and the entry form information of "ID=1" in FIG. 15. Next, the entry form acquisition unit 131 uses the acquired entry form information to configure the entry form information to be sent. The entry form information to be sent is information acquired from the entry form information of "ID=1" in Fig. 15, and is a form for inputting <first name><last name><telephone number><email address>. Next, the entry form output unit 142 sends the acquired entry form information to the terminal device 2 of user X.

[0283] Also, the entry form acquisition unit 131 acquires the group identifier "G2" corresponding to the entry instruction having the user identifier "U102" from the user management table (FIG. 14). Next, the entry form acquisition unit 131 determines that the group identifier "G2" matches the entry condition of "ID=2". Next, the entry form acquisition unit 131 acquires the form identifier "F02" and the entry form information of "ID=2". Next, the entry form acquisition unit 131 uses the acquired entry form information to configure the entry form information to be sent. Note that the entry form information to be sent is information acquired from the entry form information of "ID=2" in FIG. 15, and is a form for inputting <first name><last name><email address>. Next, the entry form output unit 142 transmits the acquired entry form information to the terminal device 2 of the user Y.

[0284] Then, the terminal device 2 of the user X receives the form information from the information processing device 1 and outputs the form of FIG.

[0285] Next, it is assumed that user X inputs <first name><last name><phone number><email address>. Next, it is assumed that the terminal device 2 of user X accepts "<first name> Tanaka <last name> A-san <phone number> 090-9876-5432 <email address> ab@x.com". Then, the terminal device 2 transmits the entry input information "<first name> Tanaka <last name> A-san <phone number> 090-9876-5432 <email address> ab@x.com" to the information processing device 1.

[0286] Next, the information receiving unit 121 of the information processing device 1 receives the entry input information from the terminal device 2 of the user X. Next, the accumulation unit 136 temporarily accumulates the entry input information in the storage unit 11 in association with the user identifier "U001."

[0287] Next, the search unit 132 acquires the user identifier "ab@x.com" contained in the entry input information. It is assumed here that the user identifier used in the search is an email address. The search unit 132 searches the external device 4 using the email address "ab@x.com" as a key, acquires the user information "<Company Name> X Co., Ltd. <Department> Sales Department <Position> Manager", and temporarily stores the user information in the storage unit 11 in association with the user identifier "U001".

[0288] Next, the processing unit 13 determines not to transmit the accumulated user information to the terminal device 2 of one or more users other than the user. Note that, based on the transmission condition "transmit user information of a person other than the organizer to the terminal device 2 of the organizer identified by the organizer identifier" and the organizer identifier "U001", the processing unit 13 determines not to transmit the user information of the user identified by the user identifier "U001" to other users.

[0289] Furthermore, the terminal device 2 of the user Y receives form information for inputting <first name>, <last name> and <email address> from the information processing device 1, and outputs it.

[0290] Next, it is assumed that user Y inputs <first name><last name><email address>. Next, it is assumed that the terminal device 2 of user Y accepts "<first name>Yamamoto <last name>Bako <email address>yb@y.co.jp". Then, the terminal device 2 transmits the entry input information "<first name>Yamamoto <last name>Bako <email address>yb@y.co.jp" to the information processing device 1.

[0291] Next, the information receiving unit 121 of the information processing device 1 receives the entry input information from the terminal device 2 of the user Y. Next, the accumulation unit 136 temporarily accumulates the entry input information in the storage unit 11 in association with the user identifier "U102."

[0292] Next, the search unit 132 acquires the user identifier "yb@y.co.jp" contained in the entry input information. It is assumed here that the user identifier used in the search is an email address. The search unit 132 searches the external device 4 using the email address "yb@y.co.jp" as a key, acquires the user information "<Company name> Y Co., Ltd. <Department> Business Department <Position> Manager", and temporarily stores the user information in the storage unit 11 in association with the user identifier "U102".

[0293] Next, the processing unit 13 determines to transmit the accumulated user information to the terminal device 2 of user X. Based on the transmission condition "transmit user information of a person other than the organizer to the terminal device 2 of the organizer identified by the organizer identifier" and the organizer identifier "U001", the processing unit 13 transmits the user information of the user identified by the user identifier "U102" to the terminal device 2 corresponding to the organizer identifier "U001".

[0294] Next, the terminal receiving unit 25 receives the user information of the user at the other end of the web conference, "<First name> Yamamoto <Last name> Bko <Email address> yb@y.co.jp <Company name> Y Co., Ltd. <Affiliation> Business Department <Position> Manager." Next, the terminal processing unit 23 composes the user information to be output. The terminal output unit 26 outputs the user information. Note that the user information may be output in any manner. It is preferable that the user information be output in association with an image of the user corresponding to the user information. An example of such an output is 2001 in FIG. 20.

[0295] Next, it is assumed that the conference support device 3 starts a web conference between user X and user Y. Then, the reception unit 12 of the information processing device 1 receives from the information processing device 1 a start instruction indicating that user X and user Y have started a web conference, the start instruction having the user identifiers "U001" and "U102."

[0296] Next, the conversation-time form acquisition unit 133 refers to the conversation-time form management table (FIG. 16) for user X and acquires conversation-time form information as follows. That is, the conversation-time form acquisition unit 133 determines that the user identifier "U001" of user X matches the conference-time condition of "ID=1", and acquires the form identifier "F11" and conversation-time form information "CF01.html" from the conversation-time form management table. Next, the conversation-time form output unit 143 acquires a file identified by "CF01.html" from the storage unit 11, and transmits the file (form information) to the terminal device 2 of user X.

[0297] Next, the terminal device 2 receives and outputs the form information. As a result, a conversation-time form is output to the terminal device 2 of the user X. An example of such output is 2002 in Fig. 20. It is assumed here that the terminal device 2 outputs the conversation-time form in addition to the video of the conference participants.

[0298] Furthermore, the question acquisition unit 134 refers to the question management table (FIG. 17) for the user X and acquires a set of questions as follows. That is, first, the question acquisition unit 134 acquires the user identifier "U001" of the user X. Furthermore, the question acquisition unit 134 acquires the group identifier "G01" that is paired with the user identifier "U001" from the user management table (FIG. 14). Furthermore, the question acquisition unit 134 acquires the title information "business negotiation with ABC company" that is a conversation attribute value corresponding to the conference from the storage unit 11. Next, the question acquisition unit 134 determines that "business negotiation with ABC company" and the group identifier "G01" match the question condition of "ID=1" in the question management table. Next, the question acquisition unit 134 acquires the question set and the input UI of "ID=1" in the question management table (FIG. 17), and configures form information to be sent to the user X using the question set and the input UI. Next, the question output unit 144 transmits the form information including the set of questions to the terminal device 2 of the user X.

[0299] Next, the terminal device 2 receives and outputs the form information. As a result, a set of questions is output to the terminal device 2 of user X. An example of such output is 2003 in FIG. 20. It is assumed here that the terminal device 2 outputs the set of questions in addition to the video of the conference participants and the conversation form. In other words, the terminal device 2 of user X also outputs his own video 2004 and a video 2005 of user Y, who is the other party in the web conference.

[0300] User X fills in the information that can be filled in the conversation form during the web conference while looking at the screen in Fig. 20. User X also checks the checkboxes for the relevant questions in the question set during the web conference while looking at the screen in Fig. 20.

[0301] Also, the processing unit 13 of the information processing device 1 acquires conversation information from the conference support device 3 at all times or at an appropriate timing during the web conference. Next, the processing unit 13 accumulates the acquired conversation information in the conversation storage unit 111. Also, the processing unit 13 performs an analysis process on the acquired conversation information. Assume that the processing unit 13 detects a question that is a partial character string "How many salespeople are there in your company?" that corresponds to the user identifier "U001". Next, the processing unit 13 detects answer information that is a partial character string "42 people" that corresponds to the user identifier "U102", acquires the answer "42" to be output, and outputs the information "42" to be substituted for the field "number of salespeople" to the field. This field is 2006 in FIG. 20. As described above, the information to be input into the field is automatically obtained from the conversation information during the conference.

[0302] Also, during the web conference, user X obtains information from the conversation with user Y and fills in the fields and checkboxes (e.g., 2007, 2008, 2009).

[0303] Next, the conference ends, and the information acquiring unit 135 determines that it is time to acquire processed information. Then, the information acquiring unit 135 refers to the storage management table (FIG. 18) and acquires the information of user Y "<First name>Yamamoto <Last name>B <Email address>yb@y.co.jp" which is information acquired using the input form corresponding to the form identifier "F01", and also acquires the telephone number "090-1234-1222" which is paired with the user identifier "U102" of user Y, by referring to the user management table (FIG. 14). Then, the storage unit 136 composes the information "<First name>Yamamoto <Last name>B <Telephone number>090-1234-1222 <Email address>yb@y.co.jp" to be stored in the external device 4. Next, the storage unit 136 obtains the device identifier "External device A" that pairs with the form identifier "F01", the storage method "UPDATE OR APPEND", the API "up_app()", the table identifier "Customer information", and attributes, and uses the API "up_app()" to add a record to the table "Customer information" with "<First name>Yamamoto" as the attribute "First name", "<Last name>Biko" as the attribute "Last name", "<Phone number>090-1234-1222" as the attribute "Phone", and "<Email address>yb@y.co.jp" as the attribute "Address" as the attribute values.

[0304] The storage unit 136 also refers to the storage management table (FIG. 18) and stores the information acquired using the conversation time form corresponding to form identifier "F11", such as the negotiation name, client name, etc., entered into each field of 2002 in FIG. 20. Here, the storage unit 136 acquires the device identifier "external device B" of "ID=2" in FIG. 18, the storage method "APPEND", the API "app()", the table identifiers "T1" and "T2", and the attributes, and stores the negotiation name, client name, etc. using the API "app()".

[0305] Furthermore, the storage unit 136 refers to the storage management table (FIG. 18), and uses the answer information for the question set 2003 to obtain answer information corresponding to the checked questions, and stores the same in the external device 4.

[0306] According to the above specific example, User X can store the sales reports that he needs to leave after the end of a business negotiation with very little effort.

[0307] As described above, according to this embodiment, information based on an online conference can be acquired efficiently.

[0308] Furthermore, according to this embodiment, information based on input information at the time of entering an online conference can be efficiently acquired.

[0309] Furthermore, according to this embodiment, by presenting to the user information acquired using the input information at the time of entry into the online conference, an effective conference can be held.

[0310] Furthermore, according to this embodiment, it is possible to obtain entry input information that matches a user who is a participant of the conference.

[0311] Furthermore, according to this embodiment, it is possible to efficiently obtain information based on conversation input information, which is information obtained during an online conference.

[0312] Furthermore, according to this embodiment, a form for inputting conversation input information can be provided to the user.

[0313] Furthermore, according to this embodiment, question information for smoothly progressing the conference can be provided to the user.

[0314] The processing in this embodiment may be realized by software. This software may be distributed by software download or the like. This software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The software for realizing the information processing device 1 in this embodiment is a program as follows. That is, this program is a program for making a computer function as an information receiving unit that acquires reception information that is received at one or more points among three or more points before an online conference, during an online conference, or after an online conference, an information acquiring unit that acquires processed information that is information based on the reception information and is not conversation information of the voice uttered by the user during the online conference, and a storage unit that accumulates the processed information.

[0315] (Embodiment 2) In this embodiment, an information system is described that includes an information processing device that receives input information including an instruction sentence entered into an interactive application, determines an access destination corresponding to the input information from two or more access destinations including one or more external devices, obtains command information corresponding to the instruction sentence, and performs processing specified by the obtained command information on the determined access destination.

[0316] In the present embodiment, an information system will be described that includes an information processing device that receives result information, which is a result of processing, from an access destination and transmits the result information to a user's terminal device.

[0317] In addition, in this embodiment, we will describe an information system that includes an information processing device that constructs a query sentence using an instruction sentence, sends the query sentence to a generation AI, obtains an answer to the query sentence from the generation AI, and obtains command information using the answer.

[0318] In addition, in this embodiment, we will describe an information system that includes an information processing device that uses an instruction sentence to select one AI template information from two or more AI template information for constructing a query sentence, and uses the one AI template information to construct a query sentence to be given to a generating AI.

[0319] In addition, in this embodiment, an information system is described that includes an information processing device that passes screen information constituting a screen having a plurality of selection items to an interactive application, receives a selection of a selection item from a user, and acquires command information in accordance with the selection.

[0320] In addition, in this embodiment, an information system including an information processing device that performs processing to select a learning model to be used by the generation AI according to a user attribute value will be described.

[0321] In addition, in this embodiment, an information system including an information processing device that determines an access destination from a command statement will be described.

[0322] In addition, in this embodiment, an information system is described that includes an information processing device that obtains an answer to an instruction sentence entered after an online conference from the audio information of the conference, obtains command information using the answer, and stores the answer in a device corresponding to the access destination.

[0323] In addition, in this embodiment, an information system including an information processing device that obtains answers from a generation AI that has learned conversation information of online conferences will be described.

[0324] Furthermore, in this embodiment, we will describe an information system that includes an information processing device that sends screen information to a user's terminal device to confirm the information to be stored in a device corresponding to the access destination, and stores the answer depending on the user's selection or permission.

[0325] In this specification, selecting information Z and determining information Z mean obtaining information Z, obtaining a pointer to information Z, obtaining the ID of information Z, setting a flag for information Z, etc., and are sufficient as long as information Z can be accessed.

[0326] 21 is a conceptual diagram of an information system B in this embodiment. The information system B includes an information processing device 5, one or more terminal devices 6, a generation AI device 7, a meeting support device 3, and one or more external devices 4.

[0327] The information processing device 5 is a device that receives an instruction from a user and performs processing according to the instruction. The information processing device 5 may have a function of supporting a conference, similar to the information processing device 1. The information processing device 1 is, for example, a cloud server or an ASP server, but the type is not important.

[0328] The terminal device 6 is a terminal used by a user. The user is usually a person who inputs input information including an instruction text. The user is, for example, a participant of a conference. The terminal device 6 is, for example, a so-called personal computer, a smartphone, or a tablet terminal, but the type is not important.

[0329] The generation AI device 7 is a device having a generation AI function. Here, the generation AI device 7 usually has a text generation AI function. The generation AI is, for example, ChatGPT or Google Bard, but it does not matter. Google is a registered trademark.

[0330] Fig. 22 is a block diagram of an information system B in this embodiment. Fig. 23 is a block diagram of an information processing device 5.

[0331] The information processing device 5 includes a storage unit 51, a reception unit 52, a processing unit 53, and an output unit 54. The storage unit 51 includes a conversation storage unit 111, and a template storage unit 511. The reception unit 52 includes an information receiving unit 521, and a result receiving unit 522. The processing unit 53 includes a decision unit 531, and a command acquisition unit 532. The command acquisition unit 532 includes, for example, a type acquisition means 5321, a template acquisition means 5322, an attribute value acquisition means 5323, a text creation means 5324, a text transmission means 5325, an answer reception means 5326, a search means 5327, a screen acquisition means 5328, a screen output means 5329, a selection reception means 5330, a candidate acquisition means 5331, and a command acquisition means 5332. The output unit 54 includes a command output unit 541, and a result transmission unit 542.

[0332] The information processing device 5 may include a generating AI device 7. The information processing device 5 may include a conference support device 3. Furthermore, the information processing device 5 may include an external device 4.

[0333] The terminal device 6 includes a terminal storage unit 61 , a terminal reception unit 62 , a terminal processing unit 63 , a terminal transmission unit 64 , a terminal reception unit 65 , and a terminal output unit 66 .

[0334] Various types of information are stored in the storage unit 51 constituting the information processing device 5. The various types of information are, for example, conversation information, template information described later, one or more pieces of user management information, an access correspondence table described later, and a command correspondence table described later. The template information is, for example, AI template information described later, or command template information described later.

[0335] The user management information here is information about a user. The user here is not necessarily a user who participates in a conference. The user here is usually a user who inputs input information. The user management information has a user identifier and one or more user attribute values.

[0336] The template storage unit 511 stores one or more pieces of AI template information. It is preferable that each of the one or more pieces of AI template information corresponds to an access destination and a command type. The template storage unit 511 stores, for example, one or more pieces of command template information.

[0337] The AI ​​template information is information for creating a query sentence to be passed to the generation AI device 7. The AI ​​template information is usually information that serves as a template for a sentence. The AI ​​template information includes, for example, one or more variables. The AI ​​template information corresponds, for example, to a device identifier and a command type. The device identifier is information that identifies the device to be accessed (for example, the external device 4 or the information processing device 5). The device identifier is, for example, the device ID, device name, or IP address.

[0338] The command template information is information for creating command information, which will be described later. The command template information is usually information that serves as a template for command information. The command template information includes, for example, one or more variables. The command template information corresponds, for example, to a device identifier and a command type.

[0339] The reception unit 52 receives various instructions and information. The various instructions and information are, for example, input information, conversation information, and various instructions, which will be described later. The various instructions are, for example, an entry instruction, a start instruction, and an exit instruction. The reception unit 52 receives various instructions and information from, for example, the terminal device 6, but may also receive various instructions and information from the conference support device 3.

[0340] The information receiving unit 521 receives input information. The information receiving unit 521 typically receives input information including an instruction input by a user to a dialogue application using the terminal device 6. The information receiving unit 521 receives the input information, for example, after an online conference.

[0341] The input information here refers to information including an instruction text input by the user to the dialogue app using the terminal device 6. The input information may include specific information that is information that specifies the access destination. The specific information is, for example, the ID of the external device 4 or the information processing device 5, the name of the external device 4 or the information processing device 5, or the IP address of the external device 4 or the information processing device 5. The input information may include information that specifies the table or record to be accessed. The specific information is, for example, "@device name." The "device name" is the name of the external device 4 or the information processing device 5.

[0342] It is preferable that the input information corresponds to a user identifier. The input information may correspond to one or more user attribute values. Since one or more user attribute values ​​that are paired with a user identifier can be obtained, the input information may be considered to correspond to one or more user attribute values. The one or more user attribute values ​​are, for example, an organization identifier, gender, age, and job title. The organization identifier is information that identifies the organization to which the user belongs. The organization may be, for example, a company, an organization, a business division, or a department, but it does not matter.

[0343] The dialogue app is software that realizes a dialogue with a user who uses the terminal device 6. The dialogue app is software that allows a user to input a natural language sentence, and is software that is installed in the terminal device 6 or available from a web browser. Examples of the dialogue app include Slack (registered trademark), Teams (registered trademark), and a chatbot.

[0344] An instruction sentence is information indicating a user's request. An instruction sentence is a sentence expressing an instruction to one or more external devices 4 and information processing devices 5. An instruction sentence is usually one or more sentences, but may be considered to be a phrase or one or more words. An instruction sentence may be the above-mentioned question information. Examples of instruction sentences are "Please change the scheduled completion date of the Johoku business meeting to 7 / 21 / 2023", "How many business meetings are there in total this month in the calendar software?", and "Please enter the date of the next business meeting decided in the web conference in the calendar software".

[0345] The result receiving unit 522 receives result information corresponding to command information, which will be described later, from the access destination. The access destination is a device determined by the determining unit 531, which will be described later. Such a device is the external device 4 or the information processing device 5.

[0346] The result receiving unit 522 receives result information from, for example, the external device 4. The result receiving unit 522 acquires result information that is a result of the processing performed by the processing unit 53 of the information processing device 5, for example.

[0347] The result information is the result of processing based on the command information, and is, for example, the result of a search process, information indicating that a record has been added, or information indicating that an attribute value has been changed.

[0348] Here, acceptance typically refers to the reception of information transmitted via a wired or wireless communication line, but may also be a concept that includes the acceptance of information input from an input device such as a keyboard, mouse, or touch panel, and the acceptance of information read from a recording medium such as an optical disk, magnetic disk, or semiconductor memory.

[0349] The processing unit 53 performs various types of processing. The various types of processing are, for example, processing performed by a determination unit 531 and a command acquisition unit 532. It is preferable that the processing unit 53 can perform all or a part of the processing performed by the processing unit 13. The two or more access destinations are two or more external devices 4, or the information processing device 5 and one or more external devices 4.

[0350] The determination unit 531 determines one access destination from two or more access destinations according to the input information. When determining the access destination using a command statement included in the input information, the determination unit 531 may determine the access destination using specific information included in the input information. (1) When determining the access destination using a directive

[0351] The determination unit 531 determines one access destination by using, for example, a command statement included in the received input information. The determination unit 531 acquires, for example, one access destination corresponding to the command statement.

[0352] When determining the access destination using an instruction sentence, for example, an access correspondence table may be used, machine learning may be used, or generation AI may be used. (1-1) When using an access correspondence table

[0353] The determination unit 531, for example, performs morphological analysis on the instruction sentence, acquires one or more words, and determines an access destination corresponding to the one or more words from an access correspondence table. The access correspondence table is stored in the storage unit 51. The access correspondence table is a table that manages the correspondence between one or more words and an access destination. The access correspondence table has two or more pieces of correspondence information that indicate the correspondence between one or more words and specific information. (1-2) When using machine learning

[0354] The determination unit 531 acquires the access destination by machine learning prediction processing using, for example, the instruction sentence and the learning model.

[0355] A learning model is information configured by a learning process of machine learning, and is information used in a prediction process of machine learning. The learning model may be called a learner, a classifier, a classification model, or the like. The learning model here is information acquired by a learning process of machine learning using two or more pieces of teacher data having a sentence and specific information. The sentence is an instruction sentence. Here, the machine learning algorithm may be deep learning, random forest, decision tree, SVM, or the like. In addition, for machine learning, various machine learning functions such as the TensorFlow (registered trademark) library, the random forest module of the R language, fastText, TinySVM, and various existing libraries can be used. (1-3) When using generative AI

[0356] The determination unit 531 constructs a sentence from the "instruction sentence", for example, "Please tell me the access destination corresponding to the 'instruction sentence'", sends the sentence to the generation AI device 7, and receives information indicating the access destination as an answer from the generation AI device 7. (2) When determining the destination to access using specific information

[0357] The determination unit 531 determines one access destination by using, for example, the specific information included in the received input information. The specific information is information that specifies the access destination. Usually, all or part of the specific information is information that indicates the access destination.

[0358] The command acquisition unit 532 acquires command information corresponding to a command statement included in the input information. The command acquisition unit 532 acquires command information corresponding to, for example, a command statement and an access destination included in the input information. The command acquisition unit 532 acquires command information corresponding to a command statement from, for example, conversation information in an online conference. Note that the command information corresponding to a command statement is information on a command for carrying out an instruction indicated by the command statement.

[0359] Command information is information for the accessed device to execute processing. The command information is, for example, an API or a command. The command information may be any information that can be interpreted and executed by the accessed device, and may be a natural language sentence. The command information is, for example, information that specifies a command to access a database. Access to a database is, for example, searching a database, adding information to a database, updating information in a database, and deleting information from a database. Note that the structure of the database does not matter, and may be a relational database, an object-oriented database, a text file, or the like. The database may also be referred to as an object, etc. The database is, for example, a Salesforce (registered trademark) object.

[0360] The type acquiring means 5321 acquires a command type that specifies the type of the instruction statement. The command type is information that identifies the command specified by the instruction statement. The command type is, for example, information that specifies a command for accessing a database of the external device 4 or the information processing device 5. The command type is, for example, "append", "update", "search", and "delete".

[0361] The method for the type acquiring means 5321 to acquire the command type includes, for example, a method using a command correspondence table, a method using machine learning, and a method using a generating AI. Note that the method using a generating AI may be considered as an example of a method using machine learning. (1) Using the command correspondence table

[0362] The type acquiring means 5321, for example, performs morphological analysis on the instruction sentence, acquires one or more words, and acquires the command type corresponding to the one or more words from the command correspondence table. In this case, the command correspondence table is stored in the storage unit 51. The command correspondence table is a table that manages the correspondence between one or more words and command types. The command correspondence table has two or more pieces of correspondence information indicating the correspondence between one or more words and command types. (2) Machine learning method

[0363] The type acquisition means 5321 acquires the command type by machine learning prediction processing using, for example, an instruction sentence and a learning model. The learning model here is information acquired by machine learning learning processing using two or more pieces of teacher data having a sentence and an instruction type. The sentence is an instruction sentence. Here, the machine learning algorithm may be deep learning, random forest, decision tree, SVM, or the like. In addition, for example, various machine learning functions such as the TensorFlow (registered trademark) library, the random forest module of the R language, fastText, TinySVM, and various existing libraries may be used for the machine learning. (3) Using generative AI

[0364] The type acquisition means 5321, for example, constructs a query sentence including an instruction sentence (for example, "Please tell me the type of instruction for <instruction sentence>"), passes the query sentence to the generation AI device 7, and receives the instruction type as an answer from the generation AI device 7.

[0365] The template acquisition means 5322 acquires AI template information from the template storage unit 511. It is preferable that the template acquisition means 5322 acquires, from the template storage unit 511, AI template information corresponding to one access destination determined by the determination unit 531 and the command type acquired by the type acquisition means 5321.

[0366] The attribute value acquisition means 5323 acquires one or more user attribute values ​​corresponding to a user identifier corresponding to the received input information. The attribute value acquisition means 5323 acquires, for example, one or more user attribute values ​​corresponding to a user identifier corresponding to the received input information from the user management information in the storage unit 51. The attribute value acquisition means 5323 acquires, for example, one or more user attribute values ​​corresponding to the received input information.

[0367] The sentence creating means 5324 creates a query sentence using the received instruction sentence and the template information acquired by the template acquiring means 5322 .

[0368] A query sentence is a sentence to be passed to the generation AI device 7. A query sentence is, for example, a sentence in which an instruction sentence is added to a position in the AI ​​template information where the instruction sentence is to be inserted. A query sentence is, for example, a sentence in which one or more terms included in the instruction sentence are written into variables of the AI ​​template information. Examples of query sentences will be described later. The sentence creation means 5324, for example, constructs a query sentence by replacing the received instruction sentence with a variable (for example, "$instruction sentence") indicating the position of the instruction sentence in the AI ​​template information.

[0369] The sentence creation means 5324 creates a query sentence including different parameters according to, for example, one or more user attribute values ​​acquired by the attribute value acquisition means 5323. The parameters are, for example, one or more user attribute values. That is, the sentence creation means 5324 creates a query sentence in which, for example, the one or more user attribute values ​​acquired by the attribute value acquisition means 5323 are added to the AI ​​template information. Here, the one or more user attribute values ​​are organization identifiers of organizations to which the user belongs.

[0370] The text sending means 5325 sends the query text created by the text creating means 5324 to the generation AI device 7.

[0371] The text sending means 5325 sends the query text created by the text creating means 5324 to the AI ​​generating device 7 that has learned the conversation information. It is preferable that such a query text includes one or more user attribute values.

[0372] The answer receiving means 5326 receives, in response to transmission of a query sentence to the generation AI device 7, an answer to the question corresponding to the query sentence from the generation AI device 7.

[0373] The search means 5327 searches the access destination using the answer received by the answer receiving means 5326, and receives a search result including one or more pieces of search information from the access destination. The search means 5327 may obtain one or more pieces of information (e.g., terms) from the received instruction text, search the access destination using the one or more pieces of information, and receive a search result including the one or more pieces of search information from the access destination. Note that, when the answer received by the answer receiving means 5326 is included in the instruction text, it can be said that the answer is information obtained from the instruction text.

[0374] When the command type is a specific type, it is preferable that the search means 5327 searches the access destination using the response received by the response receiving means 5326, and receives a search result including one or more pieces of search information from the access destination. The specific type is, for example, "update". The specific type is, for example, "update" and "append". The specific type is, for example, other than "search". The search of the access destination is, for example, a partial match search, or a match search. The search information is, for example, an attribute value of one attribute, attribute values ​​of two or more attributes, or a record.

[0375] When the search result received by the search means 5327 includes two or more pieces of search information, the screen acquisition means 5328 acquires screen information constituting a screen for selecting one or more pieces of search information from the two or more pieces of search information. The search information is, for example, an attribute value of the search result. It is preferable that the one or more pieces of search information selected here are one piece of search information. The screen information is, for example, information described in HTML or XML, but the data structure and implementation means are not important.

[0376] For example, when the candidate acquisition means 5331 described later acquires two or more pieces of candidate information, the screen acquisition means 5328 acquires screen information constituting a screen for selecting one or more pieces of candidate information from the two or more pieces of candidate information.

[0377] The screen output means 5329 passes to the dialogue application the screen information acquired by the screen acquisition means 5328. As a result, a screen configured based on the screen information is output to the dialogue application of the user's terminal device 6.

[0378] The selection receiving means 5330 receives a selection of one or more pieces of search information corresponding to the screen information. The selection receiving means 5330 normally receives a selection of one or more pieces of search information from the terminal device 6. It is preferable that the selection receiving means 5330 receives a selection of one piece of search information from the terminal device 6.

[0379] The candidate acquiring means 5331 acquires one or more pieces of candidate information constituting the command information.

[0380] The candidate acquiring means 5331 acquires, for example, one or more pieces of candidate information that are included in the conversation information and that constitute the command information.

[0381] The candidate acquisition means 5331, for example, passes the received instruction sentence to the generation AI device 7 that has learned the conversation information, and receives candidate information of one or more answers corresponding to the instruction sentence from the generation AI device 7. The candidate acquisition means 5331 passes the received instruction sentence and one or more user attribute values ​​(e.g., organization identifiers) to the generation AI device 7 that has learned the conversation information, and receives candidate information of one or more answers corresponding to the instruction sentence from the generation AI device 7. In this case, the generation AI device 7 acquires one or more answers corresponding to the instruction sentence using a learning model corresponding to one or more user attribute values.

[0382] The candidate acquiring means 5331, for example, analyzes conversation information and acquires one or more candidate answer information items corresponding to the received instruction sentence.

[0383] The command acquiring means 5332 acquires command information. For example, the command acquiring means 5332 acquires command information by using a response received by the response receiving means 5326. For example, the command acquiring means 5332 acquires command information by using one or more pieces of search information corresponding to the selection accepted by the selection accepting means 5330.

[0384] The command acquisition means 5332 acquires command template information from the template storage unit 511. Next, the command acquisition means 5332 replaces answer or search information corresponding to one or more variables (which may also be called parameters) contained in the command template information with the variables, and acquires command information.

[0385] It is preferable that the command acquiring means 5332 acquires, from the template storage unit 511, command template information corresponding to the device identifier, which is the access destination determined by the determining unit 531, and the command type acquired by the type acquiring means 5321.

[0386] When the command output unit 541 passes the command information acquired by the command acquisition unit 532 to the access destination, the command acquisition unit 532 may acquire the attribute name of the attribute value that is a necessary attribute value included in the record corresponding to the command information but does not exist. The command type of the command information is, for example, "append" or "update", but may also be "search" or the like. The command acquisition unit 532 sends the acquired command information (for example, "UPDATE negotiation table SET negotiation date = July 24, 2023 WHERE company name = Johoku", "INSERT INTO negotiation table (company name, negotiation date) VALUES (Johoku, July 24, 2023") to the access destination. Then, the access destination device detects that a required attribute value (for example, the attribute "recipient") does not exist in the negotiation table, and transmits the attribute name of the required attribute value (for example, "recipient") to the information processing device 5. The information processing device 5 may manage the required attribute values ​​in the table.

[0387] Next, the command acquisition unit 532 acquires the attribute name (for example, "recipient"), and then uses the attribute name to construct an input request (for example, "enter "recipient") for prompting input of the attribute value, and passes the input request to the dialogue application.

[0388] Next, the terminal device 6 on which the dialogue application is running receives the input request and outputs the input request in the dialogue application. Next, it is assumed that the user inputs "X Corporation" in response to the output input request and transmits it to the information processing device 5.

[0389] Next, the command acquisition unit 532 accepts an input (for example, "X Corporation") in response to the input request, and configures command information using the input. For example, the command acquisition unit 532 configures command information "UPDATE negotiation table SET negotiation date=July 24, 2023, other party='X Corporation' WHERE company name=Johoku", and passes the command information to the access destination.

[0390] The accessed device then accepts the command information and executes the command information, resulting in the appropriate record being stored containing the required attribute values.

[0391] The output unit 54 outputs various types of information. The various types of information include, for example, command information and result information. The output here is usually transmission.

[0392] The command output unit 541 passes the command information to the access destination determined by the determination unit 531. Then, the access destination, that is, the external device 4 or the information processing device 5, performs processing according to the command information.

[0393] The result transmission unit 542 transmits, to the dialogue application, the result information accepted by the result acceptance unit 522. The result information is a result of the external device 4 or the information processing device 5 performing a process according to the command information. The result information is, for example, a search result.

[0394] Various types of information are stored in the terminal storage unit 61 constituting the terminal device 6. The various types of information are, for example, a user identifier. The user identifier in the terminal storage unit 21 may be information input by a user and temporarily stored.

[0395] The terminal reception unit 62 receives various information, instructions, etc. Examples of the various information, instructions, etc. include input information, selection of one or more pieces of search information from two or more pieces of search information, selection of one or more pieces of candidate information from two or more pieces of candidate information, user voice information, and user video.

[0396] The means for inputting various information and instructions may be anything, such as a touch panel, a keyboard, a mouse, a menu screen, a camera, a microphone, etc.

[0397] The device processing unit 63 performs various types of processing. For example, the various types of processing are processing for converting received information, instructions, etc. into information or instructions with a structure to be transmitted. For example, the various types of processing are processing for converting received information into information with a structure to be output. For example, the device processing unit 63 composes a screen to be output from received screen information.

[0398] The terminal transmission unit 64 transmits various information, instructions, and the like to the information processing device 5. The various information, instructions, and the like are, for example, input information, selection of one or more pieces of search information from two or more pieces of search information, selection of one or more pieces of candidate information from two or more pieces of candidate information, user voice information, and user video. The terminal transmission unit 64 transmits, for example, the user voice information and user video to the conference support device 3. Note that the user voice information and user video are information acquired in an online conference.

[0399] The terminal receiving unit 65 receives various types of information, such as various types of screen information, audio information, and video.

[0400] The terminal output unit 66 outputs various types of information, such as various screens, audio information, and video.

[0401] Here, output is a concept that includes display on a display, projection using a projector, printing on a printer, sound output, transmission to an external device, storage on a recording medium, and handing over the processing results to other processing devices or other programs, etc.

[0402] The storage unit 51, the template storage unit 511, and the terminal storage unit 61 are preferably non-volatile recording media, but may also be realized as volatile recording media.

[0403] There is no restriction on the process by which information is stored in the storage unit 51, etc. For example, information may be stored in the storage unit 51, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 51, etc., or information inputted via an input device may be stored in the storage unit 51, etc.

[0404] It is preferable that the reception unit 52 and the result reception unit 522 are realized by wireless or wired communication means, but they may also be realized by a means for receiving broadcasts, a device driver for an input means such as a touch panel or keyboard, or control software for a menu screen.

[0405] Information receiving unit 521, selection receiving means 5330, and terminal receiving unit 65 are usually realized by wireless or wired communication means, but may be realized by means for receiving broadcast.

[0406] The processing unit 53, the deciding unit 531, the command acquiring unit 532, the type acquiring means 5321, the template acquiring means 5322, the attribute value acquiring means 5323, the text creating means 5324, the answer receiving means 5326, the searching means 5327, the screen acquiring means 5328, the screen output means 5329, the candidate acquiring means 5331, the command acquiring means 5332, and the terminal processing unit 63 can be usually realized by a processor, a memory, etc. The processing procedure of the processing unit 53, etc. is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit). The processor is a CPU, an MPU, a GPU, etc., and the type does not matter.

[0407] The output unit 54 and the command output unit 541 are realized, for example, by wireless or wired communication means, but may also be realized by means for receiving broadcasts.

[0408] The text transmitting means 5325, the result transmitting unit 542, and the terminal transmitting unit 64 are usually realized by wireless or wired communication means, but may also be realized by broadcasting means.

[0409] The terminal reception unit 62 can be realized by a device driver for an input means such as a touch panel or a keyboard, or control software for a menu screen.

[0410] The terminal output unit 66 may or may not include an output device such as a display, a speaker, etc. The terminal output unit 66 may be realized by driver software for an output device, or a combination of driver software for an output device and an output device, etc.

[0411] Next, an example of the operation of the information processing device 5 will be described with reference to the flowchart of Fig. 24. In the flowchart of Fig. 24, the description of the same processes as those of the information processing device 1 will be omitted.

[0412] (Step S2401) The information receiving unit 521 judges whether or not it has received input information input to the dialogue application in the terminal device 6. If it has received the input information, it returns to step S2402, and if it has not received the input information, it returns to step S2401. The information receiving unit 521 may receive the input information from the terminal device 6, for example, or may receive the input information from a server (not shown) of the dialogue application, etc.

[0413] (Step S2402) The determining unit 531 determines one access destination from two or more access destinations according to the input information. An example of such access destination determination processing will be described with reference to the flowchart of FIG.

[0414] (Step S2403) The command acquisition unit 532 acquires command information corresponding to the instruction statement included in the input information. An example of such command acquisition processing will be described with reference to the flowchart of FIG.

[0415] (Step S2404) The command output unit 541 passes the command information acquired in step S2403 to the access destination determined in step S2402. The command output unit 541 executes the command information, which is, for example, an API.

[0416] (Step S2405) Result receiving unit 522 judges whether or not result information has been received. If result information has been received, proceed to step S2406, and if result information has not been received, return to step S2405. Note that result information is information that specifies the result of execution of command information in the accessed device in response to the transmission of command information in step S2404.

[0417] Here, the result receiving unit 522 receives, for example, the result information from the external device 4. The result receiving unit 522 acquires, for example, the result information acquired by the processing unit 53 of the information processing device 5.

[0418] (Step S2406) Processing unit 53 judges whether or not to transmit the result information to the user's terminal device 6. If it is to be transmitted to terminal device 6, proceed to step S2407; if it is not to be transmitted to terminal device 6, return to step S2401. Note that, for example, if the result information is specific information, processing unit 53 does not transmit the result information to user's terminal device 6. The specific information is, for example, the result information being only "information indicating normal completion." Also, processing unit 53 does not transmit the result information to user's terminal device 6 if the command type is, for example, specific information. The specific information is, for example, "update," "add," or "delete."

[0419] (Step S2407) The result transmitting unit 542 transmits the result information acquired in step S2406 to the terminal device 6. Here, the result transmitting unit 542 transmits the result information so that it is output by the dialogue application of the terminal device 6.

[0420] In the flowchart of FIG. 24, the process ends when the power is turned off or an interrupt occurs to end the process.

[0421] Next, an example of the access destination determination process in step S2402 will be described with reference to the flowchart in FIG.

[0422] (Step S2501) The determination unit 531 acquires the input information received in step S2401.

[0423] (Step S2502) The determination unit 531 judges whether or not specific information is present in the input information acquired in step S2501. If specific information is present, the process proceeds to step S2503, and if specific information is not present, the process proceeds to step S2504. Note that specific information is, for example, a character string from the character next to "@" in the input information to a space.

[0424] (Step S2503) The determining unit 531 obtains the specific information from the input information obtained in step S2501. The determining unit 531 obtains an access destination paired with the specific information from the storage unit 51. The process returns to the upper level process. In this case, an access correspondence table, which is a correspondence table between one or more pieces of specific information and access destinations, is stored in the storage unit 51. The specific information may also be the access destination.

[0425] (Step S2504) The determination unit 531 acquires the instruction statement in the input information acquired in step S2501.

[0426] (Step S2505) The determination unit 531 obtains an access destination corresponding to the instruction statement obtained in step S2504. The process returns to the upper level process. As described above, when obtaining the access destination, the determination unit 531 may use an access correspondence table, may use machine learning, or may use a generation AI.

[0427] Next, an example of the command acquisition process in step S2403 will be described with reference to the flowchart in FIG.

[0428] (Step S2601) The instruction acquisition unit 532 determines whether or not the AI ​​inquiry conditions are met. If the AI ​​inquiry conditions are met, the process proceeds to step S2602, and if the AI ​​inquiry conditions are not met, the process proceeds to step S2611.

[0429] An AI query condition is a condition for determining whether or not to query the generation AI device 7. An AI query condition is, for example, that AI template information corresponding to the acquired access destination does not exist in the template storage unit 511. An AI query condition is, for example, that AI template information corresponding to the acquired access destination and command type does not exist in the template storage unit 511. An AI query condition may be, for example, that when attempting to process steps S2611 and S2612, parameters necessary for configuring the command could not be acquired.

[0430] (Step S2602) The template acquisition unit 5322 etc. acquires AI template information. An example of such template acquisition processing will be described with reference to the flowchart in FIG.

[0431] (Step S2603) The sentence creation means 5324 etc. use the AI ​​template information acquired in step S2602 to create a query sentence to be passed to the generation AI device 7. An example of such a query sentence creation process will be described with reference to the flowchart in FIG.

[0432] (Step S2604) The text sending means 5325 passes the query text constructed in step S2603 to the generation AI device 7. The process of passing the query text to the generation AI device 7 is, for example, a process of executing the API of the generation AI device 7 with the query text as a parameter.

[0433] (Step S2605) The answer receiving means 5326 judges whether or not one or more answers have been received from the generation AI device 7. If an answer has been received, the process proceeds to step S2606, and if an answer has not been received, the process returns to step S2605.

[0434] (Step S2606) The screen acquisition means 5328 judges whether or not the answer received in step S2605 matches the user inquiry condition. If it matches the user inquiry condition, the process proceeds to step S2607, and if it does not match, the process proceeds to step S2610. Note that the user inquiry condition is, for example, that there are two or more answers. Also, the user inquiry condition is, for example, that there are two or more pieces of specific information included in one or more answers.

[0435] Here, the search means 5327 may use the response received in step S2605 to search for the device to be accessed, receive the search results, and determine whether the search results match the user inquiry conditions. This determination is also made by the screen acquisition means 5328 as to whether the response received in step S2605 matches the user inquiry conditions.

[0436] (Step S2607) The screen acquisition means 5328 uses the answer received in step S2605 to configure screen information for the user to select one or more pieces of candidate information from two or more pieces of candidate information. The screen information is, for example, information of a screen having a selection object corresponding to each of the two or more pieces of candidate information. The selection object is, for example, a button, a check box, or a menu item. The two or more pieces of candidate information are, for example, two or more answers, or two or more pieces of search information included in the search result.

[0437] (Step S2608) The screen output means 5329 transmits the screen information configured in step S2607 to the terminal device 6. Here, the screen output means 5329 transmits the screen information so that the screen indicated by the screen information is output to the dialogue application of the terminal device 6 of the user.

[0438] (Step S2609) The selection receiving means 5330 judges whether or not a selection of one or more pieces of candidate information has been received from two or more pieces of candidate information. If a selection of candidate information has been received, the process proceeds to step S2610, and if a selection of candidate information has not been received, the process returns to step S2609. The candidate acquiring means 5331 acquires the one or more pieces of selected candidate information.

[0439] (Step S2610) The command acquisition means 5332 etc. configures command information using the acquired information. Then, the process returns to the upper level. An example of such command configuration process will be described with reference to the flowchart in FIG.

[0440] (Step S2611) The command acquisition unit 532 acquires the received command statement.

[0441] (Step S2612) The command acquisition unit 532 acquires one or more parameters from the instruction statement. Then, the process proceeds to step S2610.

[0442] The command acquisition unit 532 performs, for example, morphological analysis on the instruction sentence, acquires one or more independent words, acquires a class of each of the one or more independent words, and acquires a parameter that the class is an independent word of a predetermined class. It is possible to acquire the class of a term by, for example, a machine learning prediction process. Since acquiring the class of a term is a publicly known technique, a detailed description will be omitted.

[0443] Next, an example of the template acquisition process in step S2602 will be described with reference to the flowchart in FIG.

[0444] (Step S2701) The template acquiring means 5322 acquires a received instruction text.

[0445] (Step S2702) The type obtaining means 5321 obtains the command type by using the instruction statement obtained in step S2701.

[0446] (Step S2703) The template obtaining means 5322 obtains the determined access destination.

[0447] (Step S2704) The template obtaining means 5322 obtains AI template information corresponding to the access destination and the command type from the template storage unit 511. The process returns to the upper level process.

[0448] Next, an example of the query sentence construction process in step S2603 will be described with reference to the flowchart in FIG.

[0449] (Step S2801) The text creating means 5324 acquires the specified text that has been received.

[0450] (Step S2802) The sentence creation unit 5324 pastes the specified sentence into the acquired AI template information. Note that the sentence creation unit 5324 replaces the variable indicating the specified sentence of the acquired AI template information with the specified sentence acquired in step S2801.

[0451] (Step S2803) The text creating means 5324 assigns 1 to a counter i.

[0452] (Step S2804) The sentence creation means 5324 judges whether or not the i-th variable exists in the AI ​​template information in step S2802. If the i-th variable exists, the process proceeds to step S2805, and if not, the process proceeds to step S2808.

[0453] (Step S2805) The sentence creation unit 5324 acquires information to be substituted into the i-th variable. Note that this information is, for example, information contained in the specified sentence. This information is, for example, an independent word contained in the specified sentence.

[0454] (Step S2806) The sentence creation means 5324 replaces the i-th variable in the AI ​​template information with the information acquired in step S2805.

[0455] (Step S2807) The text creation means 5324 increments the counter i by 1. The process returns to step S2804.

[0456] (Step S2808) The attribute value acquiring means 5323 judges whether or not to acquire the user attribute value by using the specification statement. If the user attribute value is to be acquired, the process proceeds to step S2809, and if not, the process returns to the upper process.

[0457] Here, the attribute value acquisition means 5323 judges whether or not the specified sentence satisfies a user attribute value acquisition condition, for example. The user attribute value acquisition condition is that the specified sentence contains a specific keyword (for example, "conference"), and the user attribute value acquisition condition is that the class of the specified sentence is a specific class (for example, a question about the contents of the conference).

[0458] (Step S2809) The attribute value acquiring means 5323 acquires a user identifier paired with the received specification statement. The attribute value acquiring means 5323 acquires one or more user attribute values ​​paired with the user identifier from the user management information in the storage unit 51.

[0459] (Step S2810) The document creation unit 5324 adds one or more user attribute values ​​acquired in step S2809 to the template information, and returns to the upper process. Note that the user attribute value is, for example, an organization identifier.

[0460] Next, an example of the command construction process in step S2610 will be described with reference to the flowchart in FIG.

[0461] (Step S 2901 ) The command obtaining means 5332 obtains the command type obtained by the type obtaining means 5321 .

[0462] (Step S2902) The command obtaining means 5332 obtains the access destination determined by the determining unit 531.

[0463] (Step S2903) The command acquiring means 5332 acquires, from the command template storage unit 511, command template information corresponding to the command type and the access destination.

[0464] (Step S2904) The instruction fetching means 5332 assigns 1 to the counter i.

[0465] (Step S2905) The command obtaining means 5332 judges whether or not the i-th parameter (variable) exists in the command template information. If the i-th parameter exists, the process proceeds to step S2906, and if not, the process returns to the upper process.

[0466] (Step S2906) The command obtaining means 5332 obtains one or more pieces of information corresponding to the i-th parameter in the command template information.

[0467] (Step S2907) The screen acquisition means 5328 judges whether the information acquired in step S2906 matches the user inquiry condition. If the information matches the user inquiry condition, the process proceeds to step S2908, and if the information does not match, the process proceeds to step S2911. Note that the user inquiry condition is, for example, that two or more pieces of information are acquired in step S2906.

[0468] (Step S2908) The screen acquisition means 5328 configures screen information using the information acquired in step S2906. Note that the screen information is information of a screen for selecting one or more pieces of candidate information from the two or more pieces of candidate information acquired in step S2906.

[0469] (Step S2909) The screen output means 5329 transmits the screen information constructed in step S2908 to the terminal device 6. Note that the transmission here is an output to the dialogue application of the terminal device 6.

[0470] (Step S2910) The selection receiving means 5330 judges whether or not a selection of candidate information has been received from the terminal device 6. If a selection of candidate information has been received, the selected one or more pieces of information are acquired and the process proceeds to step S2911, and if a selection of candidate information has not been received, the process returns to step S2910.

[0471] (Step S2911) The command obtaining means 5332 sets the information obtained in step S2906 or the information corresponding to the selection received in step S2910 as the i-th parameter.

[0472] (Step S2912) The instruction fetching means 5332 increments the counter i by 1. The process returns to step S2905.

[0473] Next, an example of the operation of the terminal device 6 will be described with reference to the flowchart of FIG.

[0474] (Step S3001) The terminal reception unit 62 judges whether or not a start instruction of the dialogue application has been received. If the start instruction has been received, step S3002 is performed. If the start instruction has not been received,

[0475] (Step S3002) The device processing unit 63 starts up the dialogue application. The process returns to step S3001.

[0476] (Step S3003) The terminal reception unit 62 judges whether or not input information has been received in the dialogue application. If input information has been received, the process proceeds to step S3004, and if input information has not been received, the process returns to step S3001.

[0477] (Step S3004) The device processing unit 63 acquires a user identifier from the terminal storage unit 61. The device processing unit 63 transmits the user identifier and the input information. As a result of this transmission, the user identifier and the input information are transmitted to the information processing device 5. Note that the route of this transmission does not matter.

[0478] (Step S3005) The terminal receiving unit 65 judges whether or not result information has been received in response to the transmission of the input information. If result information has been received, the process proceeds to step S3006, and if not, the process proceeds to step S3007.

[0479] (Step S3006) The device processing unit 63 composes result information to be output using the result information received in step S3005. The terminal output unit 66 outputs the result information. Return to step S3001.

[0480] (Step S3007) The terminal receiving unit 65 judges whether or not screen information has been received from the information processing device 5. If screen information has been received, the process proceeds to step S3008, and if not, the process proceeds to step S3011.

[0481] (Step S3008) The device processing unit 63 composes a screen to be output using the screen information received in step S3007. The terminal output unit 66 outputs the screen.

[0482] (Step S3009) The terminal reception unit 62 determines whether or not a selection of information has been accepted, which is a selection on the output screen. If a selection of information has been accepted, the process proceeds to step S3010, and if not, the process returns to step S3009.

[0483] (Step S3010) The terminal processing unit 63 acquires the selected one or more pieces of information. The terminal transmission unit 64 transmits the one or more pieces of information. Return to step S3005. The transmitted one or more pieces of information are received by the information processing device 5. In this case, the information transmission route is not important.

[0484] (Step S3011) The device processing unit 63 judges whether or not a timeout has occurred. If a timeout has occurred, the process proceeds to step S3001. If a timeout has not occurred, the process proceeds to step S3005. Note that a timeout occurs when a time equal to or greater than a threshold has elapsed since the screen information was received.

[0485] In the flowchart of FIG. 30, the process ends when the power is turned off or an interrupt occurs to end the process.

[0486] A specific example of the operation of the information system B in this embodiment will be described below. A conceptual diagram of the information system B is shown in FIG.

[0487] Currently, the template management table of Fig. 31 is stored in the template storage unit 511 of the information processing device 5. The template management table is a table that manages AI template information and command template information. The template management table manages one or more records that have an "ID", "device identifier", "command type", "AI template information", and "command template information". "-" is NULL. Note that "$" in "AI template information" and "command template information" indicates a variable.

[0488] Also, it is assumed that the template storage unit 511 stores AI template information "Teach me '$instruction sentence' from the meeting information of '$organization identifier'" in association with the term "meeting."

[0489] Also, assume that user A, who belongs to organization identifier "company a," held a conference and performed an operation to have the generation AI device 7 learn the conference information of that conference. Also, assume that user B, who belongs to organization identifier "company b," held a different conference and performed an operation to have the generation AI device 7 learn the conference information of that conference.

[0490] In this situation, the following two specific examples will be described. Specific Example 1 is a case where template information is used to execute a process desired by the user. Specific Example 2 is a case where conversation information according to a user attribute value is used.

[0491] (Example 1) Assume that user A, who has launched a conversation app (here, for example, "Slack") installed on terminal device 6, inputs the input information "@Device X Change the scheduled completion date of Johoku's business negotiation to 2023 / 7 / 21" into his / her terminal device 6. Here, "@Device X" in the input information is specific information, and "Change the scheduled completion date of Johoku's business negotiation to 2023 / 7 / 21" is an instruction statement.

[0492] Next, the terminal device 6 accepts the input information, and transmits the input information to the information processing device 5 in a pair with the user identifier "User A."

[0493] Next, the information receiving unit 521 of the information processing device 5 receives the input information from the terminal device 6. Next, the determining unit 531 acquires one access destination "device X" according to the input information. The access destination here is a device identifier.

[0494] Next, the command acquisition unit 532 of the information processing device 5 determines that the input information matches the AI ​​query conditions because AI template information paired with the access destination "device X" corresponding to the input information exists in the template management table (Figure 31).

[0495] Next, the template acquisition means 5322 acquires AI template information paired with the access destination "device X" from the template management table (FIG. 31). That is, the template acquisition means 5322 acquires AI template information of "ID=1" in the template management table (FIG. 31).

[0496] Next, the sentence creation means 5324 substitutes the received instruction sentence "Change the scheduled completion date of Johoku's business negotiation to 2023 / 7 / 21" into the variable "$instruction sentence" of the acquired AI template information, and constructs a query sentence to be given to the generation AI device 7. The query sentence is "Convert "Change the scheduled completion date of Johoku's business negotiation to 2023 / 7 / 21" into a sentence "Search table (a) in device X with (b) and (e) the attribute value of attribute name (c) with (d)", and return (a)(b)(c)(d)(e)." Next, the sentence sending means 5325 passes the query sentence to the generation AI device 7.

[0497] Next, the generation AI device 7 receives the query sentence, provides the query sentence to the generation AI program, obtains the answer “(a) business negotiation (b) Johoku (c) expected completion date (d) 7 / 21 / 2023 (e) update”, and transmits the answer to the information processing device 5.

[0498] Next, the answer receiving means 5326 of the information processing device 5 receives the five answers from the generation AI device 7 in response to the transmission of the query sentence.

[0499] Next, the search means 5327 acquires "Select($a,$b)" which constitutes the command template information of "ID=1" in Fig. 31. Next, the search means 5327 substitutes "(a) Negotiation (b) Johoku" for each of "$a" and "$b" in the command information "Select($a,$b)" to construct "Select(Negotiation, Johoku)" and executes the command information on "Device X", which is the access destination. Note that "Select(Negotiation, Johoku)" is command information which instructs searching for "Johoku" from the table "Negotiation".

[0500] Then, the search means 5327 acquires attribute values ​​including "JOHOKU" from the table "Business Negotiations" in the device X. As a result, the search means 5327 acquires "JOHOKU NETWORK CO., LTD.", "JOHOKU TECHNOLOGY CO., LTD.", and "JOHOKU SCIENCE CO., LTD."

[0501] Next, the screen acquisition means 5328 judges that the acquired answer matches the user inquiry condition (here, the presence of two or more attribute values). Next, the screen acquisition means 5328 uses the acquired three attribute values ​​"JOHOKU NETWORK CO., LTD.", "JOHOKU TECHNOLOGY CO., LTD.", and "JOHOKU SCIENCE CO., LTD." to configure screen information constituting a screen for selecting one attribute value from the three attribute values. Next, the screen output means 5329 transmits the screen information to the terminal device 6.

[0502] Next, the terminal device 6 receives the screen information, composes a screen, and outputs it. The screen is output in a dialogue application. An example of such a screen is shown in FIG. 32. Then, it is assumed that user A selects the selection button 3201. Next, the terminal device 6 accepts the selection, and transmits information or an attribute value "JOHOKU NETWORK CO., LTD." that identifies the selection button 3201.

[0503] Next, the selection receiving means 5330 of the information processing device 5 receives the selection of one or more pieces of candidate information from among the two or more pieces of candidate information, and acquires the attribute value "JOHOKU NETWORK CO., LTD.".

[0504] Next, the command acquisition means 5332 configures command information using the acquired information. That is, the command acquisition means 5332 acquires the command template information "Update ($a, $b, $c, $d)" corresponding to "(e) Update" of "ID=1" from the template management table (FIG. 31). Next, the command acquisition means 5332 assigns values ​​to each of the parameters ($a, $b, $c, $d) of the command template information, and acquires the command information "Update (business negotiation, Johoku Network Co., Ltd., expected completion date, 2023 / 7 / 21)".

[0505] Next, the command output unit 541 passes the command information "Update (business negotiation, Johoku Network Co., Ltd., scheduled completion date, 2023 / 7 / 21)" to the access destination "device X."

[0506] Next, the external device 4, "Device X", receives the command information "Update (Business Negotiation, Johoku Network Co., Ltd., Expected Completion Date, 7 / 21 / 2023)", executes the command, and updates the attribute value of the attribute name "Expected Completion Date" of the record that matches "Johoku Network Co., Ltd." in the table "Business Negotiation" to "7 / 21 / 2023".

[0507] (Example 2) It is assumed that user C who has launched a dialogue application (here, for example, "Slack") inputs input information "When is the next business meeting for user A, which was decided in the previous meeting?" into his / her terminal device 6. It is also assumed that the company to which user C belongs is "company a." Next, the terminal device 6 accepts the input information and transmits the input information to the information processing device 5 in a pair with the user identifier "user C."

[0508] Next, the information receiving unit 521 of the information processing device 5 receives the input information from the terminal device 6. Next, it is assumed that the determination unit 531 is unable to determine an access destination according to the input information. In addition, the processing unit 53 determines to inquire of the generation AI device 7 in order to obtain an answer from the conversation information based on the term "conference" contained in the input information "When is the next business meeting for user A, which was decided at the previous conference?"

[0509] Next, it is assumed that the attribute value acquiring means 5323 of the information processing device 5 acquires, from the user management information in the storage unit 51, the organization identifier “company a” that is paired with the user identifier “user C”.

[0510] Next, the text creation means 5324 acquires the AI ​​template information "Tell me '$instruction' from the conference information of '$organization identifier'" that is paired with the term "conference" from the storage unit 51. Next, the command acquisition unit 532 assigns "company a" to the variable "$organization identifier" of the AI ​​template information, assigns the received instruction to the variable "$instruction", and acquires the query text "Tell me 'When is user A's next business meeting?' from the conference information of 'company a'". Next, the text sending means 5325 passes the query text to the generation AI device 7.

[0511] Next, the generation AI device 7 receives the query sentence, provides the query sentence to the generation AI program, obtains the answer “August 2, 2023”, and transmits the answer to the information processing device 5.

[0512] Next, the answer receiving means 5326 of the information processing device 5 receives the answer from the generation AI device 7 in response to the transmission of the query sentence.

[0513] Next, the output unit 54 transmits the answer "August 2, 2023" to the terminal device 6 of "User C."

[0514] Next, the terminal device 6 of "user C" receives and outputs the answer. The answer is output to the dialogue application.

[0515] As described above, according to this embodiment, a user can easily operate two or more devices by inputting input information including a natural language sentence.

[0516] Furthermore, according to the present embodiment, by the user inputting input information including a natural language sentence, it is possible to easily operate two or more devices, and the user can receive the results of the operation.

[0517] Furthermore, according to this embodiment, a query can be given to the generation AI, and the obtained answer can be used to easily obtain command information to be given to the accessed device.

[0518] Furthermore, according to this embodiment, a query can be obtained for the generation AI using AI template information corresponding to the instruction text.

[0519] Furthermore, according to this embodiment, before performing an operation using command information on an access destination, a screen for selecting search information is output to the user using two or more pieces of search information obtained from the access destination, and appropriate command information can be obtained using the search information selected by the user using that screen.

[0520] Furthermore, according to this embodiment, an appropriate answer using a learning model according to the user attribute value can be obtained from the generation AI.

[0521] Furthermore, according to this embodiment, the access destination can be automatically determined from the received instruction text.

[0522] Furthermore, according to this embodiment, after an online conference, command information can be constructed using information obtained from conversation information of the online conference.

[0523] Furthermore, according to this embodiment, after an online conference, command information can be constructed using information obtained from a generation AI that has learned conversation information from the online conference.

[0524] The software for realizing the information processing device 5 in this embodiment is the following program. That is, this program causes a computer to function as an information receiving unit that receives input information including an instruction sentence input by a user to a dialogue application using a terminal device, a determining unit that determines one access destination according to the input information from two or more access destinations including one or more external devices, a command obtaining unit that obtains command information according to the instruction sentence included in the input information and the access destination, and a command output unit that passes the command information to the access destination.

[0525] Fig. 33 shows the appearance of a computer that executes the program described in this specification to realize the information processing device 5 and the like of the various embodiments described above. The above-mentioned embodiments can be realized by computer hardware and a computer program executed thereon. Fig. 33 is an overview of this computer system 300, and Fig. 34 is a block diagram of the system 300.

[0526] In FIG. 33, computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.

[0527] 34, computer 301 includes, in addition to CD-ROM drive 3012, MPU 3013, bus 3014 connected to CD-ROM drive 3012 etc., ROM 3015 for storing programs such as a boot-up program, RAM 3016 connected to MPU 3013 for temporarily storing instructions of application programs and providing temporary storage space, and hard disk 3017 for storing application programs, system programs, and data. Although not shown here, computer 301 may further include a network card for providing connection to a LAN.

[0528] A program for causing computer system 300 to execute the functions of information processing device 5 and the like of the above-mentioned embodiment may be stored on CD-ROM 3101, inserted into CD-ROM drive 3012, and further transferred to hard disk 3017. Alternatively, the program may be transmitted to computer 301 via a network (not shown) and stored on hard disk 3017. The program is loaded into RAM 3016 at the time of execution. The program may be loaded directly from CD-ROM 3101 or the network.

[0529] The program does not necessarily include an operating system (OS) or a third party program that causes the computer 301 to execute the functions of the information processing device 5 of the above-mentioned embodiment. The program only needs to include an instruction portion that calls appropriate functions (modules) in a controlled manner and achieves a desired result. How the computer system 300 operates is well known, and a detailed description will be omitted.

[0530] In addition, in the above program, the steps of transmitting information and receiving information do not include processing performed by hardware, such as processing performed by a modem or interface card in the transmitting step (processing that is performed only by hardware).

[0531] The program may be executed by a single computer or a plurality of computers. That is, the program may be executed by a centralized processing or a distributed processing.

[0532] Furthermore, in each of the above embodiments, it goes without saying that two or more communication means present in one device may be physically realized by one medium.

[0533] In each of the above embodiments, each process may be realized by centralized processing in a single device, or may be realized by distributed processing in a plurality of devices.

[0534] The present invention is not limited to the above-described embodiment, and various modifications are possible, and it goes without saying that these modifications are also included within the scope of the present invention. [Industrial Applicability]

[0535] As described above, the information processing device 5 according to the present invention has an effect that a user can easily operate two or more devices by inputting input information including a natural language sentence, and is useful as a server or the like. [Explanation of symbols]

[0536] B. Information Systems 3. Meeting Support Equipment 4 External device 5. Information processing equipment 6 Terminal Equipment 7 Generation AI device 51 Storage area 52 Reception 53 Processing section 54 Output section 61 Terminal storage section 62 Terminal Reception 63 Terminal Processing Section 64 Terminal transmitter 65 Terminal receiving section 66 Terminal Output Unit 111 Conversation Storage Unit 511 Template Storage Section 521 Information Receiving Department 522 Results Reception Department 531 Decision Section 532 Instruction acquisition part 541 Command output section 542 Result transmission unit 5321 Type acquisition method 5322 Template acquisition means 5323 Attribute value acquisition means 5324 Writing tools 5325 Text transmission means 5326 Response receiving means 5327 Search methods 5328 Screen acquisition means 5329 Screen output means 5330 Selection acceptance means 5331 Candidate acquisition means 5332 Instruction acquisition means

Claims

1. an information receiving unit that receives input information including an instruction input by a user to the dialogue application using a terminal device; a determination unit that determines one access destination from two or more access destinations including one or more external devices according to the input information; a command acquisition unit that acquires command information corresponding to the instruction statement and the access destination included in the input information; and a command output unit that passes the command information to the access destination.

2. a result receiving unit that receives result information corresponding to the command information from the access destination; The information processing apparatus according to claim 1 , further comprising a result transmission unit that transmits the result information to the dialogue application.

3. A template storage unit in which AI template information for creating a query sentence to be sent to the generation AI device is stored; The command acquisition unit is A template acquisition means for acquiring the AI ​​template information of the template storage unit; A sentence creation means for creating a query sentence by using the instruction sentence and the AI ​​template information; A sentence transmission means for transmitting the query sentence created by the sentence creation means to the generation AI device; an answer receiving means for receiving an answer to a question corresponding to the query sentence from the generating AI device in response to the transmission of the query sentence; 3. The information processing apparatus according to claim 1, further comprising: a command acquisition means for acquiring said command information by using said response.

4. The command acquisition unit is A type acquiring means for acquiring a command type that identifies a type of the instruction sentence, The template storage unit includes: Two or more AI template information corresponding to an access destination and a command type are stored, The template acquisition means includes:

4. The information processing apparatus according to claim 3, further comprising: acquiring AI template information corresponding to the one access destination determined by the determination unit and the command type acquired by the type acquisition means.

5. The command acquisition unit is a search means for searching the access destination using the response received by the response receiving means and receiving a search result including one or more pieces of search information from the access destination; a screen acquisition means for acquiring screen information constituting a screen for selecting one or more pieces of search information from the two or more pieces of search information when the search result includes two or more pieces of search information; a screen output means for transmitting the screen information to the dialogue application; a selection receiving means for receiving a selection of one or more pieces of search information corresponding to the screen information; The command acquisition means includes:

5. The information processing apparatus according to claim 3, wherein the command information is acquired by using the one or more pieces of search information corresponding to the selection accepted by the selection accepting means.

6. The command acquisition unit is when the command output unit passes the command information to the access destination, it obtains an attribute name of a necessary attribute value included in a record corresponding to the command information but which does not exist, passes an input request to the dialogue application using the attribute name to prompt input of the attribute value, accepts input in response to the input request, and obtains the command information using the input; The command output unit is 2. The information processing apparatus according to claim 1, wherein the command information using the input is passed to the access destination.

7. The input information is associated with a user identifier, The command acquisition unit is The method further includes: acquiring one or more user attribute values ​​corresponding to the user identifier; The document creation means includes: The information processing device according to claim 3 , further comprising: a step of generating the query sentence including different parameters in accordance with the one or more user attribute values ​​acquired by the attribute value acquisition means.

8. The determination unit is 2. The information processing apparatus according to claim 1, wherein the one access destination is determined by using the instruction statement.

9. The information receiving unit receiving the input information after the online conference; The command acquisition unit is The information processing apparatus according to claim 1 , wherein command information corresponding to the instruction sentence is acquired from conversation information in the online conference.

10. The command acquisition unit is A template acquisition means for acquiring AI template information from a template storage unit; A sentence creation means for creating a query sentence by using the instruction sentence and the AI ​​template information; A sentence transmission means for transmitting the query sentence created by the sentence creation means to a generation AI device that has learned the conversation information; an answer receiving means for receiving an answer to a question corresponding to the query sentence from the generating AI device in response to the transmission of the query sentence; 10. The information processing apparatus according to claim 9, further comprising: a command acquisition means for acquiring the command information by using the response.

11. The command acquisition unit is a candidate acquisition means for acquiring one or more pieces of candidate information constituting command information, the candidate information being included in the conversation information; a screen acquisition means for acquiring screen information constituting a screen for selecting one or more pieces of candidate information from the two or more pieces of candidate information when the candidate acquisition means acquires two or more pieces of candidate information; a screen output means for transmitting the screen information to the dialogue application; a selection receiving means for receiving a selection of one or more pieces of candidate information corresponding to the screen information; 10. The information processing apparatus according to claim 9, further comprising: a command acquiring unit configured to acquire the command information by using the one or more pieces of candidate information corresponding to the selection accepted by the selection accepting unit.

12. An information processing method implemented by an information receiving unit, a determining unit, a command acquiring unit, and a command output unit, comprising: an information receiving step in which the information receiving unit receives input information including an instruction sentence input by a user to the dialogue application using a terminal device; a determination step in which the determination unit determines one access destination according to the input information from two or more access destinations including one or more external devices; an instruction acquisition step in which the instruction acquisition unit acquires instruction information corresponding to the instruction statement and the access destination included in the input information; and a command output step in which the command output unit passes the command information to the access destination.

13. Computer, an information receiving unit that receives input information including an instruction input by a user to the dialogue application using a terminal device; a determination unit that determines one access destination from two or more access destinations including one or more external devices according to the input information; a command acquisition unit that acquires command information corresponding to the instruction statement and the access destination included in the input information; A program for causing the program to function as a command output unit for passing the command information to the access destination.

Images