Program, method, information processing device, and system

Abstract

This program to be executed by an information processing terminal for storing information in a face-to-face meeting composed of a plurality of participants and comprising a processor, a storage unit, and a microphone executes: a voice acquisition step for acquiring voice information inputted from the microphone in the face-to-face meeting; a schedule acquisition step for acquiring predetermined schedule information relating to the face-to-face meeting; a participant input step for receiving input of participant information relating to participants in the face-to-face meeting; a generation step for generating meeting information relating to the face-to-face meeting on the basis of the schedule information acquired in the schedule acquisition step and the participant information received in the participant input step; and a transmission step for transmitting the voice information acquired in the voice acquisition step and the meeting information generated in the generation step in association with each other.

Description

Program, Method, Information Processing Apparatus, System

[0001] The present disclosure relates to a program, a method, an information processing apparatus, and a system.

[0002] Information processing techniques related to calls made between a user and a customer are known. In Patent Document 1, a program, an information processing system, and an information processing method for storing data related to calls made between a user and a customer in a CRM system at a timing suitable for each process such as individual user or organizational business activities are disclosed.

[0003] Japanese Unexamined Patent Application Publication No. 2023-076391

[0004] There is a problem that information on face-to-face meetings cannot be suitably managed. Therefore, the present disclosure has been made to solve the above problems, and its object is to provide a technique for suitably managing information on face-to-face meetings.

[0005] A program for causing an information processing terminal including a processor, a storage unit, and a microphone to store information in a face-to-face meeting composed of a plurality of participants, wherein the processor executes an audio acquisition step of acquiring audio information input from the microphone in the face-to-face meeting, a schedule acquisition step of acquiring predetermined schedule information related to the face-to-face meeting, a participant input step of receiving input of participant information related to the participants in the face-to-face meeting, a generation step of generating meeting information related to the face-to-face meeting based on the schedule information acquired in the schedule acquisition step and the participant information received in the participant input step, and a transmission step of transmitting the audio information acquired in the audio acquisition step in association with the meeting information generated in the generation step.

[0006] According to the present disclosure, information on face-to-face meetings can be suitably managed.

[0007] This is a block diagram showing the functional configuration of System 1. This is a block diagram showing the functional configuration of Server 10. This is a block diagram showing the functional configuration of User Terminal 20. This is a diagram showing the data structure of User Table 1012. This is a diagram showing the data structure of Group Table 1013. This is a diagram showing the data structure of Meeting Table 1014. This is a diagram showing the data structure of Label Table 1015. This is a diagram showing the data structure of Voice Interval Table 1016. This is a diagram showing the data structure of Customer Table 1021. This is a diagram showing the data structure of Schedule Table 1022. This is a flowchart showing the operation of meeting processing. This is a flowchart showing the operation of scheduling processing. This is an example screen showing the operation of meeting processing. This is a block diagram showing the basic hardware configuration of Computer 90.

[0008] The embodiments of this disclosure will be described below with reference to the drawings. In all the drawings illustrating the embodiments, common components are denoted by the same reference numerals, and repeated explanations are omitted. The following embodiments are not intended to unduly limit the content of this disclosure as described in the claims. Not all components shown in the embodiments are necessarily essential components of this disclosure. Also, each drawing is a schematic diagram and is not necessarily a strict illustration.

[0009] <Configuration of System 1> System 1 in this disclosure is an information processing system for storing and managing information in a face-to-face meeting consisting of multiple participants. System 1 comprises a server 10, a user terminal 20, and an information processing device of a CRM system 60, all connected via a network N. Figure 1 is a block diagram showing the functional configuration of System 1. Figure 2 is a block diagram showing the functional configuration of Server 10. Figure 3 is a block diagram showing the functional configuration of User Terminal 20.

[0010] Each information processing device consists of a computer equipped with an arithmetic unit and a memory device. The basic hardware configuration of the computer and the basic functional configuration of the computer realized by said hardware configuration will be described later. For each of the server 10, user terminal 20, and CRM system 60, explanations that overlap with the basic hardware configuration and basic functional configuration of the computer described later will be omitted.

[0011] <Configuration of Server 10> Server 10 is an information processing device that provides an information processing service for storing and managing information in face-to-face meetings involving multiple participants. Server 10 comprises a storage unit 101 and a control unit 104.

[0012] <Configuration of the storage unit 101 of the server 10> The storage unit 101 of the server 10 includes an application program 1011, a user table 1012, a group table 1013, a meeting table 1014, a label table 1015, an audio segment table 1016, a customer table 1021, and a schedule table 1022.

[0013] Application program 1011 is a program that causes the control unit 104 of server 10 to function as various functional units. Application program 1011 includes applications such as a web browser application.

[0014] User table 1012 is a table that stores and manages information about member users (hereinafter referred to as "users") who use the service. When a user registers to use the service, their information is stored in a new record in user table 1012. This allows the user to use the service related to this disclosure. User table 1012 is a table that has User ID as the primary key and has columns for User ID, User Name, and User Data. Figure 4 is a diagram showing the data structure of user table 1012.

[0015] The User ID is an item that stores user identification information to identify a user. User identification information is an item that is set to a unique value for each user. The Username is an item that stores the user's name. The Username may be set to any string, such as a nickname, instead of a real name. User data includes unique information about the individual user and attribute information about the user's characteristics and background. Unique information about the user includes unique information about the user such as the user's date of birth (age) and gender. Attribute information about the user includes information such as the user's educational history (highest level of education, major, graduation year), occupation, work history, interests, place of residence, and languages.

[0016] The group table 1013 is a table that stores and manages information about the groups to which a user belongs (group information). Groups can include any group, such as companies, corporations, corporate groups, clubs, and various organizations. Groups may also be defined for more detailed subgroups, such as company departments (sales department, general affairs department, customer support department). The group table 1013 has a group ID as its primary key and has columns for group ID, group name, and group attributes. Figure 5 shows the data structure of the group table 1013.

[0017] The Group ID is an item that stores group identification information to identify a group. Each group has a unique value assigned to it. The Group Name is an item that stores the name of the group. The Group Name can be any string. The Group Attributes are items that store information about the group's attributes, such as the group type (company, corporate group, other organization, etc.) and industry (real estate, finance, etc.).

[0018] The meeting table 1014 is a table for storing and managing information (meeting information) related to meetings held between users and customers. The meeting table 1014 has a primary key of meeting ID and has columns for meeting ID, user ID, schedule ID, participant ID, and audio data. Figure 6 shows the data structure of the meeting table 1014.

[0019] The Meeting ID is an item that stores meeting identification information to identify a meeting. The meeting identification information is an item with a unique value set for each meeting. The User ID is an item that stores user identification information to identify a user in a meeting between a user and a customer. Multiple user IDs may be associated with each meeting. The Schedule ID is an item that stores schedule identification information related to the meeting. The Participant ID is an item that stores user identification information and customer identification information to identify users and customers participating in a meeting in a meeting between a user and a customer. Multiple user IDs and multiple customer IDs may be associated with each meeting. The Participant ID may also be an item that stores customer identification information to identify customers participating in a meeting in the CRM system 60. The Audio Data is an item that stores audio data collected by a microphone. It may also store reference information (path) to an audio data file located elsewhere. The audio data format may be any data format such as AAC, ATRAC, mp3, mp4, etc. The audio data may be in a format in which the user's voice and the customer's voice are each assigned identifiers that allow for independent identification. In this case, the control unit 104 of the server 10 can perform independent analysis processing on the user's voice and the customer's voice. Furthermore, the user ID and customer ID can be identified based on the user's and customer's audio data. In this disclosure, video data containing audio information may be used instead of audio data. For example, video data captured by a camera may be stored. It may also store reference information (path) to a video data file located elsewhere. The video data format may be any data format such as MP4, MOV, WMV, AVI, AVCHD, etc. The audio data in this disclosure also includes audio data contained in video data.

[0020] The label table 1015 is a table for storing and managing information about labels (label information). The label table 1015 is a table that has columns for meeting ID and label data. Figure 7 shows the data structure of the label table 1015.

[0021] The meeting ID is an item that stores meeting identification information to identify a meeting. The label data is an item that stores label information for managing meetings. Label information is additional information for managing meeting information, such as classification names, labels, classification labels, and tags. Label data may be a string indicating the name of the label information, or it may be a label ID, etc., to reference the name of the label information stored in other tables. Label data may include classification information according to the emotional state of speakers in a particular meeting. Classification data may include classification information for classifying the quality of speakers' responses in a particular meeting.

[0022] The audio segment table 1016 is a table for storing and managing information about multiple audio segments included in the conference information (audio segment information). The audio segment table 1016 has a segment ID as its primary key and has columns for segment ID, conference ID, speaker ID, start date and time, end date and time, segment audio data, segment video data, and segment spoken text. Figure 8 shows the data structure of the audio segment table 1016.

[0023] The Section ID is an item that stores section identification information to identify an audio section. The section identification information is an item with a unique value set for each audio section. The Meeting ID is an item that stores meeting identification information to identify the meeting to which the audio section information is associated. The Speaker ID is an item that stores speaker identification information to identify the speaker to whom the audio section information is associated. Specifically, the Speaker ID is an item that stores multiple user IDs and customer IDs that participated in the meeting. The Start Date and Time is an item that stores the start date and time of the audio section and video section. The End Date and Time is an item that stores the end date and time of the audio section and video section. The Section Audio Data is an item that stores the audio data included in the audio section. It may also store reference information (path) to an audio data file located elsewhere. Alternatively, it may store a reference to the audio data for the period from the start date and time to the end date and time of the audio data in the meeting table 1014 based on the start date and time and the end date and time. Furthermore, the Section Audio Data may also include the audio data included in the Section Video Data. The audio data format can be any data format such as AAC, ATRAC, mp3, or mp4. The section video data is an item that stores the video data included in the audio section. It may also store reference information (path) to video data files located elsewhere. It may also store references to video data for the period from the start date and time to the end date and time of the video data in conference table 1014 based on the start date and time and end date and time. The video data format can be any data format such as MP4, MOV, WMV, AVI, or AVCHD. The section spoken text is an item that stores text information of the content spoken by the speaker in the section audio data included in the audio section. Specifically, the section spoken text may be generated manually, using any machine learning model, deep learning, etc., based on the section audio data and section video data.

[0024] The customer table 1021 is a table for storing and managing information about customers (customer information). In this disclosure, as an example, customer information is stored in the storage unit 101 of the server 10, but a CRM system 60 may be used instead of the customer table 1021. In this case, the customer table 1021 may be omitted. Alternatively, the CRM system 60 may be omitted, and only the customer table 1021 may be provided. The customer table 1021 is a table with customer ID as the primary key and columns for customer ID, customer name, user ID, telephone number, and customer group ID. Figure 9 shows the data structure of the customer table 1021.

[0025] The Customer ID is an item that stores customer identification information to identify a customer. Customer identification information is an item with a unique value set for each customer. The Customer Name is an item that stores the customer's name. The Customer Name can be set to any string. The User ID is an item that stores user identification information. The Telephone Number is an item that stores the customer's telephone number. The Customer Group ID is an item that stores customer group identification information to identify a customer group. By associating and storing one or more customer group identification values ​​for each customer, it is expressed that a customer belongs to one or more customer groups.

[0026] The schedule table 1022 is a table for storing and managing information (schedule information) related to the schedule of some event, such as a meeting or conference. The schedule table 1022 has a schedule ID as its primary key and has columns for schedule ID, start date and time, end date and time, participant ID, location, and schedule details. Figure 10 shows the data structure of the schedule table 1022. The schedule table 1022 may also include metadata as appropriate, such as information related to the schedule, including information about the event title, links and file data of materials related to the event, participant contact information such as telephone numbers and email addresses, and attendance information regarding each participant's attendance plans.

[0027] The Schedule ID is an item that stores schedule identification information to identify a schedule. The schedule identification information is an item with a unique value set for each schedule. Note that the schedule identification information may include any information included in the schedule table 1022, such as the event title and start time, or any other identification information that is stored in association with the schedule information. The Start Date and Time is an item that stores the start date and time of the schedule (event). The End Date and Time is an item that stores the end date and time of the schedule (event). The Participant ID is an item that stores user identification information and customer identification information to identify users and customers participating in the schedule. Multiple user IDs and multiple customer IDs may be associated with each schedule. The Participant ID may also be an item that stores customer identification information to identify customers participating in a meeting in the CRM system 60. The Location is an item that stores information indicating the location where the schedule (event), such as a meeting, is held. The Schedule Content is an item that stores information indicating the content of the event, such as a meeting. For example, strings such as "New Project Proposal Meeting (Considering the Implementation of the ●● System)", "Requirements Definition Phase Kick-off Meeting", and "Strategic Partnership Review Meeting" are stored.

[0028] <Configuration of the control unit 104 of the server 10> The control unit 104 of the server 10 includes a user registration control unit 1041. The control unit 104 realizes each functional unit by executing the application program 1011 stored in the storage unit 101.

[0029] The user registration control unit 1041 processes information of users who wish to use the services related to this disclosure and stores it in the user table 1012. The information stored in the user table 1012 is obtained when a user opens a web page operated by the service provider from any information processing terminal, enters information into a predetermined input form, and sends it to the server 10. The user registration control unit 1041 stores the received information in a new record in the user table 1012, and user registration is completed. As a result, users stored in the user table 1012 can use the service. Prior to the registration of user information in the user table 1012 by the user registration control unit 1041, the service provider may perform a predetermined review and restrict whether or not a user can use the service. The user ID may be any string or number that can identify the user, and may be any string or number that the user wishes, or the user registration control unit 1041 may automatically set an arbitrary string or number.

[0030] <Configuration of User Terminal 20> The user terminal 20 is an information processing device operated by a user of the service. The user terminal 20 may be, for example, a mobile device such as a smartphone or tablet, or a stationary PC (Personal Computer) or laptop PC. It may also be a wearable device such as an HMD (Head Mount Display) or a smartwatch, or a peripheral device that connects wirelessly or via wired connection to a communication device such as a smartphone. The information processing terminal related to the user terminal 20 is an information processing terminal that stores information in a business negotiation involving the user of the information processing terminal and one or more of the user's customers. The user terminal 20 comprises a storage unit 201, a control unit 204, an input device 206, and an output device 208.

[0031] <Configuration of the storage unit 201 of the user terminal 20> The storage unit 201 of the user terminal 20 includes a user ID 2011 and an application program 2012.

[0032] User ID 2011 is the user's account ID. The user sends User ID 2011 from the user terminal 20 to the server 10. The server 10 identifies the user based on User ID 2011 and provides the services related to this disclosure to the user. User ID 2011 includes information such as a session ID that is temporarily assigned by the server 10 to identify the user using the user terminal 20. User ID 2011 may also use information such as an email address or telephone number to identify the user.

[0033] The application program 2012 may be pre-stored in the memory unit 201, or it may be configured to be downloaded from a web server operated by the service provider via a communication interface. The application program 2012 includes applications such as a web browser application. The application program 2012 includes an interpreter-type programming language such as JavaScript® that is executed on the web browser application stored in the user terminal 20.

[0034] <Configuration of the control unit 204 of the user terminal 20> The control unit 204 of the user terminal 20 comprises an input control unit 2041 and an output control unit 2042. The control unit 204 realizes each functional unit, including communication with the server 10, etc., by executing the application program 2012 stored in the storage unit 201.

[0035] <Configuration of the input device 206 of the user terminal 20> The input device 206 of the user terminal 20 includes a camera 2061, a microphone 2062, a position information sensor 2063, a motion sensor 2064, and a touch device 2065. Not all elements of the input device 206 do not necessarily have to be included in the user terminal 20, and some or all elements may be configured using peripheral devices connected via a wireless or wired network as appropriate.

[0036] <Configuration of the output device 208 of the user terminal 20> The output device 208 of the user terminal 20 includes a display 2081 and a speaker 2082.

[0037] <Configuration of CRM System 60> CRM System 60 is an information processing device operated by a different business operator from the conference service provider related to this disclosure. CRM System 60 is an information processing device that provides information processing services, etc., operated by the said business operator. CRM System 60 may also be an information processing device managed and operated by a business operator that provides CRM (Customer Relationship Management) services (CRM business operator). Examples of CRM services include SalesForce, HubSpot, Zoho CRM, and kintone. CRM System 60 may also be an information processing device managed and operated by a business operator that provides chat services. Examples of chat services include Slack, Chatwork, and LINE Works.

[0038] <System 1 Operation> The following describes each process of System 1. Figure 11 is a flowchart of the operation of the meeting process. Figure 12 is a flowchart of the operation of the scheduling process. Figure 13 is an example screen showing the operation of the meeting process.

[0039] <Meeting Processing> This refers to the process of storing information in a face-to-face meeting involving multiple participants. In this disclosure, an example is provided in which the control unit 204 of the user terminal 20 performs the meeting processing, but it is not limited to this. For example, the control unit 104 of the server 10 may execute each step in response to a request sent from the control unit 204 of the user terminal 20. In addition, some or all of the steps of the meeting processing may be executed using some of multiple information processing terminals, any cloud service, etc.

[0040] <Overview of Meeting Processing> Meeting processing is a series of processes that involves launching the meeting application, receiving input of schedule information for the meeting, receiving input of information about the meeting participants, acquiring audio information for the meeting, generating meeting information for the meeting, sending the meeting information along with the audio information, and analyzing the audio information.

[0041] <Details of Meeting Procedures> The details of meeting procedures are explained below.

[0042] <Application Launch> In step S101, the user selects and executes an application program related to conference processing by operating the input device 206 of the user terminal 20. The application program may be stored in the storage unit 201 of the user terminal 20. Alternatively, the application program may be executed by accessing a website provided by the server 10 using a browser program such as a web browser on the user terminal 20. The application program may include a web application executed by a web browser.

[0043] The user executes an application program by operating the input device 206 of the user terminal 20, and displays the meeting processing page on the display 2081 of the user terminal 20. In this disclosure, it is preferable for the user to place the user terminal 20 on a desk in a meeting room or connect to a microphone 2062, camera 2061, etc. placed on the desk via a wireless or wired network during face-to-face meetings such as business negotiations with one or more customers. The user terminal 20 can record the content of speech during face-to-face meetings or videotape the meeting using the microphone 2062, camera 2061, etc. The user terminal 20 can also record the content of speech during face-to-face meetings or videotape the meeting using the microphone 2062, camera 2061, etc. Furthermore, other users (colleagues, etc.) belonging to the same company or organization as the user, as well as persons other than customers, may be present at the face-to-face meeting.

[0044] The control unit 204 of the user terminal 20 executes an application program, thereby displaying a meeting screen D1 on the display 2081 of the user terminal 20. The meeting screen D1 includes a first input field D101 for entering schedule information, a second input field D111 for entering participant information, and a meeting start button D141. In this disclosure, an example is disclosed in which the user selects the first input field D101, the second input field D111, and the meeting start button in that order, and the processes of steps S102, S103, and S104 are executed, but this is not limited to this example. The selection order of the first input field D101, the second input field D111, and the meeting start button D141 may be any order. Also, steps S102, S103, and S104 may be executed in any order. Furthermore, it is not necessary to input into the first input field D101 and the second input field D111, and input may be skipped. Alternatively, the user may only input in the first input field D101 and leave the second input field D111 blank. The meeting screen D1 includes schedule content D102, an automatic upload selection button D121, a detailed settings button D131, a recording button D151, a list button D161, and a settings button D171. Schedule content D102 is an item that displays an overview of the schedule content related to the schedule information entered in the first input field D101. If the automatic upload selection button D121 is enabled, the audio information acquired by the audio information acquisition process in step S104 is automatically uploaded to the server 10. If the automatic upload selection button D121 is disabled, the audio information acquired by the audio information acquisition process in step S104 is uploaded to the server 10 separately in response to a selection input such as an "upload" button from the user. The detailed settings button D131 is an item that changes the detailed settings of the microphone 2062 of the user terminal 20. For example, the user can select a recording mode suitable for face-to-face meetings by operating the detailed settings button D131. The recording button D151 is used to check the recording status of the microphone 2062 on the user terminal 20.The list button D161 is a button for displaying a list of meeting information that has been stored through past meeting processing and is stored in the storage unit 201 of the user terminal 20 or the meeting table 1014 of the server 10. The setting button D171 is a button for changing various settings of the application program.

[0045] The control unit 204 of the user terminal 20 transmits a request including the user ID 2011 for holding a meeting to the server 10. The control unit 104 of the server 10 creates a new record of meeting information in the meeting table 1014 and stores the user ID 2011 in the user ID item. Note that the processing from steps S101 to S105 may be executed by the control unit 204 of the user terminal 20, and the processing after step S106 may be executed by the server 10. In this case, when generating and transmitting the meeting information in step S105, the control unit 104 of the server 10 creates a new record of the meeting information in the meeting table 1014.

[0046] <Schedule Information Input> In step S102, the control unit 204 of the user terminal 20 executes a schedule acquisition step of acquiring predetermined schedule information regarding a face-to-face meeting. The control unit 104 of the server 10 executes a user storage step of storing the user identification information of the user in the storage unit. Specifically, the user selects the first input field by operating the input device 206 of the user terminal 20. The control unit 204 of the user terminal 20 transmits a request including the user ID 2011 for acquiring schedule information to the server 10. When the control unit 104 of the server 10 receives the request, it searches the participant ID item of the schedule table 1022 based on the user ID 2011 and acquires the schedule information including the user ID 2011 as a participant. The control unit 104 of the server 10 transmits the acquired schedule information to the user terminal 20.

[0047] The schedule acquisition step involves acquiring schedule information from a schedule management service connected to the information processing terminal via a network. The schedule information does not need to be stored in the storage unit 101 of the server 10; it may also be configured to acquire schedule information stored in an external calendar service (Google Calendar, Outlook Calendar, etc.). The control unit 204 of the user terminal 20 may acquire schedule information directly from the external calendar service, or it may acquire schedule information acquired by the control unit 104 of the server 10 from the external calendar service via the server 10. In this disclosure, an example of acquiring one piece of schedule information is disclosed, but multiple pieces of schedule information may be acquired. The schedule information associated with the meeting information record in the meeting table 1014 does not necessarily have to be one piece; multiple pieces of schedule information may be associated with it.

[0048] <Schedule Information Input (First Embodiment)> In step S102, the schedule acquisition step executes a step of acquiring schedule information based on at least any one of the holding date and time of the face-to-face meeting, the holding place, and the participant information received in the participant input step. Specifically, the control unit 204 of the user terminal 20 may be configured to acquire schedule information in which the start date and time or the end date and time is substantially the same as the current date and time among the schedule information stored in the schedule table 1022 or the calendar service. For example, it may be configured to acquire a schedule within a predetermined period (10 minutes, 20 minutes, 30 minutes, etc.) before or after the current date and time as the start date and time. Also, it may be configured to acquire one or more pieces of schedule information whose holding date of the schedule information is the same as the holding date of the face-to-face meeting. Also, by executing step S103 before step S102, when the participant information has already been input, the control unit 204 of the user terminal 20 may acquire schedule information including the participant ID related to the participant information received in step S103 among the schedule information stored in the schedule table 1022 or the calendar service. Also, the control unit 204 of the user terminal 20 may acquire schedule information related to a holding place within a predetermined distance from the position information acquired by the position information sensor 2063 etc. of the user terminal 20 among the schedule information stored in the schedule table 1022 or the calendar service. In addition, schedule information may be acquired by combining at least any one or more of the holding date and time, the holding place, and the participant information. Thereby, since the schedule information specified based on the information such as the holding date and time, the holding place, and the participants of the face-to-face meeting can be acquired, the input burden on the user can be reduced.

[0049] In the first embodiment of schedule information input, the system may identify and acquire one schedule that is closest in date and time (most suitable). In the first embodiment of schedule information input, the system may identify and acquire one schedule that matches all or part of the participant information. In the first embodiment of schedule information input, the system may identify and acquire one schedule that is closest in location (most suitable). The system may also present the acquired schedule information as schedule candidates, as shown in the second embodiment of schedule information input, and accept selection from the user. The acquisition of schedule information may be controlled by the control unit 204 of the user terminal 20, or by the control unit 104 of the server 10.

[0050] <Schedule Information Input (Second Embodiment)> In step S102, the control unit 204 of the user terminal 20 executes a schedule candidate acquisition step in which it acquires one or more schedule candidate information from a schedule management service connected to the information processing terminal via a network, based on user identification information. Specifically, the control unit 204 of the user terminal 20 acquires one or more schedule information that includes the user associated with user ID 2011 as a participant, by referring to the participant ID in the schedule table 1022 of the server 10 or an external calendar service, based on user ID 2011.

[0051] The control unit 204 of the user terminal 20 executes a schedule candidate presentation step, which presents one or more schedule candidate information acquired in the schedule candidate acquisition step to the information processing terminal. Specifically, the control unit 204 of the user terminal 20 displays the received one or more schedule information as a selectable list on the display 2081 of the user terminal 20 and presents them.

[0052] The control unit 204 of the user terminal 20 executes a schedule selection step in which it receives a selection operation from the user to choose a predetermined schedule from the one or more schedule candidate information presented in the schedule candidate presentation step. Specifically, the user selects a predetermined schedule from the one or more schedule information presented on the display 2081 of the user terminal 20 by operating the input device 206 of the user terminal 20 and performing a selection operation such as clicking or tapping.

[0053] The schedule acquisition step involves acquiring predetermined schedule information received in the schedule selection step. Specifically, the control unit 204 of the user terminal 20 identifies and acquires the schedule information selected by the user. It is also possible to configure the system to identify and acquire multiple schedule information items.

[0054] The control unit 204 of the user terminal 20 transmits the user ID 2011 and the acquired schedule information to the server 10. The control unit 104 of the server 10 acquires and accepts the schedule information. The control unit 104 of the server 10 stores the schedule ID of the acquired schedule information in the schedule ID field of the new record in the meeting table 1014 created in step S101. The control unit 104 of the server 10 may also be configured to store all or part of the schedule information in a column of the meeting table 1014 that stores schedule information (not shown). Alternatively, the user may directly input each item of the schedule information using the input device 206 of the user terminal 20 and transmit it to the server 10, thereby storing all or part of the schedule information in a column of the meeting table 1014 that stores schedule information (not shown).

[0055] <Participant Information Input> In step S103, the control unit 204 of the user terminal 20 executes a participant input step to receive input of participant information regarding the participants of the face-to-face meeting. The participant input step executes a step of receiving customer information of one or more customers from the user as participant information. The control unit 104 of the server 10 executes a user storage step to store the user's user identification information in the storage unit. Specifically, the user selects the second input field by operating the input device 206 of the user terminal 20. The user inputs information about the participant, such as a user or customer, by operating the input device 206 of the user terminal 20. Specifically, the user inputs information into the input fields for inputting information about the participant, such as name, affiliated organization (company name), and remarks, which are included in the second input field, and transmits it by operating the input device 206 of the user terminal 20. The control unit 104 of the server 10 may also be configured to store the user or customer information of the participant in a column of the conference table 1014 that stores participant information (not shown).

[0056] <Participant Information Input (First Embodiment)> In step S102, the control unit 204 of the user terminal 20 executes a customer candidate acquisition step in which it acquires one or more customer candidate information from a customer management service connected to the information processing terminal via a network, based on the user identification information. Specifically, the user selects a second input field by operating the input device 206 of the user terminal 20. The control unit 204 of the user terminal 20 sends a request to the server 10 that includes a user ID 2011 for acquiring customer information. When the control unit 104 of the server 10 receives the request, it searches the user ID field in the customer table 1021 based on the user ID 2011 and acquires one or more customer information associated with the user ID 2011. The control unit 104 of the server 10 sends the acquired one or more customer information to the user terminal 20.

[0057] The customer candidate acquisition step involves acquiring customer information from a customer management service connected to the information processing terminal via a network. The customer information does not need to be stored in the server 10's storage unit 101; it may be acquired by referencing schedule information stored in an external customer management service (CRM service), etc. Examples of CRM (Customer Relationship Management) services include SalesForce, HubSpot, Zoho CRM, and kintone. The control unit 204 of the user terminal 20 may acquire customer information directly from the external CRM service, or it may acquire schedule information acquired by the server 10's control unit 104 from the external CRM service via the server 10.

[0058] In step S103, the control unit 104 of the server 10 executes a customer candidate presentation step in which it presents one or more customer candidate information acquired in the customer candidate acquisition step to the information processing terminal. Specifically, the control unit 204 of the user terminal 20 displays the received one or more customer information in a selectable list on the display 2081 of the user terminal 20 and presents them.

[0059] In step S103, the participant input step performs a step of receiving participant information relating to one or more customers by receiving a selection operation from the user for customer information of one or more customers from the one or more customer candidate information presented in the customer candidate presentation step. Specifically, the user selects predetermined customer information from the one or more customer information presented on the display 2081 of the user terminal 20 by performing a selection operation such as clicking or tapping. The control unit 204 of the user terminal 20 identifies and acquires the customer information selected by the user. Note that the configuration may also include identifying and acquiring multiple pieces of customer information. In addition, participants other than face-to-face participants may be included as participants. For example, participants may join the meeting via online meeting applications such as Zoom, Google Meet, or Microsoft Teams.

[0060] The control unit 204 of the user terminal 20 transmits the user ID 2011 and the acquired customer information to the server 10. The control unit 104 of the server 10 acquires and accepts the customer information. The control unit 104 of the server 10 stores the customer ID of the acquired customer information in the participant ID field of the new record in the conference table 1014 created in step S101. The control unit 104 of the server 10 may also be configured to store all or part of the customer information in a column of the conference table 1014 that stores customer information (not shown).

[0061] <Participant Information Input (Second Embodiment)> In step S103, the control unit 104 of the server 10 compares the participant information included in the predetermined schedule information with the customer information stored in the customer management service connected to the information processing terminal via the network, and performs a customer identification step to identify predetermined customer information. The participant input step is a step of receiving input of participant information relating to the customer identified based on the predetermined customer information identified in the customer identification step. Specifically, the control unit 104 of the server 10 stores the customer ID, user ID, etc., identified based on the participant ID included in the schedule information acquired in step S102, in the participant ID field of a new record in the meeting table 1014 created in step S101. The control unit 104 of the server 10 may search the customer ID field in the customer table 1021 and the user ID field in the user table 1012 based on the participant ID included in the schedule information acquired in step S102, acquire customer information and user information, and store the acquired customer information and user information in a column of the meeting table 1014 that stores participant information (not shown). This allows for a reduction in user input burden by utilizing customer information stored in external calendar services and external customer relationship management (CRM) services.

[0062] <Participant Information Input (Third Embodiment)> In step S103, if participant information has not been entered after the acquisition of voice information is complete, the control unit 204 of the user terminal 20 executes an input prompting step to prompt the user to input participant information. Specifically, if the second input field is empty after the voice information acquisition in step S104, the control unit 204 of the user terminal 20 may display a prompt to input in the second input field on the display 2081 of the user terminal 20. In this case, the control unit 204 of the user terminal 20 may be configured to execute participant information input (first embodiment) or (second embodiment) after executing step S104. The control unit 204 of the user terminal 20 displays a string such as "Participant information is not entered in the second input field. Please enter participant information" on the display 2081 of the user terminal 20 as a prompt to input in the second input field.

[0063] <Audio Information Acquisition> In step S104, the control unit 104 of the server 10 performs an audio acquisition step to acquire audio information input from the microphone during the face-to-face meeting. Specifically, the user selects the meeting start button by operating the input device 206 of the user terminal 20. The control unit 204 of the user terminal 20 performs recording of spoken audio during the face-to-face meeting by controlling the microphone 2062 of the user terminal 20. The control unit 204 of the user terminal 20 may also record the face-to-face meeting as a video by controlling the camera 2061. The control unit 204 of the user terminal 20 hides the "meeting start button" and displays the "meeting end button" on the display 2081 of the user terminal 20.

[0064] The control unit 204 of the user terminal 20 transmits the recorded audio data and recorded video data to the server 10. The control unit 104 of the server 10 stores the received recorded audio data and video data in the audio data field of a new record in the conference table 1014 created in step S101. The control unit 204 of the user terminal 20 may transmit the recorded audio data and video data to the server 10 as real-time streaming data, or it may transmit the recorded audio data and video data to the server 10 at predetermined intervals (10 seconds, 30 seconds, 1 minute, 5 minutes, 10 minutes, etc.). The control unit 104 of the server 10 stores the received recorded audio data and video data in the audio data field.

[0065] The control unit 104 of the server 10 executes an audio acquisition termination step to end the acquisition of audio information. Specifically, the user selects the conference end button by operating the input device 206 of the user terminal 20. The control unit 204 of the user terminal 20 stops recording by the microphone 2062 of the user terminal 20. The control unit 204 of the user terminal 20 stops recording by the camera 2061.

[0066] <Conference Information Generation and Transmission> In step S105, the control unit 204 of the user terminal 20 executes a generation step to generate conference information related to the face-to-face meeting based on the schedule information acquired in the schedule acquisition step and the participant information received in the participant input step. In this disclosure, the record relating to the conference information is disclosed as an example of being created when the application program is executed in step S101, but is not limited to this. For example, it may be created in response to the input of schedule information in step S102, or in response to the input of participant information in step S103. It may also be created after the acquisition of audio information. Furthermore, it is not necessarily required that the conference information be stored in the conference table 1014 of the server 10. The storage unit 201 of the user terminal 20 may have a table similar to the conference table 1014 and store the information in that table. In this case, it is not necessary to transmit the schedule information, participant information, and audio information to the server 10 in each of steps S102, S103, and S104, and they may be stored in the conference table 1014 of the storage unit 201 of the user terminal 20. Specifically, the control unit 204 of the user terminal 20 stores the schedule ID of the schedule information entered in the first input field in step S102 and the participant ID of the participant information entered in the second input field in step S103 in the schedule ID and participant ID fields of a new record in the conference table of the user terminal 20, respectively. Alternatively, the control unit 204 of the user terminal 20 may store the schedule information entered in the first input field in step S102 and the participant information entered in the second input field in step S103 in columns not shown in the new record of the conference table of the user terminal 20.

[0067] In step S105, the control unit 204 of the user terminal 20 performs a transmission step in which it transmits the audio information acquired in the audio acquisition step and at least a portion of the meeting information generated in the generation step, in association with each other. The transmission step performs the step of transmitting the meeting information to the dialogue management service connected to the information processing terminal via a network. Specifically, the control unit 204 of the user terminal 20 stores information that can identify a meeting (meeting identification information), such as schedule information, participant information, and audio information, as meeting information in the meeting table of the user terminal 20. The control unit 204 of the user terminal 20 transmits the stored meeting information to the server 10 (information processing device having a dialogue management service function).

[0068] In step S105, the transmission step includes a notification step that notifies other members of the group to which the user of the information processing terminal belongs that the audio information and meeting information have been transmitted. Specifically, when the control unit 104 of the server 10 receives meeting information from the user terminal 20, it stores it in the meeting table 1014 of the server 10. Based on the participant IDs included in the meeting information, the control unit 104 of the server 10 searches the user ID item in the user table 1012 and obtains user information. The user information includes information that identifies the user, such as an email address and a chat service (messaging service) ID. The control unit 104 of the server 10 sends a notification to each user identified based on the user-identifying information that the meeting information has been stored in the meeting table 1014. For example, it notifies the email address of the user related to the participant, the chat service (messaging service) room, etc. This allows the user related to the participant to confirm that the meeting information has been stored in the meeting table 1014. The audio information and meeting information can then be shared with other members of the group to which the user belongs. If recording data is being sent to server 10 in parallel with the meeting, once the meeting ends, the details of the business negotiation can be automatically notified and reported to other members of the team within the company or organization to which the user belongs.

[0069] <Meeting Information Generation and Transmission (Variation 1)> In step S105, the generation step performs the step of generating meeting information based on the meeting title, date and time, location, and participant information included in the schedule information. Specifically, the meeting information may include the meeting title, date and time, location, and participant information included in the schedule information. An overview of the meeting information can be grasped from the meeting title, date and time, and location.

[0070] <Conference Information Generation and Transmission (Modification 2)> In step S105, the control unit 204 of the user terminal 20 performs a display step in which it displays keywords extracted based on the audio information acquired in the audio acquisition step on the information processing terminal. Specifically, the control unit 204 of the user terminal 20 transcribes the audio information acquired in the audio information acquisition step of step S104 into text by applying any text recognition technology. The control unit 204 of the user terminal 20 extracts one or more keywords by applying any morphological analysis technology to the transcribed text. Note that the keywords may also be extracted based on the output information that is output when the audio data, transcribed text data, etc. are input to the generation AI 50. The control unit 204 of the user terminal 20 displays and presents the extracted one or more keywords on the display 2081 of the user terminal 20 in a selectable format.

[0071] In step S105, the control unit 104 of the server 10 executes a keyword selection step in which it receives selection operations for keywords displayed in the display step from at least some of the participants. Specifically, any participant, including the user, performs an operation to select a keyword displayed on the display 2081 of the user terminal 20. The control unit 204 of the user terminal 20 transmits the selected keyword to the server 10. The control unit 104 of the server 10 receives and accepts the keyword.

[0072] In step S105, the generation step performs the step of generating meeting information that includes the keywords selected in the keyword selection step. Specifically, the control unit 104 of the server 10 stores the received keywords in an item (not shown) of a newly created record in the meeting table 1014. This allows the keywords extracted during the meeting to be presented to the participants, and their selections to be accepted. The keywords whose selections have been accepted can be stored as meeting information.

[0073] <Audio Analysis Processing> In step S106, the control unit 104 of the server 10 performs analysis processing on the audio data stored in the audio data item of the conference table 1014. The control unit 104 of the server 10 performs an audio extraction step to extract multiple segment audio data for each utterance segment from the audio data. The audio extraction step includes a step of identifying the speaker for each of the multiple segment audio data. Specifically, the control unit 104 of the server 10 acquires (receives) the conference ID and audio data (including video data) stored in the conference table 1014. From the acquired (received) audio data and video data, the control unit 104 of the server 10 detects segments (utterance segments) in which utterances are continuously present, and extracts audio data and video data for each utterance segment as segment audio data and segment video data, respectively. For example, segment audio data and segment video data may be extracted by dividing the audio data and video data into silent segments where there are no utterances. Alternatively, the spoken content contained in the audio and video data may be divided into sentence units such as segments, sentences, and paragraphs, and segment audio data and segment video data may be extracted. Each segment audio data and segment video data is associated with the speaker's user ID, the start date and time of the speech segment, and the end date and time of the speech segment.

[0074] The control unit 104 of the server 10 executes a text generation step for each of the multiple section audio data extracted in the speech extraction step, generating multiple section speech texts, which are text information of what the speaker said. Specifically, the control unit 104 of the server 10 performs text recognition on the spoken content of the extracted section audio data and section video data to convert the section audio data and section video data into section speech texts, which are written characters (text), and transcribes them into text. The specific method of text recognition is not particularly limited. For example, it may be converted using signal processing technology, machine learning or deep learning using AI (artificial intelligence), etc.

[0075] The control unit 104 of the server 10 stores the dialogue ID to be processed, the speaker's user ID (user ID 2011), start date and time, end date and time, section audio data, section video data, and section text reading in the dialogue ID, speaker ID, start date and time, end date and time, section audio data, section video data, and section text reading fields of the new record in the audio section table 1016, respectively.

[0076] The audio segment table 1016 stores the segment text for each utterance segment of the audio data, associated with the start date and time and the speaker, as continuous time-series data. By checking the segment text stored in the audio segment table 1016, the user can confirm the content of the dialogue as text information without having to check the content of the audio data.

[0077] Furthermore, during the text recognition process, it is also possible to configure the system to exclude information that is meaningless to understanding the conversation between the user and the customer, such as fillers included in the text, and to store the speech recognition information in the speech section table 1016. In addition, when converting section speech data and section video data into section spoken text, some words may be excluded or multiple words may be merged to create the text, or the data may be converted into text that includes keywords in the form of a summary or minutes.

[0078] The control unit 104 of the server 10 may identify the participant ID of the participant corresponding to the speaker for each item of audio data stored in the conference table 1014, section audio data, section video data, section text (hereinafter referred to as audio data, etc.) stored in the audio section table 1016, and store the participant associated with the audio data, etc. For example, the control unit 104 of the server 10 may identify the participant corresponding to the speaker of the audio data based on information that identifies the user or customer of the participant, which is output by inputting the audio data, etc. into an arbitrary machine learning model, deep learning model, artificial intelligence model, etc. For example, for each utterance section of the audio data, the participant who uttered that utterance section may be identified.

[0079] The control unit 104 of the server 10 may transmit information identifying participants in voice data, etc., to an external customer management service. The customer management service stores the voice data, etc., information in association with customer information stored in the customer management service based on the received information. This allows the customer management service to manage the content spoken by each customer. Alternatively, the customer table 1021 of the server 10 may store the voice data, etc., information in association with each customer.

[0080] <CRM Linkage Processing> In step S107, the control unit 104 of the server 10 executes a CRM linkage step in which it stores meeting information such as audio data and audio segment information in association with customer information in the CRM system 60. Specifically, the control unit 104 of the server 10 sends a request to the CRM system 60 to store meeting information, including the customer ID of the participant, the meeting ID, segment ID, etc. When the CRM system 60 receives the request, it identifies the record in which the customer information is stored based on the customer ID, and stores the meeting ID and segment ID in that record. As a result, meeting information such as audio data and audio segment information of the meeting conducted in the meeting processing is associated with the customer information of the participant stored in the CRM system 60. This allows the user to check the meeting information of meetings in which the customer participated as a participant, even from the customer information.

[0081] <Schedule Processing> This process involves registering schedule information for meetings, etc.

[0082] <Overview of Schedule Processing> Schedule processing is a series of processes that involves opening the schedule screen, accepting input of the schedule date and time, accepting input of participant information, and storing the schedule information.

[0083] <Details of Schedule Processing> The details of the schedule processing are explained below.

[0084] In step S301, the user opens the schedule page by operating the input device 206 of the user terminal 20, running a browser application or the like, and entering the URL of a web page (schedule page) for executing the schedule processing. The control unit 204 of the user terminal 20 sends a request to the server 10 that includes the user ID 2011 to open the schedule page. Alternatively, the user may open the schedule page by operating the input device 206 of the user terminal 20 and opening the URL of a website such as an external calendar service.

[0085] When server 10 receives a request, it generates a schedule page and sends it to user terminal 20. The control unit 204 of user terminal 20 displays and presents the schedule page on the user terminal 20's display 2081. The schedule page includes input fields for start date and time, end date and time, participants, location, and schedule details.

[0086] In step S302, the user operates the input device 206 of the user terminal 20 and enters the start date and end date and time of the schedule (event) into the input fields for the start date and end date and time included in the schedule page.

[0087] In step S303, the user operates the input device 206 of the user terminal 20 and enters user and customer information for the participants into the participant input fields included on the schedule page. For example, the control unit 204 of the user terminal 20 may be configured to display a list of multiple users stored in the user table 1012 and select a predetermined user from among them. The control unit 204 of the user terminal 20 may also be configured to display a list of multiple customers stored in the customer table 1021 and select a predetermined customer from among them. Similarly, it may be configured to display a list of multiple customers stored in an external customer management service and select a predetermined customer from among them. The participants may include the user themselves.

[0088] In step S304, the user operates the input device 206 of the user terminal 20 and enters information about the location and schedule content into the input fields for location and schedule content included on the schedule page. The user operates the input device 206 of the user terminal 20 and selects the "Save" button included on the schedule page. The control unit 204 of the user terminal 20 sends the information entered in the input fields for start date and time, end date and time, participants, location, and schedule content to the server 10. The control unit 104 of the server 10 stores the received start date and time, end date and time, participants, location, and schedule content in the respective fields of the schedule table 1022. As a result, information about the schedule (event) is stored in the schedule table 1022.

[0089] In addition, the user terminal 20 may be configured to have some or all of the items (columns) of the group table 1013, meeting table 1014, label table 1015, voice interval table 1016, customer table 1021, and schedule table 1022 stored in the storage unit 101 of the server 10. For example, the user terminal 20 may be configured to have some or all of the control unit of the control unit 104 of the server 10 according to this disclosure. In this case, the server 10 may be omitted to configure the information processing system. In this disclosure, the control unit 104 for executing the text generation step, which generates text for each voice data extracted in the voice analysis processing (S106), is shown as being provided in the server 10, but is not limited to this. For example, the user terminal 20 may be configured to have a conversion unit that converts the voice data acquired by the control unit 204 into text. In other words, the control unit 204 of the user terminal 20 may execute the voice analysis processing (step S106). In this case, in the transmission step performed in step S105 shown in Figure 11, the control unit 204 of the user terminal 20 transmits the text information converted by the conversion unit along with at least a portion of the meeting information generated in the generation step. In other words, the control unit 204 of the user terminal 20 may be configured to transmit the meeting information, such as the audio data and audio segment data, to the server 10 after performing the audio analysis processing in step S106, along with the generation and transmission of the meeting information in step S105. Furthermore, the control unit 104 of the server 10 and the control unit 204 of the user terminal 20 may exclude some words or phrases from the text during conversion, or merge multiple words or phrases, or convert it into a short text that includes keywords extracted in the form of a summary or minutes. In addition, the control unit 104 of the server 10 and the control unit 204 of the user terminal 20 may apply arbitrary text processing to the audio spoken by participants in the meeting so that it can be easily handled for analysis or reference by other users at a later date. This makes it possible to use the meeting information more favorably at a later date.

[0090] <Basic Hardware Configuration of Computer> Figure 14 is a block diagram showing the basic hardware configuration of computer 90. Computer 90 includes at least a processor 901, main memory 902, auxiliary storage 903, and a communication IF 991 (interface). These are electrically connected to each other by a communication bus 921.

[0091] The processor 901 is hardware for executing the instruction set described in a program. The processor 901 consists of an arithmetic unit, registers, peripheral circuits, etc.

[0092] The main memory 902 is for temporarily storing programs and data processed by programs, etc. For example, it is a volatile memory such as DRAM (Dynamic Random Access Memory).

[0093] The auxiliary storage device 903 is a storage device for storing data and programs. Examples include flash memory, HDD (Hard Disc Drive), magneto-optical disk, CD-ROM, DVD-ROM, semiconductor memory, etc.

[0094] A communication interface (IF991) is an interface for inputting and outputting signals for communication with other computers via a network using wired or wireless communication standards. The network consists of various mobile communication systems, such as the Internet, LANs, and wireless base stations. For example, networks include 3G, 4G, and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks (e.g., Wi-Fi®) that can connect to the Internet via designated access points. When connecting wirelessly, communication protocols include, for example, Z-Wave®, ZigBee®, and Bluetooth®. When connecting via wired connections, the network also includes connections made directly via USB (Universal Serial Bus) cables, etc.

[0095] Furthermore, by distributing all or part of each hardware configuration across multiple computers 90 and connecting them to each other via a network, a computer 90 can be virtually realized. Thus, the concept of computer 90 includes not only a computer 90 housed in a single enclosure or case, but also a virtualized computer system.

[0096] <Basic Functional Configuration of Computer 90> The functional configuration of the computer realized by the basic hardware configuration of computer 90 (Figure 14) is described below. The computer comprises at least one functional unit: a control unit, a memory unit, and a communication unit.

[0097] Furthermore, the functional units of computer 90 can also be realized by distributing all or part of each functional unit across multiple computers 90 interconnected via a network. The concept of computer 90 includes not only a single computer 90 but also a virtualized computer system.

[0098] The control unit is realized when the processor 901 reads various programs stored in the auxiliary storage device 903, loads them into the main memory device 902, and executes processing according to those programs. The control unit can realize various functional units that perform information processing depending on the type of program. In this way, the computer is realized as an information processing device that performs information processing.

[0099] The functions realized by the components described herein may be implemented in a circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), CPUs (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to realize the described functions. A processor, including transistors and other circuits, is considered a circuitry or processing circuitry. A processor may be a programmed processor that executes a program stored in memory. In this specification, circuitry, unit, and means are hardware programmed to realize or perform the described functions. Such hardware may be any hardware disclosed herein, or any hardware known to be programmed to realize or perform the described functions. If such hardware is a processor that is considered a type of circuitry, then such circuitry, means, or unit is a combination of hardware and software used to constitute such hardware and / or processor.

[0100] The memory unit is implemented by a main memory 902 and an auxiliary memory 903. The memory unit stores data, various programs, and various databases. The processor 901 can also reserve memory areas corresponding to the memory unit in the main memory 902 or the auxiliary memory 903 according to the program. The control unit can also cause the processor 901 to perform addition, update, and deletion operations on data stored in the memory unit according to the various programs.

[0101] The term "database" refers to a relational database, which is used to manage and associate data sets called tables and masters, which are structured in a tabular format defined by rows and columns. In a database, tables are called tables, masters are called masters, the columns of tables are called columns, and the rows of tables are called records. In a relational database, relationships can be established and linked between tables and masters. Typically, each table and each master has a primary key column to uniquely identify a record, but setting a primary key for a column is not mandatory. The control unit can cause the processor 901 to add, delete, and update records in specific tables and masters stored in the storage unit according to various programs. Furthermore, by storing data, various programs, and various databases in the storage unit, the information processing device and information processing system described in this disclosure can be considered manufactured.

[0102] Furthermore, the databases and masters in this disclosure may include any data structures (lists, dictionaries, associative arrays, objects, etc.) in which information is structurally defined. Data structures also include data that can be considered as data structures by combining data with functions, classes, methods, etc., written in any programming language.

[0103] The communication unit is implemented by the communication IF 991. The communication unit implements the function of communicating with other computers 90 via the network. The communication unit can receive information transmitted from other computers 90 and input it to the control unit. The control unit can cause the processor 901 to perform information processing on the received information according to various programs. The communication unit can also transmit information output from the control unit to other computers 90.

[0104] <Note> The matters described in each of the above embodiments are noted below.

[0105] (Note 1) A program to be executed by an information processing terminal for storing information in a face-to-face meeting consisting of multiple participants, comprising a processor, a memory unit, and a microphone, wherein the processor executes: an audio acquisition step (S104) for acquiring audio information input from the microphone in a face-to-face meeting; a schedule acquisition step (S102) for acquiring predetermined schedule information relating to the face-to-face meeting; a participant input step (S103) for receiving input of participant information relating to the participants of the face-to-face meeting; a generation step (S105) for generating meeting information relating to the face-to-face meeting based on the schedule information acquired in the schedule acquisition step and the participant information received in the participant input step; and a transmission step (S105) for transmitting the audio information acquired in the audio acquisition step and the meeting information generated in the generation step in association with each other. This makes it possible to store audio information discussed in a face-to-face meeting in association with schedule information and participant information. This allows for the appropriate management of information in a face-to-face meeting.

[0106] (Note 2) The program as described in Note 1, wherein the information processing terminal is an information processing terminal that stores information in a business negotiation consisting of the user of the information processing terminal and one or more of the user's customers, the schedule acquisition step (S102) is a step of acquiring schedule information from a schedule management service connected to the information processing terminal via a network, the participant input step (S103) is a step of receiving customer information of one or more customers from the user as participant information, and the transmission step (S105) is a step of transmitting meeting information to a dialogue management service connected to the information processing terminal via a network. This makes it possible to store voice information discussed in face-to-face business negotiations in association with schedule information and customer information. The input burden on the user can be reduced by using schedule information stored in an external schedule management service.

[0107] (Note 3) The program as described in Note 2, wherein the processor performs a user storage step (S103) in which the user's user identification information is stored in a storage unit, a customer candidate acquisition step (S103) in which one or more customer candidate information is acquired from a customer management service connected to the information processing terminal via a network based on the user identification information, and a customer candidate presentation step (S103) in which the one or more customer candidate information acquired in the customer candidate acquisition step is presented to the information processing terminal, and the participant input step (S103) is a step in which the user selects customer information for one or more customers from the one or more customer candidate information presented in the customer candidate presentation step, thereby accepting input of participant information relating to the one or more customers.

[0108] (Note 4) The program as described in Note 2, wherein the processor performs a user storage step (S102) in which the user's user identification information is stored in a storage unit; a schedule candidate acquisition step (S102) in which one or more schedule candidate information is acquired from a schedule management service connected to the information processing terminal via a network based on the user identification information; a schedule candidate presentation step (S102) in which one or more schedule candidate information acquired in the schedule candidate acquisition step is presented to the information processing terminal; and a schedule selection step (S102) in which the user selects a predetermined schedule information from the one or more schedule candidate information presented in the schedule candidate presentation step, the schedule acquisition step (S102) being a step to acquire the predetermined schedule information received in the schedule selection step.

[0109] (Note 5) The program as described in Note 4, wherein the processor performs a customer identification step (S103) in which it compares participant information included in predetermined schedule information with customer information stored in a customer management service connected to the information processing terminal via a network to identify predetermined customer information, and the participant input step (S103) is a step in which the input of participant information relating to a customer identified based on the predetermined customer information identified in the customer identification step is accepted. This reduces the input burden on the user by using customer information stored in Five's schedule management service and an external customer management service (CRM, etc.).

[0110] (Note 6) The program described in Note 1 is a step in which the schedule acquisition step (S102) acquires schedule information based on at least one of the following: the date and time of the face-to-face meeting, the location of the meeting, and the participant information received in the participant input step. This makes it possible to acquire schedule information that can be identified based on information such as the date and time of the face-to-face meeting, the location of the meeting, and the participants, thereby reducing the input burden on the user.

[0111] (Note 7) The program described in Note 1, wherein the processor performs a voice acquisition termination step (S104) in which it terminates the acquisition of voice information, and an input prompting step (S103) in which, if participant information has not been entered after the completion of voice information acquisition, it prompts the user to enter participant information.

[0112] (Note 8) The program described in Note 1 is a step in which the generation step (S105) generates meeting information based on the meeting title, date and time, location, and participant information included in the schedule information. This makes it possible to generate meeting information based on the meeting title, date and time, location, and participant information included in the schedule information.

[0113] (Note 9) The program as described in Note 1, wherein the processor performs a display step (S105) in which it displays keywords extracted based on the voice information acquired in the voice acquisition step on an information processing terminal, and a keyword selection step (S105) in which it receives selection operations for the keywords displayed in the display step from at least some of the multiple participants, and the generation step (S105) is a step in which it generates meeting information that includes the keywords selected in the keyword selection step. This makes it possible to present keywords extracted during the meeting to the participants and to accept their selections. The keywords that have been accepted for selection can be stored as meeting information.

[0114] (Note 10) The program described in Note 1 includes a transmission step (S105) which notifies other members of the group to which the user of the information processing terminal belongs that voice information and meeting information have been transmitted. This allows the voice information and meeting information to be shared with other members of the group to which the user belongs.

[0115] (Note 11) A method to be performed on an information processing device comprising a processor and a memory unit, wherein the processor performs all the steps performed in the invention according to any of Notes 1 to 10. This makes it possible to store audio information discussed in a face-to-face meeting in association with schedule information and participant information. This allows for the appropriate management of information from face-to-face meetings.

[0116] (Note 12) An information processing apparatus comprising a processor and a memory unit, wherein the processor performs all the steps performed in the invention according to any of Notes 1 to 10. This makes it possible to store audio information discussed in a face-to-face meeting in association with schedule information and participant information. Information from face-to-face meetings can be managed appropriately.

[0117] (Note 13) A system comprising means for performing all steps carried out in any of the inventions described in Notes 1 to 10. This allows audio information discussed in a face-to-face meeting to be stored in association with schedule information and participant information. Information from face-to-face meetings can be managed effectively.

[0118] 1 System, 10 Server, 101 Storage Unit, 104 Control Unit, 106 Input Device, 108 Output Device, 20 User Terminal, 201 Storage Unit, 204 Control Unit, 206 Input Device, 208 Output Device, 60 CRM System, 601 Storage Unit, 604 Control Unit, 606 Input Device, 608 Output Device

Claims

1. A program to be executed by an information processing terminal for storing information in a face-to-face meeting consisting of multiple participants, comprising a processor, a memory unit, and a microphone, wherein the processor executes: an audio acquisition step of acquiring audio information input from the microphone in the face-to-face meeting; a schedule acquisition step of acquiring predetermined schedule information relating to the face-to-face meeting; a participant input step of receiving input of participant information relating to the participants of the face-to-face meeting; a generation step of generating meeting information relating to the face-to-face meeting based on the schedule information acquired in the schedule acquisition step and the participant information received in the participant input step; and a transmission step of transmitting the audio information acquired in the audio acquisition step and the meeting information generated in the generation step in association with each other.

2. The program according to claim 1, wherein the information processing terminal is an information processing terminal that stores information in a business negotiation consisting of a user of the information processing terminal and one or more of the user's customers, the schedule acquisition step is a step of acquiring the schedule information from a schedule management service connected to the information processing terminal via a network, the participant input step is a step of receiving customer information of one or more of the customers from the user as participant information, and the transmission step is a step of transmitting the meeting information to a dialogue management service connected to the information processing terminal via a network.

3. The program according to claim 2, wherein the processor performs a user storage step of storing the user's user identification information in a storage unit; a customer candidate acquisition step of acquiring one or more customer candidate information from a customer management service connected to the information processing terminal via a network based on the user identification information; and a customer candidate presentation step of presenting the one or more customer candidate information acquired in the customer candidate acquisition step to the information processing terminal, and the participant input step is a step of receiving input of participant information relating to one or more customers by receiving a selection operation from the user for customer information of one or more customers from the one or more customer candidate information presented in the customer candidate presentation step.

4. The program according to claim 2, wherein the processor performs: a user storage step of storing the user's user identification information in a storage unit; a schedule candidate acquisition step of acquiring one or more schedule candidate information from a schedule management service connected to the information processing terminal via a network based on the user identification information; a schedule candidate presentation step of presenting the one or more schedule candidate information acquired in the schedule candidate acquisition step to the information processing terminal; and a schedule selection step of receiving a selection operation from the user for predetermined schedule information from the one or more schedule candidate information presented in the schedule candidate presentation step, wherein the schedule acquisition step is a step of acquiring the predetermined schedule information received in the schedule selection step.

5. The program according to claim 4, wherein the processor performs a customer identification step of comparing participant information included in the predetermined schedule information with customer information stored in a customer management service connected to the information processing terminal via a network, and identifying predetermined customer information, and the participant input step is a step of receiving input of participant information relating to a customer identified based on the predetermined customer information identified in the customer identification step.

6. The program according to claim 1, wherein the schedule acquisition step is a step of acquiring schedule information based on at least one of the date and time of the face-to-face meeting, the location of the meeting, and the participant information received in the participant input step.

7. The program according to claim 1, wherein the processor performs a voice acquisition termination step of terminating the acquisition of voice information, and an input prompting step of prompting the input of participant information if participant information has not been entered after the completion of voice information acquisition.

8. The program according to claim 1, wherein the generation step is a step of generating meeting information based on the meeting title, date and time, location, and participant information included in the schedule information.

9. The program according to claim 1, wherein the processor performs a display step of displaying keywords extracted based on the voice information acquired in the voice acquisition step on the information processing terminal, and a keyword selection step of receiving selection operations from at least some of the plurality of participants for the keywords displayed in the display step, and the generation step is a step of generating the meeting information including the keywords selected in the keyword selection step.

10. The program according to claim 1, wherein the transmission step includes a notification step of notifying other members of the group to which the user of the information processing terminal belongs that the voice information and the meeting information have been transmitted.

11. A method to be performed on an information processing apparatus comprising a processor and a storage unit, wherein the processor performs all steps performed in any of the inventions according to claims 1 to 10.

12. An information processing apparatus comprising a processor and a storage unit, wherein the processor performs all steps performed in any of the inventions according to claims 1 to 10.

13. A system comprising means for performing all steps performed in any of the inventions according to claims 1 to 10.

Images