Information processing device, information processing method, program, and recording medium

Abstract

To improve the ease of subsequently confirming interview details about a minute book to be created on the basis of recorded data of an interview.SOLUTION: An information processing device according to the present invention includes a data generation processing part for generating minute book data added with a voice link showing a link to a voice data portion of a speech place corresponding to summary sentences in recorded data to the summary sentences as minute book data including the summary sentences when an instruction to turn the minute book data created by transferring the summary sentences into data in a minute book creation screen including a summary list display area for displaying the summary sentences being data generated on the basis of the recorded data of an interview and obtained by converting a summary of topics in the interview into text in a list, and a minute book creation area in which the summary sentences displayed in the summary list display area can be transferred as a constitution sentence of the minute book of the interview and a sentence as the minute book can be created.SELECTED DRAWING: Figure 20

Description

[Technical field]

[0001] The present invention relates to an information processing device and method, a program, and a recording medium, and in particular to a technology for improving the ease of subsequent confirmation of the details of an interview in minutes created based on recorded data of the interview. [Background technology]

[0002] For example, in the case of business negotiations conducted as part of a company's sales activities, or other interviews that involve dialogue between multiple speakers, it is common practice to prepare minutes summarizing the contents of the meetings.

[0003] Furthermore, as described in Patent Document 1 below, there is a technique for converting the speech of each person in a business negotiation into text based on audio recording data of the business negotiation, and for creating a summary of the content of the business negotiation based on the converted speech data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7223469 Summary of the Invention [Problem to be solved by the invention]

[0005] Here, suppose that a user who actually conducted an interview finds a note of interest in the minutes when checking the contents of the interview after the fact by referring to the minutes. In this case, it would be sufficient to understand the details from the text of the note, but it is possible that the details cannot be understood from the text alone. It is possible to save audio recordings of the interview as in Patent Document 1, and if the audio recordings are available, the user can play back the relevant part of the audio recordings to check the details. However, it takes a lot of work for the user to search for the relevant part in the recorded data using only the text in the minutes as a clue.

[0006] The present invention has been made in consideration of the above circumstances, and has an object to improve the ease of subsequent confirmation of the details of an interview in minutes created based on audio recording data of the interview. [Means for solving the problem]

[0007] The information processing device of the present invention has a minutes creation screen which includes a main point list display area which displays a list of main point sentences, which are text data of the main points discussed in the interview, generated based on recorded data of an interview, and a minutes creation area in which the main point sentences displayed in the main point list display area can be transcribed as constituent sentences of the minutes of the interview to create text for the minutes.When an instruction is given to digitize the minutes created by transcribing the main point sentences, the information processing device is configured to generate minutes data including the main point sentences, in which an audio link is added to the main point sentences, indicating a link to the audio data portion of the speech portion in the recorded data that corresponds to the main point sentence. By generating minutes data in the form in which audio links are attached to key sentences as described above, when a key sentence is specified in the minutes, it is possible to smoothly play back the audio data of the speech point corresponding to the specified key sentence by playing back the audio data portion indicated by the audio link. Effect of the Invention

[0008] According to the present invention, it is possible to improve the ease of subsequent confirmation of the details of an interview in minutes created based on audio recording data of the interview. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of a minutes-taking support system according to an embodiment. [Diagram 2]FIG. 1 is a block diagram illustrating an example of a hardware configuration of an information processing device according to an embodiment. [Diagram 3] FIG. 2 is a block diagram showing an example of the hardware configuration of a device for displaying a minutes-of-meeting creation screen and a minutes-of-meeting management server in an embodiment. [Figure 4] 13 is a diagram showing an example of a registration screen for interview participant information in an embodiment. FIG. [Diagram 5] FIG. 13 is a diagram showing an example of a recording list screen in the embodiment. [Figure 6] FIG. 13 is a diagram showing an example of a minutes creation screen (before speaker / group setting and analysis processing are performed) in an embodiment. [Figure 7] FIG. 13 is a diagram showing an example of a setting screen for setting speaker groups in the embodiment. [Figure 8] 13 shows an example of information displayed in the speaker / group setting area after information has been set. FIG. [Figure 9] FIG. 13 is a diagram showing an example of a minutes creation screen (after setting speakers and groups) in an embodiment. [Figure 10] 13 is a diagram showing an example of display of analysis result text data on the minutes creation screen. FIG. [Figure 11] 11 is an explanatory diagram of a display method for a subcategory of gist sentences in a gist list display area in the embodiment. FIG. [Figure 12] FIG. 13 is an explanatory diagram of a display method for subcategory gist sentences in the gist list display area of ​​the embodiment. [Figure 13] 11 is an explanatory diagram of an example of an operation for instructing audio playback of an utterance portion corresponding to a gist sentence. FIG. [Figure 14] FIG. 13 is a diagram for explaining the transcription of a gist sentence into a minutes-taking area. [Figure 15] FIG. 13 is a diagram for explaining the transcription of a summary sentence into the minutes creation area. [Figure 16] 13 is an explanatory diagram of uploading of minutes data with an audio link and processing related to the uploaded minutes data. FIG. [Figure 17] 13 is a diagram for explaining an example of an audio playback instruction operation while viewing minutes. FIG. [Figure 18] FIG. 2 is an explanatory diagram of functions of an information processing apparatus according to an embodiment; [Figure 19] 10 is an explanatory diagram of a process for generating key sentences for each category according to an embodiment, and an audio link that is assigned to each generated key sentence. FIG. [Figure 20] 11 is a flowchart of a process for generating minutes data in an embodiment. [Figure 21] 10 is a flowchart of a process relating to access authentication in the embodiment. [Figure 22] FIG. 13 is a diagram illustrating a configuration example of a modified example of a minutes-taking support system. [Diagram 23] FIG. 11 is an explanatory diagram of an example of contract content management information. [Figure 24] FIG. 13 is an explanatory diagram of an example of a link button to a location where minutes are stored. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present invention will be described in the following order. <1. System configuration example> <2. Example of each device configuration> <3. Flow from interview to creating minutes and example of display screen> <4. Example of functional configuration> <5. Processing Procedure> <6. Variations> <7. Summary of the embodiment>

[0011] <1. System configuration example> FIG. 1 is a block diagram showing an example of the configuration of a minutes-of-meeting support system 100 including an information processing device according to an embodiment of the present invention. As shown in the figure, the minutes-of-meeting support system 100 as an embodiment includes a voice management server 1, an information processing device 20, and a minutes-of-meeting management server 30. The voice management server 1, the information processing device 20, and the minutes-of-meeting management server 30 are configured with a microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The voice management server 1, the information processing device 20, and the minutes-of-meeting management server 30 are capable of mutual data communication via a network NT, which is a predetermined communication network such as the Internet.

[0012] The minutes-taking support system 100 of the embodiment has a function of recording the interview, a function of analyzing the contents of the interview based on the recorded interview data, and a function of presenting text data showing the results of the analysis (hereinafter referred to as "analysis result text data") to the user on a screen. The user can use the analysis result text data presented on the screen to create minutes of the interview.

[0013] In this embodiment, a "business negotiation" between a sales representative Hs and a customer Hc is taken as an example of an "interview." The sales representative Hs is a person who participates in business negotiations, performs sales activities, and takes minutes of the business negotiations. The sales representative Hs can be considered as the host in a business meeting, while the customer Hc can be considered as the guest in the meeting.

[0014] The information processing device 20 is a computer device such as a PC (personal computer), a tablet terminal, or a smartphone used by the sales representative Hs. In this example, the business negotiation is conducted face-to-face between a sales representative Hs and a customer in a real space, such as a conference room, in which the information processing device 20 is placed. Here, the participants in the interview are multiple sales representatives Hs and multiple customers Hc (specifically, in the example, two of each), but the business negotiation may be conducted with the participation of at least one sales representative Hs and one customer Hc.

[0015] The information processing device 20 has a function of recording the business negotiations. The recorded data of the business negotiations obtained by the recording function is transmitted to the voice management server 1 via the network NT, and is stored (accumulated) in the voice management server 1.

[0016] The voice management server 1 has a function of converting recorded data of an interview (a business negotiation in this example) into text, i.e., a function of generating text data by transcribing (writing down) the spoken portion of the interview, and an analysis function of performing an analysis process related to the contents of the interview based on the text data obtained by the text conversion function (hereinafter referred to as "interview text data"). Specifically, the analysis function in this embodiment includes a function of performing a summary creation process that creates a summary of the interview, a main point list creation process that creates a list of main points of the contents (topics) of the interview, and a question and answer identification process that identifies the question portion and the answer portion of the interview.

[0017] In the minutes support system 100, text data showing the analysis results obtained by the analysis function of the voice management server 1 as described above (hereinafter referred to as "analysis result text data") is displayed on a screen to the sales representative Hs as a user. Specifically, the analysis result text data showing the analysis results as a summary, a list of key points, questions and answers, etc. as described above is displayed on a screen and presented to the user. In this example, the screen display of such analysis result text data is performed by the information processing device 20.

[0018] The user can efficiently create minutes by using the analysis result text data of the interview presented as described above. Specifically, the user can transcribe (copy) text data such as a summary presented as the analysis result text data and use it to create minutes, which eliminates the need to create minutes from scratch, improving the efficiency of creating minutes.

[0019] In this embodiment, the analysis result text data is presented by displaying one minutes-creation screen (editing screen Ge, described later) including an analysis result display area for displaying the analysis result text data and a minutes-creation area for creating minutes. Here, the minutes-creation area is an area in which the analysis result text data (particularly, data on main points, described later) can be transcribed as constituent sentences of the minutes, and sentences for the minutes can be created.

[0020] Since a minutes creation screen including not only an analysis result display area but also a minutes creation area is displayed, it is possible to eliminate the need to open a separate editing screen when creating minutes using the analysis result text data. Therefore, the effort required for the user to create minutes of an interview can be reduced, and the ease of creating minutes can be improved. In particular, when the interview is a business meeting as in this example, the improved ease of creating minutes can improve the efficiency of business negotiation activities and contribute to an improvement in the success rate.

[0021] <2. Example of each device configuration> An example of the configuration of each device constituting the minutes-taking support system 100 according to the embodiment will be described with reference to FIGS.

[0022] FIG. 2 is a block diagram showing an example of the hardware configuration of the voice management server 1. As shown in FIG. 2, the voice management server 1 includes a CPU 1a, a ROM 2, and a RAM 3. The CPU 1a functions as an arithmetic processing unit that performs various processes, and executes the various processes according to a program stored in the ROM 2 or a program loaded from the storage unit 9 to the RAM 3. The RAM 3 also stores data and the like required for the CPU 1a to execute the various processes, as appropriate.

[0023] The CPU 1 a, the ROM 2, and the RAM 3 are connected to each other via a bus 4.

[0024] In addition, the bus 4 is connected to an analysis unit 1b. The analysis unit 1b is configured as, for example, a signal processor or an AI (Artificial Intelligence) processor, and performs various analytical processes related to the contents of the interview based on the recorded data of the interview. Specifically, in this example, the analysis unit 1b performs at least the summary creation process, the main point list creation process, and the question and answer identification process described above. The details of the analysis process executed by the analysis unit 1b, including the summary creation process, the main point list creation process, and the question and answer identification process, will be described later.

[0025] Here, the analysis process by the analysis unit 1b may be executed by the CPU 1a, particularly when the analysis process is performed as a rule-based process rather than an AI process.

[0026] In addition, an input / output interface (I / F) 5 is also connected to the bus 4 .

[0027] To the input / output interface 5, an input section 6 including an operator and an operating device is connected. For example, the input unit 6 may be various types of operators or operation devices such as a keyboard, a mouse, keys, a dial, a touch panel, a touch pad, or a remote controller. An operation by a user is detected by the input unit 6, and a signal corresponding to the input operation is interpreted by the CPU 1a.

[0028] Further, the input / output interface 5 is connected, either integrally or separately, to a display unit 7 formed of an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) panel, and an audio output unit 8 formed of a speaker, etc. The display unit 7 is used to display various types of information, and is configured, for example, by a separate display device connected to the voice management server 1, or a display device provided in the housing of the voice management server 1, or the like.

[0029] The display unit 7 displays various information on the display screen based on instructions from the CPU 1 a. The display unit 7 can display various operation menus, icons, messages, etc., that is, GUI (Graphical User Interface), based on instructions from the CPU 1 a.

[0030] The input / output interface 5 may be connected to a storage unit 9 and a communication unit 10 . The storage unit 9 is configured with a hard disk drive (HDD) or a solid state drive (SSD) and stores various types of information.

[0031] The communication unit 10 performs communication processing via a transmission path such as the Internet, and communication with various devices via wired / wireless communication, bus communication, and the like.

[0032] A drive 12 is also connected to the input / output interface 5 as required, and a removable recording medium 11 such as a memory card or an optical disk is appropriately attached thereto.

[0033] The drive 12 allows data files such as programs used in each process to be read from the removable recording medium 11. The read data files are stored in the storage unit 9, and images and sounds contained in the data files are output on the display unit 7 and the audio output unit 8. In addition, the computer programs and the like read from the removable recording medium 11 are installed in the storage unit 9 as necessary.

[0034] In the voice management server 1 having the above hardware configuration, for example, software for the processing of this embodiment can be installed via network communication by the communication unit 10 or the removable recording medium 11. Alternatively, the software may be stored in the ROM 2, the storage unit 9, etc. in advance.

[0035] The CPU 1a performs processing operations based on various programs, thereby executing information processing and communication processing required to realize the processing of the voice management server 1 as an embodiment described later.

[0036] The voice management server 1 is not limited to being configured as a single computer device as shown in Fig. 2, but may be configured as a system of multiple computer devices. The multiple computer devices may be systemized using a LAN (Local Area Network) or the like, or may be located in a remote location using a VPN (Virtual Private Network) using the Internet or the like. The multiple computer devices may include computer devices as a server group (cloud) available through a cloud computing service.

[0037] FIG. 3 is a block diagram showing an example of the hardware configuration of the information processing device 20 and the minutes management server 30. As shown in FIG. In the following description, parts that are similar to parts that have already been described will be given the same reference numerals and description thereof will be omitted.

[0038] The difference from the voice management server 1 shown in FIG. 2 is that the voice management server 1 does not include the analysis unit 1b. Hereinafter, for convenience of explanation, the CPU 1a included in the information processing device 20 will be referred to as "CPU 20a," and the CPU 1a included in the minutes management server 30 will be referred to as "CPU 30a."

[0039] Furthermore, the information processing device 20 includes, as the input unit 6, not only an operation input unit for performing operation input, but also a voice input unit for inputting a voice signal detected by a microphone. In the information processing device 20, various types of operators and operation devices such as a keyboard, a mouse, a key, a touch panel, a touch pad, a dial, a remote controller, etc. are assumed as the operation input unit in the input unit 6. An operation is detected by the operation input unit, and a signal corresponding to the detected operation is interpreted by the CPU 20a. In the input unit 6 of the information processing device 20, the audio signal inputted by the audio input unit is stored in a storage device such as the storage unit 9 under the control of the CPU 20a. The audio signal inputted by the audio input unit may be converted from an electrical signal to sound via the audio output unit 8 and output (emitted as sound).

[0040] In addition, in the minutes management server 30, it is not essential that the input unit 6 includes a voice input unit.

[0041] The minutes management server 30 is not limited to being configured as a single computer device as shown in Fig. 3, but may be configured as a system of multiple computer devices. The multiple computer devices may be configured as a system using a LAN or the like, or may be located in a remote location using a VPN or the like using the Internet or the like. The multiple computer devices may include computer devices as a server group (cloud) available through a cloud computing service.

[0042] <3. Flow from interview to creating minutes and example of display screen> As described above, the business negotiation in this example is conducted face-to-face in real space between the sales representative Hs and the customer Hc, and the content of the conversation during the business negotiation is recorded by the information processing device 20 used by the sales representative Hs. In this example, a software program (hereinafter referred to as "recording software") for realizing various functions related to recording business negotiations is installed in the information processing device 20, and sales representative Hs conducts business negotiations with the recording software running. It is also possible to adopt a so-called SaaS (Software as a Service) type configuration in which recording software is stored on the voice management server 1 side and the software is provided as a service.

[0043] FIG. 4 shows an example of a participant information registration screen Gr displayed by the recording software. As shown in the figure, the registration screen Gr has input boxes b1, b2, and b3 for inputting the company name, name, and position of the customer Hc (customer) in the business negotiation, and an input box b4 for inputting the name of the sales representative Hs himself. Input boxes b1, b2, and b3 for customer Hc can be added by operating an add button B1 in the figure, making it possible to input information for a plurality of customers Hc. The registration screen Gr also includes a start button B2 for issuing an instruction to start recording.

[0044] In this registration screen Gr, the sales representative Hs inputs the company name, name, and job title of the customer Hc, as well as his own name, and operates the start button B2 to cause the information processing device 20 to start recording the business negotiation.

[0045] Although not shown in the figure, after recording has started, a recording stop button is displayed on the screen of the recording software, and the sales representative Hs can operate the stop button to instruct the information processing device 20 to stop recording the business negotiation (end instruction).

[0046] When an instruction to stop recording the business negotiation is given, the CPU 20a of the information processing device 20 performs processing to accumulate the recording data of the business negotiation and the participant information in the voice management server 1. Specifically, the recording data of the business negotiation, the information on the name and job title of the customer Hc entered on the registration screen Gr, and the information on the name of the sales representative Hs are transmitted to the voice management server 1 and stored therein.

[0047] The voice management server 1 associates the recorded data of the business negotiation sent from the information processing device 20 with participant information (information on the company name, name, position, and name of the sales representative Hs of the customer Hc) and stores them in a predetermined storage device, such as the memory unit 9. At this time, the voice management server 1 assigns a business negotiation ID, which serves as an identifier for the business negotiation, to each recording data of the business negotiation.

[0048] In the minutes-of-business support system 100 of this embodiment, a list of recording data of business negotiations stored in the voice management server 1 can be presented to a user as a sales representative Hs. Specifically, in this example, the above-mentioned recording software is provided with a function for accessing such a list of recorded data. In other words, in this case, the user as the sales representative Hs can receive a recording list screen Gi, which is a list screen of recorded data, by operating an access button (not shown) to the recorded data list displayed on the screen of the recording software.

[0049] FIG. 5 shows an example of the recording list screen Gi. As shown in the figure, the recording list screen Gi displays, for each business negotiation ID, date and time information of the business negotiation, a play button Bp, participant information, and an input memo field. In this example, the date and time information of the business negotiation includes the start time (recording start time) and end time (recording end time). The playback button Bp is a button for issuing an instruction to play back the recording data of the business negotiation associated with the business negotiation ID. The information entered on the registration screen Gr is displayed as the participant information. If there are any items on the registration screen Gr that have not been entered, a message to that effect is displayed. In the input memo field, memo information after the business negotiation can be input as text. The information input in the input memo field is stored in association with the business negotiation ID.

[0050] In addition, the recording list screen Gi can also display links to data on materials used in business negotiations (see "Main Materials" in the figure).

[0051] On the recording list screen Gi, a minutes creation button B3 for instructing the start of software as a minutes editor is displayed for each business negotiation ID. The minutes editor is software that displays the above-mentioned editing screen Ge (minutes creation screen), i.e., a screen for creating minutes that includes an analysis result display area and a minutes creation area, and performs processes related to creating minutes. The user as the sales representative Hs operates the minutes creation button B3 displayed on the recording list screen Gi for the business negotiation for which the minutes are to be created.

[0052] In this example, the minutes editor software is stored on the voice management server 1 side, and the minutes editor function is provided to users in the form of SaaS. Therefore, in response to the operation of the minutes creation button B3, the information processing device 20 requests the voice management server 1 to start the minutes editor service.

[0053] In the above example, the recording software is given the ability to access the recording list screen Gi, but access to the recording list screen Gi can be achieved by specifying the URI (Uniform Resource Identifier) ​​of the recording list screen Gi, regardless of the software used.

[0054] In addition, in the above example, the recorded data of the business negotiation is sent to the voice management server 1 in response to the recording stop operation, but the recorded data may also be sent to the voice management server 1 in response to an operation by the sales representative Hs after the business negotiation that is separate from the recording stop operation, or may be sent in real time during the business negotiation.

[0055] As described below, in this embodiment, the editing screen Ge displayed by the minutes editor is provided with a button that instructs the execution of analytical processing related to the contents of the business negotiation, such as creating the summary described above, and the voice management server 1 executes the analytical processing in response to the operation of the button.

[0056] Fig. 6 is a diagram showing an example of an edit screen Ge. Specifically, the edit screen Ge shown in Fig. 6 is the edit screen Ge before an instruction to execute an analysis process is given. In response to a request to start the minutes editor service from the information processing device 20 in response to the operation of the minutes creation button B3 described above, the voice management server 1 performs processing to display the editing screen Ge shown in Figure 6 on the information processing device 20. As will be described later, in the editing screen Ge of this example, a display area for analysis result text data (analysis result display area Aa, which will be described later) is displayed on the screen in response to an instruction to execute an analysis process.

[0057] The editing screen Ge in Figure 6 includes an utterance text display area At, a recording period display area Ap, a speaker / group setting area Ah, and a text editing area Ae, as well as an input box b5, a play button Bp, copy (transcription) buttons B10 and B11, an analysis execution button B13, a speaker setting instruction button B14, an output button B15, a clipboard copy button B16, an upload button B17, and a post button B18.

[0058] As shown in the figure, in the editing screen Ge of this example, the speech text display area At, the speaker / group setting area Ah, and the text editing area Ae are arranged from left to right on the screen in the following order: utterance text display area At, speaker / group setting area Ah, text editing area Ae. In this example, the recording period display area Ap is disposed below the spoken text display area At.

[0059] The spoken text display area At is an area in which text data of spoken portions in an interview are displayed for each speaker. In this embodiment, the text data of the spoken portion is generated by converting the recorded data of the interview into text by the voice management server 1. Here, the technology for converting the recorded voice data into text, i.e., the technology for converting voice data into text data, is a well-known technology, and a detailed description thereof will be omitted here.

[0060] In this example, the voice management server 1 divides the text data (text data of the utterance) into predetermined block units and manages them. The block units here are topic units. That is, the blocking process (text block process) of the text data in this example is a process of detecting topic boundaries from the text data. Such text block process can be performed using a machine-learned AI (artificial intelligence) model. Alternatively, the text block process can be realized as a rule-based process using keyword matching. For details, please refer to, for example, "JP Patent Publication No. 2023-34235" and the like. The text blocking process identifies at least the start time (start time of speech) of each block of text data. By identifying the start time of a block, a position (playback position) corresponding to the start time of the block is also identified for the recorded audio data (audio data).

[0061] Moreover, the voice management server 1 in this embodiment also performs speaker separation processing based on the recorded data. The speaker separation process is a process for identifying which speaker each speech portion is from. Specifically, for example, a voiceprint analysis process is performed on the recorded data, and a speaker ID is associated with each detected voiceprint. For example, speaker A, speaker B, speaker C, etc. are assigned as this speaker ID. Then, a process for identifying the speech section for each speaker is performed on the recorded data based on the voiceprint information for each speaker. By identifying the speech section for each speaker in this way for the recorded data, it is possible to identify the correspondence between the speech portion and the speaker for the speech text data obtained by converting the recorded data into text.

[0062] The CPU 1a in the voice management server 1 causes the information processing device 20 to display the text data of the speech part for each speaker in the speech text display area At based on the text data of each speech part obtained by the speaker separation processing as described above and information indicating the correspondence between each speech part and the speaker ID.

[0063] In this example, the text data of the speech portions of each speaker are displayed in chronological order from the top to the bottom of the screen. In this example, the speech text display area At displays speaker icons as identifiers of the speech parts for each speaker. At the stage of Fig. 6, the speaker icons display speaker IDs such as speakers A, B, and C that are assigned in the speaker separation process.

[0064] In addition, in the utterance text display area At at the stage of Fig. 6, all the text data of the utterance portion for each speaker is displayed close to either the left or right end of the utterance text display area At. Specifically, in this example, the text data is displayed close to the left end of the utterance text display area At as shown in the figure.

[0065] The recording period display area Ap is an area for displaying recording period display information that graphically represents the recording period of the interview. As shown in the figure, in the recording period display area Ap in this example, information showing the recording period of the interview in the form of a bar extending in the left-right direction is displayed as recording period display information. In the recording period display area Ap, a seek bar Sb for indicating the playback position of the recording data is displayed in association with the recording period display information. This seek bar Sb can be moved left and right, and the user can operate the play button Bp after moving the seek bar Sb to instruct the information processing device 20 to play the recording data from the playback position indicated by the position of the seek bar Sb.

[0066] In this embodiment, the recording period display information displays information showing the distribution of the speech period for each speaker. Specifically, the recording period display information in this case displays a bar indicating the recording period for each speaker, and displays the information on the speech period for each speaker in a different display format for each speaker. For example, it is possible to display the color or pattern of the speech period for each speaker differently. In this example, the bar for each speaker indicating the recording period displays the speaker icon of the corresponding speaker.

[0067] By displaying information showing the distribution of speaking periods for each speaker as recording period display information as described above, the user can easily grasp information such as which speaker spoke for how long during what period during the interview, and the amount of speech each speaker made, allowing the user to easily understand which speaker's speech they should pay attention to when creating the minutes. In addition, by displaying the speech period information for each speaker in a different display format for the recording period display information, the user can intuitively identify the speech period for each speaker. When a user wants to check the recording data for a specific speech portion of a specific speaker during the process of creating minutes, the ease of searching for that speech portion can be improved.

[0068] In addition, in the recording period display area Ap of this embodiment, the negotiation process period information Is is displayed in association with the recording period display information. Here, the negotiation process period information Is is information indicating the name and period of each negotiation process when the negotiation is divided into processes according to a predetermined criterion.

[0069] The voice management server 1 in this embodiment performs a process of identifying the negotiation process in the negotiation based on the recording data of the negotiation. The negotiation process here refers to the progress stage of the negotiation, and can be information classified into categories such as "ice breaker (small talk)", "needs analysis", "proposal / explanation", and "negotiation". In this example, the analysis unit 1b in the voice management server 1 performs the process of identifying the negotiation process. Specifically, the analysis unit 1b performs a process of identifying the negotiation process based on the text data obtained by converting the audio recording data of the negotiation into text. This process may be performed by an AI model generated by performing machine learning such as deep learning using the text data as learning input data and the correct answer data of the negotiation process as teacher data. Alternatively, it may be realized as a rule-based process in which frequently occurring words for each negotiation process are predetermined as process identification keywords, and the negotiation process is identified based on the results of matching with the process identification keywords.

[0070] The CPU 1a of the voice management server 1 performs processing to display the negotiation process period information Is on the information processing device 20 based on the period information for each negotiation process (information indicating the start time and end period of the process) obtained as a result of the processing to identify the negotiation process by the analysis unit 1b as described above. The figure shows an example of the negotiation process period information Is when five processes, ST1 to ST5, are identified as the negotiation process.

[0071] By displaying the negotiation process period information Is, i.e., the chronological information of the negotiation process, in association with the recording period display information as described above, when a user wishes to check the recording data of a specific speech portion of a specific speaker during the process of creating minutes, it is possible to improve the ease of searching for that speech portion.

[0072] Although the recording period display information has been described above as an example in which the recording period is displayed in the form of a bar, the recording period may also be displayed in another shape, such as an arc. When the recording period is represented in an arc shape, it is possible to use, as the playback position indication operator, an operator that performs rotational movement, rather than an operator that performs sliding movement, such as the seek bar Sb.

[0073] The speaker / group setting area Ah is an area for setting the name of each speaker and the group for each speaker. Group setting here means setting which group each speaker belongs to, with respect to two groups: the self-group, which is the group on the sales representative Hs (user)'s side, and the other-side group, which is the group on the sales representative Hs's other side (i.e., the customer Hc's side).

[0074] In the speaker / group setting area Ah, when group setting has not been performed, each speaker is displayed to indicate that she belongs to the other party's group, as shown in the figure. Note that the display mode when group setting has not been performed is not limited to this, and other display modes may be adopted, such as a display indicating that each speaker belongs to the user's group.

[0075] In the editing screen Ge of this example, when the user wishes to set the speaker's name and group, he or she operates the speaker setting instruction button B14. Then, a setting screen Gs for setting speakers and groups as shown in FIG. 7 is displayed on the speaker / group setting area Ah. As shown in the figure, the setting screen Gs is provided with an input box b6 for inputting the speaker's name, a selection operator for selecting one's own group / other party group, a save button B20, and a cancel button B21. Here, an example of the setting screen Gs for speaker B is shown, but in this example, in response to the operation of the speaker setting instruction button B14 described above, the setting screen Gs is displayed in sequence starting from speaker A, and the user can input name information in the input box b6 starting from speaker A, set a group by operating the selection operator s1, and perform a save operation using the save button B20, thereby inputting the name and setting a group for each speaker. When the cancel button B21 is operated on the setting screen Gs, the CPU 1a ends the acceptance of the setting operation of the speaker name and group, and performs processing to return the screen display state to that of FIG. 6, for example.

[0076] FIG. 8 shows an example of display information in the speaker / group setting area Ah when the name and group for each speaker are set on the setting screen Gs described above. As shown in the figure, the set name information is displayed for each speaker. As shown in Figure 6, before the name of each speaker is set in the speaker / group setting area Ah, the speaker icon displays information such as A, B, or C indicating the speaker ID. However, after the name is set, the speaker icon displays the first letter of the name, as shown in Figure 8. In this case, in the speaker / group setting area Ah, the speaker set to the other party's group is displayed in the other party's group area, and the speaker set to your own group is displayed in your own group area, according to the group settings for each speaker.

[0077] In this example, the settings of the name and group for each speaker in the speaker / group setting area Ah are also reflected in the speech text display area At. Specifically, as in the editing screen Ge shown in FIG. 9, the speaker icon for each speaker in the spoken text display area At is changed to a notation showing the initials of the set name.

[0078] Furthermore, in the utterance text display area At, in accordance with the group setting for each speaker, the utterance text data of the speaker belonging to the self-group is moved toward one of the left and right ends, and the utterance text data of the speaker belonging to the other-group is moved toward the other left and right end. Specifically, in this example, the utterance text data of the speaker belonging to the self-group is moved toward the right end of the utterance text display area At, and the utterance text data of the speaker belonging to the other-group is moved toward the left end of the utterance text display area At.

[0079] This allows the user to intuitively distinguish between the utterances of the user's group and the other party's group in the interview text data displayed in the utterance text display area At. In particular, when the interview is a business negotiation as in this embodiment, the user as the sales representative Hs can refer to the utterances of the person on the other party's group, the customer Hc, all at once, which increases convenience.

[0080] In this example, the setting information of the name of each speaker in the speaker / group setting area Ah is also reflected in the display information in the recording period display area Ap. Specifically, the display of the speaker icon displayed in each bar for each speaker will be changed from the speaker ID to the initials of the speaker's name.

[0081] Furthermore, in this example, the setting information of the name of each speaker in the speaker / group setting area Ah is also reflected in the text editing area Ae. Specifically, in this example, as shown by "X" in the figure, the name information for each speaker set in the speaker / group setting area Ah (i.e., the names of the interview participants) is automatically transcribed (copied) into the text editing area Ae. Here, in addition to the setting information in the speaker / group setting area Ah, it is possible to automatically input information about the date and time of the interview as shown in the figure into the text editing area Ae. Furthermore, it is also possible to automatically input text information (for example, "Materials used:" as shown in the figure) to prompt the user to enter the URI (Uniform Resource Identifier) ​​of the storage destination of the materials used in the interview (business negotiation). In this example, the interview information, including the name information of each speaker and the interview date and time, is automatically copied to the top of the text editing area Ae.

[0082] In the edit screen Ge, the analysis execution button B13 is a button for instructing the execution of analysis processes related to the contents of the interview, such as the creation of a summary, etc. Specifically, in this example, it is a button for instructing the execution of at least the summary creation process, the main points list creation process, and the question and answer identification process.

[0083] In this example, the analysis processes as the summary creation process, the main point list creation process, and the question and answer identification process are performed by the analysis unit 1b in the speech management server 1. Therefore, when the analysis execution button B13 is operated, the CPU 1a issues an instruction to execute the analysis process to the analysis unit 1b.

[0084] The CPU 1a displays the analysis result text data obtained as a result of the analysis process performed by the analysis unit 1b on the editing screen Ge.

[0085] FIG. 10 shows an example of the analysis result text data displayed on the editing screen Ge. In the editing screen Ge, the analysis result text data is displayed in the analysis result display area Aa. As shown in the figure, the analysis result display area Aa in this example is disposed in a position between the speaker / group setting area Ah and the text editing area Ae in the left-right direction of the screen.

[0086] The analysis result display area Aa is provided with a summary display area A1, a main points list display area A2, and a question and answer display area A3, as well as copy buttons B25, B26, and B27. The summary display area A1 is an area for displaying text data as a summary created in the summary creation process.

[0087] The main point list display area A2 is an area for displaying the text data of the main point list created in the main point list creation process, that is, the text data showing each main point. The main points here refer to the main points of the topic in the interview. Therefore, the text data showing each main point can be said to be data in which the main points of the topic in the interview have been converted into text. Hereafter, such data that is a text version of the main points discussed in interviews will be referred to as "main points."

[0088] The question and answer display area A3 is an area for displaying text data of the question portion and text data of the answer portion identified in the question and answer identification process.

[0089] A copy button B25 is provided for the summary display area A1. The copy button B25 is a button for instructing copying of the text data displayed in the summary display area A1 to the text editing area Ae. In the summary display area A1 of this example, it is possible to perform a range designation operation of the text data, and the user can designate the range of the text data to be copied in the summary display area A1, and then operate the copy button B25 to instruct copying of the text data in the designated range to the text editing area Ae.

[0090] In this example, the gist list display area A2 is capable of displaying gist sentences for each topic category. In this example, multiple categories with different levels of abstraction are defined as topic categories in an interview, and key point sentences to be displayed in the key point list display area A2 are generated for each category. Specifically, in this example, three levels of abstraction for topic categories are defined: large, medium, and small (large being the most abstract). In other words, there are three topic categories: "large category," "medium category," and "small category."

[0091] In this example, the major categories are classified based on the concept of the "sales negotiation process" described above, and the key points of the major categories would be, for example, any of the above-mentioned "icebreaker," "needs analysis," "proposal / explanation," and "negotiation." The main point sentences of the medium category are less abstract than the main point sentences of the large category. For example, the main point sentences of the medium category for the large category = "icebreaker" can be "golf" or "weather" as specific topics in the negotiation process of "icebreaker", and the main point sentences of the medium category for the large category = "needs analysis" can be "current situation confirmation" or "challenges" as specific topics in the negotiation process of "needs analysis". The main point sentences of the minor category are less abstract than the main point sentences of the medium category. For example, the main point sentences of the minor category for the medium category "golf" are "I played last week" and "I want a new club," while the main point sentences of the minor category for the medium category "weather" are "My head hurts on rainy days" and "If it's sunny next week, I'll go golfing."

[0092] In addition, since an example of a method for identifying a sales negotiation process as a major classification has already been explained, a duplicate explanation will be avoided. A specific example of a method for generating gist sentences for medium and small categories will be described later.

[0093] 10 shows an initial display state of the analysis result display area Aa that is displayed in response to analysis of the converted text data. The initial display state here refers to the state at the stage when the display starts. In this embodiment, in the gist list display area A2 in the initial display state, gist sentences of categories with a certain level of abstraction or higher are displayed, and gist sentences of categories with a level of abstraction below the certain level are hidden. Specifically, in this example, as shown in the figure, only gist sentences of major and medium categories are displayed, and gist sentences of minor categories are hidden. If the main point sentences of all categories were displayed from the beginning, the display content of the main point list display area would become cluttered, and there is a risk that the user would have difficulty in grasping the outline of the main points of the interview. By displaying only the main point sentences of categories with a certain level of abstraction or higher as described above and hiding the main point sentences of the remaining categories, it is possible to prevent the display content of the main point list display area from becoming cluttered, and to make it easier for the user to grasp the outline of the main points of the interview.

[0094] In this example, in which services for analyzing text data using the editing screen Ge and supporting the creation of minutes are provided in a SaaS format, display control of the key points list display area A2 as described above is primarily performed by the CPU 1a of the voice management server 1.

[0095] Although an example in which the number of categories of gist sentences is three is given here, the number of categories may be, for example, two (a major category and a minor category), or may be four or more categories.

[0096] In the gist list display area A2 in the initial display state, a check box cb for selecting a gist sentence is displayed for each of the displayed gist sentences of the major and medium categories.

[0097] In the gist list display area A2, a freely expandable box ex is provided for the display area of ​​the subcategory under each medium category. The user can display the gist sentence of the subcategory under the medium category by selecting the freely expandable box ex under any medium category. Figure 11 shows an example of a case where an expandable box ex located under the medium category "Golf" in the large category "Icebreaker" is selected (Figure 11A), and in response, the main points of the small categories under "Golf" are displayed (Figure 11B). Furthermore, Figure 12 shows an example of a case where an expandable box ex located under the medium category "Weather" is then selected (Figure 12A), and the main points of the subcategory under "Weather" are displayed (Figure 12B). As shown in FIGS. 11 and 12, check boxes cb are displayed for the main points of the displayed small categories as well, in the same way as for the main points of the large and medium categories. In addition, when the freely expandable box ex is expanded to display the main points of the subcategories as shown in Figures 11B and 12B, a close button (marked with an x ​​in the figure) is displayed as shown, making it possible to return the main points of the subcategories to a hidden state.

[0098] In the present example, the gist sentences are arranged in the gist display area A2 in the chronological order of the original utterances. Specifically, in the present example, the gist sentences are arranged in the upper part of the gist display area A2 in the order of the chronological order of the original utterances.

[0099] In addition, in the gist list display area A2 in this example, when an operation for selecting a gist sentence is performed, the voice data of the utterance portion corresponding to the selected gist sentence is reproduced. FIG. 13 illustrates an example in which the main point sentence for minor category = "played last week" has been selected. In the main point list display area A2 in this example, it is possible to perform the operation of selecting each main point sentence in this manner, and when the user performs the operation of selecting one of the main point sentences, the CPU 1a performs a process of playing back the audio data of the speech portion in the recorded data that corresponds to the selected main point sentence.

[0100] In this way, by making it possible to play back the audio data of the utterance portion corresponding to the gist sentence on the editing screen Ge, when the user wants to clarify the contents of the gist sentence whose details are unclear when creating the minutes, the user does not need to search for the corresponding audio data portion by himself / herself. Therefore, the efficiency of creating the minutes can be improved.

[0101] Here, in the minutes support system 100 of this embodiment, in order to enable smooth playback of audio data of the speech points corresponding to such key point sentences, the audio management server 1 performs a pre-link assignment process in which a link to the audio data, specifically an "audio link" which is a link to the audio data portion of the speech point corresponding to the key point sentence, is assigned to the key point sentence before the key point sentence is displayed in the key point list display area A2. This advance link assignment process will be explained later.

[0102] In FIG. 10, a copy button B26 is provided for the main point list display area A2. The user can check any one of the check boxes cb for each gist sentence and then operate the copy button B26 to instruct copying of the gist sentence corresponding to the checked check box cb into the text editing area Ae.

[0103] A specific example will be described with reference to Figures 14 and 15. For convenience of illustration, Figures 14 and 15 mainly show only the analysis result display area Aa and the text editing area Ae of the editing screen Ge. 14 shows a state in which the main point sentences of the intermediate category "golf" in the main point list display area A2 are selected as the target for copying. When the copy button B26 is operated in this state, at least the selected main point sentences of "golf" are copied to the text editing area Ae, as shown in FIG. In this example, if the key sentence selected to be copied is a key sentence from a medium category, as shown in the figure, the key sentences from the large category corresponding to the key sentence from the medium category, as well as all key sentences from the small category corresponding to the key sentence from the medium category, are copied. Although not illustrated, when a main point sentence of a large category is selected as a copy target, the main point sentence of all the medium and small categories under the large category are copied together with the main point sentence of the large category. For example, when a main point sentence of the large category = "Icebreaker" is selected, the main point sentence of the large category "Icebreaker", the main point sentence of the medium category "Golf" under the large category, the main point sentences of the small category under that "I played last week" and "I want a new club", the main point sentence of the other medium category under the large category "Weather", and the main point sentences of the small category under that "My head hurts on rainy days" and "If it's sunny next week, I'll go golfing" are copied to the text editing area Ae. In addition, when a main point sentence of a small category is selected as a copy target, the main point sentence of the medium category and the main point sentence of the large category corresponding to the main point sentence of the small category are copied together with the main point sentence of the small category. For example, when the main point sentence of the small category = "If it's sunny next week, I'll go golfing", which is the main point sentence of the small category, "Weather", which is the main point sentence of the medium category corresponding to the small category, and "Icebreaker", which is the main point sentence of the large category corresponding to the small category, are copied to the text editing area Ae.

[0104] As mentioned above, in this example, the interview schedule information is automatically copied to the top of the text editing area Ae, but the text data copied from the analysis result display area Aa, including the main points list display area A2, is placed below the schedule information. At this time, it is conceivable to insert title information indicating that this is the main subject of the minutes, such as the text "Minutes Contents" illustrated in the figure, between the event information and the copied text data.

[0105] In FIG. 10, in the question and answer display area A3, text data showing the question portion and text data showing the answer portion are displayed. In the question and answer display area A3 of this example, text data showing the question portion and text data showing the answer portion are arranged in the chronological order of the original utterances.

[0106] The copy button B27 is a button for instructing copying of the text data in the question and answer display area A3 to the text editing area Ae. In question and answer display area A3, it is possible to perform an operation to specify the text data of the question portion and answer portion to be copied. After performing an operation to specify the range of the text data of the question portion and answer portion to be copied in question and answer display area A3, the user can issue an instruction to copy the specified text data to text editing area Ae by operating copy button B27.

[0107] In the question and answer display area A3 of this example, the text data showing the question portion and the text data showing the answer portion are displayed so that it is possible to identify whether the comment is made by the user's group or the other party's group. Specifically, in this example, the question and answer portions of your group are displayed as text data toward the right edge of question and answer display area A3, and the question and answer portions of the other group are displayed as text data toward the left edge of question and answer display area A3. This makes it easier for a user to understand specific questions or answers from his or her own or the other party when creating minutes of a meeting.

[0108] Here, the analysis result display area Aa is provided with a summary display area A1, a main points list display area A2, and a question and answer display area A3, and in the analysis result display area Aa in this embodiment, the summary display area A1 is arranged on the uppermost side of these display areas. In other words, the CPU 1a performs display processing of the analysis result display area Aa so that the text data showing the summary is arranged on the uppermost side.

[0109] Here, "upper right" means the direction toward the top of the screen when the speech text display area At, the analysis result display area Aa, and the text editing area Ae are arranged in the left-right direction of the screen, and means the direction toward the left of the screen when the speech text display area At, the analysis result display area Aa, and the text editing area Ae are arranged in the top-bottom direction of the screen. Information displayed on the screen tends to be viewed starting from the top of the screen. Therefore, by arranging the "Summary" at the topmost part of the screen as described above, it is possible to have the user check the contents of the "Summary" before referring to the "Main Points" and "Questions and Answers." In other words, it is possible to have the user understand the overall outline of the interview from the "Summary" before referring to the "Main Points" and "Questions and Answers," making it easier for the user to understand the contents of the "Main Points" and "Questions and Answers." Therefore, the time required for the user to understand the contents of the interview can be shortened, and the efficiency of creating minutes can be improved.

[0110] In addition, in the analysis result display area Aa, the arrangement order of the summary display area A1, the main points list display area A2, and the question and answer display area A3 may be changeable based on a user operation (customizable by the user). On the other hand, it is also possible to make it impossible for the user to customize the arrangement order of the spoken text display area At, the analysis result display area Aa, and the text editing area Ae on the editing screen Ge.

[0111] Next, the text editing area Ae serving as an area for creating minutes will be described. The text editing area Ae is an area where the analysis result text data in the analysis result display area Aa, including the main points, can be transcribed as constituent sentences of the minutes, and a sentence for the minutes can be created. In this text editing area Ae, the copied analysis result text data can be edited, for example, by deleting a part of it or adding an addition. Since the analysis result text data in the analysis result display area Aa as key points or summaries can be copied and used in the sentences that make up the minutes, there is no need to input the text for creating the minutes from scratch, which improves the efficiency of creating the minutes.

[0112] In the text editing area Ae in this embodiment, if the transcribed analysis result text data is corrected, the corrected text portion is displayed in a different display mode from the uncorrected text portion. Although not shown, if the analysis result text data copied from the analysis result display area Aa is corrected by a user operation in the text editing area Ae, for example, the corrected text portion is displayed in a different color from the uncorrected text portion. Alternatively, only the corrected text portion may be displayed in bold or underlined.

[0113] In addition, in this embodiment, when the transcribed analysis result text data is corrected in the text editing area Ae, the CPU 1a performs a process of recording the text data before and after correction in a memory device in association with each other. The text data before and after the correction is recorded in a memory device in the voice management server 1, such as the storage unit 9 of the voice management server 1. Alternatively, the text data before and after the correction may be recorded in a memory device in the minutes management server 30 or the information processing device 20.

[0114] Here, the text data before and after the correction in the text editing area Ae can be used for training the minutes creator. Specifically, for example, by having a low performer who is inexperienced in minutes creation refer to the text data before and after the correction by a high performer who is skilled in minutes creation, it becomes possible for the low performer to learn the skills related to minutes creation, which can contribute to the development of the low performer creator. In particular, if the interview is a business meeting, it becomes possible for the low performer to learn the skills of a high performer sales representative, which can contribute to the improvement of the sales representative's skills. In addition, the above-mentioned pre- and post-correction text data can also be used to improve and enhance the functionality of the analysis process for obtaining analysis result text data from the recorded interview data. Specifically, for example, when the analysis process is performed using an AI model, the pre- and post-correction text data can be used as learning data for machine learning to improve and enhance the analysis process. Or, even when the analysis process is performed using rule-based processing, the pre- and post-correction text data can be referred to in updating the analysis process, which can contribute to improving and enhancing the functionality of the analysis process.

[0115] In addition, there are other benefits to saving the text data before and after the correction as described above, such as the following: For example, if a sales representative Hs1 modifies the minutes of a meeting to read "President xx likes golf," while another sales representative Hs2 modifies the same summary by deleting the phrase "likes golf," then sales representative Hs1 will be recognized as a high performer and sales representative Hs2 as a low performer based on past sales negotiation results, etc. Additionally, we envision an AI that automatically creates meeting minutes. In this case, by saving the text data before and after the correction, the AI ​​can learn the correction tendency of the sales representative Hs1 (always recording the customer's hobbies in the minutes, and adding a subject so that it is clear whose hobby it is). Specifically, it is possible to make the AI ​​acquire the function of recording the customer's hobbies in the minutes, including the name of the person.

[0116] In the editing screen Ge, an output button B15 and a clipboard copy button B16 are provided as operators related to the text editing area Ae. The output button B15 is a button for instructing to output the text created in the text editing area Ae (text as minutes) in a predetermined file format such as Word. Specifically, it is a button for instructing to save the text created in the text editing area Ae in the information processing device 20 (i.e. locally) in a predetermined file format. The clipboard copy button B16 is a button for instructing copying of the text created in the text editing area Ae to the clipboard of the information processing device 20. Providing the output button B15 and the clipboard copy button B16 enables the user to save the created text data as the minutes in a predetermined file format in the information processing device 20. Therefore, the output button B15 and the clipboard copy button B16 can be said to be buttons for instructing local saving of the created minutes.

[0117] Furthermore, in the editing screen Ge of this example, an upload button B17 and a submission button B18 are also provided as operators related to the text editing area Ae. The upload button B17 is a button for instructing uploading of the text created in the text editing area Ae to a cloud server of a predetermined sales information management service (for example, Salesforce: registered trademark). Specifically, the upload button B17 is a button for instructing uploading of the text created in the text editing area Ae to the minutes management server 30. For example, when the upload button B17 is operated, a dialog is displayed, and when the user specifies his / her company name on the dialog and performs an operation to instruct uploading, the text created in the text editing area Ae is uploaded to the minutes management server 30.

[0118] The post button B18 is a button for instructing to post the text created in the text editing area Ae as shared information in a specified sales communication service (e.g., Slack: registered trademark). For example, when the post button B18 is operated, a dialog is displayed, and when the user designates a channel related to information sharing on the dialog and performs an operation to instruct to execute posting, the text created in the text editing area Ae is posted as shared information in the designated channel in the above-mentioned sales communication service.

[0119] In addition, in the editing screen Ge of this example, a copy button B10 and a copy button B11 are also provided as operators for issuing a copy instruction to the text editing area Ae. The copy button B10 is a button for instructing to copy the text data in the speech text display area At to the text editing area Ae. The copy button B11 is a button for instructing to copy the speaker name information and group setting information set in the speaker / group setting area Ah to the text editing area Ae. By providing these copy buttons, the efficiency of creating minutes can be further improved.

[0120] Here, when an instruction to digitize the minutes is given, such as by operating the upload button B17, the voice management server 1 (CPU 1a) in this embodiment performs a process of generating minutes data in which the above-mentioned voice link is added to the main points as the minutes data (hereinafter referred to as the "minutes data generation process"). By generating minutes data in this manner in which audio links are attached to key sentences, when a key sentence is specified in the minutes, the audio data portion indicated by the audio link can be played back, making it possible to smoothly play back the audio data of the speech portion corresponding to the specified key sentence. Therefore, when realizing audio playback of the speech points corresponding to the key points in the minutes, it is no longer necessary for the user to search for the relevant audio data portions from the recorded data, which makes it easier to check the minutes later to confirm the details of the interview.

[0121] As described above, in this example, before the gist sentences are displayed in the gist list display area A2, the process of attaching audio links to the gist sentences is performed in advance by the pre-link attaching process. Therefore, when generating the minutes data, the audio links attached by the pre-link attaching process are inherited. Since it is only necessary to take over the already assigned voice link, the minutes data can be generated quickly.

[0122] In addition, in this embodiment, when an instruction is given to digitize the minutes data, such as by using the upload button B17 described above, in order to convert the minutes data into cloud-stored data, which is data stored in a specified server device other than the device displaying the editing screen Ge (minutes creation screen), the audio management server 1 generates minutes data in a format in which an audio link is attached to the main points, while when an instruction is given to digitize the minutes data into local-stored data, which is data stored in the device displaying the editing screen Ge, such as by using the output button B15 described above, the audio management server 1 generates minutes data in a format in which an audio link is not attached to the main points. Specifically, in this example, the instruction to convert the data to cloud storage corresponds to an instruction to upload on the dialog that is displayed in response to the operation of the upload button B17, and an instruction to post on the dialog that is displayed in response to the operation of the post button B18. In addition, the instruction to make the data locally stored corresponds to the operation of the output button B15 and the operation of the clipboard copy button B16 in this example.

[0123] By being able to toggle between including and not including audio links as described above, if a user thinks that the audio link function is unnecessary for minutes, they can simply instruct the created minutes to be saved locally.Because the user can choose whether to create minutes data with or without an audio link, it is possible to prevent situations where the user is only able to create and save minutes data with audio links even though they think that the audio link function is unnecessary.

[0124] Here, in this embodiment, it is possible to modify the gist text copied into the text editing area Ae, but in this example, the audio management server 1 will not release the audio link when generating the minutes data, even if the transcribed gist text is modified, as long as even one character before the modification remains.

[0125] FIG. 16 is an explanatory diagram of uploading of minutes data with an audio link and processing related to the uploaded minutes data. First, the minutes data created through the editing screen Ge is <1> As shown in the figure, the uploaded file is uploaded to the minutes management server 30. In this example, this upload is performed based on the operation of the upload button B17.

[0126] The minutes data uploaded to the minutes management server 30 can be viewed by the sales representative Hs as a user (see FIG. <2> In this example, the minutes data is viewed using the information processing device 20 (i.e., the device that created the minutes), but the minutes data can also be viewed using a device other than the information processing device 20.

[0127] While viewing the minutes data, the user can select a key sentence in the minutes and play back the audio data corresponding to the selected key sentence (see FIG. <3> reference).

[0128] FIG. 17 is a diagram for explaining an example of an audio playback instruction operation while the minutes are being viewed. As shown in Fig. 17A, the transcribed main point sentence can be selected in the minutes. Fig. 17A shows an example in which the main point sentence "The customer played last week", which is a subcategory under "Golf" in the minutes, is selected.

[0129] When an operation to select a key sentence in the minutes is performed, a dialog showing a list of processes that can be executed on the selected key sentence is displayed, as shown in Fig. 17B. Specifically, this dialog has an audio playback button for instructing audio playback of the corresponding part of the selected key sentence, and a correction button for instructing a transition to a correction mode for the selected key sentence, and if the user wishes to play back the audio, he or she need only operate the audio playback button. The dialog also includes a back button, and the user can close the dialog by operating the back button.

[0130] As shown in FIG. 16, in this embodiment, the recorded data of the interview is stored in the voice management server 1. Therefore, when reproducing the voice data according to the voice link added to the gist sentence, an access request is generated to the voice management server 1 (see FIG. 16). <4> 3. This access request is made by the device displaying the minutes (here, the information processing device 20) to the voice management server 1 based on the voice link added to the gist sentence.

[0131] When an access request is made to the voice management server 1 for accessing the corresponding voice data based on the voice link in the minutes data, the voice management server 1 performs an access authentication process (see FIG. <5> ).

[0132] In this example, it is assumed that a contract has already been made between the administrator of the voice management server 1 and the user (or the company to which the user belongs) to enjoy the service of audio playback of the parts corresponding to the gist sentence. In the access authentication process here, at least a process of determining whether the user who has accessed is a legitimate user who has made the contract is performed. For example, in the access authentication process, the voice management server 1 requests the information processing device 20 to transmit the user account and password information determined at the time of the contract, and determines whether the transmitted information matches the user account and password information registered for the legitimate user. If the information matches, the authentication is successful, and if it does not match, an authentication error occurs.

[0133] If the authentication is successful, the voice management server 1 permits the device that has made the access request to access the voice data (recorded data). In the event of an authentication error, the voice management server 1 denies access to the voice data by the device that made the access request, and performs appropriate error processing, such as displaying a message on the device indicating that access is not permitted.

[0134] <4. Example of functional configuration> With reference to FIG. 18, a functional configuration for realizing the information processing method according to the embodiment described above will be described. Figure 18A is a functional block diagram showing various functions as an embodiment of the CPU 1a in the voice management server 1, and Figure 18B is a functional block diagram showing various functions as an embodiment of the analysis unit 1b in the voice management server 1.

[0135] As shown in FIG. 18A, the CPU 1a has functions as an audio recording data recording processor F11, a screen control processor F12, a minutes generation and transmission processor F13, and an access authentication processor F14.

[0136] The audio recording processing unit F11 performs a process of recording the audio recording data of the interview. Specifically, this is a process of storing the audio recording data of the interview (the audio data detected by the microphone described above) transmitted from the information processing device 20 in the storage unit 9 of the audio management server 1.

[0137] The screen control processing unit F12 performs control processing for displaying an editing screen Ge as a minutes creation screen on the information processing device 20, and various processing in response to operations on the editing screen Ge. In particular, the screen control processing unit F12 in this embodiment, as described above in Figure 10, performs processing to display key sentences of categories with a certain level of abstraction or higher in the initial display state of the key point list display area A2, and to hide key sentences of categories with a level of abstraction below the certain level.

[0138] The minutes generation / transmission processing unit F13 performs processing for generating minutes data and transmitting the generated minutes data to a predetermined server device other than the device that displays the editing screen Ge in response to a user operation. As mentioned above, when generating minutes data, if an instruction is given to store the data in the cloud, minutes data with an audio link is generated, and if an instruction is given to store the data locally, minutes data without an audio link is generated. In addition, when an instruction to make the data cloud-stored is given, the generated minutes data is sent to a server device according to the operation. Specifically, when an instruction to upload is given on the dialog displayed by operating the upload button B17, the generated minutes data is sent to the minutes management server 30, and when an instruction to post is given on the dialog displayed by operating the post button B18, the generated minutes data is sent to a predetermined server device other than the minutes management server 30.

[0139] The access authentication processing unit F14 performs the above-mentioned access authentication processing when an access request to the recorded data is made from an external device based on the audio link in the minutes data. Note that a specific example of the access authentication processing has already been explained, so a duplicate explanation will be avoided.

[0140] In FIG. 18B, the analysis unit 1b has functions as a text block processing unit F21, a summary creation processing unit F22, a main point list creation processing unit F23, and a question and answer identification processing unit F24.

[0141] The text block processing unit F21 performs the above-mentioned text block processing on the text data obtained by converting the recorded data of the interview into text. Specifically, the text block processing in this example is processing for detecting topic breaks from the text data. As described above, such text block processing can be realized as processing using a machine-learned AI model or rule-based processing using keyword matching. The text blocking process identifies at least the start time (start time of speech) of each block of text data. By identifying the start time of a block, a position (playback position) corresponding to the start time of the block is also identified for the recorded audio data (audio data).

[0142] The summary creation processing unit F22 performs a summary creation process, which is a process of creating a summary of the interview, based on the text data of the audio recording data of the interview. The main point list creation processing unit F23 performs a main point list creation process, which is a process for creating a list of main points of the interview, based on the text data of the audio recording data of the interview. The question and answer identification process unit F24 performs a question and answer identification process, which is a process for identifying the question portion and the answer portion in the interview, based on the text data of the recorded data of the interview.

[0143] Here, it is considered that the summary creation process and the question and answer determination process are each realized as a process using an AI model. For example, it is considered that the summary creation process uses an AI model trained by machine learning such as deep learning so that text data is input data and a summary sentence is output data. Similarly, it is considered that the question and answer determination process uses an AI model trained by machine learning such as deep learning so that text data is input data and a question portion and an answer portion are output data. Alternatively, the summary creation process and the question and answer identification process may be realized as rule-based processes. For example, the summarization process may be realized by a processing method of identifying and deleting unnecessary sentences in spoken language, while the question / answer process may be realized by a processing method of identifying a question portion in speech based on recorded data, on the premise that the ending of a sentence rises in a question portion, and identifying the portion of the speech that follows the question portion as the answer portion.

[0144] Here, in this embodiment, the gist list creation processing unit F23 generates gist sentences for each of the aforementioned categories (in this example, large, medium, and small categories) to create a gist list, and also performs the aforementioned "pre-link assignment processing" for each of the generated gist sentences. It is possible that the process of generating the gist sentences and the provision of audio links to the gist sentences may be performed automatically, for example, after the business negotiations are completed.

[0145] Referring to FIG. 19, a process for generating gist sentences for each category as an embodiment performed by the gist list creation processor F23 and a voice link added to each generated gist sentence will be described.

[0146] As described above, in this example, the main points of the major categories are assigned to the negotiation stages identified in the negotiation stage identification process.

[0147] In the figure, the playback start position information Spl of the audio link assigned to the main point sentence of the large category is illustrated. The playback start position information Spl corresponding to the main point sentence of the large category may be, for example, playback position information indicating the start timing of the period of the negotiation process (the negotiation process period) of the large category.

[0148] In this example, the main point sentences of the middle category are generated based on the results of keyword matching using keywords defined corresponding to the topics of the middle category (hereinafter referred to as "middle category keywords"). In this example, a plurality of middle category keywords are defined for each topic of the large category, i.e., for each negotiation process in this example, and the main point sentences of the middle category are generated based on the results of keyword matching using the middle category keywords corresponding to each section of the topic of the large category in the text data (i.e., for each negotiation process period in this example). For example, in generating the main point sentences of the middle category in the topic section of the large category = "icebreaker", keywords such as "golf" and "weather" are defined as the middle category keywords corresponding to the "icebreaker", and if there is a section in which the word "golf" is detected in the text data of the negotiation process period of "icebreaker", the main point sentence of "golf" is generated as the main point sentence of the middle category under "icebreaker". In addition, if the word "weather" is detected in the text data of the "icebreaker" section in a section other than the section in which the word "golf" was detected, a further key sentence of "weather" is generated as a key sentence of the intermediate category under "icebreaker". For example, using this method, it is possible to generate gist sentences for each major category and its subordinate medium categories.

[0149] At this time, the breaks between the intermediate categories, i.e., the start and end of the period of each intermediate category, follow the breaks between blocks detected in the text block processing. Within the negotiation process period of each major category, the intermediate categories are identified by keyword matching, starting from the first block of the negotiation process period. A block in which a word matching a previously prepared keyword is detected (the keyword at this time is designated as the first keyword) is identified as a block of the intermediate category to which the first keyword should be applied (designated as the first block). If a word matching another keyword (designated as the second keyword) is detected in a subsequent block, the block is identified as a block of the intermediate category to which the second keyword should be applied (designated as the second block). At this time, if there is a block between the first block and the second block in which no keyword is detected (designated as a non-detection block), the period from the first block to the non-detection block (the period to the last non-detection block if there are multiple non-detection blocks) is identified as the period of the intermediate category to which the first keyword should be applied. This process of identifying the period of the intermediate classification is carried out for each major classification.

[0150] In the figure, the playback start position information Spm of the audio link to be assigned to the main point sentence of the medium classification is shown as an example. In this case, the main point sentence of the medium classification is "golf" and "weather", and the playback start position information Spm is shown as "Spm1" and "Spm2". The playback start position information Spm corresponding to the gist sentence of the medium classification may be, for example, playback position information indicating the start timing of the period of the medium classification.

[0151] The main point sentences in the small category are generated with a lower level of topic abstraction than the main point sentences in the medium category. Specifically, in this example, the main points of the subcategory are generated by summarizing the text for each block detected in the text blocking process. That is, in this example, the main points of the subcategory can be said to be summaries for each block. The generation of the summary sentences here can be performed using the same method as the generation of the summary sentences in the summary creation process described above.

[0152] In the figure, the playback start position information Sps of the audio link to be assigned to the main point sentence of the minor category is shown as an example. In this case, five pieces of playback start position information Sps, "Sps1" to "Sps5", are shown in accordance with the case where the number of blocks during the major category = "Ice Break" period is five, i.e., five main point sentences are generated as main point sentences of the minor category under the major category = "Ice Break". The reproduction start position information Spm corresponding to the gist sentence of a small category may be, for example, reproduction position information indicating the start timing of the period of that small category (that is, the start timing of the corresponding block).

[0153] In addition, while Figure 11 and other figures show two examples of key sentences for the minor category under the medium category "Golf", namely "I played last week" and "I want a new club", Figure 19 shows an example of a case where three blocks exist in the period of the medium category "Golf", i.e., three key sentences should be generated for the minor category under the medium category "Golf". However, Figure 19 ignores consistency with Figure 11 and other figures.

[0154] In the above, it is assumed that one small category is composed of one block, but it is also possible that one small category is composed of a predetermined number of blocks. In that case, the audio link attached to the gist sentence of the small category may be a link indicating the start position of the first block of the corresponding predetermined number of blocks.

[0155] The method of determining the playback start position is not limited to the above method. The playback start position of each gist sentence may be determined so as to include at least the voice data portion of the utterance of the gist sentence.

[0156] In Figure 18B, in generating key point sentences, the key point list creation processing unit F23 in this embodiment generates key point sentences for categories with a certain level of abstraction or above based on the speech content of both the one's own group and the other party's group, and generates key point sentences for categories with a level of abstraction below a certain level based only on the latter of the two groups' speeches. Specifically, the main point list creation processing unit F23 in this example generates main point sentences for the major categories based on the speech contents of both the self-side group and the other-side group, while generating main point sentences for the medium and small categories based only on the speech of the other-side group among the speeches of the self-side group and the other-side group. For confirmation, it is possible to identify which group's speaker each utterance in the text data belongs to by performing the above-mentioned speaker separation process and setting groups for each speaker in the speaker / group setting area Ah.

[0157] Regarding key sentences, in order to properly classify high levels of abstraction, it is advantageous to have a large number of utterances as a basis, and it is desirable to use the utterances of both one's own group and the other group when generating key sentences for high levels of abstraction, as described above. On the other hand, for key sentences for low levels of abstraction, if they are to be recorded in the minutes, the content of the other party's utterances is important (especially in the case of business negotiations, medical interviews, educational interviews, etc.), so it is desirable to use only the utterances of the other party's group when generating key sentences for low levels of abstraction, as described above. Therefore, by selecting the utterance on which the main point sentence is generated from either the user's own utterance or the other person's utterance depending on the level of abstraction of the classification as described above, the utterance used to generate the main point sentence can be appropriately selected depending on the level of abstraction of the classification of the main point sentence to be generated, thereby ensuring that the main point sentence is generated appropriately.

[0158] Here, the voice management server 1 in this embodiment has a function of converting recorded interview data into text, and in this example, this text conversion function is a function possessed by the analysis unit 1b. Furthermore, the voice management server 1 in this embodiment performs the above-mentioned speaker separation process, and the speaker separation process is also performed by the analysis unit 1b.

[0159] <5. Processing Procedure> An example of a specific process procedure to be executed to realize an information processing method according to an embodiment will be described with reference to the flowcharts of FIGS. In this example, the processes shown in FIGS. 20 and 21 are executed by the CPU 1a in the voice management server 1 based on a program stored in the ROM 2 or the storage unit 9, for example.

[0160] FIG. 20 is a flowchart of a process for generating minutes data. First, in step S101, the CPU 1a waits for an instruction to digitize the minutes data. That is, in this example, the CPU 1a waits for any of the following: the operation of the Output button B15, the operation of the Clipboard Copy button B16, the operation of the Upload button B17 to instruct upload execution on the dialog box displayed when the Upload button B17 is operated, and the operation of the Post button B18 to instruct posting execution on the dialog box displayed when the Post button B18 is operated.

[0161] When an operation to instruct the digitalization of the minutes data is performed, the CPU 1a proceeds to step S102 and judges whether it is a cloud storage operation. That is, it judges whether the operation detected in step S101 is either an upload execution instruction operation on a dialog displayed in response to the operation of the upload button B17 or a posting execution instruction operation on a dialog displayed in response to the operation of the post button B18.

[0162] In step S102, if the operation is a cloud storage operation, the CPU 1a proceeds to step S103 and generates minutes data that inherits the audio link attachment state for the gist sentences of each classification.

[0163] Then, in step S104 following step S103, the CPU 1a performs upload processing of the generated minutes data. That is, if the operation detected in step S101 is an operation to instruct upload execution on the dialog displayed in response to the operation of the upload button B17, the CPU 1a transmits the generated minutes data to the minutes management server 30, and if the operation detected in step S101 is an operation to instruct posting execution on the dialog displayed in response to the operation of the post button B18, the CPU 1a transmits the generated minutes data to a predetermined server device as a posting destination.

[0164] On the other hand, if the operation is not a cloud storage operation in step S103, the CPU 1a proceeds to step S105 and generates minutes data in which the audio link state for the gist sentences of each classification is released. For example, the CPU 1a generates minutes data that inherits only the text data of the gist sentences.

[0165] In step S106 following step S105, the CPU 1a performs processing to transmit the generated minutes data to the information processing device 20. This allows the information processing device 20 to output the text data of the minutes to Word or copy it to the clipboard.

[0166] The CPU 1a ends the series of processes shown in FIG. 20 in response to execution of either step S104 or S106.

[0167] The minutes editor service may be provided by software stored locally in the information processing device 20. 20 is executed by the CPU 20a of the information processing device 20. At this time, it goes without saying that a process of outputting the text data of the minutes to Word or copying it to the clipboard should be executed instead of the transmission process in step S106.

[0168] FIG. 21 is a flowchart of a process relating to access authentication. First, in step S110, the CPU 1a waits for an access request to the voice data, and when the access request is received, the CPU 1a proceeds to step S111 and performs authentication processing. That is, the above-mentioned access authentication processing is executed. Note that an example of the access authentication processing has already been explained, so a duplicate explanation will be avoided.

[0169] In step S112 following step S111, the CPU 1a judges whether the authentication is successful or not. If the authentication is successful, the CPU 1a proceeds to step S113 and performs access permission processing. That is, the access request of step S110 is permitted. This enables the device that has made the access request in accordance with the audio link of the minutes data (the information processing device 20 in this example) to reproduce and output the audio data of the portion corresponding to the selected gist sentence.

[0170] On the other hand, if it is determined in step S112 that the authentication has not been successful, the CPU 1a proceeds to step S114 to perform error processing, i.e., to deny access to the voice data by the device that has made the access request, and to perform corresponding error processing, such as processing for displaying a message on the device indicating that the access has not been permitted.

[0171] The CPU 1a ends the series of processes shown in FIG. 21 in response to execution of either step S113 or S114.

[0172] <6. Variations> The embodiment is not limited to the specific example described above, and various modified configurations may be adopted. For example, in the above example, the interview is a business negotiation, but interviews to which the technology of the present invention can be applied include various opportunities for people to communicate with each other, such as business negotiations, as well as interviews in the medical field, counseling, lectures, speeches, education, and entertainment.

[0173] In the above example, the main categories of the main points sentences are business negotiation processes, but the main categories can be categories other than business negotiation processes. For example, any category can be defined as the main categories, and multiple keywords can be prepared for each category. If a matching keyword is detected in the text, the wording of the keyword can be adopted as the wording of the main points sentence of the main category, making it possible to accommodate any category. Regarding major classification, rather than simply adopting the wording of the matched keywords as the main points, it is also possible to adopt a method in which, for example, when keywords such as "what will be done in the future," "who will be the decision maker," and "how to proceed going forward" are detected, a more abstract wording such as "next action" can be applied.

[0174] In addition, although the above describes an example in which the interview is conducted face-to-face, the technology of the present invention can also be suitably applied to cases in which the interview is conducted as an online interview. 22 illustrates an example in which an online interview is conducted between an information processing device 20 used by a sales representative Hs and an information processing device 40 such as a PC used by a customer Hc. Here, the online interview can be realized by exchanging at least audio between the information processing device 20 and the information processing device 40. Of course, a camera can be provided to conduct an online interview with video. By using video data as well as audio recording data as information relating to the interview, it becomes possible to perform analysis based on the speaker's gestures and hand movements in the analysis process, thereby improving analytical performance.

[0175] In Figure 22, dotted arrows indicate the transmission path of the voices of the interview participants, with the voice of sales representative Hs being picked up by the microphone of information processing device 20, and the voice of customer Hc being picked up by the microphone of information processing device 40, and then input to information processing device 20 via network NT. In this case, the information processing device 20 records the interview not only the voice data picked up by its own microphone, but also the voice data acquired from the information processing device 40. Then, the information processing device 20 transmits (records) the recorded data to the voice management server 1 as shown by the solid arrow in the figure. After the recorded data is recorded in the voice management server 1, the processing of the voice management server 1 and the information processing device 20 is the same as in the case of a face-to-face interview, so duplicate explanations will be avoided.

[0176] In addition, in the above example, the device (voice management server 1) that performs the analysis process of the interview content based on the recorded data and the device (information processing device 20) that performs the display process of the editing screen Ge are separate devices, but it is also possible to configure the analysis process and the display of the editing screen Ge to be performed by the same information processing device.

[0177] In addition, in the above-mentioned Figures 16 and 21, it was mentioned that authentication processing based on a contract is performed for access to the recorded interview data, but the contents of the contract in this case can be various. For example, there may be a case where there are multiple contracts with different restrictive conditions for allowing access to the recorded data (audio playback). One example is a case where there is a contract with a time limit that allows use only for a specified period such as one month or one year, and a contract with no time limit. In this case, the voice management server 1 performs a corresponding process for each user who has made an access request, according to the contents of the contract. Specifically, in this case, when the voice management server 1 responds to the access request, it refers to the contract content management information Ic as shown in Fig. 23. As shown in the figure, the contract content management information Ic is information indicating the correspondence between the user and the contract content. In the figure, examples of the contract content are shown as a contract with a time limit, a contract without a time limit, and no contract. In this case, when an access request specifying a user ID is made from a user terminal such as the information processing device 20, the voice management server 1 identifies the contract contents based on the user ID and the contract content management information Ic, and performs a corresponding process according to the identified contract contents. For example, if there is no contract, or if there is a time limit but the time limit condition is violated, the voice management server 1 does not permit playback of the voice data, and if there is a time limit but the time limit condition is not violated, or if there is a contract without a time limit, the voice management server 1 performs a process to permit playback of the voice data. For users without a contract, a screen may be presented urging them to make a contract, such as "If you make a contract, you will be able to play audio."

[0178] Here, the voice management server 1 and the minutes management server 30 may be managed by different companies. In this case, the minutes creation support service using the editing screen Ge and the interview recording data storage service may be under separate contracts. For example, multiple contract plans may be offered, such as a minutes editing plan, a minutes editing + recording data storage (limited storage) plan, and a minutes editing + recording data storage (unlimited storage) plan. Alternatively, the same company may manage both the minutes creation support service and the recording data storage service of the interview. In that case, the functions of the voice management server 1 and the minutes management server 30 may be integrated into one server device.

[0179] In addition, on the recording list screen Gi described in Figure 5, a link button to the minutes of an interview for which the minutes have been uploaded to the minutes management server 30 can also be displayed in the display area of ​​the interview (for example, see link button B30 in Figure 24). In this way, by displaying the link button B30 to the minutes on the recording list screen Gi, the user can access the minutes via the recording list screen Gi.

[0180] In addition, in the explanation so far, it has been said that the audio link assigned to each key point sentence is information indicating the start position of the speech of the corresponding key point sentence in the recorded data of the interview (the series of recorded data of that interview). However, if it is assumed that the audio data for each key point sentence will be saved in its own individual URI, then the audio link could be information indicating the URI where the corresponding audio data is saved.

[0181] In addition, in the explanation so far, we have given an example of adding audio links to all key sentences regardless of whether they are from your own group or the other group. However, it is also possible to add audio links only to key sentences whose original utterance is an utterance from the other group (hereinafter referred to as "key sentences based on the other group's utterance"). In interviews, what the other party says is often more important than what you say. In particular, in business negotiations, it is important to confirm the other party's potential desires, so what the other party (customer) says is more important than what you say. By assigning audio links only to key sentences that are based on the other party's utterance as described above, it is possible to smoothly confirm the other party's latent desires, while reducing the processing burden associated with assigning audio links and reducing the required data capacity. When an audio link is provided only to the gist sentences based on the other party's utterance, it is possible to save only the speech data of the other party's group without saving the speech data of the one's group.

[0182] Moreover, as an information processing device related to the present technology, an information processing device having the following configuration is also considered. In other words, the information processing device is equipped with a link assignment processing unit that assigns an audio link to a key point sentence, which is data that has been generated based on recorded data of an interview and that is a text version of the main points discussed in the interview, and which indicates a link to the audio data portion of the speech that corresponds to the key point sentence in the recorded data. With such an information processing device, when a user wishes to play back and check the audio of an utterance portion corresponding to a gist sentence, it becomes unnecessary for the user to search for the corresponding audio data portion from within the recorded data. Therefore, it is possible to improve the ease of checking the details of the interview after the fact.

[0183] <7. Summary of the embodiment> As described above, the information processing device (voice management server 1 or information processing device 20) as an embodiment is provided with a data generation processing unit (minutes generation / transmission processing unit F13) that generates minutes data including the key point sentences, which are text data of the main points discussed in the interview, generated based on the recorded data of the interview, on a minutes creation screen (editing screen Ge) including a key point list display area that displays a list of key point sentences, which are data that transcribes the key point sentences displayed in the key point list display area into text data of the main points discussed in the interview, and a minutes creation area in which the key point sentences displayed in the key point list display area can be transcribed as constituent sentences of the minutes of the interview, so that text can be created as minutes. By generating minutes data in the form in which audio links are attached to key sentences as described above, when a key sentence is specified in the minutes, it is possible to smoothly play back the audio data of the speech point corresponding to the specified key sentence by playing back the audio data portion indicated by the audio link. Therefore, when realizing audio playback of the speech points corresponding to the key points in the minutes, it is no longer necessary for the user to search for the relevant audio data portions from the recorded data, which makes it easier to check the minutes later to confirm the details of the interview.

[0184] In addition, in the information processing device according to the embodiment, the data generation processing unit performs processing for transmitting the generated minutes data to an instructed server device. It is considered that the recording data of the interview is stored in a server device different from the server device in which the minutes data is stored. According to the above configuration, in such a case where the recording data and the minutes data are stored in different server devices, the minutes data can be appropriately transmitted to the server device for storing them. At this time, since an audio link is attached to the main points contained in the minutes data, even if the recording data and the minutes data are stored in different server devices, it is possible to smoothly play back the audio of the utterances corresponding to the main points.

[0185] Furthermore, in the information processing device as an embodiment, a pre-link assignment processing unit (main point list creation processing unit F23) is provided which assigns an audio link to the main point sentence before it is displayed in the main point list display area, and the data generation processing unit inherits the audio link assigned by the pre-link assignment processing unit as the audio link to be assigned to the main point sentence when generating the minutes data. By providing an audio link to the gist sentence before displaying it in the gist list display area as described above, the gist sentence designated by the user can be played back in audio even during the work of creating the minutes, and the audio playback can be performed smoothly. Also, since the audio link is provided to the gist sentence before the creation of the minutes data begins, the process of generating the minutes data only needs to inherit the audio link already provided for the gist sentence included in the minutes, and the minutes data can be generated quickly.

[0186] Furthermore, in the information processing device as an embodiment, when an instruction to digitize the minutes data is given to convert the minutes data to local stored data, which is data stored on the device displaying the minutes creation screen, the data generation processing unit generates minutes data in a format that does not add an audio link to the main points, and when an instruction to digitize the minutes data is given to convert the minutes data to cloud stored data, which is data stored on a specified server device other than the device displaying the minutes creation screen, the data generation processing unit generates minutes data in a format in which an audio link is added to the main points. This allows the user to select whether to create minutes data with or without audio links, and therefore prevents situations in which the user is only able to create and save minutes data with audio links even though the user does not think the audio link function is necessary.

[0187] In the information processing device according to the embodiment, the interview is assumed to be a business negotiation. This makes it easier to check the details of business negotiations after the fact in the minutes of the business negotiations.

[0188] Furthermore, in the information processing device as an embodiment, multiple categories with different levels of abstraction are defined as topic classifications, and key point sentences are generated for each category. A screen control processing unit (F12) is provided for controlling the display of the minutes creation screen, and in the initial display state of the key point list display area, the screen control processing unit displays key point sentences of categories with a level of abstraction equal to or higher, and hides key point sentences of categories with a level of abstraction below the certain level. In the case where key point sentences are generated for each category with different levels of topic abstraction, if key point sentences for all categories were displayed from the beginning, the display content in the key point list display area would become cluttered, and the user would have difficulty in grasping the outline of the key points of the interview. By displaying only key point sentences for categories with a certain level of abstraction or higher as described above and hiding key point sentences for the remaining categories, it is possible to prevent the display content in the key point list display area from becoming cluttered, and to make it easier for the user to grasp the outline of the key points of the interview.

[0189] Furthermore, in the information processing device as an embodiment, the groups of speakers participating in the interview are a self-side group, which is the group of the user who is creating the minutes, and an other-side group, which is the group that is the other side of the self-side group, and a number of categories with different levels of abstraction are defined as topic categories.The device is equipped with a main point generation processing unit (main point list creation processing unit F23) that generates main point sentences for each category based on the text data of the recorded data, and the main point generation processing unit generates main point sentences for categories with a level of abstraction or above based on the speech content of both the self-side group and the other-side group, and generates main point sentences for categories with a level of abstraction below the certain level based only on the speech of the self-side group and the other-side group. Regarding key sentences, in order to properly classify high levels of abstraction, it is advantageous to have a large number of utterances as a basis, and it is desirable to use the utterances of both one's own group and the other group when generating key sentences for high levels of abstraction, as described above. On the other hand, for key sentences for low levels of abstraction, if they are to be recorded in the minutes, the content of the other party's utterances is important (especially in the case of business negotiations, medical interviews, educational interviews, etc.), so it is desirable to use only the utterances of the other party's group when generating key sentences for low levels of abstraction, as described above. According to the above configuration, it is possible to appropriately select utterances to be used in generating a gist sentence depending on the classification abstraction level of the gist sentence to be generated, thereby enabling the gist sentence to be generated appropriately.

[0190] An information processing method as an embodiment is an information processing method in which an information processing device performs processing on a minutes creation screen including a main point list display area that displays a list of main point sentences, which are text data of the main points discussed in the interview, generated based on recorded data of the interview, and a minutes creation area in which the main point sentences displayed in the main point list display area can be transcribed as constituent sentences of the interview minutes to create text as minutes, and when an instruction is given to digitize the minutes created by transcribing the main point sentences, the information processing device performs processing to generate minutes data including the main point sentences, in which an audio link is added to the main point sentences, indicating a link to the audio data portion of the speech portion in the recorded data that corresponds to the main point sentence. With such an information processing method, it is possible to obtain the same functions and effects as those of the information processing device according to the above embodiment.

[0191] The program as the embodiment is a program that causes a computer device such as a CPU 1a to execute all or part of the processing shown in FIG. In other words, the program as an embodiment is a program readable by a computer device, and causes the computer device to execute a process of generating minutes data including the key point sentences, which are generated based on the recorded data of the interview and are converted into text data of the main points discussed in the interview, on a minutes creation screen including a key point list display area that displays a list of key point sentences, which are text data of the main points discussed in the interview, and a minutes creation area in which the key point sentences displayed in the key point list display area can be transcribed as constituent sentences of the interview minutes to create text as minutes, when an instruction is given to digitize the minutes created by transcribing the key point sentences, as minutes data including the key point sentences, in which audio links are attached to the key point sentences, indicating links to the audio data portions of the speech points in the recorded data that correspond to the key point sentences.

[0192] Such a program can realize a computer device that executes all or part of the functions of the voice management server 1 described in the embodiment. For example, by downloading the program to a personal computer, a portable information processing device, or the like, the personal computer can be made to function as the voice management server 1.

[0193] Such a program can be pre-recorded in a HDD as a recording medium built into a device such as a computer device, or in a ROM in a microcomputer having a CPU, etc. Also, the program can be provided by being temporarily or permanently recorded on a removable recording medium such as a memory card or an optical disk. Such programs can also be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet. [Explanation of symbols]

[0194] 1. Voice Management Server 20 Information processing device Hs Sales Representative Hc customer 1a,20a,30a CPU 1b Analysis Department 6 Input section 7 Display section F11 Audio data recording processing unit F12 Screen control processing part F13 Minutes generation and transmission processing unit F14 Access authentication processing section F21 Text block processing section F22 Summary creation processing section F23 Summary list creation processing section F24 Question and Answer Specific Processing Unit 40 Information processing device Group registration screen Gi Recording list screen B3 Minutes creation button Ge Editing Screen At Speech text display area Ah Speaker / Group Settings Area Ae Text Editing Area Ap Recording period display area Aa Analysis result display area b5 Input box B13 Analysis execution button B14 Speaker setting instruction button B15 Output button B16 Clipboard copy button B17 Upload button B18 Post button Sb Seek Bar Is Negotiation process period information Gs setting screen b6 Input box s1 Selection operator B20 Save button B21 Cancel button A1 Summary display area A2 Main points list display area A3 Question and answer display area B25, B26, B27 Copy button cb Checkbox Ex Expandable Box

Claims

1. The minutes creation screen includes a main point list display area for displaying a list of main point sentences, which are data that have been generated based on recorded data of an interview and are converted into text of main points of the topics discussed in the interview, and a minutes creation area in which the main point sentences displayed in the main point list display area can be transcribed as constituent sentences of the minutes of the interview to create text as the minutes. When an instruction is given to convert the minutes created by transcribing the main point sentences into data, the screen includes a data generation processing unit for generating minutes data including the main point sentences, in which audio links indicating links to audio data portions of utterances in the recorded data corresponding to the main point sentences are added to the main point sentences. Information processing device.

2. The data generation processing unit includes: The generated minutes data is transmitted to a designated server device. The information processing device according to claim 1 .

3. a pre-link adding processing unit that adds the audio link to the gist sentence before the gist sentence is displayed in the gist list display area, The data generation processing unit includes: When generating the minutes data, the audio link assigned by the pre-link assignment processing unit is taken over as the audio link to be assigned to the gist sentence. The information processing device according to claim 1 .

4. The data generation processing unit includes: When an instruction for digitizing the minutes data is given to convert the minutes data to local storage data, which is data stored in a device displaying the minutes creation screen, the minutes data is generated in a format in which the audio link is not added to the gist sentence, and when an instruction for digitizing the minutes data is given to convert the minutes data to cloud storage data, which is data stored in a predetermined server device other than the device displaying the minutes creation screen, the minutes data is generated in a format in which the audio link is added to the gist sentence. The information processing device according to claim 1 .

5. The above interview is a business meeting. The information processing device according to claim 1 .

6. A plurality of categories having different levels of abstraction are defined as the topic categories, and the gist sentence is generated for each of the categories; A screen control processing unit that controls the display of the minutes creation screen, The screen control processing unit: In an initial display state of the gist list display area, the gist sentences of the classifications whose abstraction level is equal to or higher than a certain level are displayed, and the gist sentences of the classifications whose abstraction level is lower than the certain level are not displayed. The information processing device according to claim 1 .

7. As groups of speakers participating in the interview, there are a self-group, which is a group on the side of the user who performs the task of creating the minutes, and a counterpart group, which is a group opposite to the self-group, A plurality of topic classifications having different levels of abstraction are defined, a gist generation processing unit that generates the gist sentence for each of the classifications based on the text data of the recorded data; The gist generation processing unit The gist sentences of the classifications having a certain degree of abstraction or more are generated based on the speech contents of both the self-group and the other-group; The gist sentences of the classifications whose abstraction level does not meet the certain level are generated based on only the latter utterance of the self-group and the other-group utterance.

7. The information processing device according to claim 1.

8. An information processing device, In a minutes-creation screen including a main point list display area for displaying a list of main point sentences, which are data that have been generated based on recorded data of an interview and that are converted into text of the main points of the topics discussed in the interview, and a minutes-creation area in which the main point sentences displayed in the main point list display area can be transcribed as constituent sentences of the minutes of the interview to create text as the minutes, when an instruction is given to digitize the minutes created by transcribing the main point sentences, a process is performed to generate minutes-creation data including the main point sentences, in which audio links indicating links to audio data portions of utterances in the recorded data corresponding to the main point sentences are attached to the main point sentences Information processing methods.

9. A computer readable program, a minutes creation screen including a main point list display area for displaying a list of main point sentences, which are data that have been generated based on recorded data of an interview and that are converted into text of main points of the topics discussed in the interview, and a minutes creation area in which the main point sentences displayed in the main point list display area can be transcribed as constituent sentences of the minutes of the interview to create text as the minutes, when an instruction is given to digitize the minutes created by transcribing the main point sentences, the computer device is caused to execute a process of generating minutes data including the main point sentences, in which audio links indicating links to audio data portions of utterances in the recorded data that correspond to the main point sentences are attached to the main point sentences program.

10. A recording medium having a computer-readable program recorded thereon, A minutes-creation screen includes a main point list display area for displaying a list of main point sentences, which are data that has been generated based on recorded data of an interview and that are converted into text of the main points discussed in the interview, and a minutes-creation area in which the main point sentences displayed in the main point list display area can be transcribed as constituent sentences of the minutes of the interview to create text as the minutes. When an instruction is given to digitize the minutes created by transcribing the main point sentences, a program is recorded on the computer device to execute a process of generating minutes-creation data including the main point sentences, in which audio links are attached to the main point sentences, which links to audio data portions of the audio data of the utterances in the recorded data that correspond to the main point sentences. Recording medium.

Images