Information processing system, program, and information processing method

Abstract

To provide an information processing device or the like configured to perform character recognition and easily extract summaries, important parts, and the like of proceedings.SOLUTION: An information processing system is provided. The information processing system comprises a control unit. The control unit is configured to execute each of the following steps. In a display control step, an input screen for creating the minutes of a meeting is displayed, including a document creation area and a script display area. The document creation area is configured to be able to display the results of non-voice input made by a user during a meeting conducted by one or more users. The script display area is configured to be able to display a conference script generated on the basis of audio of the conference. In a specifying step, a part of a script corresponding to the time at which the non-voice input was input is specified when a selection pertaining to the non-voice input displayed in the document creation area is received. In a transfer step, when a selection pertaining to the script displayed in the script display area is received, the selected portion is transferred into the document creation area.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, a program, and an information processing method. [Background technology]

[0002] 2. Description of the Related Art Conventionally, systems are known that perform voice recognition from speech uttered by people during a meeting, and similarly, systems are known that assist in creating minutes of a meeting by using voice recognition.

[0003] Patent Document 1 discloses a prior art technique that recognizes voice and assists in creating minutes of a meeting. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2013-105374 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, even if speech is recognized and converted into text, including the technology disclosed in Patent Document 1, it is not possible to effectively extract portions of the minutes that one wishes to keep as a record.

[0006] In view of the above circumstances, the present invention provides an information processing device and the like that can perform character recognition and can easily extract summaries and important parts of minutes. [Means for solving the problem]

[0007] According to one aspect of the present invention, there is provided an information processing system. The information processing system includes a control unit. The control unit is configured to execute the following steps. In the display control step, an input screen for creating minutes of a meeting is displayed, the input screen including a document creation area and a script display area. The document creation area is configured to be capable of displaying the results of non-voice input by users during a meeting held by one or more users. The script display area is configured to be capable of displaying a script of the meeting generated based on the voice of the meeting. In the identification step, when a selection is received for the non-voice input displayed in the document creation area, a part of the script corresponding to the time when the non-voice input was input is identified. In the transcription step, when a selection is received for the script displayed in the script display area, the selected part is transcribed into the document creation area.

[0008] This makes it possible to perform character recognition and easily extract summaries and important parts of minutes. [Brief description of the drawings]

[0009] [Figure 1] 1 is an example of a configuration diagram illustrating an information processing system 1. FIG. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a server 2. [Diagram 3] 2 is a block diagram showing an example of a hardware configuration of a user terminal 3. FIG. [Figure 4] 2 is an example of a block diagram showing functions realized by a control unit 23 of the server 2. FIG. [Diagram 5] 2 is an example of an activity diagram illustrating an overview of information processing executed by the information processing system 1. FIG. [Figure 6] 13 is an example of the minutes displayed on the display unit 34 of the user terminal 3 immediately after the end of the conference and before voice recognition in the first embodiment. [Figure 7] 4 is an example of the minutes displayed on the display unit 34 of the user terminal 3 after speech recognition in the first embodiment. [Figure 8]13 is an example of the minutes displayed on the display unit 34 of the user terminal 3 when a script is specified in the first embodiment. [Figure 9] 13 is an example of the minutes displayed on the display unit 34 of the user terminal 3 when a script is transcribed in the first embodiment. [Figure 10] 13 is an example of the minutes displayed on a display unit 34 of a user terminal 3 in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] [Embodiment] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various characteristic features shown in the following embodiments can be combined with each other.

[0011] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client terminal (so-called cloud computing).

[0012] In addition, in this embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, in this embodiment, various information is handled, and this information is represented, for example, by physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit collection consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculation can be performed on the circuit in the broad sense.

[0013] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. In other words, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0014] [Embodiment 1] 1. Hardware Configuration In Section 1, the hardware configuration of the first embodiment will be described.

[0015] 1.1 Information Processing System 1 1 is an example of a configuration diagram showing an information processing system 1. The information processing system 1 includes a server 2, a user terminal 3 (e.g., user terminal 3-1, user terminal 3-2, . . ., user terminal 3-n), a sound collection device 4 (e.g., microphone 4-1, microphone 4-2, . . ., microphone 4-n), and an image capture device 5 (e.g., camera 5-1, camera 5-2, . . ., camera 5-n), which are connected via a network. These components will be further described. Here, the system exemplified as the information processing system 1 is composed of one or more devices or components.

[0016] 1.2 Server 2 2 is an example of a block diagram showing a hardware configuration of the server 2. The server 2 has a communication unit 21, a storage unit 22, and a control unit 23, and these components are electrically connected via a communication bus 20 inside the server 2. Each component will be further described.

[0017] The communication unit 21 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt, wired LAN network communication, etc., but may also include wireless LAN network communication, mobile communication such as LTE / 3G, Bluetooth (registered trademark) communication, etc. as necessary. In other words, it is more preferable to implement it as a collection of multiple communication means. That is, the server 2 communicates various information with the sound collection device 4 and the user terminal 3 via the communication unit 21 through the network. In particular, the server 2 is configured to accept voice data including user voice data from the sound collection device 4. These details will be described later.

[0018] The storage unit 22 stores various information defined by the above description. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs related to the server 2 executed by the control unit 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the calculation of the program. Also, a combination of these may be used. In particular, the storage unit 22 stores voice data as recorded data. The storage unit 22 also stores various programs related to the server 2 executed by the control unit 23.

[0019] The control unit 23 performs processing and control of the overall operation related to the server 2. The control unit 23 is, for example, a central processing unit (CPU) not shown. The control unit 23 realizes various functions related to the server 2 by reading out a predetermined program stored in the storage unit 22. That is, information processing by the software stored in the storage unit 22 can be specifically realized by the control unit 23, which is an example of hardware, and executed as each functional unit included in the control unit 23. These will be described in more detail in the next section. Note that the control unit 23 is not limited to being single, and may be implemented with multiple control units 23 for each function. Also, a combination of these may be used.

[0020] 1.3 User terminal 3 3 is an example of a block diagram showing a hardware configuration of the user terminal 3. The user terminal 3 has a communication unit 31, a storage unit 32, a control unit 33, a display unit 34, and an input unit 35, and these components are electrically connected via a communication bus 30 inside the user terminal 3. Descriptions of the communication unit 31, the storage unit 32, and the control unit 33 are omitted because they are substantially similar to the communication unit 21, the storage unit 22, and the control unit 23 in the server 2.

[0021] The display unit 34 may be included in the housing of the user terminal 3, or may be externally attached, for example. The display unit 34 displays a screen of a graphical user interface (GUI) that can be operated by the user. This is preferably implemented by using display devices such as a CRT display, a liquid crystal display, an organic EL display, and a plasma display, depending on the type of the user terminal 3. Here, the display unit 34 will be described as being included in the housing of the user terminal 3.

[0022] The input unit 35 may be included in the housing of the user terminal 3 or may be externally attached. For example, the input unit 35 may be implemented as a touch panel integrated with the display unit 34. If it is a touch panel, the user can input a tap operation, a swipe operation, or the like. Of course, a switch button, a mouse, a QWERTY keyboard, or the like may be adopted instead of the touch panel. That is, the input unit 35 accepts an operation input made by the user. The input is transferred as a command signal to the control unit 33 via the communication bus 30, and the control unit 33 can execute a predetermined control or calculation as necessary.

[0023] 1.4 Sound collection device 4 The sound collection device 4 is a so-called microphone that is configured to convert external sounds into signals. The sound collection device 4 may be provided by directly connecting the sound collection device 4 to the server 2, but is provided in or connected to the user terminal 3, for example.

[0024] The sound collection device 4 is configured to generate voice data by collecting the user's speech. The voice data is temporarily stored in a memory in the user terminal 3, and does not have to be stored in a non-volatile manner in the storage unit 32. The voice data generated by the sound collection device 4 is configured to be transferable to the server 2 via a network.

[0025] The sound collection device 4 collects, but is not limited to, at least sounds in the human audible range, sounds with frequencies between 20 Hz and 20,000 Hz, and converts them into electrical signals. The sounds may be recorded in monaural or stereo. The sampling rate for digitally processing the sound data may be, for example, 48000 Hz, 44100 Hz, 32000 Hz, 22050 Hz, 16000 Hz, 11025 Hz, 11000 Hz, 8000 Hz, etc. The sampling rate may be within any of the ranges of values ​​exemplified here. By increasing the sampling rate, the temporal timing of the sound can be discretized more precisely, and the accuracy of voice recognition can be improved.

[0026] The data collected by the sound collection device 4 may be appropriately compressed by the control unit 33 of the user terminal 3, and the compression format at this time may be any of MP3, AAC, WMA, Vorbis, AC3, MP2, FLAC, TAK, etc. By compression, it is possible to reduce communication traffic due to data transfer from the user terminal 3 to the server 2.

[0027] 1.5 Imaging Equipment5 The photographing device 5 is a so-called camera that is configured to be able to extract information of the outside world as an image. The photographing device 5 may be provided by directly connecting the photographing device 5 to the server 2, but is provided in or connected to the user terminal 3, for example.

[0028] The photographing device 5 is configured to generate video data by collecting fragmented images. The video data generated by the photographing device 5 is configured to be transferable to the server 2 via a network.

[0029] 2. Functional configuration This section describes the functional configuration of embodiment 1. As described above, information processing by the software stored in the storage unit 22 is specifically realized by the control unit 23, which is an example of hardware, and can be executed as each functional unit included in the control unit 23.

[0030] 4 is an example of a block diagram showing functions realized by the control unit 23 of the server 2. Specifically, the control unit 23 of the server 2, which is an example of the information processing system 1, includes a receiving unit 231, a display control unit 232, an output unit 233, a time recording unit 234, a sound recording unit 235, a voice recognition unit 236, an identifying unit 237, a transcription unit 238, and a playback unit 239.

[0031] The receiving unit 231 receives data such as instructions, voice data, and video data transmitted from the user terminal 3 via the network. Note that the voice data is temporarily stored in the memory of the server 2 and is not recorded in the storage unit 22.

[0032] The display control unit 232 controls the screen data to be displayed on the display unit 34 of the user terminal 3. Note that the screen data may be visual information itself, such as a screen, an image, an icon, text, etc., generated in a manner that is visible to the user, or may be rendering information for displaying visual information, such as a screen, an image, an icon, text, etc., on various terminals.

[0033] The output unit 233 performs processes related to output, such as transmitting data to the user terminal 3 and storing the minutes in the storage unit 22 via the network.

[0034] The time recording unit 234 associates the time data of the voice data with the time data of the script data. In this embodiment, the time data is data representing the time counted from the start of the conference. In other embodiments, the time data may simply be data representing the current time.

[0035] The recording unit 235 stores the audio data acquired via the sound collection device 4 in the storage unit 22 as recorded data.

[0036] The voice recognition unit 236 recognizes the recorded data and converts the recorded data into script data. The script data is data obtained by transcribing the recorded data. In other words, the script data is data obtained by transcribing the voice data during the conference.

[0037] The identification unit 237 identifies a part of the information in a document creation area and a script display area, which will be described later.

[0038] The transcription unit 238 transcribes document data in the document creation area (to be described later) to the script display area, and also transcribes script data in the script display area to the document creation area.

[0039] The playback unit 239 plays back the recording data recorded by the recording unit 235 .

[0040] 3. Information processing method 3.1. Overview of data processing method In this section, an overview of the information processing method of the information processing system 1 described above will be given.

[0041] FIG. 5 is an example of an activity diagram showing an overview of information processing executed by the information processing system 1. As shown in FIG.

[0042] First, the pre-conference process will be described with reference to A1 and A5. At A1, the control unit 33 of the user terminal 3 accepts input of a user ID and a password and a participation request via a user's operation on the input unit 35. Upon accepting the participation request, the control unit 33 transmits the user ID, the password, and the participation request to the server 2 via the communication unit 31 of the user terminal 3 and the network.

[0043] In A2, the reception unit 231 of the server 2 acquires a user ID, a password, and a participation request from the user terminal 3 via the network and the communication unit 21 of the server 2. The reception unit 231 determines whether the user ID and password match the account information stored in the storage unit 22, and if they match, performs login authentication. A user who has been authenticated by login is registered as a participant of the conference. The display control unit 232 of the server 2 controls the screen data so that a message is displayed on the display unit 34 indicating that the login authentication has been successful and that the user can participate in the conference. The output unit 233 of the server 2 then transmits the screen data to the user terminal 3 via the communication unit 21 and the network.

[0044] In A3, the control unit 33 of the user terminal 3 accepts a request to start a conference from the host user via the host user's operation of the input unit 35. Upon accepting the request to start a conference, the control unit 33 transmits the request to start a conference to the server 2 via the communication unit 31 of the user terminal 3 of the host user and the network.

[0045] In A4, the reception unit 231 of the server 2 receives a conference start request from the user terminal 3 of the host user via the network and the communication unit 21 of the server 2. When the reception unit 231 receives the conference start request, a process for starting the online conference is performed. Specifically, the reception unit 231 makes the server 2 ready to receive data such as voice data and video data transmitted from each user terminal 3. The display control unit 232 of the server 2 makes the server 2 ready to control screen data such that an input screen for creating minutes of the conference is displayed, including a document creation area 62 and a script display area 63 (see FIG. 6 for the document creation area 62 and the script display area 63). Here, the document creation area 62 is configured to be capable of displaying document data of non-voice input by a user during a conference held by one or more users. The script display area 63 is configured to be capable of displaying script data 635 of the conference generated based on the voice of the conference (see FIG. 7 for the script data 635). Furthermore, the output unit 233 of the server 2 makes the data, such as screen data, audio data, and video data, available for transmission to each user terminal 3.

[0046] At A5, the control unit 23 of the server 2 starts the conference. Specifically, the output unit 233 starts transmitting the screen data controlled by the display control unit 232 of the server 2 to all users via the communication unit 21 of the server 2 and the network. Also, the output unit 233 starts transmitting the audio data or video data acquired by the sound collection device 4 or the image capture device 5 of one user to other users.

[0047] Next, the process during the conference will be described with reference to A6 and A7. At A6, the control unit 33 of each user's user terminal 3 accepts text input to the minutes during the conference via the user's operation on the input unit 35. The control unit 33 also acquires voice data from each user via the sound collection device 4. Furthermore, the control unit 33 acquires video data via the photographing device 5. The control unit 33 transmits the text input data, voice data, and video data to the server 2 via the communication unit 31 and the network.

[0048] In A7, the reception unit 231 of the server 2 receives character input data, voice data, and video data from each user terminal 3 via the network and the communication unit 21 of the server 2. The time recording unit 234 and the display control unit 232 of the server 2 appropriately process the character input data in a text input area 622 and a time stamp display area 623 described later. In addition, the sound recording unit 235 of the server 2 stores the acquired voice data as recorded data in the storage unit 22. Furthermore, the output unit 233 of the server 2 may transmit screen data controlled by the display control unit 232 to all users via the communication unit 21 and the network, or may transmit voice data or video data acquired by a sound collection device 4 or a shooting device 5 of one user to another user.

[0049] Finally, post-conference processing will be described with reference to A8 to A15. At A8, the control unit 33 of the user terminal 3 accepts a voice recognition request from the user via the input unit 35. Upon accepting the voice recognition request, the control unit 33 transmits the voice recognition request to the server 2 via the communication unit 31 of the user terminal 3 and the network.

[0050] At A9, the reception unit 231 of the server 2 acquires the voice recognition request via the network and the communication unit 21 of the server 2. The voice recognition unit 236 of the server 2 that has received the voice recognition request recognizes the recorded data during the conference.

[0051] In A10, the voice recognition unit 236 of the server 2 outputs script data 635 based on the recognized recorded data. The display control unit 232 controls the screen data so that the output script data 635 is displayed on the display unit 34. The output unit 233 transmits the screen data to the user terminal 3 via the communication unit 21 of the server 2 and the network. A9 and A10 will be described with reference to Figs. 6 and 7.

[0052] At A11, the control unit 33 of the user terminal 3 receives the screen data via the network and the communication unit 31 of the user terminal 3. The control unit 33 causes the display unit 34 to display the screen data.

[0053] In A12, the control unit 33 of the user terminal 3 accepts a character input or an area operation through a user's operation of the input unit 35. When the control unit 33 accepts the character input or the area operation, the control unit 33 transmits information related to the character input or the area operation to the server 2 via the communication unit 31 and the network.

[0054] In A13, the receiving unit 231 of the server 2 receives information related to the character input or area operation via the network and the communication unit 21 of the server 2. A13 will be described with reference to FIGS.

[0055] At A14, the control unit 33 of the user terminal 3 transmits a save instruction to the server 2 via the communication unit 31 and the network.

[0056] In A15, the control unit 23 of the server 2 causes the memory unit 22 to store the minutes via the network and the communication unit 21 of the server 2. For details of A15, please refer to the processing related to the Save button 602 and the Complete button 603 in FIG.

[0057] 3.2 Details of data processing In this section, the information processing method of the above-mentioned information processing system 1 will be described in detail. In addition, in this embodiment, the processing of receiving data such as instructions from the user terminal 3 to the server 2, the transmission of data such as screen data from the server 2 to the user terminal 3, and the processing related to communication between the server 2 and the user terminal 3 are omitted unless specifically necessary.

[0058] FIG. 6 shows an example of the minutes displayed on the display unit 34 of the user terminal 3 immediately after the end of the conference and before voice recognition in the first embodiment. The screen displays a minutes area 6. The minutes area 6 includes an agenda area 601, a save button 602, a complete button 603, a layout button 604, a minutes summary display area 61, a document creation area 62, a script display area 63, and a playback area 64. The agenda area 601 is an area where the agenda of the meeting, such as "Sales Headquarters Regular Meeting," is displayed.

[0059] The save button 602 is a button for temporarily saving the minutes. That is, the accepting unit 231 of the server 2 accepts the selection of the save button 602. When the selection of the save button 602 is accepted, the output unit 233 of the server 2 causes the storage unit 22 to store the current minutes. This allows minutes that are in the process of being created to be temporarily saved.

[0060] The completion button 603 is a button for completing the minutes. That is, the reception unit 231 receives the selection of the Complete button 603. When the Complete button 603 is operated, the output unit 233 of the server 2 converts the current minutes into an arbitrary extension such as PDF and stores it in the storage unit 22 in the specified save destination. Thereafter, the output unit 233 transmits the minutes to the user terminals 3 of the members related to the meeting, such as participants and absentees, via the communication unit 21 and the network. This allows the completed minutes to be saved in any desired location, in any desired format, and shared with any desired members.

[0061] The layout button 604 is a button for changing the layout of the minutes area 6. That is, the reception unit 231 receives an operation of the layout button 604. When the layout button 604 is operated, the display control unit 232 controls the screen data so as to change the layout of the minutes area 6 to be displayed on the display unit 34 of the user terminal 3. This allows you to freely change the layout of the minutes.

[0062] The minutes summary display area 61 is an area where the minutes summary is displayed. The minutes summary includes data such as the date and time when the meeting was held, such as "2022 / 4 / 23 10:00-11:00", the URL of the storage destination of the materials used in the meeting, such as "https: / / meeting...", attendees, absentees, and the creator of the minutes. That is, the receiving unit 231 receives an input to the proceedings summary display area 61. When an input is made to the proceedings summary display area 61, the display control unit 232 controls the screen data so as to change the description in the proceedings summary display area 61. This allows the summary of the minutes to be edited as desired.

[0063] The document creation area 62 further includes a document creation assistant area 621, a document input area 622, a time stamp display area 623, and a specific button 624. The document creation assistant area 621 is configured to be able to display the data displayed in the document creation area 62, such as "agenda," in one word.

[0064] The text input area 622 is configured to be capable of displaying the results of text input by one or more users before, during, and after a conference, where text input is an example of non-voice input. That is, the accepting unit 231 accepts character input to the text input area 622. When character input to the text input area 622 is accepted, the display control unit 232 controls the screen data so as to cause the user terminal 3 to display the character input as text data. This allows the minutes to be edited as desired.

[0065] The time stamp display area 623 is configured to be capable of displaying time data relating to the elapsed time of the conference when characters are input in the text input area 622 . That is, the reception unit 231 receives character input into the text input area 622. The time recording unit 234 records time data in the time stamp display area 623 regarding the time when the character was input into the text input area 622. Thereafter, the display control unit 232 controls screen data to be displayed on the user terminal 3. For example, it is assumed that the reception unit 231 receives character input into the text input area 622 when 1 minute and 16 seconds have elapsed since the start of the conference. In this case, the time recording unit 234 records the time data "1:16" in a location in the time stamp display area 623 next to the location where the character was input. This makes it possible to record the time when text is entered into the minutes.

[0066] The specification button 624 is a button for reflecting a part of the document input in the document input area 622 in a specified item such as a determined item or a ToDo item. That is, the receiving unit 231 receives the selection of document data in the document input area 622 by dragging the cursor, etc. The identifying unit 237 identifies the selected document data. Furthermore, the receiving unit 231 receives the selection of the Identify button 624. When these are received, the display control unit 232 controls screen data for displaying the identified document data on the user terminal 3 so that the identified items, such as decisions and ToDo items, can be recognized. This allows any document data to be designated as a specific item.

[0067] Furthermore, the script display area 63 includes a script display auxiliary area 631, a script area 632, a search area 633, and a specification button 634. The script display auxiliary area 631 is configured to be able to display the data displayed in the script area 632, such as "transcription," in a single word. The script area 632 is configured to be able to display script data 635 of the conference generated based on the audio of the conference. At least one of the text input area 622 and the script area 632 may be displayed as a pop-up. The processing performed in the script area 632 will be described in detail with reference to FIG.

[0068] The search area 633 is an area for searching for any keyword within the script area 632 . That is, the receiving unit 231 receives input of a keyword into the search area 633. The output unit 233 determines whether or not a corresponding keyword exists. If a corresponding keyword exists, the display control unit 232 displays the corresponding keyword in the script area 632 in a highlighted manner. This allows any data in the script to be found conveniently.

[0069] The Identify button 634 is a button for reflecting a part of the document input in the script area 632 in the determined items, ToDo items, etc. The process related to the Identify button 634 will be described in detail with reference to FIG.

[0070] In the playback area 64, objects for playing back recorded data, such as play and stop, are displayed.

[0071] FIG. 7 is an example of the minutes displayed on the display unit 34 of the user terminal 3 after speech recognition in the first embodiment. 6 in that the transcription has been completed and script data 635 has been generated. The script data 635 is data obtained by transcribing the recorded data.

[0072] That is, during the conference, the reception unit 231 receives voice data of the conference. The recording unit 235 stores (records) the voice data of the conference in the storage unit 22 as recorded data. After the conference, the reception unit 231 of the server 2 acquires a voice recognition request via the network and the communication unit 21 of the server 2. The voice recognition unit 236 of the server 2 that receives the voice recognition request recognizes the recorded data during the conference. The voice recognition unit 236 outputs script data 635 to the script area 632 based on the recognized recorded data. The display control unit 232 controls the script area 632 to display the output script data 635. That is, the script area 632 is configured to be able to display the script data 635 generated based on the recorded data. This allows the recorded data to be extracted as script data.

[0073] FIG. 8 shows an example of the minutes displayed on the display unit 34 of the user terminal 3 when the script data 635 is specified in the first embodiment. 7 in that a time highlight region 625 and a script highlight region 636 are displayed. The time highlight region 625 is displayed when data in the time stamp display region 623 is selected. For example, when "11:32" is selected, the time highlight region 625 changes color and is displayed. The script highlight region 636 is displayed when data in the time stamp display region 623 is selected. For example, when "11:32" is selected, the script highlight region 636 changes color and is displayed. The seek bar in the playback region 64 can be configured to move to "11:32," which is the start time of the agenda, in accordance with the selection of data in the time stamp display region 623.

[0074] That is, the accepting unit 231 accepts a selection of information displayed in the document input area 622 or the time stamp display area 623. When the identifying unit 237 accepts a selection of document data or time data displayed in the document input area 622 or the time stamp display area 623, the identifying unit 237 identifies script data 635 in the script area 632 that corresponds to the time at which the document data or time data was input. Here, the document data or time data is an example of non-voice input. Also, the script data 635 is identified as a certain range. The display control unit 232 controls the screen data so that the selected document data in the document input area 622 or the time data in the time stamp display area 623 and the corresponding script data 635 are displayed in a highlighted manner. This makes it possible to easily refer to the document data or the script data corresponding to the time stamp display area.

[0075] Furthermore, in this embodiment, the audio recording data can be played back. In this case, the minutes area 6 may be provided with an object for playing back the audio recording data. That is, the receiving unit 231 receives a selection of information displayed in the document input area 622 or the time stamp display area 623. When a selection for document data or time data is received, the identifying unit 237 identifies the script data 635 in the script area 632 that corresponds to the time when the document data or time data was input. Next, the receiving unit 231 receives an operation for the object to be played. By receiving the operation for the object to be played, the playback unit 239 plays back the sound recording data that corresponds to a certain range of the script data 635 identified by the identifying unit 237. This makes it possible to play back the recorded data of the conference that corresponds to the identified script data.

[0076] FIG. 9 is an example of the minutes displayed on the display unit 34 of the user terminal 3 when the script data 635 is transcribed in the first embodiment. 7 in that a document highlight area 626 and a script highlight area 637 are displayed. The script highlight area 637 is displayed by dragging data in the script data 635 with the cursor. For example, the script highlight area 637 is displayed by dragging the part "Please give feedback to attendees on the new plan" with the cursor, and the color of the part changes. The document highlight area 626 is displayed by operating the specification button 634 while the script highlight area 637 is displayed. For example, the document highlight area 626 is transcribed as a specific item such as a decision item or a ToDo item by operating the specification button 634 while "Please give feedback to attendees on the new plan" is highlighted. The seek bar of the playback area 64 can be configured to move to "11:32", which is the start time of the agenda, in accordance with the selection of the script data 635.

[0077] The reception unit 231 receives a selection of a script in the script display area 63. When the transcription unit 238 receives a selection for the script displayed in the script display area 63, it transcribes the selected portion into the document creation area 62. At this time, it is possible to transcribe as a specific item such as a decision item, a ToDo item, etc. In other words, the transcription unit 238 transcribes at least a portion of the script displayed in the script display area 63 as a description of a specific item such as a decision item, a ToDo item, etc. The display control unit 232 controls the screen data to display the transcribed data. This makes it possible to easily set specific items such as decision items for the transcribed script data.

[0078] [Embodiment 2] In the first embodiment, an example in which the creator creates the minutes has been described. In the second embodiment, an example in which the minutes are created in a chat format will be described. The hardware configuration and the functional configuration are omitted because they are the same as those in the first embodiment. Also, this chat may be realized by linking with an existing chat service or chat application.

[0079] FIG. 10 is an example of the minutes displayed on the display unit 34 of the user terminal 3 in the second embodiment. 10 shows the minutes area 7. The minutes area 7 includes an agenda summary area 71 and a main point summary area 72. The minutes area 7 may also include the title of the minutes, an outline of the minutes, decisions made at the meeting, and ToDo items for the meeting. The agenda summary area 71 includes an agenda information area 711 and a playback area 712.

[0080] The number of posts, the number of decisions, the number of ToDo items, and the time spent on each agenda are displayed for each agenda in the agenda information area 711. Also, the playback area 712 displays objects for playing recorded data, such as play and stop. That is, the receiving unit 231 receives an operation on the agenda information area 711. When an operation is performed on the agenda information area 711, the identifying unit 237 identifies the sound recording data corresponding to the agenda. Furthermore, the receiving unit 231 receives an operation on the playback area 712. When an operation on the playback area 712 is received, the playback unit 239 plays back the sound recording data corresponding to the selected agenda information area 711. This makes it possible to play back the recorded data of the meeting corresponding to the specified topic.

[0081] The summary area 72 includes an important data area 721 and a heading data area 722 . The important data area 721 is configured to be capable of displaying the results of inputs inputted by users via chat during a conference held by one or more users. The header data area 722 is configured to be capable of displaying a portion of script data corresponding to each agenda item of the conference generated by the voice recognition unit 236 based on the voice of the conference. Here, the important data area 721 and the header data area 722 are examples of a document creation area and a script display area. Here, the inputs inputted via chat are an example of non-voice inputs.

[0082] Heading data area 722 may be selected to display all of the script information corresponding to each agenda item. That is, the receiving unit 231 receives the selection of any subject in the header data area 722. When the selection of a subject is received, the display control unit 232 controls the screen data so as to display all of the script data corresponding to that subject. This allows smooth reference to the script data.

[0083] Also, similarly to the first embodiment, identification and transcription may be performed. Specifically, the display control unit 232 controls the screen data to display an input screen for creating the minutes of a meeting, including an important data area 721 and a heading data area 722. When the identification unit 237 receives a selection for the important data area 721, it identifies the heading data area 722 corresponding to the time when the important data area 721 was input, and expands the script data. When the transcription unit 238 receives a selection for the expanded script data, it transcribes the selected portion into the important data area 721. This makes it possible to easily set specific items such as decision items for the transcribed script data.

[0084] According to such an embodiment, minutes can be created more efficiently, that is, character recognition can be performed, and summaries, important parts, etc. can be easily extracted from the minutes.

[0085] [others] The information processing system 1 according to this embodiment may adopt the following aspects.

[0086] The sound collection device 4 may be directly connected to the communication unit 21 in the server 2 via a network, without going through the user terminal 3, and may be configured to be able to transfer collected voice data to the server 2. In this case, it is preferable to record which user the collected voice data is associated with, from which sound collection device 4. Although it is possible to associate voice data with multiple users, if there is one sound collection device 4 for each user, it is possible to record which user's utterance is the voice data. The same applies to the imaging device 5.

[0087] Furthermore, the present invention may be provided in the following manner. In the information processing system, the non-voice input is character input by the user. In the information processing system, in the identifying step, the script is identified as a certain range. In the information processing system, in the transcription step, at least a part of the script displayed in the script display area is transcribed as a description of a specific item. In the information processing system, the control unit is configured to further execute a recording step, in which the audio of the conference is recorded as recording data, and the script display area is configured to be capable of displaying a script generated based on the recording data. In the information processing system, the control unit is configured to further execute a playback step, and in the playback step, recorded data corresponding to a certain range of the script identified in the identification step is configured to be playable. In the information processing system, the non-voice input is an input input via chat. A program for causing a computer to function as a control unit of the information processing system. An information processing method executed by a computer, the information processing method comprising the steps of the information processing system. Of course, this is not the case.

[0088] Finally, although various embodiments of the present invention have been described, these are presented as examples and are not intended to limit the scope of the invention. The novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments and their modifications are within the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0089] 1: Information processing system 2: Server 3: User terminal 3-1: User terminal 3-2: User terminal 3-n: User terminal 4: Sound collection device 4-1: Mike 4-2: Mike 4-n: Microphone 5: Imaging device 5-1: Camera 5-2: Camera 5-n: Camera 6: Minutes area 7: Minutes area 20: Communication bus 21: Communications Department 22: Storage section 23: Control section 30: Communication bus 31: Communications Department 32: Storage section 33: Control section 34: Display section 35: Input section 61:Agenda summary display area 62: Document creation area 63: Script display area 71 :Area 72 :Area 231: Reception 232: Display control unit 233: Output section 234: Time Recording Section 235: Recording Department 236: Voice recognition unit 237: Specific part 238: Transcription section 239: Playback Department 601 :Agenda area 602: Save button 603: Completed button 604: Layout button 621: Document Creation Support Area 622: Text input area 623:Time stamp display area 624: Specific button 625 : Time highlight area 626 : Document highlight area 631: Script display auxiliary area 632: Script area 633: Search area 634: Specific button 635: Script data 636 : Script highlighting area 637 : Script highlighting area 711 :Agenda information area 712: Play area 721: Important data area 722: Header data area

Claims

1. An information processing system, A control unit is provided, The control unit is configured to execute the following steps: In the display control step, an input screen for creating the minutes of the meeting, including a document creation area and a script display area, is displayed. the document creation area is configured to display results of non-speech input by one or more users during a conference conducted by the users; the script display area is configured to be able to display a script of the conference generated based on the audio of the conference, In the identification step, when a selection is received for the non-speech input displayed in the document creation area, a part of the script corresponding to a time when the non-speech input is input is identified; In the transcription step, when a selection for the script displayed in the script display area is accepted, the selected portion is transcribed into the document creation area.

2. 2. The information processing system according to claim 1, An information processing system, wherein the non-voice input is character input by the user.

3. 3. The information processing system according to claim 1, In the identifying step, the script is identified as a certain range.

4. 4. The information processing system according to claim 1, In the transcription step, at least a part of the script displayed in the script display area is transcribed as a description of a specific item.

5. 5. The information processing system according to claim 1, The control unit is configured to further perform a recording step; In the recording step, audio of the conference is recorded as recording data; The script display area is configured to be capable of displaying a script generated based on the recorded data.

6. 6. The information processing system according to claim 5, The control unit is configured to further execute a reproducing step; The information processing system is configured such that, in the reproducing step, recorded data corresponding to a certain range of the script identified in the identifying step can be reproduced.

7. 7. The information processing system according to claim 1, An information processing system, wherein the non-voice input is input entered via chat.

8. A program, A program for causing a computer to function as a control unit of the information processing system according to any one of claims 1 to 7.

9. An information processing method performed by a computer, comprising the steps of: An information processing method comprising the steps of the information processing system according to any one of claims 1 to 7.

Images