Information processing system, information processing method and program

Abstract

To provide an information processing system, method and program for improving convenience of preparation of a minute.SOLUTION: In an information processing system 1 in which a server 2 and a plurality of user terminals 3 each having a sound collection device 4 are connected through a network, a control unit of the server displays a picture for preparing an agenda of a conference including an input area and a transcription area in a display control step, The input area is constituted to display input information. The transcription area is constituted to display transcription information and the transcription information is transcribed on the basis of sound of the conference. In an output control step, output information is output on the basis of first transcription information in the input area by accepting a predetermined operation of the first transcription information from the inside of the transcription area to the input area. The first transcription information includes first time information and the first time information is information of time when sound of characters linked with the first transcription information is uttered in the conference.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing system, an information processing method, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, systems are known that perform voice recognition from speech uttered by people during a meeting, and similarly, systems are known that assist in creating minutes of a meeting by using voice recognition.

[0003] Patent Document 1 discloses a prior art technique that recognizes voice and assists in creating minutes of a meeting. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2013-105374 A Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention provides a technique capable of supporting the creation of minutes in a more convenient manner. [Means for solving the problem]

[0006] According to one aspect of the present invention, an information processing system is provided. The information processing system includes at least one processor that executes the following steps. In the display control step, a screen for creating minutes of a meeting is displayed, the screen including an input area and a transcription area. The input area is configured to be able to display input information, and the input information is information to be input. The transcription area is configured to be able to display transcription information, and the transcription information is information of characters transcribed based on the speech of the meeting. In the output control step, output information is output based on the first transcription information in the input area by accepting a predetermined operation from the transcription area to the input area for the first transcription information, and the output information is information to be output. The first transcription information includes first time information, and the first time information is information of a time during the meeting when the speech of the characters associated with the first transcription information was uttered.

[0007] According to the present disclosure, it is possible to provide a technology that can assist in creating minutes in a more convenient manner. [Brief description of the drawings]

[0008] [Figure 1] 1 is an example of a configuration diagram illustrating an information processing system 1. FIG. [Diagram 2] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a server 2. [Diagram 3] 2 is a block diagram showing an example of a hardware configuration of a user terminal 3. FIG. [Figure 4] 2 is an example of a block diagram showing functions realized by a processor 23 of the server 2. FIG. [Diagram 5] 2 is an example of an activity diagram illustrating an overview of information processing executed by the information processing system 1. FIG. [Figure 6] FIG. 1 is a diagram showing an example of a meeting minutes recording screen 6 during a meeting. [Figure 7] FIG. 13 is a diagram showing an example of a minutes recording screen 7 after a meeting. [Figure 8] FIG. 13 is a diagram showing an example of the proceedings recording screen 7 after drag and drop. [Figure 9] FIG. 13 is a diagram showing an example of performing various operations on the proceedings recording screen 7. [Figure 10] FIG. 10 is a diagram showing an example of a management screen 1005. [Figure 11] FIG. 10 is a diagram showing an example of a template creation screen 1006. [Figure 12] FIG. 10 is a diagram showing an example of a minutes recording screen 1007 before a meeting. [Figure 13] FIG. 10 is a diagram showing an example of a meeting minutes recording screen 1008 after a meeting. [Figure 14] FIG. 10 is a diagram showing an example of a CRM screen 1009. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] [Embodiment] Hereinafter, embodiments of the present invention will be described with reference to the drawings. Various features shown in the following embodiments can be combined with each other.

[0010] Incidentally, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable recording medium, or may be provided so as to be downloadable from an external server, or may be provided so that the program is started on an external computer and its functions are realized on a client device (so-called cloud computing).

[0011] In this embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by a circuit in the broad sense and software information processing that can be specifically realized by these hardware resources. In addition, this embodiment handles various information, which may be represented by, for example, physical values ​​of signal values ​​representing voltage and current, high and low signal values ​​as a binary bit set consisting of 0 or 1, or quantum superposition (so-called quantum bits), and communication and calculation may be performed on a circuit in the broad sense.

[0012] In addition, a circuit in the broad sense is a circuit realized by at least appropriately combining a circuit, circuitry, a processor, a memory, etc. In other words, it includes an application specific integrated circuit (ASIC), a programmable logic device (e.g., a simple programmable logic device (SPLD), a complex programmable logic device (CPLD), and a field programmable gate array (FPGA)), etc.

[0013] 1. Hardware Configuration Section 1 describes the hardware configuration.

[0014] 1.1. Information Processing System 1 FIG. 1 is an example of a configuration diagram showing an information processing system 1. The information processing system 1 includes a server 2, user terminals 3 (e.g., user terminal 3-1, user terminal 3-2, . . ., user terminal 3-n), and sound collection devices 4 (e.g., sound collection devices 4-1, sound collection devices 4-2, . . ., sound collection devices 4-n), which are connected via a network. These components will be further described. Here, the system exemplified by the information processing system 1 is composed of one or more devices or components.

[0015] 1.2.Server 2 2 is an example of a block diagram showing a hardware configuration of the server 2. The server 2 has a communication unit 21, a storage unit 22, and a processor 23, and these components are electrically connected via a communication bus 20 inside the server 2. Each component will be further described.

[0016] The communication unit 21 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., but may also include wireless LAN network communication, mobile communication such as LTE / 3G, Bluetooth (registered trademark) communication, etc. as necessary. In other words, it is more preferable to implement it as a collection of multiple communication means. That is, the server 2 communicates various information with the user terminal 3 via the communication unit 21 and the network. In particular, the server 2 is configured to accept the voice data acquired by the sound collection device 4 from the user terminal 3. These details will be described later.

[0017] The storage unit 22 stores various information defined by the above description. This can be implemented, for example, as a storage device such as a solid state drive (SSD) that stores various programs and the like related to the server 2 executed by the processor 23, or as a memory such as a random access memory (RAM) that stores temporarily required information (arguments, arrays, etc.) related to the calculation of the program. Also, a combination of these may be used. In particular, the storage unit 22 stores voice data. The storage unit 22 also stores various programs and the like related to the server 2 executed by the processor 23.

[0018] The processor 23 performs processing and control of the overall operations related to the server 2. The processor 23 is, for example, a central processing unit (CPU) not shown. The processor 23 realizes various functions related to the server 2 by reading out a predetermined program stored in the storage unit 22. That is, information processing by software stored in the storage unit 22 can be specifically realized by the processor 23, which is an example of hardware, and executed as each functional unit included in the processor 23. These will be described in more detail in the next section. The processor 23 is not limited to being single, and may be implemented by having multiple processors 23 for each function. Also, a combination of these may be used.

[0019] 1.3. User terminal 3 3 is an example of a block diagram showing a hardware configuration of the user terminal 3. The user terminal 3 has a communication unit 31, a storage unit 32, a processor 33, a display unit 34, and an input unit 35, and these components are electrically connected via a communication bus 30 inside the user terminal 3. Descriptions of the communication unit 31, the storage unit 32, and the processor 33 are omitted because they are substantially similar to the communication unit 21, the storage unit 22, and the processor 23 in the server 2.

[0020] The display unit 34 may be included in the housing of the user terminal 3, or may be externally attached, for example. The display unit 34 displays a screen of a graphical user interface (GUI) that can be operated by the user. This is preferably implemented by using display devices such as a CRT display, a liquid crystal display, an organic EL display, and a plasma display, depending on the type of the user terminal 3. Here, the display unit 34 will be described as being included in the housing of the user terminal 3.

[0021] The input unit 35 may be included in the housing of the user terminal 3 or may be externally attached. For example, the input unit 35 may be implemented as a touch panel integrated with the display unit 34. If it is a touch panel, the user can input a tap operation, a swipe operation, or the like. Of course, a switch button, a mouse, a QWERTY keyboard, or the like may be adopted instead of the touch panel. That is, the input unit 35 accepts an operation input made by the user. The input is transferred as a command signal to the processor 33 via the communication bus 30, and the processor 33 can execute a predetermined control or calculation as necessary.

[0022] 1.4.Sound collection device 4 The sound collection device 4 is a so-called microphone that is configured to convert external sounds into signals. The sound collection device 4 may be provided by directly connecting the sound collection device 4 to the server 2, but is provided in or connected to the user terminal 3, for example.

[0023] The sound collection device 4 is configured to generate voice data by collecting the user's speech. The voice data is temporarily stored in a memory in the user terminal 3, and does not have to be stored in a non-volatile manner in the storage unit 32. The voice data generated by the sound collection device 4 is configured to be transferable to the server 2 via a network.

[0024] The sound collection device 4 collects, but is not limited to, at least sounds in the human audible range, sounds with frequencies between 20 Hz and 20,000 Hz, and converts them into electrical signals. The sounds may be recorded in monaural or stereo. The sampling rate for digitally processing the sound data may be, for example, 48000 Hz, 44100 Hz, 32000 Hz, 22050 Hz, 16000 Hz, 11025 Hz, 11000 Hz, 8000 Hz, etc. The sampling rate may be within any of the ranges of values ​​exemplified here. By increasing the sampling rate, the temporal timing of the sound can be discretized more precisely, and the accuracy of voice recognition can be improved.

[0025] The data collected by the sound collection device 4 may be appropriately compressed by the processor 33 of the user terminal 3, and the compression format at this time may be any of MP3, AAC, WMA, Vorbis, AC3, MP2, FLAC, TAK, etc. By compression, it is possible to reduce communication traffic due to data transfer from the user terminal 3 to the server 2.

[0026] 2. Functional configuration In this section, the functional configuration of the embodiment will be described. As described above, information processing by the software stored in the storage unit 22 is specifically realized by the processor 23, which is an example of hardware, and can be executed as each functional unit included in the processor 23 as shown in Fig. 4. Fig. 4 is an example of a block diagram showing functions realized by the processor 23 of the server 2. As shown in Fig. 4, the processor 23 includes an information transmission / reception unit 230, a display control unit 231, a storage control unit 232, a voice recognition unit 233, an output control unit 234, a reproduction unit 235, a filter control unit 236, a selection unit 237, and an identification unit 238 as functional units.

[0027] The information transmitting / receiving unit 230 receives information via the network and the communication unit 21. The information transmitting / receiving unit 230 also transmits information via the communication unit 21 and the network.

[0028] The display control unit 231 controls the screen data to be displayed on the display unit 34 of the user terminal 3. Note that the screen data may be visual information itself, such as a screen, an image, an icon, text, etc., generated in a manner that is visible to the user, or may be rendering information for displaying visual information, such as a screen, an image, an icon, text, etc., on various terminals.

[0029] The memory control unit 232 stores various information such as minutes template information and audio data acquired via the sound collection device 4 in the memory unit 22. Hereinafter, the memory control unit 232 may have a function referred to as a recording unit.

[0030] The voice recognition unit 233 recognizes the voice data and generates transcription information 730 from the voice data.

[0031] The output control unit 234 generates information to be output to an arbitrary output destination and outputs the information to the output destination. The output control unit 234 outputs information to an input area (hereinafter, the "input area" may be referred to as an information input area). The output control unit 234, as a transcription unit, includes a process related to transcription of characters from the transcription area to the input area.

[0032] The playback unit 235 plays back the audio data recorded by the storage control unit 232 .

[0033] The filter control unit 236 exercises control so that the transcription information is used as filter application information.

[0034] The selection unit 237 selects one or more pieces of transcription information from among the multiple pieces of transcription information that have been transcribed.

[0035] The specification unit 238 specifies the output destination.

[0036] 3. Information processing method 3.1. Overview of data processing method In this section, an overview of the information processing method of the information processing system 1 described above will be given.

[0037] With reference to FIG. 5, an example will be described in which a secretary user creates minutes using a minutes creation support tool in a conference in which a secretary user and a participating user participate, by the information processing system 1 of this embodiment. FIG. 5 is an example of an activity diagram showing an outline of information processing executed by the information processing system 1. The secretary user is the creator of the minutes, and uses a user terminal 3-1 and a sound collection device 4-1. The participating user is a participant of the conference, and uses a user terminal 3-2 and a sound collection device 4-2. There may be multiple participating users. The minutes creation support tool of this embodiment is a tool that can, for example, record the elapsed time of the conference when a document is input, and reflect the transcription results in the minutes.

[0038] First, information processing during a conference will be described while showing A1 to A8. In A1, the display control unit 231 displays the minutes recording screen 6 for creating the minutes of the meeting, including the input area 61 and the transcription area 63, on the display unit 34 of the user terminal 3-1 of the clerk user. The minutes recording screen 6 will be described in detail with reference to FIG. 6. The display control unit 231 may generate the minutes recording screen 6 using a template based on the minutes template information. That is, the information transmission / reception unit 230 accepts an instruction to create the minutes using a template from the user terminal 3-1 of the clerk user. The display control unit 231 displays the minutes recording screen 7 for creating the minutes of the meeting, including the input area 71 and the transcription area 73, on the display unit 34 of the user terminal 3-1 of the clerk user, based on the minutes template. The template may include the concept of a template.

[0039] The template is created on a template creation screen 1006, which will be described later. The template creation screen 1006, as will be described later, is a screen including objects for creating a template in which a template area for API linkage with a CRM tool is embedded. The display control unit 231 causes the template creation screen 1006 to be displayed on the display unit 34 of the user terminal 3-1. The processor 23 creates a template in response to receiving an instruction from the user terminal 3-1 of the secretary user to embed a template area in the minutes template. The storage control unit 232 causes the minutes template to be recorded for use when linking with the other tool.

[0040] Next, in A2, the processor 23 of the server 2 starts processing during the conference in response to receiving an instruction to start the conference from the user terminal 3-1 of the secretary user. That is, as processing during the conference, the processor 23 starts counting the elapsed time of the conference and starts recording by the minutes creation support tool. The processor 23 executes information processing of A6 in parallel with A3.

[0041] Next, in A3, the information transmitting / receiving unit 230 receives, via the network and the communication unit 21, from the user terminal 3, the voice data acquired by the sound collection device 4 of each user (the secretary user and the participating users).

[0042] Next, in A4, the information transmitting / receiving unit 230 transmits the voice data received from a certain user to a user terminal 3 owned by a user other than the certain user. For example, the information transmitting / receiving unit 230 transmits the voice data received from the user terminal 3-2 owned by the joining user to the user terminal 3-1 of the secretary user. After that, the processor 33 of the joining user outputs the voice data.

[0043] Next, in A5, the storage control unit 232 (sound recording unit) records the voice data received from the user terminal 3 of each user in the storage unit 22. After that, the processor 23 advances the process to A2. The processor 23 repeatedly executes the information processes of A3 to A5 until the conference ends.

[0044] Next, at A6, the processor 33 of the user terminal 3-1 of the clerk user accepts character input to the input area 61 via the clerk user's operation on the input unit 35. The information transmitting / receiving unit 230 accepts character data from the user terminal 3-1 of the clerk user.

[0045] Next, in A7, the display control unit 231 appropriately executes processing related to the input of this character in the input area 61 and the time stamp area 62 described below. After that, the processor 23 advances the process to A6. The processor 23 repeatedly executes the information processing of A6 to A7 until the conference ends.

[0046] Next, at A8, the processor 23 accepts an instruction to end the conference, such as pressing a conference end button or the like, from the user terminal 3 of the secretary user. The processor 23 ends counting the elapsed time of the conference and ends recording by the minutes creation support tool.

[0047] Next, the post-conference information processing will be described with reference to A9 to A12. First, in A9, the processor 33 of the user terminal 3-1 of the clerk user accepts a transcription request from the clerk user via the input unit 35. When the processor 33 accepts the transcription request, it transmits the transcription request to the server 2 via the communication unit 31 of the user terminal 3-1 and the network. The information transmitting / receiving unit 230 accepts the transcription request from the user terminal 3. When the server 2 accepts the transcription request, the voice recognition unit 233 executes transcription based on the recorded voice data. When the transcription is completed, the display control unit 231 displays the transcription information 730 shown in FIG. 7 in the transcription area 73.

[0048] Next, at A10, the processor 33 of the user terminal 3 accepts a predetermined operation through a user's operation of the input unit 35. When the processor 33 accepts the predetermined operation, it transmits information identifying the predetermined operation to the server 2 via the communication unit 31 and the network. The processor 23 accepts information identifying the predetermined operation from the user terminal 3 via the network and the communication unit 21.

[0049] Next, if the predetermined operation accepted by A10 is an operation including an instruction to complete the minutes, the processor 23 advances the information processing to A12. If the predetermined operation accepted by A10 is not an operation including an instruction to complete the minutes, the processor 23 advances the information processing to A11.

[0050] Next, at A11, the processor 23 of the server 2 executes information processing related to a predetermined operation. After that, the processor 23 advances the information processing to A10 again. A11 will be described with reference to FIGS.

[0051] Finally, in A12, the processor 23 of the server 2 stores the minutes in the storage unit 22. Thereafter, the processor 23 ends the information processing.

[0052] In A1, A8 to A10, the information transmitting / receiving unit 230 may receive an instruction to specify another tool to be linked with the minutes-taking tool from the user terminal 3-1 of the secretary user via the management screen 5. The linkage may be performed, for example, by API (Application Programming Interface) linkage, CSV linkage, etc. In this embodiment, a process is performed by API linkage to reflect part of the data output by the minutes-taking tool as data in some items on a CRM (Customer Relationship Management) tool.

[0053] A CRM tool is a tool that supports customer relationship management. CRM tools are used for centralized management of customer information, customer information analysis, promotion management, etc. CRM tools may be, for example, Salesforce Sales Cloud, Mazrica Sales, Zoho CRM, GENIEE SFA / CRM, Kintone, etc. The output destination tool may include any tool. In addition to a CRM tool, the output destination tool may include a document creation tool, a presentation tool for supporting presentations, a spreadsheet tool for creating spreadsheets, a project management tool for managing projects, a communication tool for supporting communication between multiple users, a design tool for supporting design creation, a code repository and version control tool for supporting source code management, a database management tool for managing databases, a CMS (Content Management System) tool for supporting website creation, an email marketing tool for supporting email distribution, a calendar tool, a recruitment management system, a storage service, a workflow tool, and the like. A document creation tool is a tool that can centrally manage the creation, management, storage, sharing, utilization, and disposal of documents. In addition, a document creation tool is used, for example, to improve the efficiency and convenience of handling documents. The document creation tool may be Notion, NotePM, Google Docs, Evernote, Zoho Learn, Cacoo, astah, Enterprise Architect, Gridraw, draw.io, etc. The communication tools may include groupware and the like. The calendar tool is a tool for registering schedules, tasks, goals, etc., visually organizing and displaying them, and sharing them with other employees. The calendar tool may be Google Calendar, Outlook, etc. The recruitment management system may be an ATS, etc. The storage service is a service that allows data to be stored in disk space for file storage on the Internet. The storage service may be Google Drive, OneDrive, etc. The workflow tool is a tool for streamlining in-house business procedures. The workflow tool may include JobCan, SmartHR, and the like.

[0054] 3.2 Details of how data is processed Next, the parts whose detailed description was omitted in the previous section will be described with reference to Figures 6 to 9. Specifically, the screen displayed in A6 will be described in detail with reference to Figure 6, the screen displayed in A9 with reference to Figure 7, and the screen displayed in A11 with reference to Figures 8 to 9.

[0055] First, the minutes recording screen 6 during a conference will be described with reference to FIG. 6. FIG. 6 is a diagram showing an example of the minutes recording screen 6 during a conference. The minutes recording screen 6 is a screen for creating minutes during a conference. The minutes recording screen 6 includes a conference information area 60, an input area 61, a timestamp area 62, and a transcription area 63. Note that the minutes recording screen 6 may be displayed by referring to a template. The template is created on a template creation screen 1006, which will be described later.

[0056] The meeting information area 60 is an area where an outline of the proceedings is displayed. The outline of the proceedings includes the name of the meeting, such as "Regular Advisory Meeting," the date and time of the meeting, such as "2023 / 4 / 20 10:00-11:00," the URL of the storage location of the materials used in the meeting, such as "https: / / meeting...," attendees, absentees, and the creator of the minutes.

[0057] The input area 61 is configured to be able to display input information. The input area 61 is configured to be able to display the result of input of input information performed by a clerk user. The input information is information input to the minutes creation tool. The input information is information including image data, audio data, video data, and other attached files in addition to character input information consisting of characters. Text information in the attached file may be treated as character input information. The input area 61 is also configured to be able to reflect information of a template of minutes. For example, the input area 61 is configured to be able to set areas such as template areas 610, 611, and 612 as templates. The input area 61 is configured to be able to further set the format of minutes (method of writing a document, appearance of the document, character style, etc.) as a template. For example, the template areas 610, 611, and 612 include information in the form of an agenda, a table, a graph, etc. The template areas 610, 611, and 612 are configured to be able to be arbitrarily set by the user. This allows the clerk user to create minutes more efficiently. This allows minutes including various information to be created.

[0058] The time stamp area 62 during the conference is configured to be capable of displaying time data relating to the elapsed time of the conference when input information is entered into the input area 61. That is, in response to accepting character input into the input area 61, the processor 23 records, in the time stamp area 62, time information on the time when the characters were entered into the input area 61. For example, it is assumed that the processor 23 accepts character input into the input area 61 when 2 minutes and 3 seconds have passed since the start of the conference. In this case, the processor 23 records time data such as "02:03" next to the location where the characters were entered and in the time stamp area 62. This makes it possible to record the time when the characters were entered into the minutes.

[0059] The transcription area 63 is configured to be able to display the transcription information 730 after transcription. Before transcription (i.e., during the conference), the transcription information 730 has not been generated, so the transcription area 63 does not display the transcription information 730.

[0060] Next, the minutes recording screen 7 after the meeting will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the minutes recording screen 7 after the meeting. The minutes recording screen 7 is a screen for creating minutes after the meeting. The minutes recording screen 7 includes a meeting information area 70, an input area 71, a time stamp area 72, a transcription area 73, a full text selection button 74, and a playback area 75. For the meeting information area 70 after the meeting, please refer to the meeting information area 60 during the meeting.

[0061] The post-conference input area 71 is configured to be able to display the result of input information input by the secretary user, similar to the input area 61 during the conference. In addition, the post-conference input area 71 is configured to be able to accept a drag-and-drop operation of the transcription information 730. When the post-conference input area 71 accepts the drag-and-drop operation, the post-conference input area 71 may be configured to be able to transcribe character information similar to the input character information of the transcription information 730. In addition, the output control unit 234 may also output output information to the input area 71 including the minutes template area. For example, the output control unit 234 may output output information to the cell by accepting a drag-and-drop operation of the first transcription information 730a from within the transcription area to a cell of the table of the template area 611.

[0062] The concept of a "drag-and-drop operation" includes the action of pointing the pointer (arrow) at a target object, pressing and holding the button of the mouse (or other pointing device), moving the pointer to the desired location (drag), and then releasing the button at the desired location (drop). When a touch panel is used, the drag-and-drop operation also includes the concept of being performed by touching with a finger, a touch pen, etc. instead of a pointer. The drag-and-drop operation is an example of a predetermined operation.

[0063] The output of the output information from the transcription area 73 to the input area 71 (including the template area 611) is not limited to a drag-and-drop operation, and may be performed by a predetermined operation such as a button operation (such as a transcription button described later). In addition, the predetermined operation may also include an operation of performing information processing using natural language processing, which will be described later, using the information in the transcription area 73.

[0064] The post-conference timestamp area 72 displays the first time information 720 recorded during the conference. The post-conference timestamp area 72 also displays the time associated with the first time information 720 of the transcription information 730 transcribed from the transcription area 73.

[0065] The post-conference transcription area 73 is configured to be able to display the results of transcription of statements made during the conference. As shown in FIG. 7, the post-conference transcription area 73 may be configured to display, without being limited thereto, a plurality of pieces of transcription information 730 arranged in chronological order by dividing the results of transcription of statements made during the conference at a predetermined timing. The predetermined timing may be any timing that allows the transcription results to be understood as a conversation flow, such as a timing when there is a breath, a timing when there is a pause, a timing at which a sentence can be determined such as a period, or a timing when the speaker changes. As a result, the transcription results are divided at predetermined timings, so that the transcription user can easily check the results.

[0066] The characters used for drag and drop may be dragged and dropped for each area divided at a predetermined timing as shown in Fig. 7, or may be dragged and dropped for the transcribed characters themselves. When dragging and dropping is performed for the characters themselves, the processor 23 may accept the designation of the characters by accepting a drag on the characters themselves, for example, and at this time, the processor 23 may accept a drag across multiple pieces of transcription information 730.

[0067] The transcription information 730 is information of characters transcribed based on the audio data of the conference. The transcription information 730 includes speaker information 731, text information 732, and time information. The speaker information 731 is information that can identify the user who made the remark derived from the transcription. The text information 732 is information of the transcribed characters. The time information is information of the time when the voice of the script linked to the script information was uttered during the conference. The time information is displayed in the time stamp area 72 when the time information is transcribed from the transcription area 73 to the input area 71. The transcription button is a button for transcribing the transcription information 730 linked to the transcription button to the input area 71 by accepting a press. The selection checkbox is configured to be able to select the transcription information 730 linked to the selection checkbox by accepting a selection.

[0068] The first transcription information 730a is one piece of transcription information 730 arbitrarily selected from the plurality of pieces of transcription information 730. The first transcription information 730a is configured so as to be treatable as information including first speaker information 731a, first character information 732a, first time information 720, first transcription button 733a, and first selection checkbox 734a. This allows various information to be included in the transcription result and the data to be used.

[0069] The output control unit 234 (transcription unit) outputs output information based on the first transcription information 730a into the input area by accepting a drag-and-drop operation of the first transcription information 730a from the transcription area 73 to the input area 71. For example, the output control unit 234 (transcription unit) transcribes the first transcription information 730a into the input area 71 by accepting a drag-and-drop operation of the first transcription information 730a from the transcription area 73 to the input area 71. The first transcription information 730a includes first time information 720. The first time information 720 is information on the time during the conference when the voice of the characters linked to the first transcription information 730a was uttered. This allows various information to be included in the transcription result and the data to be used for the minutes.

[0070] Output information is information that is output. The output information may include transcription information 730 that is transcribed directly from the transcription area 73. The output information may also include information generated by a large-scale language model, as described below.

[0071] The selection checkboxes are configured to be able to select multiple pieces of transcription information 730 by accepting a selection. The full text selection button 74 is able to select all of the transcription information 730 by accepting a selection. The selection unit 237 selects multiple pieces of transcription information 730 by accepting checks on multiple selection checkboxes or by accepting the full text selection button 74. The output control unit 234 (transcription unit) transcribes the selected multiple pieces of transcription information 730 into the input area 71 in response to a drag-and-drop operation on the selected multiple pieces of transcription information 730. This makes it possible to transcribe multiple pieces of transcription information 730 from the transcription area 73 to the input area 71 at once.

[0072] The playback area 75 is configured to be able to play back the audio during the conference. The processor 23 may generate a conference link that allows playback from any time during the conference (for example, playing back the audio of the conference from 7 minutes 35 seconds) and allows reference to the minutes recording screen 7. This allows users to share the minutes in a manner that makes it easy to play back parts of important remarks made during the conference.

[0073] FIG. 8 is a diagram showing an example of the minutes recording screen 7 after drag and drop. The display control unit 231 displays the time when the voice associated with the output information was generated. For example, the display control unit 231 displays information on the time when the voice of the character associated with the first transcription information 730a transcribed in the input area 71 was generated in a manner that is visible to the user. For example, if the time when the sound of "gu" in "I would like to conduct a customer survey" associated with the first transcription information 730a was generated was 12 minutes and 43 seconds after the start of the conference, the display control unit 231 displays "12:43" in the time stamp area 72. This allows the user to easily check the time related to the transcription information.

[0074] In response to a user's clicking on the first time information 720, the playback unit 235 plays back the audio of the conference from the time associated with the first time information 720. The playback of the audio of the conference may end when the audio portion associated with the first transcription information 730a has been completely played back, or all audio from the time associated with the first time information 720 onward may be played back continuously. This allows the user to easily play back the audio from the time associated with the transcription information 730.

[0075] Furthermore, when the first transcription information 730a transcribed in the input area 71 is mouse-overed, the display control unit 231 displays the speaker information 710 linked to the first transcription information 730a. The speaker information 710 is information about the speaker, such as a name, a user name, a user ID, or the like, that can identify the speaker. The speaker information 710 may be information that can be changed by the user. Instead of mouse-over, the display control unit 231 may display the speaker information 710 linked to the first transcription information 730a when the first transcription information 730a is dragged. This makes it possible to easily check the speaker linked to the transcription information 730 transcribed from the transcription area 73 to the input area 71.

[0076] FIG. 9 is a diagram showing an example of a case where various operations are to be performed on the proceedings recording screen 7. The filter application information 735 is information on characters obtained by removing characters that have no meaning as the contents of the conference from the characters included in the transcription information 730. That is, each piece of transcription information 730 may include characters that have no meaning as the contents of the conference, called 'keba'. The characters that have no meaning as the contents of the conference include some interjections, such as "eh", "ah", "um", "er", "well", and "um". That is, the filter control unit 236 controls one or more pieces of transcription information 730 as one or more pieces of filter application information 735. The output control unit 234 (transcription unit) outputs (transcribes in this embodiment) the first filter application information 735a into the input area 71 by accepting an operation of dragging and dropping the first filter application information 735a from the transcription area to the input area 71. The first filter application information 735a includes the second time information 721, and the second time information 721 is information on the time during the conference when the voice of the character associated with the first filter application information 735a was uttered. That is, the second time information 721 may be moved down as a result of removing characters that are meaningless as the contents of the conference from the first utterance, as compared with the first time information 720. This allows transcription from the transcription area 73 to the input area 71 in a form in which meaningless characters are removed. The first filter application information 735a is an example of the filter application information 735, and is arbitrarily selected from the filter information in this embodiment. The output first filter application information 735a may be processed directly by a large-scale language model, which will be described later.

[0077] The input area 71 is configured to be able to display linkage information 711, and the linkage information 711 is information that changes in association with the operation of a certain service. The linkage information 711 includes information output from any service, such as a CRM tool, a document creation tool, a presentation tool, a spreadsheet tool, a project management tool, a communication tool, a design tool, a code repository & version management tool, a database management tool, a CMS tool, and an email marketing tool. The processor 23 also updates the linkage information 711 in the input area 71 in association with an update of information on the service. In the example shown in FIG. 9, the linkage information 711 is information displayed in the form of a line graph in which sales amount indicating a sales service is displayed in a time series. When the sales amount information is updated in the sales service, the processor 23 also updates the linkage information 711 in the input area 71 in association with the update. The processor 23 specifies the minutes recording screen 8 on the minutes taking tool that displays data acquired from a tool different from the minutes taking tool. The processor 23 displays the data via the minutes recording screen 8 of the minutes taking tool. In this case, the tool other than the minutes-taking tool is an example of the first tool, and the minutes-taking tool is an example of the second tool. This makes it possible to include information linked to various services in the minutes.

[0078] [Information processing using natural language processing] The output control unit 234 may output output information generated using a large-scale language model to the input area based on at least one of the information in the CRN tool, the input information in the input area 71, and the transcription information 730. For example, in response to an instruction from a user via the input unit 35, the output control unit 234 executes various information processing exemplified below for the selected transcription information 730 or the selected input information in the input area 71 by using a large-scale language model (BERT, ChatGPT, etc.), and outputs the result of the processing. The processor 23 may accept the selection of the input information in the input area 71 by dragging a character string in the input area 71 to select a range. In addition, the output control unit 234 may output output information using a large-scale language model in response to a drag-and-drop operation from the transcription area 73 to the input areas 61, 71 (including the template areas 610, 611, 612, 1710, 1711). Furthermore, the output control unit 234 may generate output information by providing a prompt such as "based on only the transcription information" or "also using CRM tool information" to the large-scale language model. The prompt provided to the large-scale language model may be freely input by the user, or a default prompt may be specified. The number of characters of the information output to the input area 71 may be arbitrarily set by the user (for example, short (about 90 characters), normal (about 300 characters), long (about 900 characters), or a specified number of characters).

[0079] The output information generated using the large-scale language model may also include the first time information 720. The first time information 720 is information associated with the output information about the time of the meeting when the input information was input, or information associated with the transcription information about the time when the voice during the meeting was uttered.

[0080] The position (output destination) where the result of using the large-scale language model is output may be a position specified by the user, or may be a position selected by the large-scale language model. The identification unit 238 identifies an output destination to which output information is output on a screen for creating the minutes of the meeting. The identification unit 238 may identify template areas 1710 and 1711, which will be described later, as the output destination, or may identify any location in the input area 71 specified by the user as the output destination. In addition, the identification of the output destination by the identification unit 238 is not limited to only on the screen for creating the minutes of the meeting. For example, the identification unit 238 may identify a location specified by the user via another tool linked to the minutes-creating tool as the output destination, or may identify a location determined by the program of another tool linked to the minutes-creating tool as the output destination. The output control unit 234 may output the output information generated using the large-scale language model to the minutes reflecting the information of the template (to the template areas 610, 611, 612, 1710, and 1711). The specification unit 238 may specify the output destination determined by the large-scale language model. The output control unit 234 outputs the generated output information to the specified output destination. The output timing may be the timing when the output information is generated or may be the timing specified by the user.

[0081] [Drag the transcripts together to extract key points] For example, the output control unit 234 generates a summary of the meeting based on the selected transcription information 730 or the selected input information in the input area 71, and outputs the summary of the meeting to the input area 71. The summary is a summary of important points of the meeting in one or more sentences in bullet points. The summary may or may not be limited to a predetermined number of characters.

[0082] [Final copy function (removing fuzz, converting written text)] For example, the output control unit 234 may perform a clean-up process on the selected transcription information 730 based on the selected transcription information 730 or the input information in the selected input area 71, and then output the transcription information 730 to the input area 71. The clean-up process may be, for example, a process of removing the above-mentioned burrs, or a process of converting spoken language into written language. The process of converting spoken language into written language includes a process of converting "so / no de / dakara / desukara" to "so / takara", a process of converting "after all" to "after all", and the like.

[0083] [Summary function] For example, the output control unit 234 generates a summary of the meeting based on the selected transcription information 730 or the selected input information in the input area 71, and outputs the summary of the meeting to the input area 71. The summary is a written sentence that briefly summarizes what was discussed in the entire meeting. The summary may or may not be restricted to a predetermined number of characters.

[0084] [Support for extracting next actions and decisions] For example, the output control unit 234 may extract a to-do item or a decision item based on the selected transcription information 730 or the input information in the selected input area 71, and output the extracted to-do item or decision item to the input area 71. A to-do item is an item that is decided during a meeting as a next action, such as creating materials. A decision item is an item that is agreed upon during a meeting, such as the date and time of the next meeting.

[0085] [Automatic Agenda Extraction] For example, the output control unit 234 extracts key topics during the meeting based on the selected transcription information 730 or the input information in the selected input area 71, and outputs the extracted topics to the post-meeting input area 71. For example, if the selected transcription information 730 is a topic related to sales performance, the processor 23 outputs a topic such as "developing new customers" to the input area 71. This process may be performed based on all of the transcription information 730 in the transcription area 73.

[0086] [Automatically reflect on seek bar] For example, when the output control unit 234 extracts an agenda item that is a key point in a meeting based on the selected transcription information 730 or the input information selected in the input area 71, the output control unit 234 may display the start time of the agenda item on the seek bar in the playback area 75. For example, when an agenda item regarding the development of new customers starts at the point 25 minutes and 00 seconds after the start time of the meeting, the processor 23 links and displays the words "development of new customers" at the position corresponding to 25 minutes and 00 seconds on the seek bar of the playback area 75. This process may be performed based on all of the transcription information 730 in the transcription area 73.

[0087] [Automatic generation of minutes outline] The output control unit 234 may generate an outline of the minutes based on the selected transcription information 730 and output it to the input area 71. The outline of the minutes includes an agenda, decisions, ToDo items, a summary, etc. This process may be performed based on all of the transcription information 730 in the transcription area 73.

[0088] [Convert to easy-to-read text] The output information may include information on characters calibrated using a large-scale language model. Calibration may include making abstract words concrete. For example, calibration may include replacing demonstratives such as "this" and "that" with concrete words. Calibration may also include replacing words indicating dates and times such as "today," "yesterday," and "two hours ago" with words indicating specific dates and times such as "March 31, 2024, 14:30." This allows the minutes to be created by replacing the characters with easy-to-read characters.

[0089] [Output for each input topic] The output information may include characters related to the contents of the meeting corresponding to the agenda. In this case, the input information may include information on the agenda of the meeting. The agenda may include a concept of a theme or purpose to be discussed at the meeting. The processor 23 identifies the time during the meeting when the input information including the agenda information was input (time information included in the time stamp area 72 may be used) and the transcription information 730 close to the time. The output control unit 234 inputs the input information and the transcription information into a large-scale language model, and outputs output information including characters related to the contents of the meeting corresponding to the agenda to the input area 71. As a result, the AI ​​automatically generates the contents of the agenda, thereby reducing the effort of the user to input the information.

[0090] [AI note completion] The output information may include character information that complements the contents of the input information. Complement means to supplement missing information. The processor 23 identifies the time during the conference when the input information was input (time information included in the time stamp area 72 may be used) and the transcription information 730 that is close to the time. The output control unit 234 inputs the input information and the transcription information into a large-scale language model, and outputs output information including characters that complement the contents of the input information to the input area 71. As a result, the AI ​​automatically complements the contents of notes, etc., that the user has simply input, thereby reducing the effort required for the user to input information.

[0091] The following explanation may overlap.

[0092] Next, the creation of a template will be described with reference to Figs. 10 and 11. Fig. 10 is a diagram showing an example of a management screen 1005. The management screen 1005 in Fig. 10 displays a list of tools that can be linked when creating a minutes template. The management screen 1005 is a screen for managing the settings of the minutes creation tool. The management screen 1005 includes at least a linking tool area 1050. The management screen 1005 may further include a button for transitioning to a screen that can manage workspaces, users, groups, and spaces.

[0093] The linked tool area 1050 is an area where information about other tools that can be linked with the minutes creation tool is displayed. The linked tool area 1050 displays names, icons, etc. that can identify the various tools described above. The linked tool area 1050 includes a linked tool area 1500 and an unlinked tool area 1501.

[0094] The linked tool area 1500 is an area where information about the minutes creation tool and linked tools is displayed. The linked tool area 1500 includes the name of the linked tool (e.g., "Service A"), information indicating that the tool is linked (e.g., "Linked"), and a template creation button 1500a. In response to the secretary user pressing the template creation button 1500a, the display control unit 231 displays the template creation screen 1006 shown in FIG. 11.

[0095] The unlinked tool area 1501 is an area where information about tools that are not linked with the minutes creation tool is displayed. The unlinked tool area 1501 includes the name of the linked tool (e.g., "Service B"), information indicating that the tool is not linked (e.g., "unlinked"), and a link button 1501a. In response to the secretary user pressing the link button 1501a, the display control unit 231 displays the login of the tool associated with the link button. In response to successful login to the tool, the processor 23 starts linking with the tool. Thereafter, the display control unit 231 changes the display of the tool that has started linking from the unlinked tool area 1501 to the linked tool area 1500. FIG. 11 is a diagram showing an example of a template creation screen 1006. The template creation screen 1006 is a screen for creating minutes template information. The minutes template information is saved in advance as information including information on the title of the minutes and information to be input in advance to the minutes, and is used as a template when creating new minutes. The template creation screen 1006 in FIG. 11 is a screen for saving information setting API collaboration between Service A as a CRM tool and a minutes creation tool as minutes template information. The template creation screen 1006 includes a collaboration option area 1060, a title area 1061, a collaboration area 1062, and a save button 1063.

[0096] The linkage option area 1060 is an area where options that can be embedded as information to be input in advance into the minutes of the minutes template information are displayed. The linkage option area 1060 in Fig. 11 is configured so that items such as purpose, background of inquiry, expected number of users, minutes format, next step, and memo can be set as template areas. The linkage option area 1060 may display different options for each linked tool.

[0097] The title area 1061 is an area where information on the title of the minutes can be set, and is configured so that, for example, "202YMMDD_[client]_new business negotiation" can be set.

[0098] The link area 1062 is configured to be able to embed various information as a template. For example, the link area 1062 is configured to be able to embed information such as characters, graphs, tables, images, and videos. The link area 1062 is also configured to be able to drag and drop items from the link option area 1060. The display control unit 231 embeds the item that has been accepted for drag and drop into the template. When drag and drop is performed on the background of the inquiry or the template area of ​​the next step, the identification unit 238 identifies the location where the drag and drop is performed as the output destination of the output information. That is, when the identification unit 238 receives a designation of the output destination of the output information on the minutes from the clerk user, it identifies the accepted output destination. The identification by the identification unit 238 is not limited to execution on the template creation screen 1006, but is also executed when the template area is embedded on the minutes recording screen 1007. With such a configuration, it is possible to specify the destination to which the output information is to be output.

[0099] The save button 1063 is a button for saving the minutes template information. In response to pressing the save button 1063, the storage control unit 232 stores the minutes template information set via the title area 1061 and the link area 1062 in the storage unit 22. That is, the storage control unit 232 stores the output destination of the meeting content information in the minutes accepted by the specification unit 238 in the storage unit 22. With this configuration, the output destination of the output information in the minutes can be saved and used for creating the next or subsequent minutes, so that it is possible to omit designation of the output destination when creating the next or subsequent minutes.

[0100] Next, referring to Fig. 12, a pre-conference minutes recording screen 1007 will be described using the minutes template information set in Fig. 11. Fig. 12 is a diagram showing an example of the pre-conference minutes recording screen 1007. The minutes recording screen 7 is a screen for creating pre-conference minutes. The minutes recording screen 1007 includes a conference information area 1070, an input area 1071, a timestamp area 1072, and a transcription area 1073.

[0101] The conference information area 1070 is an area where an outline of the proceedings is displayed. In the example of Fig. 12, "202YMMDD_[Customer]_New Business Negotiation" is displayed as the name of the conference. The information entered in the title area 1061 in Fig. 11 is entered in the conference information area 1070 in Fig. 12.

[0102] The input area 1071 is configured to be able to display the result of character input by a scribe user. The input information is treated as input information. The input information is information input via a minutes-taking tool. The input information is information including image data, audio data, video data, and other attached files in addition to character input information consisting of characters. Text information in the attached file may be treated as character input information. The character input information may be information in bulleted form as long as it includes characters, or may be information only of numbers, symbols, words, etc. The template area input in the link area 1062 in FIG. 11 is input in the input area 1071 in FIG. 12. That is, the input area 1071 is configured to be able to embed the template areas 1710 and 1711. The template areas 1710 and 1711 are areas into which data is input in a predetermined manner. In the example of FIG. 8, the template area 1710 is an area into which background information of an inquiry is input as output information, and the template area 1711 is an area into which information of a next step is input as output information.

[0103] The time stamp area 1072 is configured to be capable of displaying time data relating to the elapsed time of the conference when information is input into the input area 1071. That is, in response to accepting input of information such as characters into the input area 1071, the processor 23 records, in the time stamp area 1072, time information of the time when the characters were input into the input area 1071. For example, it is assumed that the processor 23 accepts input of characters into the input area 1071 when 2 minutes and 3 seconds have elapsed since the start of the conference. In this case, the processor 23 records time data such as "02:03" next to the location where the characters were input and in the time stamp area 1072. This makes it possible to record the time when the characters were input into the minutes.

[0104] The transcription area 1073 is configured to be able to display the transcription information 1830 after transcription. In a state before transcription (i.e., before or during the conference), the transcription information 1830 is not generated, so the transcription area 73 does not display the transcription information 1830.

[0105] Next, the after-conference minutes recording screen 1008 will be described with reference to Fig. 13. Fig. 13 is a diagram showing an example of the after-conference minutes recording screen 1008. The after-conference minutes recording screen 1008 is a screen for creating the minutes of a meeting after the meeting. The after-conference minutes recording screen 1008 includes a meeting information area 1080, an input area 1081, a time stamp area 1082, a transcription area 1083, and a playback area 1085. For the after-conference meeting information area 1080, please refer to the before-conference meeting information area 1070.

[0106] The post-conference input area 1081 is configured to be able to display the results of character input by a secretary user, similar to the pre-conference input area 1071. In addition, the post-conference input area 1081 is configured to be able to accept drag and drop, as described below. The post-conference input area 1081 also includes linked template areas 1810 and 1811 to which output information has been output.

[0107] The output information includes text information generated based on the audio data, the transcription information 1830, or the input information in the input area 1071. The output information includes textual conference content information indicating the contents of the conference. The conference content information is information generated by applying a large-scale language model to the transcription information 1830 or the information in the input area 1071. The conference content information includes information on a summary of the conference, information on post-conference ToDos, information on decisions made at the conference, information on a list of key points of the conference, information on remarks made by speakers participating in the conference, information on names of people involved in the conference, or information on questions and answers at the conference. For example, a summary is a written statement that briefly summarizes what was discussed during the entire meeting. The summary may or may not be constrained to a certain number of characters. ToDo items are matters that are decided during the meeting as the next action, such as creating materials. Decisions are matters agreed upon during the meeting, such as the date and time of the next meeting. A gist is a one or more sentence summary of the important points from a meeting, using bullet points. A gist may or may not be constrained to a certain number of characters. Information on the utterances of speakers participating in the conference may be information summarized by speaker, information separated by utterance, information in which the information on each utterance is linked with time, or information in which the information on each utterance is linked with the speaker. The participants in the meeting are information on the names or surnames of people who will appear in the meeting. The participants in the meeting may be specified along with their job titles, titles, company names, department names, and the like. A question and answer session is information on a combination of questions and answers during a conference. The information on a question and answer session during a conference may be information in which one question is associated with multiple answers, or information in which one answer is associated with multiple questions. With this configuration, information on various aspects of the meeting, such as post-meeting ToDos, decisions made at the meeting, names of people involved in the meeting, a summary of the meeting, a list of key points of the meeting, remarks made by speakers participating in the meeting, or questions and answers at the meeting, can be output as output information.

[0108] The generated meeting content information changes depending on the output destination. For example, when the background and next step of the inquiry are set as the linkage template areas 1810 and 1811, the output control unit 234 generates a list of the main points of the meeting corresponding to the background and next step of the inquiry. In addition, the output control unit 234 outputs the list of the main points of the meeting corresponding to the background and next step of the inquiry to the linkage template areas 1810 and 1811, respectively. In addition, for another example of the meeting content information that changes depending on the output destination, the output control unit 234 generates the meeting content information in a format corresponding to the tool that is the output destination identified by the identification unit 238. For example, when the output destination is a column in which the information of the number of people is the main focus, the output control unit 234 generates the meeting content information in the format of the number of people (3 people). In addition, when the output destination is a column in which information such as probability (10%, etc.), amount (10,000 yen, 100 dollars, etc.), date (September 15, 2023, etc.) is the main focus, the output control unit 234 generates the meeting content information in a format corresponding to them. According to this configuration, it is possible to generate output information appropriately according to the output destination.

[0109] In addition, the output control unit 234 may generate each of the meeting content information after the speech recognition unit 233 completes the transcription. Thereafter, the identification unit 238 may identify an output destination based on the information indicated by the meeting content information. For example, the identification unit 238 identifies one or more other tools linked with the minutes-taking tool, and further identifies a location on the identified other tool that is linked with the meeting content information of the minutes-taking tool as an output destination. For example, the identification unit 238 identifies that information on the gist of the meeting is set as an output destination in a specific item on a CRM tool linked with the minutes-taking tool. The output control unit 234 outputs the meeting content information to the output destination of the meeting content information on the other tool identified by the identification unit 238. According to such a configuration, the output information can be output to an appropriate output destination based on the content of the output information.

[0110] In addition to the above-mentioned functions, the time stamp area 1082 is configured to be able to display time data related to the elapsed time of the conference from the start of the speech of the related transcription information 1830 when the conference content information is output to the input area 1081. Also, the time stamp area 1082 is configured to be able to display time data related to the elapsed time of the conference input to the related input area 1081 when the conference content information is output to the input area 1081. When the information is input to the input area 1081 in bullet points, the processor 23 assigns a time stamp (such as "00:30" or "02:45" in the example of FIG. 9) to each bullet point item. In response to the time stamp being selected, the playback unit 235 moves the position of the seek bar to the elapsed time of the conference associated with the time stamp. Also, the time may be displayed on the seek bar in the playback area 1085. For example, if 30 seconds have elapsed since the start of a meeting and the background to the inquiry is something like, "I heard about it from an acquaintance and it caught my attention so I searched for it," processor 23 may link and display the words "Developing new customers" at the position corresponding to 30 seconds on the seek bar of playback area 1085.

[0111] FIG. 14 is a diagram showing an example of the CRM screen 1009. The CRM screen 1009 is a screen of the CRM tool that is API-linked with the minutes-taking tool. That is, the CRM screen 1009 may include some information of the minutes-taking screen 1008. In the example of FIG. 14, the information included in the linkage template areas 1810 and 1811 is displayed as linkage information 1090 and 1091 on the CRM screen 1009. The linkage information 1090 includes output information indicating the background of the inquiry in the linkage template area 1810, and the linkage information 1091 includes output information indicating the next step of the linkage template area 1810. The CRM screen 1009 may display some of the minutes-taking screens 1007 and 1008 displayed on the minutes-taking tool as they are. That is, the processor 23 specifies the CRM screen 1009 on the CRM tool that displays the data acquired from the minutes-taking tool. The processor 23 displays the data via the CRM screen 1009 of the CRM tool. In this case, the minutes-taking tool is an example of a first tool, and the CRM tool is an example of a second tool.

[0112] According to the present disclosure, it is possible to provide a technology that can assist in creating minutes in a more convenient manner.

[0113] [others] The information processing method includes each step of the information processing system 1. The program is a program that causes at least one computer to execute each step of the information processing system 1.

[0114] In the above embodiment, the server 2 performs various storage and control, but multiple external devices may be used instead of the server 2. In other words, various information and programs may be distributed and stored in multiple external devices using block chain technology or the like.

[0115] In a modified example, the second time information 721 may be information indicating the same time as the first time information 720, regardless of whether or not characters that have no meaning in the content of the conference have been removed.

[0116] In a modified example, the selection unit 237 may select information in the transcription area 73 by dragging a character string to select a range instead of or in addition to the selection checkboxes.

[0117] The server 2 may be in an on-premise form or in a cloud form. As the server 2 in a cloud form, the above functions and processes may be provided in the form of, for example, SaaS (Software as a Service) or cloud computing.

[0118] At least one of the devices included in the information processing system 1 may be installed outside Japan. For example, the server 2 may be installed outside Japan, and each terminal device may be installed inside Japan. Similarly, a user may access the server 2 installed inside Japan from outside Japan using his / her own terminal device. According to such an embodiment, a more convenient experience can be provided to the user through various management forms.

[0119] In the embodiment, an example has been described in which the output control unit 234 continuously generates the output information (meeting content information) as a series of steps after the transcription is completed. However, in a modified example, the output control unit 234 may generate the output information (meeting content information) when an instruction is received from the user after the transcription is completed.

[0120] Furthermore, the present invention may be provided in the following manner.

[0121] (1) An information processing system comprising at least one processor that executes each of the following steps: a display control step displays a screen for creating minutes of a meeting, the screen including an input area and a transcription area, the input area being configured to be able to display input information, the input information being information to be input, the transcription area being configured to be able to display transcription information, the transcription information being text information transcribed based on audio of the meeting, and an output control step outputs output information based on the first transcription information into the input area by accepting a predetermined operation from within the transcription area to the input area to output first transcription information, the output information being information to be output, the first transcription information including first time information, the first time information being information on a time during the meeting when the audio of the text associated with the first transcription information was uttered.

[0122] (2) In the information processing system described above in (1), in the display control step, a time when a sound associated with the output information was uttered is displayed.

[0123] (3) An information processing system according to (2) above, further comprising a playback step, in which the audio of the conference is played back from a time associated with the first time information in response to the first time information being selected.

[0124] (4) An information processing system according to any one of (1) to (3) above, further comprising: a filter control step, in which the transcription information is converted into filter application information, and the filter application information is character information obtained by removing characters that have no meaning in the content of the conference from characters included in the transcription information; and in the output control step, first filter application information is output into the input area by accepting the specified operation from within the transcription area to the input area, and the first filter application information includes second time information, and the second time information is information on a time during the conference when the voice of the characters associated with the first transcription information was uttered.

[0125] (5) In the information processing system according to any one of (1) to (4) above, the input area is configured so that information on a template for minutes of a meeting can be reflected.

[0126] (6) In the information processing system described in (5) above, the template includes a form in which the input area includes a table, and in the output control step, the output information is output to the cell by accepting the specified operation of the first transcription information from within the transcription area to a cell of the table in the input area.

[0127] (7) In the information processing system described in any one of (1) to (6) above, the input area is configured to be able to display linkage information, and the linkage information is information that changes in conjunction with the operation of a certain service.

[0128] (8) An information processing system according to any one of (1) to (7) above, further comprising a selection step, in which a selection of a plurality of pieces of transcription information is accepted in the selection step, and in which output information based on each of the selected pieces of transcription information is output within the input area in response to the drag-and-drop operation on the selected pieces of transcription information in the output control step.

[0129] (9) In an information processing system described in any one of (1) to (8) above, in the display control step, when the output information in the input area is moused over, information about the speaker linked to the first transcription information is displayed.

[0130] (10) The information processing system according to any one of (1) to (9) above, wherein the output information includes character information calibrated using a large-scale language model.

[0131] (11) In the information processing system according to any one of (1) to (10) above, the predetermined operation is drag and drop.

[0132] (12) An information processing system comprising at least one processor that executes each of the following steps: a display control step displays a screen for creating minutes of a meeting, the screen including an input area and a transcription area, the input area being configured to be able to display input information, the transcription area being configured to be able to display transcription information, the transcription information being text information transcribed based on audio of the meeting, the input information being information to be input; an output control step outputs output information generated using a large-scale language model based on at least one of the input information and the transcription information to the input area, the output information being information to be output, the output information including first time information, the first time information being information of the time of the meeting when the input information was input, which is linked to the output information, or information of the time when audio during the meeting, which is linked to the transcription information, was uttered.

[0133] (13) In the information processing system described above in (12), the output information includes character information that complements the content of the input information.

[0134] (14) In the information processing system described in (12) or (13) above, the input information includes information on an agenda for the meeting, and the output information includes text related to the content of the meeting corresponding to the agenda.

[0135] (15) In the information processing system described in any one of (12) to (14) above, the input area is configured to be able to reflect information of a minutes template, and in the output control step, output information generated using a large-scale language model is output to the minutes reflecting the information of the template.

[0136] (16) An information processing method comprising the steps of the information processing system according to any one of (1) to (15) above.

[0137] (17) A program causing at least one computer to execute each step of the information processing system according to any one of (1) to (15) above. Of course, this is not the case.

[0138] The following aspect may also be adopted. (1) An information processing system comprising at least one processor that executes each of the following steps: a display control step displays a screen for creating minutes of a meeting, the screen including a character input area and a transcription area, the character input area being configured to be capable of displaying the results of character input by one or more users, the transcription area being configured to be capable of displaying one or more pieces of transcription information in chronological order, the transcription information being character information transcribed based on audio of the meeting; and a transcription step transcribing first transcription information into the character input area by accepting a drag-and-drop operation from within the transcription area to the character input area, the first transcription information including first time information, the first time information being information on a time during the meeting when the audio of the characters associated with the first transcription information was uttered.

[0139] (2) In the information processing system described in (1) above, in the display control step, information on the time when the voice of the characters linked to the first transcription information output in the character input area was uttered is displayed in a manner that is visible to the user.

[0140] (3) An information processing system according to (2) above, further comprising a playback step, in which the audio of the conference is played back from a time associated with the first time information in response to a user's selection of the first time information.

[0141] (4) An information processing system according to any one of (1) to (3) above, further comprising: a filter control step, in which one or more pieces of transcription information are controlled as one or more pieces of filter application information in the filter control step; the filter application information being character information in which characters included in the transcription information have been removed that have no meaning in the content of the conference; and in the transcription step, the first filter application information is transcribed into the character input area by accepting the drag-and-drop operation from within the transcription area to the character input area; the first filter application information includes second time information, and the second time information is information on a time during the conference when the voice of the characters associated with the first transcription information was uttered.

[0142] (5) In the information processing system described in any one of (1) to (4) above, the character input area is configured so that a format can be set based on template information, and the template information is information relating to a template for a minutes format.

[0143] (6) In the information processing system described in any one of (1) to (5) above, the character input area is configured to be able to display linkage information, and the linkage information is information that changes in conjunction with the operation of a certain service.

[0144] (7) An information processing system according to any one of (1) to (6) above, further comprising a selection step, in which a plurality of pieces of transcription information are selected in the selection step, and in which the selected plurality of pieces of transcription information are transcribed into the character input area in accordance with the drag-and-drop operation on the selected plurality of pieces of transcription information in the transcription step.

[0145] (8) In an information processing system described in any one of (1) to (7) above, in the display control step, when the mouse is hovered over the first transcription information transcribed in the character input area, information about the speaker linked to the first transcription information is displayed.

[0146] (1) A program that causes a computer to execute the following steps: in a specification step, an output destination to which character generation information is output on a screen for creating meeting minutes is specified; the character generation information is character information generated based on transcription information or input information; the transcription information is character information transcribed from the content of an utterance; and the input information is information input by a user; and in an output control step, the character generation information is output to the output destination.

[0147] With this configuration, the convenience of the minutes-taking tool can be further improved.

[0148] (2) In the program described in (1) above, the character generation information includes meeting content information indicating the content of the meeting, and the meeting content information includes information on ToDos after the meeting, information on decisions made at the meeting, information on names of people involved in the meeting, information on remarks made by speakers attending the meeting, information on a summary of the meeting, information on a list of key points of the meeting, or information on questions and answers at the meeting.

[0149] With this configuration, information on various aspects of the meeting, such as post-meeting ToDos, decisions made at the meeting, names of people involved in the meeting, a summary of the meeting, a list of key points of the meeting, remarks made by speakers attending the meeting, or questions and answers at the meeting, can be output as character generation information.

[0150] (3) In the program according to (2) above, the conference content information changes depending on the output destination identified in the identifying step.

[0151] According to this configuration, it is possible to generate character generation information appropriately according to the output destination.

[0152] (4) The program according to (2) above, wherein in the specifying step, the output destination is specified based on information indicated by the conference content information.

[0153] According to this configuration, it is possible to output the character generation information to an appropriate output destination based on the contents of the character generation information.

[0154] (5) A program according to (1) above, wherein in the identifying step, when a designation of the output destination of the character generation information on the minutes is received from a user, the received output destination is identified.

[0155] According to this configuration, it is possible to specify the destination to which the character generation information is to be output.

[0156] (6) A program according to the above (5), which executes a storage control step, and in the storage control step, stores the output destination of the character generation information on the minutes accepted in the identification step.

[0157] According to this configuration, the output destination of the character generation information on the minutes can be stored and used for creating the next and subsequent minutes.

[0158] (7) The program according to (1) above, wherein the output destination is a CRM tool or a document creation tool.

[0159] According to such a configuration, it is possible to more appropriately output information in a CRM tool or a document creation tool, and to further improve convenience.

[0160] (8) A program for causing a computer to execute the following steps: in an identification step, a screen on a second tool for displaying data obtained from a first tool is identified; in an output control step, the data is displayed via the screen on the second tool; and one of the first tool and the second tool is a tool for creating minutes of a meeting.

[0161] With this configuration, the convenience of the minutes-taking tool can be further improved.

[0162] Finally, although various embodiments of the present invention have been described, these are presented as examples and are not intended to limit the scope of the invention. The novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. The embodiments and their modifications are included within the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0163] 1: Information processing system 2: Server 20: Communication bus 21: Communications Department 22: Storage section 23: Processor 230: Information transmission and reception unit 231: Display control unit 232: Memory control unit 233: Voice recognition unit 234: Transcription department 235: Playback section 236: Filter control section 237: Selection section 3: User terminal 30: Communication bus 31: Communications Department 32: Storage section 33: Processor 34: Display section 35: Input section 4: Sound collection device 6: Recording of proceedings 60: Meeting information area 61: Input area 610: Template information 611: Template information 612: Template information 62: Timestamp area 63: Transcription area 7: Recording of proceedings 70: Meeting information area 71: Input area 710: Speaker information 711: Collaboration information 72: Timestamp area 720: 1st hour information 721: 2nd Hour Information 73: Transcription area 730: Transcription information 730a: First transcript 731: Speaker information 731a: First speaker information 732: Text information 732a: First character information 733a: First posting button 734a: First selection checkbox 735: Filter application information 735a: First filter application information 74: Full text selection button 75: Play area 1005: Management screen 1050: Collaboration tool area 1500: Linked tools area 1500a: Template creation button 1501: Unlinked tool area 1501a: Link button 1006: Template creation screen 1060: Linkage options area 1061: Title area 1062: Collaboration Area 1063: Save button 1007: Recording of proceedings 1070: Meeting information area 1710: Template region 1711: Template region 1071: Input area 1072: timestamp area 1073: Transcription area 1008: Recording of proceedings 1080: Meeting information area 1081: Input area 1810: Collaboration template area 1811: Collaboration template area 1082: timestamp area 1083: Transcription area 1830: Transcription information 1085: Play area 1009 :CRM screen 1090: Collaboration information 1091: Collaboration information

Claims

1. An information processing system, at least one processor that performs the steps of: In the display control step, a screen for creating minutes of a meeting, including an input area and a transcription area, is displayed. the input area is configured to be capable of displaying input information, the input information being information to be input; the transcription area is configured to be able to display transcription information, the transcription information being information of text transcribed based on the audio of the conference; In the output control step, a predetermined operation is received from the transcription area to the input area to output output information based on the first transcription information into the input area, and the output information is information to be output. The first transcription information includes first time information, and the first time information is information about a time during the conference when a voice of the characters associated with the first transcription information was uttered. Information processing system.

2. 2. The information processing system according to claim 1, In the display control step, a time when the sound associated with the output information was uttered is displayed. Information processing system.

3. 3. The information processing system according to claim 2, performing a further regeneration step; In the reproducing step, in response to the selection of the first time information, the audio of the conference is reproduced from a time associated with the first time information. Information processing system.

4. 2. The information processing system according to claim 1, further performing a filter control step; In the filter control step, the transcription information is converted into filter application information, and the filter application information is character information obtained by removing characters that have no meaning as to the contents of the conference from characters included in the transcription information, In the output control step, the first filter application information is output into the input area by receiving the predetermined operation from the transcription area to the input area, and The first filter application information includes second time information, and the second time information is information about a time during the conference when a voice of a character associated with the first transcription information was uttered. Information processing system.

5. 2. The information processing system according to claim 1, The input area is configured to allow information of a minutes template to be reflected. Information processing system.

6. 6. The information processing system according to claim 5, The template includes a form in which a table is included in the input area, In the output control step, the output information is output to a cell of the table in the input area by accepting the predetermined operation from within the transcription area to the cell of the table in the input area. Information processing system.

7. 2. The information processing system according to claim 1, The input area is configured to be capable of displaying link information, and the link information is information that changes in association with an operation of a certain service. Information processing system.

8. 2. The information processing system according to claim 1, Performing further selection steps; The selecting step includes accepting a selection of a plurality of pieces of transcription information; In the output control step, output information based on each of the selected pieces of transcription information is output into the input area in response to the drag-and-drop operation on the selected pieces of transcription information. Information processing system.

9. 2. The information processing system according to claim 1, In the display control step, when a mouse is hovered over the output information in the input area, information on a speaker associated with the first transcription information is displayed. Information processing system.

10. 2. The information processing system according to claim 1, the output information includes character information calibrated using a large-scale language model; Information processing system.

11. 2. The information processing system according to claim 1, The predetermined operation is drag and drop. Information processing system.

12. An information processing system, at least one processor that performs the steps of: In the display control step, a screen for creating minutes of a meeting, including an input area and a transcription area, is displayed. the input area is configured to be capable of displaying input information; the transcription area is configured to be able to display transcription information, the transcription information being information of text transcribed based on the audio of the conference, and the input information being information to be input, In the output control step, output information generated using a large-scale language model based on at least one of the input information and the transcription information is output to the input area, and the output information is information to be output; The output information includes first time information, and the first time information is information associated with the output information, the information being information on a time of the meeting when the input information was input, or information associated with the transcription information, the information being information on a time when a voice was uttered during the meeting. Information processing system.

13. 13. The information processing system according to claim 12, the output information includes text information that complements the content of the input information; Information processing system.

14. 13. The information processing system according to claim 12, The input information includes an agenda for the meeting; The output information includes text related to the content of the meeting corresponding to the agenda. Information processing system.

15. 13. The information processing system according to claim 12, The input area is configured to allow information of a minutes template to be reflected, In the output control step, output information generated using a large-scale language model is output to the minutes reflecting the information of the template. Information processing system.

16. 1. An information processing method, comprising: The information processing system according to any one of claims 1 to 15, Information processing methods.

17. A program, A method for causing at least one computer to execute each step of the information processing system according to any one of claims 1 to 15. program.

Images