Voice management system, voice recording management device, voice recording management method and program

Abstract

To provide a voice management system, a voice record management device, a voice record management method, and a program that can provide a user with easy-to-read text information when voice information concerning voice uttered by the user is converted to text information to use, because the conversion is performed corresponding to organizational information of an organization to which the user belong.SOLUTION: A voice record management device 5 receives voice information transmitted by a communication terminal 3 concerning voice uttered by a user using the communication terminal 3 S62, and transmits corrected text information including a corrected text recognized from the received voice information among texts recognized from received voice information, which is corrected based on organizational information of an organization to which the user belongs, to one or more communication terminals 3 S66, S70. The communication terminal 3 that received the corrected text information displays on its display the voice record indicated by the corrected text information transmitted by the voice record management device 5, S71.SELECTED DRAWING: Figure 13

Description

[Technical field]

[0001] The present invention relates to a voice management system, a voice recording management device, a voice recording management method, and a program. [Background technology]

[0002] A technique for analyzing speech information and converting it into text information has been known for some time. In this technique, for example, a method is known in which a dictionary describing a plurality of text sequences that are candidates for speech recognition results is referenced to determine text corresponding to the speech.

[0003] For example, when creating minutes by converting voice information into text data based on a dictionary, a technique is known in which the accuracy of voice recognition is improved by switching the dictionary (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, when converting voice information related to a voice uttered by a user into text information for use, there was no idea of ​​converting the text information into text information in association with organizational information of the organization to which the user belongs. This resulted in a problem that the converted text information was easily converted into text information with a different meaning from the content of the user's utterance, making the converted text information difficult to read. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the invention according to claim 1 provides a voice management system including one or more communication terminals capable of displaying text information converted based on voice information, and a voice record management device that communicates with the communication terminals and manages the text information displayed on the communication terminals as a voice record, wherein the voice record management device has a receiving means for receiving voice information transmitted by the one or more communication terminals, the voice information being related to a voice spoken by a user using a specific communication terminal among the one or more communication terminals, and a transmitting means for transmitting corrected text information including corrected text corrected based on organizational information of an organization to which the user belongs among text recognized from the received voice information to the one or more communication terminals, and the one or more communication terminals display the voice record indicated by the corrected text information transmitted by the voice record management device on a display means. Chronological display and displaying a playback operation section for playing back a specific voice recording selected by the user among one or more voice recordings displayed in chronological order on the display means, in the vicinity of the specific voice recording selected by the user. Display control means and , a playback means for playing back the audio related to the specific audio recording selected by the user in response to the operation of the playback operation unit; The present invention provides a voice management system comprising: Effect of the Invention

[0006] As described above, according to the present invention, when audio information related to a voice spoken by a user is converted into text information for use, the conversion is made in correspondence with organizational information of the organization to which the user belongs, thereby achieving the effect of providing the user with easy-to-read text information. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 illustrates an example of an overall configuration of a communication system. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a communication terminal and a voice recognition server. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a voice recording management device. [Figure 4] FIG. 1 illustrates an example of a functional configuration of a communication system. [Diagram 5] FIG. 13 is a conceptual diagram illustrating an example of a login management table. [Figure 6] FIG. 13 is a conceptual diagram showing an example of a record bibliographic information management table. [Figure 7] FIG. 13 is a conceptual diagram illustrating an example of a text information management table. [Figure 8] FIG. 13 is a conceptual diagram illustrating an example of a capture information management table. [Figure 9A] FIG. 13 is a conceptual diagram illustrating an example of a user information management table. [Figure 9B] FIG. 13 is a conceptual diagram illustrating another example of a user information management table. [Figure 10A] FIG. 13 is a conceptual diagram illustrating an example of a correction information management table. [Figure 10B] FIG. 11 is a conceptual diagram illustrating another example of the correction information management table. [Figure 10C] FIG. 11 is a conceptual diagram illustrating another example of the correction information management table. [Figure 10D] FIG. 11 is a conceptual diagram illustrating another example of the correction information management table. [Figure 11] 11 is a sequence diagram showing an example of application startup, authentication processing, and session establishment processing. FIG. [Figure 12] FIG. 11 is a sequence diagram showing an example of a process of starting recording and recording and editing recorded bibliographic information. [Figure 13] 13 is a sequence diagram showing an example of an automatic correction process and a display process of a recording screen. FIG. [Figure 14] 10 is a flowchart showing details of an automatic correction process. [Figure 15] FIG. 11 is a sequence diagram showing an example of a screen display process after updating. [Figure 16] FIG. 13 is a sequence diagram showing an example of a process of ending recording and editing recorded bibliographic information. [Figure 17] 13 is an example of a screen display when an application is started on a communication terminal. [Figure 18] 13 is an example of a screen display of a recording start instruction on a communication terminal. [Figure 19] 13 is an example of a screen display of a recording end instruction on a communication terminal. [Figure 20] 13 is an example of a screen display during recording on the communication terminal. [Figure 21] 13 is an example of a screen display when recording ends on the communication terminal. [Figure 22] 13 is a sequence diagram showing an example of a process from receiving a record selection to displaying a record viewing and editing screen. FIG. [Figure 23] 13 is an example of a screen display when selecting recording on the communication terminal. [Figure 24] 13 is an example of a screen display for viewing and editing records on a communication terminal. [Diagram 25] FIG. 11 is a sequence diagram showing an example of a reproducing process of audio information. [Figure 26] 13 is an example of a screen display when a play mark is executed on a communication terminal. [Figure 27] FIG. 11 is a sequence diagram illustrating an example of a bookmark process. [Figure 28] FIG. 11 is a sequence diagram illustrating an example of a bookmark process. [Figure 29] 13 is an example of a screen display when a bookmark is executed on the communication terminal. [Diagram 30] FIG. 11 is a sequence diagram showing an example of a process of recording and editing correction information. [Diagram 31] 10 is a flowchart showing details of a correction information recording and editing process. [Diagram 32] 13 is a screen display example for setting whether or not a user registration instruction is required and the validity period of a correction target in a communication terminal. [Diagram 33] 10 is a flowchart showing details of a correction information recording and editing process according to a modified example of the embodiment. [Diagram 34] 13 is an example of a screen display of a registration necessity dialogue in a communication terminal according to a modified example of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals, and the description of overlapping parts will be omitted.

[0009] [First embodiment] The first embodiment will be described with reference to FIGS. 1 to 32. FIG.

[0010] [Overall configuration of communication system] <System configuration example> FIG. 1 is a diagram showing an example of the overall configuration of a communication system. As shown in FIG. 1, the communication system 1 has devices including one or more communication terminals 3, a voice recording management device 5, and a voice recognition server 7. The communication terminals 3, the voice recording management device 5, and the voice recognition server 7 are connected to each other via a communication network 100. Here, the communication network 100 is a communication network in which an unspecified number of communications are performed, and is constructed by the Internet, an intranet, a LAN (Local Area Network), or the like. Note that the communication network 100 may include not only wired communication but also communication networks using wireless communication such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), WiMAX (Worldwide Interoperability for Microwave Access), and LTE (Long Term Evolution). Furthermore, the communication system 1 includes a voice management system 2 constructed by the communication terminal 3 and the voice recording management device 5. The communication terminals 3 and the voice recording management device 5 may be connected to each other via a dedicated in-house network, or may be connected to each other inside the communication network 100 via a firewall.

[0011] <Communication terminal> The communication terminal 3 is realized by one or more information processing devices (computer systems) for communication equipped with a general OS, etc. The communication terminal 3 is capable of communicating with the voice recording management device 5 via a communication network 100. As shown in Fig. 1, the communication terminal 3 is composed of one or more communication terminals including a communication terminal 3(A), a communication terminal 3(B), and a communication terminal 3(C).

[0012] The communication terminal 3(A) has installed thereon, for example, a browser application for communicating with the voice recording management device 5 and a record management application for creating minutes and the like based on text information transmitted by the voice recording management device 5. Furthermore, the communication terminal 3(A) is a communication terminal capable of participating in a predetermined event such as a conversation and creating minutes between the communication terminal 3(B) and the communication terminal 3(A) by using a general-purpose tool (herein referred to as a "conversation tool") for remote work, video conferencing, instant messaging, group chat, and the like. In this way, the communication terminal 3(A) functions as, for example, a minutes-creation terminal.

[0013] The communication terminal 3(B) uses the communication terminal 3(A) and the above-mentioned general-purpose tool to participate in a predetermined event together with a user (e.g., user A) who uses the communication terminal 3(A) and is participating in the predetermined event. In this way, the communication terminal 3(B) functions as, for example, an event participation terminal.

[0014] The communication terminal 3(A) and the communication terminal 3(B) can hear each other's voice by speaking at a specific event using the general-purpose tool as described above. Therefore, the communication terminal 3(A) as a minutes-taking terminal can acquire not only the voice of user A who uses the communication terminal 3(A) but also the voice of a user (e.g., user B) who uses the communication terminal 3(B) as an event participation terminal communicating with the communication terminal 3(A).

[0015] The communication terminal 3(C) is a terminal for viewing minutes created based on a predetermined event in which the communication terminals 3(A) and 3(B) participated. In this case, the communication terminal 3(C) is able to view the minutes of the predetermined event via a browser even if a record management application is not installed. In this way, the communication terminal 3(C) functions as, for example, a minutes viewing terminal.

[0016] In this embodiment, unless otherwise specified, it will be simply referred to as "communication terminal 3". Note that the communication terminal 3 may be a communication terminal having a communication function, such as a commonly used PC (Personal Computer), a portable notebook PC, a mobile phone, a smartphone, a tablet terminal, or a wearable terminal (sunglasses type, wristwatch type, etc.). Furthermore, the communication terminal 3 may be a communication device or a communication terminal capable of operating software such as browser software.

[0017] <Audio recording management device> The voice recording management device 5 is realized by one or more information processing devices (computer systems) equipped with a general server OS or the like. The voice recording management device 5 executes a dedicated application program and manages text information displayed on the communication terminal 3 via the communication network 100 as voice records. Note that the voice recording management device 5 may manage, as text information, not only minutes of a meeting as an example of an event, but also voice memos for any individual or group activity, voice history of an operator answering the phone, work records at a specific location such as a factory, and the like as subjects of voice recording.

[0018] Furthermore, the voice recording management device 5 manages users who use the communication terminal 3 with user identification information. In the communication system 1 in this embodiment, the users can log in to the voice recording management device 5 using the user identification information. Therefore, the user identification information has a function that allows the voice recording management device 5 to uniquely identify the user. The user identification information includes information that can uniquely identify the user, such as an email address, ID, and telephone number.

[0019] The voice recording management device 5 may be constructed by a single computer, or may be constructed by multiple computers to which each section (function or means) such as storage is divided and arbitrarily assigned. In addition, all or part of the functions of the voice recording management device 5 may be a server computer existing in a cloud environment, or a server computer existing in an on-premise environment.

[0020] <Speech recognition server> The voice recognition server 7 is realized by one or more information processing devices (computer systems) equipped with a general server OS or the like. When the voice recognition server 7 receives voice information (data) transmitted by the voice recording management device 5, it has a function of activating a voice recognition engine to convert the voice information (data) into text data, and returning (transmitting) the converted text data to the voice recording management device 5. That is, in the communication system 1 of this embodiment, the voice recognition server 7 has a function of converting voice information into text information. The voice recognition server 7 may use a service provided by another company that enables voice recognition.

[0021] ●About terminology● In this embodiment, a user refers to a person who falls under any of the following categories. For example, users include participants who speak at a specific event, machines equipped with AI capable of generating language spoken by humans, humanoid robots, etc. In this embodiment, the term "user" is used for the sake of convenience.

[0022] Furthermore, in this embodiment, the event refers to various occasions, events, etc. For example, the event includes a meeting, a conference, a lecture, a speech, a competition, etc.

[0023] Furthermore, in this embodiment, text refers to information obtained by converting voice information related to a voice uttered by a user into visible characters, numbers, etc. Similarly, in this embodiment, text information refers to, for example, a sentence including text. Furthermore, text information may refer to a document having multiple sentences including text information.

[0024] [Hardware configuration] Next, the hardware configuration of the device or terminal constituting the communication system according to the embodiment will be described with reference to Figures 2 and 3. Note that components may be added or deleted from the hardware configuration of the device or terminal shown in Figures 2 and 3 as necessary.

[0025] <Hardware configuration of communication terminal and voice recognition server> Fig. 2 is a diagram showing an example of the hardware configuration of the communication terminal and the voice recognition server. As shown in Fig. 2, the communication terminal 3 is constructed by, for example, a computer. The communication terminal 3 includes a CPU 301, a ROM 302, a RAM 303, an EEPROM 304, a CMOS (Complementary Metal Oxide Semiconductor) sensor 305, an image sensor I / F (Inter face) 306, a media I / F 309, a bus line 310, a network I / F 312, an antenna 312a of the network I / F 312, a CMOS sensor 313, an image sensor I / F 314, a microphone 315, a speaker 316, a sound input / output I / F 317, a display 318, an external device connection I / F 319, a short-range communication circuit 320, an antenna 320a of the short-range communication circuit 320, and a touch panel 321.

[0026] Among these, the CPU 301 controls the overall operation of the communication terminal 3. The ROM 302 stores a program used in the processing of the CPU 301. The RAM 303 is used as a work area for the CPU 301. The EEPROM 304 reads or writes various data such as applications under the control of the CPU 301. The CMOS sensor 305 (313) is a type of built-in imaging means that captures an object under the control of the CPU 301 to obtain image data or video data. The imaging means may be an imaging means configured of a CCD (Charge Coupled Device) sensor or the like instead of a CMOS sensor. The imaging element I / F 306 (314) is a circuit that controls the driving of the CMOS sensor 305 (313). The media I / F 309 controls the reading or writing (storing) of data from or to a recording medium 308 such as a flash memory. The bus line 310 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 301 shown in FIG. 2.

[0027] The network I / F 312 is a communication interface for communicating various data (information) with other devices via the communication network 100. At this time, the network I / F 312 may communicate using an antenna 312a of the network I / F 312. The microphone 315 is a built-in circuit that converts sound into an electric signal, and acquires voice or sound waves emitted from an external speaker or the like to acquire information using the electric signal. The speaker 316 is a built-in circuit that converts an electric signal into physical vibration to generate sound such as music or voice. The sound input / output I / F 317 is a circuit that processes input and output of sound signals between the microphone 315 and the speaker 316 under the control of the CPU 301. The display 318 is a type of display means such as a liquid crystal or organic EL (Electro Luminescence) that displays images of subjects, characters, various icons, etc. The external device connection I / F 319 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory or the like. The short-range communication circuit 320 is a communication circuit for performing data communication with a communication device having a wireless communication interface such as NFC (Near Field Communication), Bluetooth (registered trademark; omitted below), or Wi-Fi (registered trademark; omitted below), or a communication terminal. The short-range communication circuit 320 is also provided with an antenna 320a for the short-range communication circuit 320. The touch panel 321 is a type of input means for operating the communication terminal 3 by a user pressing, clicking, tapping, or the like on a predetermined button, icon, or the like arranged on the display 318.

[0028] The communication terminal 3 may be a communication device or a communication terminal capable of running a program such as browser software.

[0029] The voice recognition server 7 includes a CPU 701, a ROM 702, a RAM 703, an EEPROM 704, a CMOS (Complementary Metal Oxide Semiconductor) sensor 705, an image sensor I / F 706, a media I / F 709, a bus line 710, a network I / F 712, an antenna 712a of the network I / F 712, a CMOS sensor 713, an image sensor I / F 714, a microphone 715, a speaker 716, an audio input / output I / F 717, a display 718, an external device connection I / F 719, a short-range communication circuit 720, an antenna 720a of the short-range communication circuit 720, and a touch panel 721. These hardware resources are similar to the respective hardware resources of the communication terminal 3, namely, the CPU 301, ROM 302, RAM 303, EEPROM 304, CMOS (Complementary Metal Oxide Semiconductor) sensor 305, image sensor I / F 306, media I / F 309, bus line 310, network I / F 312, antenna 312a of the network I / F 312, CMOS sensor 313, image sensor I / F 314, microphone 315, speaker 316, sound input / output I / F 317, display 318, external device connection I / F 319, short-range communication circuit 320, antenna 320a of the short-range communication circuit 320, and touch panel 321, and therefore description thereof will be omitted.

[0030] <Hardware configuration of the voice recording management device> Fig. 3 is a diagram showing an example of the hardware configuration of the voice recording management device. As shown in Fig. 3, the voice recording management device 5 is constructed by, for example, a computer, and includes a CPU 501, a ROM 502, a RAM 503, an EEPROM 504, an HD 505, an HDD (Hard Disk Drive) controller 506, a display 507, a short-distance communication I / F 508, a CMOS sensor 509, an image sensor I / F 510, a network I / F 511, a keyboard 512, a pointing device 513, a media I / F 515, an external device connection I / F 516, an audio input / output I / F 517, a microphone 518, a speaker 519, and a bus line 520.

[0031] Among these, the CPU 501 controls the operation of the entire voice recording management device 5. The ROM 502 stores a program used to drive the CPU 501. The RAM 503 is used as a work area for the CPU 501. The EEPROM 504 reads or writes various data such as applications under the control of the CPU 501. The HD 505 stores various data such as programs. The HDD controller 506 controls the reading or writing of various data from the HD 505 under the control of the CPU 501. The display 507 displays various information such as a cursor, a menu, a window, a character, or an image. The short-distance communication I / F 508 is a communication circuit for performing data communication with a communication device or a communication terminal having a wireless communication interface such as NFC (Near Field Communication), Bluetooth (registered trademark, omitted below), or Wi-Fi (registered trademark, omitted below). The CMOS sensor 509 is a type of built-in imaging means that captures an image of a subject under the control of the CPU 501 to obtain image data or video data. The imaging means may be an imaging means configured with a CCD (Charge Coupled Device) sensor, etc. The imaging element I / F 510 is a circuit that controls the driving of the CMOS sensor 509.

[0032] The network I / F 511 is an interface for data communication using the communication network 100. The keyboard 512 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. The pointing device 513 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The media I / F 515 controls reading and writing (storing) data from and to a recording medium 514 such as a flash memory. The external device connection I / F 516 is an interface for connecting various external devices. In this case, the external device is, for example, a USB (Universal Serial Bus) memory, etc. The sound input / output I / F 517 is a circuit that processes input and output of sound signals between a microphone 518 and a speaker 519 under the control of the CPU 501. The microphone 518 is a built-in circuit that converts sound into an electric signal, and obtains voice and sound waves emitted from an external speaker, etc., and obtains information using the electric signal. The speaker 519 is a built-in circuit that converts an electrical signal into physical vibrations to produce sounds such as music and voice. The bus line 520 is an address bus, a data bus, or the like for electrically connecting the components such as the CPU 501 shown in FIG.

[0033] Furthermore, the voice recording management device 5 may notify (send) data (information) to the communication terminal 3 by push notification (transmission). In this case, the voice recording management device 5 can realize the push notification by using, for example, FCM (Firebase Cloud Messaging), which is an example of a push notification server. The voice recording management device 5 may be a commonly used PC (Personal Computer). Furthermore, the voice recording management device 5 may be a communication device or communication terminal capable of running software such as browser software.

[0034] Furthermore, the above program may be recorded in a computer-readable recording medium as an installable or executable file, or may be distributed by downloading via a network. Examples of recording media include CD-Rs (Compact Disc Recordable), DVDs (Digital Versatile Disks), Blu-ray Discs, SD cards, USB memories, etc. Furthermore, the recording media may be provided domestically or internationally as a program product. For example, the voice recording management device 5 realizes the voice recording management method according to the present invention by executing the program according to the present invention.

[0035] [Functional configuration of communication system] Next, the functional configuration of this embodiment will be described with reference to Fig. 4 to Fig. 10. Fig. 4 is a diagram showing an example of the functional configuration of a communication system.

[0036] <Functional configuration of communication terminal> As shown in Fig. 4, the communication terminal 3 has a transmission / reception unit 31, an operation reception unit 32, a sound / image acquisition unit 33, a display control unit 34, a playback unit 36, an application launch unit 38, and a storage / readout unit 39. Each of these functional units is a function or means realized by operating any of the hardware resources shown in Fig. 2 by an instruction from the CPU 301 according to a program for the communication terminal 3 expanded from at least one of the ROM 302 and the EEPROM 304 to the RAM 303. The communication terminal 3 also has a storage unit 3000 constructed by at least one of the ROM 302 and the EEPROM 304 shown in Fig. 2. Furthermore, the storage unit 3000 stores a communication program (communication application) for communicating with the voice recording management device 5 via the communication network 100, and a browser application, a record management application, etc. for generating minutes based on voice information.

[0037] <<Functional configuration of communication terminal>> Next, a detailed description will be given of each functional configuration of the communication terminal 3. The transmission / reception unit 31 of the communication terminal 3 shown in Fig. 4 is mainly realized by the processing of the CPU 301 on the network I / F 312 and the short-range communication circuit 320 shown in Fig. 2, and transmits and receives various data (or information) with the voice recording management device 5 via the communication network 100. In this embodiment, the transmission / reception unit 31 functions as an example of at least one of a transmitting means and a receiving means.

[0038] The operation reception unit 32 is realized mainly by the CPU 301 processing signals generated by various operations received by the touch panel 321 shown in Fig. 2. Note that the operation reception unit 32 may use an input means such as a keyboard or a pointing device instead of the touch panel 321. In this embodiment, the operation reception unit 32 functions as an example of a reception means.

[0039] The sound / image acquisition unit 33 is mainly realized by the processing of the CPU 301 on the microphone 315, the sound input / output I / F 317, the CMOS sensor 313, and the image sensor I / F 314 shown in FIG. 2, and collects voice (sound) related to the voice etc. uttered by the user who uses the communication terminal 3 to acquire voice information (voice data) or sound information (sound data). Furthermore, the sound / image acquisition unit 33 captures an image related to an image such as the face of the user to acquire image information (image data). The voice information includes speech voice information indicating the voice uttered by a human being, and artificial voice information which is an artificial voice generated by a machine equipped with AI, a humanoid robot, etc. In this embodiment, the sound / image acquisition unit 33 functions as an example of an acquisition means.

[0040] 2, and controls the display of various screens and information (data) in the communication terminal 3. In the present embodiment, the display control unit 34 functions as an example of a display control means.

[0041] 2 and the processing of the CPU 301 with respect to the speaker 316 and the sound input / output I / F 317, and performs sound reproduction in the communication terminal 3. In the present embodiment, the reproduction unit 36 ​​functions as an example of a reproduction means.

[0042] The application launch unit 38 is mainly realized by the processing of the CPU 301 shown in Fig. 2, and launches various applications between the application launch unit 38 and the voice recording management device 5 via the communication network 100. The application launch unit 38 also operates a recording management application and a browser application for editing and managing various text information managed by the voice recording management device 5 in a predetermined working area of ​​the RAM 303. In this embodiment, the application launch unit 38 functions as an example of a launching means.

[0043] 2, and stores various data (or information) in the storage unit 3000 and reads various data (or information) from the storage unit 3000. In this embodiment, the storage unit 39 functions as an example of a storage and reading means.

[0044] <Functional configuration of the voice recording management device> As shown in FIG. 4, the voice recording management device 5 has a transmission / reception unit 51, an acquisition / extraction unit 52, a calculation unit 53, a display control unit 54, a judgment unit 55, an authentication unit 56, a correction / generation unit 57, a registration unit 58, and a storage / readout unit 59. Each of these functional units is a function or means realized by operating any of the hardware resources shown in FIG. 3 by an instruction from the CPU 501 according to a program for the voice recording management device 5 expanded from at least one of the ROM 502 and the HD 505 to the RAM 503. The voice recording management device 5 also has a storage unit 5000 constructed by at least one of the ROM 502 and the HD 505 shown in FIG. 3. Furthermore, the storage unit 5000 stores a communication program (communication application) for communicating with the communication terminal 3 and the voice recognition server 7 via the communication network 100, and a browser application, a record management application, etc. executed between the communication terminal 3 and the voice recognition server 7.

[0045] ●Login management table● Fig. 5 is a conceptual diagram showing an example of a login management table. A login management DB 5001 configured by a login management table as shown in Fig. 5 is constructed in the memory unit 5000. In the login management table, terminal identification information, IP address of the participating communication terminal, user identification information (user ID), password, and user name are stored and managed in association with each session ID. Of these, the session ID is information for identifying a session established by communication between the voice recording management device 5 and one or more communication terminals 3, and is given, for example, SE001, SE002, etc.

[0046] The terminal identification information is information for identifying the communication terminal 3, and is given, for example, as T001, T002, etc. The IP address of the participating communication terminal indicates the IP address of the communication terminal that participated in a specific event, and is unique information given in correspondence with the version such as IPv4, IPv6, etc. In this embodiment, it is given, for example, as information such as 1.2.1.3, 1.2.2.4, etc. The user identification information is information for identifying the user, and in this embodiment, for example, an email address such as "taroh.r@ricoh.ex.com" is given as the user ID of the user. The password is identification information associated with the user identification information at the time of logging in when using the communication system 1, and is set by the user at the time of the first startup (login), etc. The password is information of multiple characters (digits) including any character string, numbers, and symbols randomly. The user name represents the name of the user indicated by the user identification information, and is given, for example, as "Rikotaro", "Magome Hanako", etc.

[0047] ●Record bibliographic information management table● Fig. 6 is a conceptual diagram showing an example of a record bibliographic information management table. A record bibliographic information management DB 5002 configured by a record bibliographic information management table as shown in Fig. 6 is constructed in the storage unit 5000. In the record bibliographic information management table, record identification information is treated as tabs, and record names, start dates and times, end dates and times, audio data paths, user identification information (user ID), and conference URLs, which are separated by each tab, are stored and managed in association with each other. The record identification information as tabs is given, for example, as "R5006", "R5007", etc.

[0048] Among these, the record name is the name of the recorded content recorded at a specific event, and for example, a name such as "Healthcare Business Performance Reporting Meeting" is given. The start date and time and the end date and time are given as the date and time when the specific event started and ended, and are, for example, information such as "2021 / 03 / 31 11:00:00" and "2021 / 03 / 31 12:00:00". The audio data path indicates the location where the audio recording data (all data) recorded at a specific event is stored and managed, and is given as path information including a file name, such as "... / 00005006 / record.mp3". The audio data path may be URL information representing a dedicated audio server device arranged in the communication system 1. The event URL indicates the location for editing the record viewing in the event, and the user can access the "record viewing and editing screen" described later by accessing this event URL. The event URL may be URL information representing a dedicated image server device arranged in the communication system 1.

[0049] Among the items managed in the record bibliographic information management table according to this embodiment, the items of end date and time and audio data path are items that are edited or added by a user or editor when the event ends.

[0050] ●Text information management table● FIG. 7 is a conceptual diagram showing an example of a text information management table. In the storage unit 5000, a text information management DB 5003 configured by a text information management table as shown in FIG. 7 is constructed. In the text information management table, the record identification information is treated as a tab, and text identification information, start time, end time, text, editing result, editor identification information, editing date and time, bookmark, and audio data path, which are separated by each tab, are stored and managed in association with each other. The record identification information as a tab is given as, for example, "R5006", "R5007", etc. The text information management DB 5003 manages specific text information including specific text, an example of which is erroneously recognized text described later, and corrected text information including corrected text, in association with each other.

[0051] Among these, the text identification information is information for identifying a specific speech content or a speech sentence as a unit when it is treated as one text (or text information), and is given, for example, as "T001", "T002", ... "T005". The start time and end time are information for managing the time when a specific speech started and ended in minutes and seconds. For example, if the date and time when a specific event started was the start date and time "2021 / 03 / 31 11:00:00" managed in the record bibliographic information management DB 5002 (see FIG. 6), the start time is managed as the time elapsed from the start date and time. In other words, if the start time is "1 minute 53 seconds", it indicates that the date and time when the speech managed in the text information "T005" started was "2021 / 03 / 31 11:01:53". The end time is managed as the time elapsed from the start date and time, like the start time. In other words, if the end time is "1 minute 58 seconds," it indicates that the date and time when the utterance managed by the text information "T005" ended was "2021 / 03 / 31 11:01:58." Therefore, in this case, the time required for the utterance managed by the text information "T005" is 58 seconds - 53 seconds = 5 seconds.

[0052] As described above, the text is the content of the speech uttered by the user between the start time and the end time replaced with text information. For example, the text is managed as "Today's meeting will be locked on." As will be described in detail later, "lock on" is a text resulting from the misrecognition of "recording," but this is because the text information transmitted by the voice recognition server 7 is managed as is without correction.

[0053] The editing results are stored and managed in the record viewing and editing screen, in which the content edited by the user is stored. As a result, the content "Today's meeting will be locked on" managed in the text item is corrected to a meaningful content such as "Today's meeting will be recorded" and managed. The editor identification information is information for identifying the person who made the edit related to the editing result, and is given, for example, as "E001", "E002", etc. The editing date and time indicates the date and time when the content related to the editing result was edited, and is given, for example, as "2021 / 03 / 31 16 / 00:00". Note that since editing takes a certain amount of time, this editing date and time is synonymous with the date and time when editing started.

[0054] Bookmarks are specific text ( This is a flag that manages whether or not to bookmark (or has bookmarked) text information, and is given, for example, as "ON" or "OFF." The initial value of the bookmark is given, for example, as "OFF."

[0055] The voice data path indicates a location where voice recording data (partial data) recorded in a specific event is stored and managed, and the voice associated with the text identification information is given as path information such as "... / 00005006 / 0005.mp3". In other words, when there are voice data paths associated with multiple text identification information in a specific event, the voice data indicated by all of the voice data paths is joined together to become the voice data (whole data) indicated by the voice data path managed in the bibliographic record information management DB 5002 (see FIG. 6). In other words, the contents of each voice data path managed in the text information management DB 5003 become part of the voice data constituting the voice data path of the bibliographic record information management DB 5002 (see FIG. 6). The voice data path may be URL information indicating a dedicated voice server device arranged in the communication system 1.

[0056] In this embodiment, the corrected text information is treated as an example of text information constituting information related to a voice recording including minutes of a specific event and speech notes of speech made by a user.

[0057] ●Capture image management table● Fig. 8 is a conceptual diagram showing an example of a capture image management table. A capture image management DB 5004 configured by the capture image management table as shown in Fig. 8 is constructed in the storage unit 5000. In this embodiment, the display control unit 34 of the communication terminal 3 has a function of capturing a viewing and editing screen displayed and edited during execution of a predetermined event at a predetermined time interval (e.g., every 30 seconds). The capture image management table is a table for storing and managing information related to the screen capture function, and in the capture image table, record identification information is treated as a tab, and an image ID, acquisition time, and image data path, which are separated by each tab, are stored and managed in association with each other.

[0058] Of these, the image ID is information for identifying the captured image, and is given, for example, as "I001", ... "I003". The acquisition time indicates the elapsed time in a specific event when the captured image was acquired, and is given, for example, as "1 minute 30 seconds", "2 minutes 0 seconds", etc. The image data path indicates the location where the captured image data captured in a specific event is stored and managed, and is given as path information such as "... / 00005006 / 0003.jpg". The image data path may be URL information indicating a dedicated image server device arranged in the communication system 1.

[0059] ●User information management table● Fig. 9A is a conceptual diagram showing an example of a user information management table. A user information management DB 5005 configured by a user information management table as shown in Fig. 9A is constructed in the storage unit 5000. In the user information management table, organization names associated with user identification information are stored and managed. In this embodiment, the organization names are given as, for example, "accounting department", "sales department", "general affairs department", etc.

[0060] Fig. 9B is a conceptual diagram showing another example of a user information management table. A user information management DB 5005 configured by a user information management table as shown in Fig. 9B is constructed in the storage unit 5000. In the user information management table shown in Fig. 9B, a tenant name is used as a tab, and organization names associated with user identification information separated by each tab are stored and managed. The tenant name is given as, for example, "A Corporation", "B Corporation", etc. In other words, the user information management table shown in Fig. 9B is table information further hierarchically separated by a tenant name tab than the user information management table shown in Fig. 9A.

[0061] ●Correction information management table● FIG. 10A is a conceptual diagram showing an example of a correction information management table. In the storage unit 5000, a correction information management DB 5006 configured by a correction information management table as shown in FIG. 10A is constructed. In the correction information management table, for example, information with an organization name, which is an example of organization information, given as "Accounting Department" is used as a tab, and for each correction ID separated by this "Accounting Department" tab, a correction text, a misrecognized text, and a registration date are associated and stored and managed. In the present embodiment, the misrecognized text is an example of a specific text identified from the voice information.

[0062] Among these, the correction ID represents identification information identified when correcting the text information transmitted by the voice recognition server 7, and is given, for example, as "C001", "C002", etc. The correction text indicates the text after correcting the misrecognized text described below (after the misrecognized text is corrected). In the correction information management table when managed by the tab of the Accounting Department, it is shown that "Science and Engineering" and "Clever" are corrected to, for example, "Performance", which is highly likely to be used in the Accounting Department.

[0063] The misrecognized text indicates the text transmitted by the voice recognition server 7, and a text converted based on the voice spoken by a user or the like is given. The misrecognized text is given, for example, as "Science and Engineering", "Clever". Note that the text transmitted by the voice recognition server 7 can be registered in this correction information management table as the misrecognized text because the voice recording management device 5 compares the correction text corrected by a user or the like with the text transmitted by the voice recognition server 7, and when there is different text information, that text information can be registered as the misrecognized text. However, the method of registering as the misrecognized text is not limited to such a method.

[0064] The registration date represents the date and time when the correction text and the misrecognized text are registered, and is given, for example, as "2020 / 07 / 21", etc. The information related to this registration date may be given on a validity period setting screen for a correction target described later.

[0065] As described above, the correction information management DB 5006 manages, for each correction ID, specific text information including specific text, such as the misrecognized text described below, correction text information including correction text, and the registration date, in association with each other.

[0066] Fig. 10B is a conceptual diagram showing another example of the correction information management table. A correction information management DB 5006 configured by the correction information management table shown in Fig. 10B is constructed in the storage unit 5000. In the correction information management table, for example, information given by the organization name "Sales Department" is treated as a tab, and the correction text, the misrecognition text, and the registration date are associated with each other and stored and managed for each correction ID divided by the tab of "Sales Department."

[0067] In the correction information management table when managed under the Sales Department tab, it is shown that "Ikusou" and "Sakaguchi" have been corrected to, for example, "Riko", which is likely to be used in the Sales Department. In this case, the misrecognized text is given as, for example, "Ikusou" and "Sakaguchi".

[0068] FIG. 10C is a conceptual diagram showing an example of a correction information management table. A correction information management DB 5006 configured by a correction information management table as shown in FIG. 10C is constructed in the storage unit 5000. In the correction information management table, for example, each piece of information given by the tenant name "A Corporation" and the organization name "Accounting Department" is treated as a tab, and correction text, misrecognition text, and registration date are associated with each correction ID divided by this tab and stored and managed. The contents of the items constituting the table are the same as those in FIG. 10A, so the description will be omitted. In other words, the correction information management table shown in FIG. 10C is table information further divided by the information constituting the tab called tenant name than the correction information management table shown in FIG. 10A.

[0069] FIG. 10D is a conceptual diagram showing an example of a correction information management table. In the storage unit 5000, a correction information management DB 5006 configured by the correction information management table shown in FIG. 10D is constructed. In the correction information management table, for example, each piece of information given by the tenant name "B Corporation" and the organization name "Sales Department" is treated as a tab, and the correction text, the misrecognition text, and the registration date are associated with each correction ID divided by this tab and stored and managed. The contents of the items constituting the table are the same as those in FIG. 10B, so the description will be omitted. In other words, the correction information management table shown in FIG. 10D is table information further divided by the information constituting the tab called the tenant name than the correction information management table shown in FIG. 10B. In this embodiment, the correction information management DB 5006 functions as an example of a correction information management means.

[0070] <<Functional configuration of the voice recording management device>> Next, each functional component of the voice recording management device 5 will be described in detail. The transmission / reception unit 51 of the voice recording management device 5 shown in Fig. 4 is mainly realized by the processing of the CPU 501 on the short-range communication I / F 508 and the network I / F 511 shown in Fig. 3, and transmits and receives various data (or information) to and from the communication terminal 3 via the communication network 100. In this embodiment, the transmission / reception unit 51 functions as an example of at least one of a transmitting means and a receiving means.

[0071] 3, and acquires text data based on text information transmitted by the voice recognition server 7, acquires capture information on the viewing and editing screen, etc. In this embodiment, the acquisition and extraction unit 52 functions as an example of an acquisition means.

[0072] 3, and calculates the start date and time when the user starts speaking in a predetermined event. In this calculation, clock information generated using a clock signal of the CPU 501 may be used. In this embodiment, the calculation unit 53 functions as an example of a calculation means.

[0073] The display control unit 54 is mainly realized by the processing of the CPU 501 on the display 507 shown in Fig. 3, and controls the display of various screens and information (data) in the voice recording management device 5. In this embodiment, the display control unit 54 functions as an example of a display control means.

[0074] The judgment unit 55 is mainly realized by the processing of the CPU 501 shown in Fig. 3, and performs various judgments in the voice recording management device 5. In this embodiment, the judgment unit 55 functions as an example of a judgment means for judging whether or not a given predetermined condition is satisfied.

[0075] 3, and performs authentication processing of the communication terminal 3 based on an authentication request made from the communication terminal 3. In this embodiment, the authentication unit 56 functions as an example of an authentication means.

[0076] The correction generation unit 57 is mainly realized by the processing of the CPU 501 shown in FIG. 3, and generates voice recording data based on text information transmitted by the voice recognition server 7. The correction generation unit 57 also generates screen data for a record viewing and editing screen displayed on the communication terminal 3. Furthermore, the correction generation unit 57 corrects the misrecognized text managed in the correction information management DB 5006 (see FIG. 10) to corrected text. In this embodiment, the correction generation unit 57 functions as an example of a correction means. In this embodiment, the correction generation unit 57 also functions as an example of a generation means.

[0077] The registration unit 58 is mainly realized by the processing of the CPU 501 shown in Fig. 3, and for example, registers authentication information of the user who uses the communication terminal 3, which is performed on the voice recording management device 5, in the storage unit 3000. Furthermore, the registration unit 58 registers correction text, bookmarks, etc., and deletes bookmarks. In this embodiment, the registration unit 58 functions as an example of a registration means.

[0078] 3, and stores various data (or information) in the storage unit 5000 and reads various data (or information) from the storage unit 5000. In this embodiment, the storage unit 59 functions as an example of a storage and reading means.

[0079] <Functional configuration of the speech recognition server> As shown in Fig. 4, the voice recognition server 7 has a transmission / reception unit 71, a voice recognition unit 76, and a storage / readout unit 79. Each of these functional units is a function or means realized by any of the hardware resources shown in Fig. 2 operating in response to an instruction from the CPU 701 in accordance with a program for the voice recognition server 7 that is loaded from at least one of the ROM 702 and the EEPROM 704 to the RAM 703. The voice recognition server 7 also has a storage unit 7000 constructed from at least one of the ROM 702 and the EEPROM 704 shown in Fig. 2. Furthermore, the storage unit 7000 stores a communication program (communication application) for communicating with the voice recording management device 5 via the communication network 100, etc.

[0080] <<Functional configuration of the speech recognition server>> Next, each functional configuration of the voice recognition server 7 will be described in detail. The transmission / reception unit 71 of the voice recognition server 7 shown in Fig. 4 is mainly realized by the processing of the CPU 701 on the network I / F 712 and the short-range communication circuit 720 shown in Fig. 2, and transmits and receives various data (or information) with the voice recording management device 5 via the communication network 100. In this embodiment, the transmission / reception unit 71 functions as an example of at least one of a transmitting means and a receiving means.

[0081] 2 and recognizes the voice data or sound data (voice information) transmitted by the voice recording management device 5 and converts it into text data (text information). In this embodiment, the voice recognition unit 76 functions as an example of a voice recognition means.

[0082] 2, and stores various data (or information) in the storage unit 7000 and reads various data (or information) from the storage unit 7000. In this embodiment, the storage and reading unit 79 functions as an example of a storage and reading means.

[0083] [Processing or Operation of the Embodiment] Next, each process or operation in the voice management system according to the first embodiment will be described with reference to Fig. 11 to Fig. 32. Fig. 11 is a sequence diagram showing an example of application activation, authentication processing, and session establishment processing.

[0084] <App launch and authentication process> First, the user of the communication terminal 3 performs an operation to start the record management application and the browser application running on the communication terminal 3. As a result, the operation acceptance unit 32 of the communication terminal 3 accepts the user's operation to start the communication application and the browser application (step S11). In this embodiment, it is sufficient that the communication terminal 3(A), which is an example of the minutes creation terminal described above, starts the record management application, and the communication terminal 3(B), which participates in a predetermined event together with the communication terminal 3(A), does not need to start the record management application. If the communication terminal 3(B) starts the above-mentioned conversation tool instead of starting the record management application, voice communication with the communication terminal 3(A) becomes possible, and as a result, each other's voice can be recognized. Furthermore, in this embodiment, it is assumed that the communication application used between the communication terminal 3 and other devices is in a state in which communication with other devices is possible by a predetermined protocol or the like.

[0085] After receiving an instruction from the user to start the recording management application and the browser application, the application starting unit 38 starts the communication application and the browser application for communicating with the voice recording management device 5 that are pre-installed in the storage unit 3000 (step S12). Then, the display control unit 34 displays an authentication screen (such as a sign-in screen) on the display 318 and waits for the user to perform an authentication operation (step S13).

[0086] After the communication application and the browser application are launched by the application launch unit 38, the user performs authentication processing (signs in) to the voice recording management device 5. In this state, the operation acceptance unit 32 accepts authentication information input by the user (step S21).

[0087] Next, the transmitting / receiving unit 31 transmits a request for authentication processing based on the received authentication information to the voice recording management device 5 (step S22). As a result, the transmitting / receiving unit 41 of the voice recording management device 5 receives the request for authentication processing transmitted by the communication terminal 3. At this time, the request for authentication processing includes a session ID for establishing a session with the voice recording management device 5, terminal identification information of the communication terminal 3, user identification information for identifying the user, and a password.

[0088] After receiving the request for authentication processing, the authentication unit 46 of the voice recording management device 5 performs login authentication processing by comparing the received session ID, terminal identification information, user identification information and password with the session ID, terminal identification information and password corresponding to the user identification information read from the login management DB 5001 (see FIG. 5) by the storage and reading unit 59 (step S23). Here, it is assumed that the login authentication processing by the user to the voice recording management device 5 has been successful. In the processing of step S23, the registration unit 58 may register the IP address of the communication terminal 3 that has performed the login authentication processing in the login management DB 5001 (see FIG. 5).

[0089] When the login authentication process by the user is successful and a communication session with the communication terminal 3 is established, the transmission / reception unit 51 transmits a response to the authentication process and a response to the participation process to the communication terminal 3 (step S24). As a result, the transmission / reception unit 31 of the communication terminal 3 receives the response to the authentication process and the response to the participation process transmitted by the voice recording management device 5. At this time, the response to the authentication process and the response to the participation process include the session ID and a participation process result that permits the participation process to the communication session with the voice recording management device 5.

[0090] The application launch unit 38 of the communication terminal 3 that has received the response to the authentication process and the response to the participation process cooperates with the memory readout unit 39 to store and register the user identification information, password, and user name as a set in a predetermined area of ​​the storage unit 3000 in order to simplify the login process and communication session establishment from the next time onwards (step S25). Note that the process of step S25 may be omitted.

[0091] In the above-mentioned example of the processing sequence, it is assumed that a browser application is installed in the communication terminal 3, but a Web service using a browser may be provided, as in the case of the communication terminal 3 (C) used as an example of a minutes viewing terminal. In the case of a Web service, the voice recording management device 5 has a Web server function, and may transmit screen data (data for displaying the entire screen or a part of the screen) such as a viewing and editing screen to the communication terminal 3 for display.

[0092] In the voice management system according to this embodiment, for example, when the processing of steps S22 and S24 described above is executed, other devices may be present between the communication terminal 3 and the voice recording management device 5. In other words, each piece of information (data) transmitted and received between the communication terminal 3 and the voice recording management device 5 may be configured to be transmitted and received once via another device. The above-mentioned configuration is applicable even if other processing steps exist between the communication terminal 3 and the voice recording management device 5.

[0093] ●Screen display example● 17 is an example of a screen display when an application is started in a communication terminal. After the process of step S12 described above is executed, the display control unit 34 of the communication terminal 3 displays an application start-up screen 3101 on the display 318 as an example of a screen when an application is started. For example, a mark representing the application (a picture of a microphone) and version information of the application are displayed on the application start-up screen 3101.

[0094] <Start of recording and recording and editing of bibliographic information> Next, the recording start and the recording and editing process of the recorded bibliographic information will be described. Fig. 12 is a sequence diagram showing an example of the recording start and the recording and editing process of the recorded bibliographic information. As shown in Fig. 12, the display control unit 34 of the communication terminal 3 displays the application start screen 3101 as shown in Fig. 17, and then displays the recording start screen on the display 318 (step S41).

[0095] After the recording start screen is displayed, the operation accepting unit 32 accepts an operation of a recording start button provided on the recording start screen as an instruction from the user to start recording (step S42).

[0096] ●Screen display example● 18 is an example of a screen display of a recording start instruction in a communication terminal. After the process of step S42 described above is executed, the display control unit 34 of the communication terminal 3 displays a recording start instruction screen 3111 as an example of a screen for a recording start instruction on the display 318. The recording start instruction screen 3111 includes, for example, an overview (agenda) field 3112, a participant field 3113, a meeting memo field 3114, a bookmark field 3115, a microphone description display field 3116, and a recording screen display field 3117.

[0097] In the summary (agenda) column 3112, for example, the agenda of an event such as a meeting to be held is input. In the participant column 3113, for example, the participants of the event are input. In the meeting notes column 3114, for example, the main agenda of the event is input. The main agenda is, for example, an outline, decisions, action items, etc. In the bookmark column 3115, the user inputs particularly important content of an event such as a meeting. In the microphone description display column 3116 and the recorded screen display column 3117, the microphone arrangement, the number of the screen to be recorded, etc. The microphone arrangement and the number of the screen to be recorded may be preset by the user in the recording management device 5, or may be arbitrarily set by the recording management device 5.

[0098] Furthermore, the display control unit 34 displays a record start button 3118 on the record start instruction screen 3111. A user of the communication terminal 3 can start recording of the contents of speech spoken at an event such as a conference by operating (pressing or tapping, etc.) the record start button 3118.

[0099] ●Screen display example● Fig. 19 is an example of a screen display of a recording end instruction in a communication terminal. When a user operates a recording start button displayed on the recording start instruction screen 3111 shown in Fig. 18, the display control unit 34 of the communication terminal 3 can display a recording end instruction screen 3121 as an example of a screen for a recording end instruction on the display 318. The recording end instruction screen 3121 includes an overview (agenda) column 3122, a participant column 3123, a meeting memo column 3124, a bookmark column 3125, and a processing message display column 3126, as in the case of Fig. 18. Here, the overview (agenda) column 3122, the participant column 3123, the meeting memo column 3124, and the bookmark column 3125 are similar to the overview (agenda) column 3112, the participant column 3113, the meeting memo column 3114, and the bookmark column 3115 shown in Fig. 18, and therefore description thereof will be omitted. In the processing message display field 3126, for example, a message such as "Preparing. Please wait a moment" is displayed, which indicates that recording of minutes or the like is being prepared.

[0100] Furthermore, the display control unit 34 displays a record end button 3127 on the record end instruction screen 3121. The user of the communication terminal 3 can end the recording of the contents of the speech spoken at an event such as a conference by operating (pressing or tapping, etc.) the record end button 3127.

[0101] 12, after accepting the recording start button 3511 operated by the user, the transmitting / receiving unit 31 of the communication terminal 3 transmits a recording start request to the voice recording management device 5 (step S43). As a result, the transmitting / receiving unit 51 of the voice recording management device 5 receives the recording start request transmitted by the communication terminal 3.

[0102] After receiving the recording start request, the transmitting / receiving unit 51 of the voice recording management device 5 transmits a voice data transmission start notification to the voice recognition server 7 (step S44). As a result, the transmitting / receiving unit 71 of the voice recognition server 7 receives the voice data transmission start notification transmitted by the voice recording management device 5.

[0103] After receiving the notification of the start of voice data transmission, the voice recognition server 7 transmits a permission response to start transmission to the voice recording management device 5 (step S45). As a result, the transmission / reception unit 51 of the voice recording management device 5 receives the permission response to start transmission transmitted by the voice recognition server 7.

[0104] After receiving the permission response to start transmission, the transceiver 51 of the voice recording management device 5 transmits a request to display the recording screen to the communication terminal 3 (step S46). As a result, the transceiver 31 of the communication terminal 3 receives the request to display the recording screen transmitted by the voice recording management device 5.

[0105] After receiving the request to display the recording screen, the display control unit 34 of the communication terminal 3 displays the recording screen on the display 318, and accepts the operation of the recording end button operated by the user (step S47).

[0106] After executing step S46, the transmitting / receiving unit 51 of the voice recording management device 5 further transmits a voice data transmission start request to the communication terminal 3 (step S48). As a result, the transmitting / receiving unit 31 of the communication terminal 3 receives the voice data transmission start request transmitted by the voice recording management device 5.

[0107] The transmitting / receiving unit 31 of the communication terminal 3 that has received the request to start transmitting the voice data transmits the voice data and the record bibliographic information to the voice recording management device 5 (step S49). As a result, the transmitting / receiving unit 51 of the voice recording management device 5 receives the voice data and the record bibliographic information transmitted by the communication terminal 3. The record bibliographic information transmitted and received at this time includes the record name, start date and time, user identification information (user ID), and event URL that are managed in the record bibliographic information management DB 5002 (see FIG. 6).

[0108] After receiving the voice data, the transmitting / receiving unit 51 of the voice record management device 5 transmits a voice recognition request to the API (Application Programming Interface) of the voice recognition server 7 (step S50). As a result, the transmitting / receiving unit 71 of the voice recognition server 7 receives the voice recognition request transmitted by the voice record management device 5. At this time, the voice recognition request includes the voice data to be converted into text by the voice recognition server. By performing the processes of steps S49 and S50, the transmitting / receiving unit 51 of the voice record management device 5 continuously transmits the voice data transmitted by the communication terminal 3 (voice data by voice streaming) to the voice recognition server 7. In this case, even if the user's speech is not detected, the transmitting / receiving unit 51 may continue to transmit the voice streaming to the voice recognition server 7 continuously. However, the voice record management device 5 may transmit the voice data individually based on the specifications of the API of the voice recognition server 7 and request voice recognition.

[0109] After executing step S50, the registration unit 58 of the voice recording management device 5 registers the record bibliographic information received in step S49 in the record bibliographic information management DB 5002 (see FIG. 6) (step S51). The record bibliographic information registered at this time includes the record name "Healthcare Business Performance Report Meeting", "Start Date and Time", "User Identification Information (User ID)", and "Event URL", as well as new information indicating "End Date and Time" and "Voice Data Path".

[0110] ●Screen display example● Fig. 20 is an example of a screen display during recording in a communication terminal. After the process of step S47 described above is executed, the display control unit 34 of the communication terminal 3 displays a recording screen 3131 as an example of a screen during recording on the display 318. The recording screen 3131 includes an overview (agenda) column 3132, a participants column 3133, a meeting memo column 3134, and a bookmark column 3135, similar to the case of Fig. 18. Here, the overview (agenda) column 3132, the participants column 3133, the meeting memo column 3134, and the bookmark column 3135 are similar to the overview (agenda) column 3112, the participants column 3113, the meeting memo column 3114, and the bookmark column 3115 shown in Fig. 18, and therefore description thereof will be omitted.

[0111] The display control unit 34 displays a text display field 3136 on the recording screen 3131. In this text display field 3136, for each user who has spoken, a facial photograph or image of the user, the date and time of the speech, and an editing field in which the speech content can be edited are displayed in chronological order. Here, in the editing field in which the speech content can be edited, text information such as "Today's meeting will be locked on" transmitted by the voice recognition server 7 is displayed.

[0112] The display control unit 34 further displays a recording stop button 3137 for instructing to end recording and a pause button 3138 for instructing to pause recording on the recording in progress screen 3131. The user can end or pause recording of speech content spoken at an event such as a conference by operating (pressing or tapping, etc.) the recording stop button 3137 or the pause button 3138. When the recording stop button 3137 is operated, the communication terminal 3 starts transmitting voice data to the voice recording management device 5.

[0113] <Automatic correction processing> FIG. 13 is a sequence diagram showing an example of the automatic correction process and the display process of the recording screen.

[0114] First, among one or more communication terminals 3, the audio / image acquisition unit 33 of the communication terminal 3(A), which is an example of a minutes-taking terminal, collects the voice or sound uttered by a user using the communication terminal 3(A) via the microphone 315 and acquires voice information (including voice data or audio data; hereinafter simply referred to as "voice information") (step S61).

[0115] Next, the transmitting / receiving unit 31 transmits the acquired voice information to the voice recording management device 5 (step S62). As a result, the transmitting / receiving unit 51 of the voice recording management device 5 receives the voice information related to the voice spoken by the user of the communication terminal 3(A) transmitted by the transmitting / receiving unit 31 of the communication terminal 3(A). In the following description, the communication terminal 3(A) will be simply referred to as the communication terminal 3.

[0116] After receiving the voice information, the transmitting / receiving unit 51 of the voice recording management device 5 transmits a voice recognition request to the voice recognition server 7 (step S63). As a result, the transmitting / receiving unit 71 of the voice recognition server 7 receives the voice recognition request transmitted by the voice recording management device 5. At this time, the voice recognition request includes the voice information transmitted by the communication terminal 3. In other words, the voice recording management device 5 also serves as an intermediary device between the communication terminal 3 and the voice recognition server 7.

[0117] The voice recognition unit 76 of the voice recognition server 7 that receives the voice information included in the voice recognition request executes voice recognition on the received voice information and converts the voice information into text information (step S64).

[0118] The transmitting / receiving unit 71 of the voice recognition server 7 that has converted the voice into text data transmits the voice recognition result to the voice recording management device 5 (step S65). As a result, the transmitting / receiving unit 51 of the voice recording management device 5 receives the voice recognition result transmitted by the voice recognition server 7. At this time, the voice recognition result includes the converted text data, the start time, and the end time. Specifically, the voice recognition server 7 transmits, for example, the content "Today's minutes will be locked on" managed in the text information management DB 5003 (see FIG. 7) and the start time (1 minute 53 seconds) at which the content was spoken to the voice recording management device 5. The start time is managed as the time elapsed from the start date and time, as explained in the text information management table. The same concept is applied to the end time.

[0119] After receiving the voice recognition result, the voice recording management device 5 performs an automatic correction process on the text data (step S66).

[0120] <Detailed flow of automatic correction process> In this embodiment, the registration unit 58 associates the correction information created or corrected by a user (login user) of a specific event such as a conference with the name of the organization to which the user belongs, and registers the correction information in the correction information management DB 5006 (see FIG. 10). When a new minutes is created by a user, the correction generation unit 57 refers to the correction information management DB 5006 (see FIG. 10) corresponding to the name of the organization to which the user belongs, and corrects the misrecognized text with the correction text.

[0121] Fig. 14 is a flowchart showing details of the automatic correction process. As shown in Fig. 14, the acquisition and extraction unit 52 breaks down the text received from the voice recognition server 7 and extracts keywords at the word level (step S66-1).

[0122] Next, the acquiring / extracting unit 52 acquires the name of the organization to which the logged-in user who is using the application belongs, using the user information management DB 5005 (see FIG. 9B) (step S66-2).

[0123] Next, the memory readout unit 59 searches the correction information management DB 5006 (see FIG. 10) using the acquired organization name as a search key to read out the corresponding erroneous recognition text (step S66-3). The erroneous recognition text read out at this time may be all erroneous recognition texts managed in the correction information management DB 5006 (see FIG. 10).

[0124] Thereafter, the voice recording management device 5 executes the following loop process from step S66-4 to step S66-7 for each keyword (step S66-4).

[0125] First, the judgment unit 55 judges whether the keyword extracted in step S66-1 is registered as an erroneous recognition text (step S66-5). If the extracted keyword is registered as an erroneous recognition text in the correction information management DB 5006 (see FIG. 10) (step S66-5: YES), the correction generation unit 57 corrects the target keyword with a correction text corresponding to the erroneous recognition text (step S66-6). Specifically, the correction generation unit 57 corrects the text (keyword) recognized from the received voice information with a correction text managed in the correction information management DB 5006 (see FIG. 10) given for each organization information (e.g., organization name) of the organization to which the user belongs.

[0126] When all the loop processes for each keyword are completed, the voice recording management device 5 ends the loop process for each keyword and exits this flow (step S66-7).

[0127] On the other hand, if the keyword has not been registered as erroneously recognized text (step S66-5: NO), the determination unit 55 proceeds to the process of step S66-7, and moves on to the process of other keywords.

[0128] In the above-mentioned process of step S66, the correction generation unit 57 of the voice recording management device 5 corrects the target keyword with the correction text corresponding to the erroneously recognized text, but the present invention is not limited to this. For example, the voice recording management device 5 may not include the correction generation unit 57, and the voice recognition server 7 may transmit the corrected text after the correction to the voice recording management device 5, and the voice recording management device 5 may receive the corrected text from the voice recognition server 7.

[0129] Returning to Fig. 13, after receiving the voice recognition result transmitted by the voice recognition server 7 in step S65, the registration unit 58 registers the received text information for one record in the text information management DB 5003 (see Fig. 7) (step S67). In this case, the text information for one record is information for the text identification information, start time, end time, text, and voice data path, and the editing result, editor identification information, and editing date and time are left blank. Furthermore, the bookmark is set to "OFF".

[0130] After executing step S67, obtaining and extracting section 52 obtains corresponding text data by searching text information management DB 5003 (see FIG. 7) using the received start time as a search key, for example (step S68).

[0131] After acquiring the text data, the calculation unit 53 calculates the start date and time (step S69). The calculation of the start date and time is performed based on the following formula. In other words, the start date and time is the sum of the start date and time managed in the record bibliographic information management DB 3002 (see FIG. 6) and the start date and time managed in the text information management DB 5003 (see FIG. 7). Specifically, 2021 / 03 / 31 11:00:00 + 00:01:53 = 2021 / 03 / 31 11:01:53 is the start date and time calculated in step S69.

[0132] Next, the transmitting / receiving unit 51 of the voice recording management device 5 transmits a recording screen update request to the communication terminal 3 (step S70). As a result, the transmitting / receiving unit 31 of the communication terminal 3 receives the recording screen update request transmitted by the recording management device 5. At this time, the recording screen update request includes corrected text information (text data) including the corrected text corrected based on the organization name, and start date and time information indicating the start date and time calculated in step S69.

[0133] The display control unit 34 of the communication terminal 3 that has received the recording screen update request causes the voice recording indicated by the corrected text information transmitted by the voice recording management device 5 to be displayed on the display 318. Specifically, the display control unit 34 causes the voice recording indicated by the corrected text information included in the recording screen as shown in Fig. 20 to be displayed (step S71).

[0134] In addition, in the communication system 1 of this embodiment, the correction generation unit 57 may be configured to be provided in the voice recording management device 5, or may be configured to be provided in another device that is capable of communicating between the voice recording management device 5 and the communication terminal 3 via the communication network 100.

[0135] Furthermore, in the voice management system according to this embodiment, for example, when the processes of steps S63 and S65 described above are executed, other devices may exist between the voice recording management device 5 and the voice recognition server 7. In other words, each piece of information (data) transmitted and received between the voice recording management device 5 and the voice recognition server 7 may be configured to be transmitted and received once via another device. The above-mentioned configuration is applicable even if other processing steps exist between the voice recording management device 5 and the voice recognition server 7.

[0136] <Screen capture processing> Next, the screen capture process will be described. Fig. 15 is a sequence diagram showing an example of the screen display process after updating. As shown in Fig. 15, the audio / image acquisition unit 33 of the communication terminal 3 performs a capture process on the record viewing / editing screen as shown in Fig. 24 described later, for example, every 30 seconds to acquire a capture image (step S81). Note that the time interval for acquiring the capture image is not limited to 30 seconds, and may be set arbitrarily, such as every 10 seconds or every minute.

[0137] After acquiring the capture image, the transmission / reception unit 31 transmits the acquired capture image to the voice recording management device 5 (step S82). As a result, the transmission / reception unit 51 of the voice recording management device 5 receives the capture image transmitted by the communication terminal 3. At this time, the information transmitted by the communication terminal 3 includes the image ID and the capture image corresponding to the image ID.

[0138] After receiving the image ID and the captured image, the registration unit 58 of the voice recording management device 5 registers information including the capture time as captured image information and the image data path indicating the storage location of the captured image in the captured image management DB 5004 (see Figure 8) (step S83).

[0139] After registering the capture image information, the transmitting / receiving unit 51 transmits a record screen update request to the communication terminal 3 (step S84). As a result, the transmitting / receiving unit 31 of the communication terminal 3 receives the record screen update request transmitted by the record management device 5.

[0140] After receiving the recording screen update request, the display control unit 34 of the communication terminal 3 updates the recording screen (step S85) and displays the updated screen on the display 318 (step S86). The recording screen updated at this time will be described in detail later with reference to FIG.

[0141] In this embodiment, the voice management system 2 repeatedly executes the processing from steps S81 to S86 described above until the specified event ends, that is, the voice management system 2 repeatedly executes the processing from steps S81 to S86 described above until the voice recording ends.

[0142] <End recording> Next, the recording end instruction will be described. Fig. 16 is a sequence diagram showing an example of the recording end and recording editing process of the recorded bibliographic information. First, the operation reception unit 22 of the communication terminal 3 receives a recording end instruction from the user (step S91). This recording end instruction is an instruction to end the recording of the contents of the speech spoken at an event such as a conference, and is issued, for example, by the user operating (pressing or tapping, etc.) the recording end button 3137 displayed on the recording screen 3131 shown in Fig. 20.

[0143] After receiving the recording end instruction, the transceiver 31 transmits a recording end request to the voice recording management device 5 (step S92). As a result, the transceiver 51 of the voice recording management device 5 receives the recording end request transmitted by the communication terminal 3.

[0144] After receiving the recording end request, the transmitting / receiving unit 51 of the voice recording management device 5 transmits a voice information transmission end notification to the voice recognition server 7 (step S93). As a result, the transmitting / receiving unit 71 of the voice recognition server 7 receives the voice information transmission end notification transmitted by the voice recording management device 5.

[0145] After receiving the voice information transmission end notification, the transmitting / receiving unit 71 of the voice recognition server 7 transmits a receipt of the voice information transmission end notification to the voice recording management device 5 (step S94). As a result, the transmitting / receiving unit 51 of the voice recording management device 5 receives the receipt of the voice information transmission end notification transmitted by the voice recognition server 7.

[0146] After receiving the voice information transmission end notification, the transceiver 51 of the voice recording management device 5 transmits a recording end screen display request to the communication terminal 3 (step S95). As a result, the transceiver 31 of the communication terminal 3 receives the recording end screen display request transmitted by the voice recording management device 5.

[0147] After receiving the recording end screen display request, the display control unit 34 of the communication terminal 3 displays a recording end screen as shown in Fig. 21 on the display 318 (step S96). Note that the process of step S96 may be omitted.

[0148] On the other hand, the correction generation unit 57 of the voice recording management device 5 that transmitted the recording end screen display request in step S95 generates voice recording data of the predetermined event where the recording end operation was performed (step S97). Specifically, the correction generation unit 57 combines multiple voice data paths (such as "... / 0005006 / 0005.mp3" and "... / 0005006 / 0006.mp3") managed in the text information management DB 5003 (see FIG. 7) to generate a voice data path (such as "... / 00005006 / record.mp3") that is the storage destination of the voice recording data managed in the record bibliographic information management DB 5002 (see FIG. 6).

[0149] Following the process of step S97, the registration unit 58 registers the recorded bibliographic information including the generated audio data path in the recorded bibliographic information management DB 5002 (see FIG. 6) (step S98). At this time, the recorded bibliographic information registered by the registration unit 58 is information including the end date and time and the audio data path ("... / 00005006 / record.mp3").

[0150] ●Screen display example● 21 is an example of a screen display at the end of recording in a communication terminal. After the process of step S96 described above is executed, the display control unit 34 of the communication terminal 3 displays a recording end screen 3141 on the display 318 as an example of a screen at the end of recording. Specifically, the display control unit 34 displays a save end mark 3142 indicating the end of saving, a conference setting button 3143 for recording a new conference, and a log confirmation button 3144 for viewing a log on the recording end screen 3141. The user can move to a screen for recording a new conference by operating the conference setting button 3143. In addition, the user can check various information (data) including the history of the contents spoken in a specific event, various buttons operated by the user, and a processing log by operating the log confirmation button 3144.

[0151] <Viewing and editing process> Next, a recording end instruction will be described. Fig. 22 is a sequence diagram showing an example of a process from receiving a recording selection to displaying the record viewing and editing screen.

[0152] In this embodiment, the record viewing and editing functions can be provided as a Web service. Therefore, the voice recording management device 5 has a Web server function, and transmits screen data for displaying the entire screen or a part of the screen to the communication terminal 3(C), which is an example of a minutes viewing terminal, and displays it in a browser application of the communication terminal 3(A), which is an example of a minutes creating terminal. The voice recording management device 5 may also transmit content data and display instruction information for various displays to the communication terminal 3(C), and display a predetermined screen in an application installed in the communication terminal 3(C).

[0153] Based on this premise, first, the operation acceptance unit 22 of the communication terminal 3 accepts a record selection by the user on a record selection screen as shown in FIG. 23 (step S101).

[0154] After accepting the recording selection, the transmitting / receiving unit 31 transmits a record viewing and editing screen request to the voice recording management device 5 (step S102). As a result, the transmitting / receiving unit 51 of the voice recording management device 5 receives the record viewing and editing screen request transmitted by the communication terminal 3. At this time, the record viewing and editing screen request includes record identification information (R0005, R0006, etc.). In steps S101 and S102, the user has already logged in and the record list screen is displayed. At this time, the user may directly enter the shared record URL into the address bar of the browser.

[0155] After receiving the record viewing and editing screen request, the acquisition and extraction unit 52 of the voice recording management device 5 searches the record bibliographic information management DB 5002 (see FIG. 6), the text information management DB 5003 (see FIG. 7), and the capture image management DB 5004 (see FIG. 8) using the record identification information as a search key, respectively, to acquire the corresponding record bibliographic information, text information, and capture information (step S103). Specifically, the acquisition and extraction unit 52 searches the record bibliographic information management DB 5002 (see FIG. 6) using the record identification information as a search key to acquire the corresponding record bibliographic information. Here, the record bibliographic information includes the record name, start date and time, user identification information, and event URL. In addition, the acquisition and extraction unit 52 searches the text information management DB 5003 (see FIG. 7) using the record identification information as a search key to acquire the corresponding text information. Here, the text information includes the text identification information, start time, end time, text, and voice data path. Furthermore, the acquiring and extracting unit 52 searches the capture image management DB 5004 (see FIG. 8) using the record identification information as a search key to acquire corresponding capture information. Here, the capture information includes an image ID, acquisition time, and image data path.

[0156] Based on the various information acquired in step S103, the correction generation unit 57 generates a record viewing and editing screen. More specifically, the correction generation unit 57 generates record viewing and editing screen data that configures the record viewing and editing screen (step S104).

[0157] After generating the record viewing and editing screen data, the transmitting / receiving unit 51 transmits the record viewing and editing screen data to the communication terminal 3 as a response to the record viewing and editing screen request (step S105). As a result, the transmitting / receiving unit 31 of the communication terminal 3 receives the record viewing and editing screen data transmitted by the voice recording management device 5.

[0158] After receiving the record viewing and editing screen data, the display control unit 34 of the communication terminal 3 displays the record viewing and editing screen shown in FIG. 24 or FIG. 26 on the display 318 (step S106).

[0159] ●Screen display example● 23 is an example of a screen display when selecting a record in a communication terminal. When the process of step S101 described above is executed, the display control unit 34 of the communication terminal 3 displays a record selection screen 3151 on the display 318 as an example of a screen when selecting a record. Specifically, the display control unit 34 displays the date and time and the event title indicating the recorded content as a set in a selectable display form on the record selection screen 3151. This allows the user to select an event represented by an arbitrary date and event title. Note that the technology and method for selecting the content displayed on the display and executing a process according to the selected content are omitted from the description, as they can be well-known technology and methods.

[0160] ●Screen display example● Fig. 24 is an example of a screen display for viewing and editing records in a communication terminal. When the process of step S106 described above is executed, the display control unit 34 of the communication terminal 3 displays a record viewing and editing screen 3161 on the display 318 as an example of a screen for viewing and editing records. The record viewing and editing screen 3161 includes an overview (agenda) column 3162, a participant column 3163, a meeting memo column 3164, and a bookmark column 3165, as in the case of Fig. 18. Here, the overview (agenda) column 3162, the participant column 3163, the meeting memo column 3164, and the bookmark column 3165 are similar to the overview (agenda) column 3112, the participant column 3113, the meeting memo column 3114, and the bookmark column 3115 shown in Fig. 18, so their explanations will be omitted.

[0161] The display control unit 34 displays text and screen display fields 3166, 3167, and 3168 on the record viewing and editing screen 3161. In the text and screen display fields 3166-3168, for each user who has spoken, the face photo or image of the user, the speech date and time, an utterance content editing field in which the utterance content can be edited, and the screen displayed at that time are displayed in a group in chronological order. Here, in the editing field in which the utterance content can be edited, text information such as "Today's meeting will be locked on" transmitted by the voice recognition server 7 is displayed. When displaying as a group, the display control unit 34 may display the screen by dividing it into 30 seconds, which is an example of the capture processing interval executed in the screen capture processing. Furthermore, the number of text and screen display fields is not limited to that exemplified in FIG. 24, and text and screen display fields for other dates and times may be displayed.

[0162] <Audio playback processing> Next, the audio playback process will be described. Fig. 25 is a sequence diagram showing an example of the audio information playback process. As shown in Fig. 25, the display control unit 34 of the communication terminal 3 displays a record viewing and editing screen on the display 318 (step S111).

[0163] After the record viewing and editing screen is displayed by the display control unit 34, when the user places the mouse near the speech content editing field as shown in FIG. 24 by performing a process such as hovering the mouse over it, the operation receiving unit 32 displays an audio playback mark at the location where the mouse is placed (step S112).

[0164] ●Screen display example● Fig. 26 is an example of a screen display when a play mark is executed in a communication terminal. When the process of step S112 described above is executed, the display control unit 34 of the communication terminal 3 displays a record viewing and editing screen 3171 on the display 318 as an example of a screen when viewing and editing a record. The record viewing and editing screen 3171 includes an overview (agenda) column 3172, a participant column 3173, a meeting memo column 3174, and a bookmark column 3175, as in the case of Fig. 24. Here, the overview (agenda) column 3172, the participant column 3173, the meeting memo column 3174, and the bookmark column 3175 are similar to the overview (agenda) column 3112, the participant column 3113, the meeting memo column 3114, and the bookmark column 3115 shown in Fig. 18, so a description thereof will be omitted.

[0165] The display control unit 34 displays text and screen display fields 3176, 317, and 3178 on the record viewing and editing screen 3171. In these text and screen display fields 3176-3178, for each user who has spoken, a facial photograph or image of the user, the date and time of the speech, an utterance content editing field in which the utterance content can be edited, and the screen displayed at that time are displayed as a whole in chronological order. Note that, similarly to Fig. 24, the number of text and screen display fields is not limited to that exemplified in Fig. 26, and text and screen display fields for other dates and times may also be displayed.

[0166] In this state, when the process of step S112 is executed, the display control unit 34 displays a play mark 3179 and a bookmark 3180 in the vicinity of the speech content editing field where the mouse is placed (where processing such as mouse over is performed). As a result, when the user operates the operation for playing a specific voice, i.e., the play mark 3179, the playback unit 36 ​​can play the voice related to the specific voice recording selected by the user. Also, when the user operates the operation for realizing a specific bookmark function, i.e., the bookmark 3180, the display control unit 34 can display the specific voice recording selected by the user in a predetermined bookmark display area. At this time, the content of the speech content editing field where the mouse is placed (where processing such as mouse over is performed) is treated as an example of the specific voice recording. In this embodiment, the play mark 3179 is an example of a playback operation unit for playing a specific voice recording, and the bookmark 3180 is an example of a bookmark operation unit for realizing a bookmark function for the specific voice recording.

[0167] Returning to FIG. 25, when the user performs an operation such as clicking on the play mark, the operation receiving unit 32 receives the operation on the play mark in the process of step S112 described above.

[0168] Thereafter, the transmitting / receiving unit 31 transmits a voice playback request to the voice recording management device 5 (step S113). As a result, the transmitting / receiving unit 51 of the voice recording management device 5 transmits the voice playback request transmitted by the communication terminal 3. At this time, the voice playback request includes the text identification information.

[0169] The calculation unit 53 of the audio recording management device 5 that receives the audio playback request calculates and identifies the corresponding start time by searching the text information management DB 5003 (see Figure 7) using the text identification information included in the received audio playback request as a search key (step S114).

[0170] After identifying the start time, the calculation unit 53 moves the start time of the audio information to be reproduced to the identified start time (date and time) (step S115).

[0171] After moving the start time of the voice information to the specified start time (date and time), the transmitting / receiving unit 51 transmits a voice playback response to the communication terminal 3 (step S116). As a result, the transmitting / receiving unit 31 of the communication terminal 3 receives the voice playback response transmitted by the voice recording management device 5. At this time, the voice playback response includes the voice information to be played back.

[0172] After receiving the audio playback response, the playback unit 36 ​​of the communication terminal 3 plays the audio to be played back from the beginning, and outputs the audio from the speaker 316 of the communication terminal 3 (step S117).

[0173] <Bookmark processing> Next, the bookmark process will be described. Fig. 27 and Fig. 28 are sequence diagrams showing an example of the bookmark process. As shown in Fig. 27, the display control unit 34 of the communication terminal 3 displays a record viewing and editing screen on the display 318 (step S121).

[0174] After the record viewing and editing screen is displayed by the display control unit 34, when the user places the mouse near the speech content editing field as shown in Figure 29 by performing a process such as hovering the mouse over it, the operation receiving unit 32 displays a bookmark at the location where the mouse is placed (step S122).

[0175] ●Screen display example● FIG. 29 is an example of a screen display when bookmarking is executed in a communication terminal. When the process of step S122 described above is executed, the display control unit 34 of the communication terminal 3 displays a record viewing and editing screen 3181 on the display 318 as an example of a screen when viewing and editing a record. The record viewing and editing screen 3181 includes an overview (agenda) column 3182, a participant column 3183, a meeting memo column 3184, and a bookmark column 3185, as in the case of FIG. 24. Here, the overview (agenda) column 3182, the participant column 3183, the meeting memo column 3184, and the bookmark column 3185 are the same as the overview (agenda) column 3112, the participant column 3113, the meeting memo column 3114, and the bookmark column 3115 shown in FIG. 18, so that the description will be omitted. In FIG. 29, the display control unit 34 further displays a remark display button 3190 in the bookmark column 3185. The remark display button 3190 is a button used in the process of step S128 described later. By operating the utterance display button 3190, the display control unit 34 scrolls the utterance content editing field including the selected text information, and displays it, for example, at the top of the utterance content displayed on the display 318. In this embodiment, the utterance display button 3190 functions as an example of a utterance display operating unit for displaying utterances indicating specific audio records.

[0176] The display control unit 34 displays text and screen display fields 3186, 318, and 3188 on the record viewing and editing screen 3181. In these text and screen display fields 3186-3188, for each user who has spoken, a facial photograph or image of the user, the date and time of the speech, an utterance content editing field in which the utterance content can be edited, and the screen displayed at that time are displayed in chronological order as a whole. Note that, as in Fig. 24, the number of text and screen display fields is not limited to that exemplified in Fig. 29, and text and screen display fields for other dates and times may also be displayed.

[0177] In this state, when the process of step S122 is executed, the display control unit 34 displays a bookmark 3180 near the utterance content editing field where the mouse is placed (where processing such as mouse over is performed). Note that a play mark may be displayed in addition to the bookmark as shown in Fig. 26. This allows the user to perform bookmark processing.

[0178] 27, when the user performs an operation such as clicking on the bookmark, the operation acceptance unit 32 accepts the operation on the bookmark in the process of step S122 described above. Specifically, the operation acceptance unit 32 accepts a mark ON operation by the user.

[0179] After that, the transmitting / receiving unit 31 transmits a bookmark registration request to the voice recording management device 5 (step S123). As a result, the transmitting / receiving unit 51 of the voice recording management device 5 transmits the bookmark registration request transmitted by the communication terminal 3. At this time, the bookmark registration request includes the text identification information.

[0180] The registration unit 58 of the voice recording management device 5 that has received the bookmark registration request registers a bookmark corresponding to the text identification information included in the received bookmark registration request (step S124). Specifically, the registration unit 58 sets the bookmark corresponding to the text identification information in the text information identification table managed in the text information management DB 5003 (see FIG. 7) to "ON."

[0181] After registering the bookmark, the transmitting / receiving unit 51 transmits a bookmark registration completion notification to the communication terminal 3 (step S125). As a result, the transmitting / receiving unit 31 of the communication terminal 3 receives the bookmark registration completion notification transmitted by the voice recording management device 5.

[0182] After receiving the bookmark registration completion notification, the display control unit 34 of the communication terminal 3 highlights the utterance content editing field including the text information that is the target of bookmark registration. The highlighting at this time may be performed by using a different color from other utterance content editing fields, or the characters of the text information to be bookmarked displayed in the utterance content editing field may be displayed in bold. Furthermore, the display control unit 34 additionally displays the text information that is the target of bookmark registration in the bookmark field provided on the display 318 (step S126).

[0183] Furthermore, the display control unit 34 accepts a bookmark selection for the text information additionally displayed in the bookmark section (step S127).

[0184] Next, when the statement display button 3190 is operated, the display control unit 34 scrolls the utterance content editing field including the text information selected in step S127, and displays it, for example, at the top of the utterance content displayed on the display 318 (step S128).

[0185] Thereafter, when the user performs an operation to turn off the bookmark for the predetermined text, the operation reception unit 32 of the communication terminal 3 receives the operation to turn off the bookmark for the predetermined text (step S129).

[0186] In Fig. 28, after accepting the operation to turn off the bookmark, the transceiver 31 transmits a bookmark deletion request to the voice recording management device 5 (step S131). As a result, the transceiver 51 of the voice recording management device 5 receives the bookmark deletion request transmitted by the communication terminal 3. At this time, the bookmark deletion request includes text identification information.

[0187] After receiving the bookmark deletion request, the registration unit 58 searches the text information management DB 5003 (see FIG. 7) using the received text identification information as a search key, and deletes the corresponding bookmark (changing it from “ON” to “OFF”) (step S132).

[0188] After deleting the bookmark, the transceiver 51 transmits a notification of completion of bookmark deletion to the communication terminal 3 (step S133). As a result, the transceiver 31 of the communication terminal 3 receives the notification of completion of bookmark deletion transmitted by the voice recording management device 5.

[0189] After receiving the notification of the completion of bookmark deletion, the display control unit 34 of the communication terminal 3 cancels the highlighting of the text information to be bookmarked that was performed in step S126, and erases the content displayed in the bookmark column (step S134).

[0190] <Recording Editing Process> Next, the recording and editing process will be described. FIG. 30 is a sequence diagram showing an example of the recording and editing process of the correction information. In FIG. 30, first, the operation acceptance unit 32 accepts text editing performed by the user in a predetermined utterance content editing field in which a predetermined text is displayed. Furthermore, the operation acceptance unit 32 cooperates with the display control unit 34 to update the screen in which the text information including the edited text is displayed (step S141). This text information including the edited text includes, for example, a corrected text obtained by the user of the communication terminal 3(A) correcting a specific text (misrecognized text).

[0191] Next, the transmitting / receiving unit 31 transmits a text editing request to the voice recording management device 5 (step S142). As a result, the transmitting / receiving unit 51 of the voice recording management device 5 receives the text editing request transmitted by the communication terminal 3. At this time, the text editing request includes the login user identification information (user identification information) and the edited text data. In the process executed in step S142, the login user identification information corresponds to the editor identification information managed in the text information management DB 5003 (see FIG. 7). The timing of transmitting the text editing request in step S142 may be, for example, the timing when a period (.) is entered in the edited sentence in the speech content editing field, or the timing when a send button separately provided on the record viewing and editing screen 3181 is operated.

[0192] After receiving the text editing request, the registration unit 58 of the audio recording management device 5 performs a registration process of the text information edited by the user (editor) in the editing result corresponding to the editor identification information managed in the text information management DB 5003 (see Figure 7) (step S143).

[0193] Following the process of registering the text information, the registration unit 58 performs a process of registering correction information (step S144).

[0194] After registering the correction information, the transmitting / receiving unit 51 transmits a text edit response to the communication terminal 3 (step S145). As a result, the transmitting / receiving unit 31 of the communication terminal 3 receives the text edit response transmitted by the voice recording management device 5.

[0195] <Details of the correction information registration process> Fig. 31 is a flow chart showing the details of the recording and editing process of the correction information. As shown in Fig. 31, the acquisition and extraction unit 52 of the voice recording management device 5 acquires the edited text (detects the editing) (step S144-1-1).

[0196] Next, obtaining and extracting unit 52 compares the "text" and the "editing result" managed in text information management DB 5003 (see FIG. 7) to extract the difference (step S144-1-2).

[0197] After the acquisition / extraction unit 52 extracts the difference, the judgment unit 55 judges whether the difference has been extracted (step S144-1-3). If the difference has been extracted (step S144-1-3: YES), the acquisition / extraction unit 52 acquires the user identification information and the organization name (step S144-1-4).

[0198] Thereafter, the registration unit 58 replaces the correction text corresponding to the organization name managed in the correction information management DB 5006 (see FIG. 10) with the extracted difference content, registers it, and exits this flow (step S144-1-5).

[0199] On the other hand, if a difference cannot be extracted (step S144-1-3: NO), the determination unit 55 returns to the process of step S144-1-1.

[0200] ●Screen display example● Fig. 32 shows an example of a screen display for setting whether or not a user registration instruction is required and the validity period of the correction target in a communication terminal. The display control unit 34 of the communication terminal 3 displays an initial setting screen 3301 as shown in Fig. 32 on the display 318 of the communication terminal 3 used by a system administrator or the like who manages the communication system 1 to which a predetermined authority has been granted. The timing for displaying the initial setting screen may be, for example, by operating a setting button provided on the screen, or may be given at any timing.

[0201] In the initial setting screen 3301, whether or not a user registration instruction is required can be selected by using a radio button. The necessity of a user registration instruction refers to whether or not it is necessary to register the corrected text edited in the speech content editing field described in Fig. 24 and the like, and can be selected by the user. The necessity of this instruction will be described in detail in the embodiment of the dialog screen display described later.

[0202] Furthermore, on the initial setting screen 3301, the display control unit 34 displays a validity period setting area in which the validity period of the correction target can be set. The validity period of the correction target is a period that is set depending on the freshness of the text to be corrected, the necessity of its usability within a specific organization, etc., and is given as a period such as "within XX months" or "XX year ●● month △▽ day to XX year XX month ◆◆ day", and these may be selected by a selection button. The above-mentioned validity period of the correction target can be set by users including the system administrator.

[0203] Furthermore, the display control unit 34 displays an OK button 3311 and a cancel button 3312 on the initial setting screen 3301 to accept a selection by a user (such as a system administrator). In this embodiment, as described above, the communication terminal 3 used by the system administrator, for example, can display the initial setting screen 3301 when starting an application and accept settings for each function. In the example shown in FIG. 32, the necessity of a user registration instruction, the validity period of a correction target, and the like can be uniquely determined in the voice management system 2. By limiting the authority to make such settings to the system administrator, it is possible to avoid duplication of settings between other users, situations in which different settings are made simultaneously, and the like.

[0204] [Major Effects of the First Embodiment] As described above, according to this embodiment, the voice record management device 5 receives voice information related to the voice uttered by the user using the communication terminal 3, transmitted from the communication terminal 3 (step S62), and transmits corrected text information including corrected text corrected based on organizational information of the organization to which the user belongs, among texts recognized from the received voice information, to one or more communication terminals (steps S66-6, S70). The display control unit 34 of the communication terminal 3 displays the voice record indicated by the corrected text information transmitted from the voice record management device 5 on the display 318 (step S71). As a result, when the voice information related to the voice uttered by the user is converted into text information and used, the conversion corresponding to the organizational information of the organization to which the user belongs is performed, so that it is possible to provide the user with easy-to-read text information.

[0205] [Modification of the first embodiment] Next, a modified example of the first embodiment will be described with reference to Fig. 33 and Fig. 34. The modified example of the first embodiment is different from the first embodiment in that a function for detecting a user's operation on a registration necessity dialog displayed on the communication terminal 3 is added when registering the correction text corrected in the recording and editing process.

[0206] Fig. 33 is a flowchart showing details of the correction information recording and editing process according to the modified embodiment. The process from steps S144-2-1 to S144-2-3 in Fig. 33 is similar to the process from steps S144-1-1 to S144-1-3 described in Fig. 31, and therefore the description will be omitted.

[0207] In the modified example of the first embodiment, the acquiring and extracting unit 52 executes the following process as the next process when the difference is extracted. In the process of registering the correction text edited in the speech content editing field of the record viewing and editing screen displayed on the communication terminal 3, the acquiring and extracting unit 52 detects an operation on the registration necessity dialog as shown in Fig. 34 (step S144-2-4).

[0208] Thereafter, the determination unit 55 determines whether or not to register the operation in the registration necessity dialogue in the correction information (step S144-2-5). Note that the execution of the processes in steps S144-2-4 and S144-2-5 can be realized by, in cooperation with the transmission / reception unit 51, receiving a signal related to the operation in the registration necessity dialogue transmitted by the communication terminal 3 at a predetermined interval, for example.

[0209] If the signal related to the operation in response to the registration necessity dialogue sent by the communications terminal 3 indicates the contents to be registered in the correction information (step S144-2-5: YES), the processes of steps S144-1-4 and S144-1-5 described in FIG. 31 are executed, and then this flow is exited.

[0210] On the other hand, if the signal related to the operation in response to the registration necessity dialogue transmitted by the communications terminal 3 indicates that the correction information should not be registered (step S144-2-5: NO), the determination unit 55 returns to the process of step S144-2-1.

[0211] ●Screen display example● 34 is a screen display example of a registration necessity dialogue in a communication terminal according to a modified example of the embodiment. When the same process as in step S106 described above is executed, the display control unit 34 of the communication terminal 3 displays a record viewing and editing screen 3191 on the display 318 as a screen example of the modified example when viewing and editing a record. The record viewing and editing screen 3191 includes an overview (agenda) column 3192, a participant column 3193, a meeting memo column 3194, and a bookmark column 3195, as in the case of FIG. 24. Here, the overview (agenda) column 3192, the participant column 3193, the meeting memo column 3194, and the bookmark column 3195 are the same as the overview (agenda) column 3162, the participant column 3163, the meeting memo column 3164, and the bookmark column 3165 shown in FIG. 24, so description thereof will be omitted. Furthermore, the record viewing and editing screen 3191 includes text / screen display columns 3196, 3197, and 3198, as in the case of FIG. 24. The text / screen display fields 3196, 3197, and 3198 are similar to the text / screen display fields 3166, 3167, and 3168 shown in FIG. 24, and therefore a description thereof will be omitted.

[0212] The display control unit 34 also displays a registration necessity dialog 3199 in the vicinity of the speech content editing field being edited by the user. The registration necessity dialog 3199 is displayed when the setting for the necessity of the user registration instruction in FIG. 32 described in the first embodiment is set (selected) to "Necessary". In the registration necessity dialog 3199, in addition to a message (notification) that the text being edited (for example, "lock on" which is an example of erroneously recognized text) has been converted into corrected text, buttons ("YES button 3211" and "NO button 3212") for selecting whether to register the corrected corrected text are displayed. The user can perform a process corresponding to the selected button by selecting one of these buttons. In this embodiment, the registration necessity dialog 3199 functions as an example of a registration necessity selection unit. The system administrator of the communication system 1 may, for example, associate the setting for the necessity of the user registration instruction in FIG. 32 with the department name of the department that hosts a specific event, and set a restriction such as limiting the setting to users belonging to that department. By setting such a restriction, it is possible to prevent, as much as possible, the possibility that the correction text is corrected to a text other than the text frequently used in the department.

[0213] [Major Effects of the Modification of the First Embodiment] As described above, according to this embodiment, the communication terminal 3 displays the registration necessity dialogue 3197 near the speech content editing field being edited by the user, and allows the user to select whether or not to register the correction information. The voice recording management device 5 receives the user's selection result in the registration necessity dialogue 3197, and performs processing according to the result. This has the effect of preventing, for example, a user from another department from carelessly replacing the text with a meaningless correction text.

[0214] [Supplementary description of the embodiment] Each function of the above-mentioned embodiment can be realized by one or more processing circuits. Here, the term "processing circuit" in this specification includes a device programmed to execute each function by software, such as a processor implemented by an electronic circuit. This device refers to, for example, a processor, an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), a SOC (System on a chip), a GPU (Graphics Processing Unit), and a conventional circuit module designed to execute each function described above.

[0215] We have described an audio management system, audio recording management device, audio recording management method, and program according to one embodiment of the present invention, but the present invention is not limited to the above-described embodiment and can be modified within the scope of what a person skilled in the art can imagine, such as adding, changing, or deleting other embodiments, and as long as the functions and effects of the present invention are achieved in any form, it is within the scope of the present invention. [Explanation of symbols]

[0216] 1. Communication Systems 2. Voice Management System 3. Communication terminals 5. Audio recording management device 7. Speech Recognition Server 31 Transmitting / receiving unit (an example of a receiving means, an example of a transmitting means) 32 Reception unit (an example of a reception means) 33 Sound / image acquisition unit (an example of an acquisition means) 34 Display control unit (an example of a display control means) 36 Reproducing unit (an example of a reproducing means) 38 Application launch section (an example of a launching means) 51 Transmitting / receiving unit (an example of a receiving means, an example of a transmitting means) 52 Acquisition and extraction unit (an example of an acquisition means) 53 Calculation unit (an example of a calculation means) 54 Display control unit (an example of a display control means) 55 Judgment unit (an example of a judgment means) 57 Correction generation unit (an example of a correction means, an example of a generation means) 58 Registration unit (an example of a registration means) 3179 Play mark (example of the playback operation part) 3180 Bookmark (an example of the bookmark operation section) 3190 Remarks display button (an example of the remarks display operation section) 3199 Registration Necessity Dialog (Example of Registration Necessity Selection Section) 5006 Correction information management DB (an example of correction information management means) [Prior art documents] [Patent documents]

[0217] [Patent Document 1] JP 2017-191533 A

Claims

1. A voice management system including one or more communication terminals capable of displaying text information converted based on voice information, and a voice record management device that communicates with the communication terminals and manages the text information displayed on the communication terminals as a voice record, The voice recording management device includes: a receiving means for receiving voice information transmitted by the one or more communication terminals, the voice information being related to a voice spoken by a user using a specific communication terminal among the one or more communication terminals; a transmission means for transmitting corrected text information, including corrected text corrected based on organizational information of the organization to which the user belongs, from among the text recognized from the received voice information, to the one or more communication terminals; having The one or more communication terminals include a display control means for displaying, in chronological order, the voice recordings indicated by the corrected text information transmitted by the voice recording management device on a display means, and displaying, among one or more voice recordings displayed in chronological order on the display means, a playback operation section for playing back a specific voice recording selected by the user in the vicinity of the specific voice recording; a playback means for playing back the audio related to the specific audio recording selected by the user in response to the operation of the playback operation unit; having 13. A voice management system comprising:

2. The organization information includes a name of an organization to which the user belongs, The transmitting means is transmitting, to the one or more communication terminals, the corrected text information including the corrected text managed in association with each of the organization names among the text recognized from the voice information; The voice management system according to claim 1 .

3. 3. The voice management system according to claim 1, the corrected text information includes the corrected text obtained by a user of a predetermined communication terminal among the one or more communication terminals correcting a specific text included in the text recognized from the voice information, The voice recording management device further comprises: a registration means for registering the corrected text in a text information management means for managing specific text information including the specific text and corrected text information including the corrected text in association with each other, 13. A voice management system comprising:

4. 4. The voice management system according to claim 3, The voice recording management device further comprises: a correction means for correcting the erroneous recognition text to the correction text when the specific text recognized from the received voice information is an erroneous recognition text due to an erroneous recognition; 13. A voice management system comprising:

5. 5. The voice management system according to claim 4, The voice recording management device further comprises: a correction information management means for managing the misrecognized text and the corrected text in association with each other; having The correction means is correcting the misrecognized text managed by the correction information management means with the correction text; 13. A voice management system comprising:

6. 6. A voice management system according to claim 1, further comprising: The display control means a bookmark operation section for realizing a bookmark function for a particular voice recording selected by the user among the one or more voice recordings displayed in chronological order on the display means is displayed near the particular voice recording, and the particular voice recording selected by the user is displayed in a predetermined bookmark display area by operating the bookmark operation section; 13. A voice management system comprising:

7. 7. The voice management system according to claim 6, The display control means a comment display operation unit for displaying comments indicating the specific voice recording after the specific voice recording is displayed in the bookmark display area, the comment display operation unit displayed in the bookmark display area being operated to display comments indicating the specific voice recording among the one or more voice recordings, scrolling said particular voice recording and displaying it on said display means; 13. A voice management system comprising:

8. 6. The voice management system according to claim 5, The display control means displaying, in the vicinity of an utterance content editing field of a specific voice recording being edited by the user among the one or more voice recordings displayed in chronological order on the display means, a notification that the erroneously recognized text has been converted into the corrected text and a registration necessity selection section for selecting whether or not to register the corrected text; When the selection of registering the correction text is made in the registration necessity selection unit, the registration means registering the converted corrected text in the correction information management means; 13. A voice management system comprising:

9. 9. The voice management system according to claim 8, The display control means a screen including a registration necessity selection section for selecting whether or not to display the registration necessity selection section and a validity period setting area for setting a validity period of the correction text is displayed on the display means; 13. A voice management system comprising:

10. The corrected text information is text information constituting information related to a voice recording including minutes of an event and a speech memo of the user's speech.

10. The voice management system according to claim 1,

11. A voice record management device that manages text information converted based on voice information as a voice record, comprising: a receiving means for receiving voice information transmitted from one or more communication terminals capable of displaying the text information, the voice information being related to a voice spoken by a user using a specific communication terminal among the one or more communication terminals; a transmission means for transmitting corrected text information, including corrected text corrected based on organizational information of the organization to which the user belongs, from among the text recognized from the received voice information, to the one or more communication terminals; a generating means for generating display screen information for displaying, in chronological order, the voice recordings indicated by the correction text information to be transmitted to the one or more communication terminals, and for generating display screen information for displaying a playback operation section for playing back a specific voice recording selected by the user from among the one or more voice recordings displayed in chronological order on the display screen of the one or more communication terminals; having The transmitting means is transmitting, to the one or more communication terminals, audio data for reproducing the audio related to the specific audio recording selected by the user, when the playback operation unit is operated in the one or more communication terminals; 1. A voice recording management device comprising:

12. A voice recording management method executed by a voice recording management device that manages text information converted based on voice information as a voice recording, comprising: receiving voice information transmitted from one or more communication terminals capable of displaying the text information, the voice information being related to a voice spoken by a user using a specific communication terminal among the one or more communication terminals; transmitting corrected text information, which includes corrected text corrected based on organization information of the organization to which the user belongs, from among the text recognized from the received voice information, to the one or more communication terminals; generating display screen information for displaying, in chronological order, the voice recordings indicated by the correction text information to be transmitted to the one or more communication terminals, and generating display screen information for displaying a playback operation section for playing back a specific voice recording selected by the user among the one or more voice recordings displayed in chronological order on the display screen of the one or more communication terminals; transmitting, to the one or more communication terminals, audio data for reproducing the audio related to the specific audio recording selected by the user, by operating the playback operation unit in the one or more communication terminals; Perform a process that includes A voice recording management method comprising:

13. A computer of a voice record management device that manages text information converted based on voice information as a voice record, a receiving step of receiving voice information transmitted from one or more communication terminals capable of displaying the text information, the voice information being related to a voice spoken by a user using a specific communication terminal among the one or more communication terminals; a transmitting step of transmitting corrected text information, the corrected text being corrected based on organization information of the organization to which the user belongs, to the one or more communication terminals, among the text recognized from the received voice information; generating display screen information for displaying, in chronological order, the voice recordings indicated by the correction text information to be transmitted to the one or more communication terminals, and generating display screen information for displaying a playback operation section for playing back a specific voice recording selected by the user among the one or more voice recordings displayed in chronological order on the display screen of the one or more communication terminals; transmitting, to the one or more communication terminals, audio data for reproducing the audio related to the specific audio recording selected by the user, by operating the playback operation unit in the one or more communication terminals; A program that executes the following.

Images