Voice information processing device

Abstract

To make it difficult for meaningful information to be illegally obtained when voice information is converted to text data by performing voice recognition using an external voice recognition engine.SOLUTION: Voice information to be recognized is divided into multiple split voice information by a voice information division section 112, and an order information assignment section 114 assigns order information indicating a playback order to each of them. A transmission control section 118 distributes and transmits each of the multiple split voice information to which the order information is assigned, to multiple voice recognition sections through an LAN I / F 101. Recognition results to which the order information is associated, which are received from each of the multiple voice recognition sections through a receiving section, are rearranged by a recognition result generating section 112 based on the order information to generate recognition results corresponding to the voice information to be recognized.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a voice information processing device that prevents leakage of meaningful information when, for example, recorded voice information of a conference or telephone call is converted into text data using an external voice recognition engine. [Background technology]

[0002] Regarding voice recognition, as disclosed in Patent Documents 1 and 2 described later, for example, a voice recognition engine is provided in a server device on the cloud so that it can be shared by various users. A user transmits recorded voice information, for example, from a meeting, to a server device on the cloud equipped with a voice recognition engine. The server device recognizes the transmitted voice information using its own voice recognition engine, converts it into text data, and returns this to the sender of the voice information. This allows a user to easily and highly accurately convert voice information that he or she holds into text using a general-purpose, highly accurate voice recognition engine provided on the cloud, without having to prepare his or her own voice recognition engine. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-91954 A [Patent Document 2] JP 2020-86048 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, as shown in Fig. 10, there is a possibility that information may be leaked due to hacking or the like on the path for transmitting voice data from the voice information processing device 100 to the server device 200 equipped with a voice recognition engine. The same is true for the path for transmitting text data from the server device 200 to the voice information processing device 100. When transmitting data, the data to be transmitted is encrypted to ensure high confidentiality, but there is still a possibility that information may be leaked. In addition, there is a possibility that information may be leaked if so-called hacking is performed on the server device 200, which processes data by decrypting the data. In view of the above, an object of the present invention is to prevent leakage of meaningful information on the transmission path or on the voice recognition engine side when voice information is converted into text data by performing voice recognition using an external voice recognition engine. [Means for solving the problem]

[0005] In order to solve the above problem, the voice information processing device of the invention described in claim 1 comprises: a voice information separation unit that separates recorded voice information of a call or conference, which includes voice information of a plurality of speakers, into voice information for each of the speakers; The voice information separating unit separates the voices for each speaker. Voice recognition target The above Audio information Each of a voice information division unit for dividing the voice information into The audio information divided by the audio information division unit For each speaker a sequence information adding unit that adds sequence information indicating a playback sequence to each of the plurality of pieces of divided audio information; The order information is given For each speaker A plurality of the divided audio information For each of the above, speaker identification information for distinguishing the speaker and a start time based on the start time of the recording are assigned, a transmission control unit that distributes and transmits the voice to a plurality of voice recognition units through a transmission unit; The sequence information received from each of the plurality of speech recognition units through a receiving unit. the speaker identification information, and the start time. is associated with For each speaker Reordering the recognition results based on the order information For each of the speakers a recognition result rearrangement unit for restoring a recognition result corresponding to the speech information of a speech recognition target; 、 a recognition result combining unit that rearranges the recognition results for each speaker, which have been rearranged by the recognition result rearrangement unit, based on the start time and makes each speaker identifiable based on the speaker identification information; The present invention is characterized by comprising: Effect of the Invention

[0007] According to this invention, the voice information to be recognized is divided into a plurality of divided voice information, and the divided voice information is distributed and transmitted to a plurality of voice recognition units, so that it is difficult to obtain meaningful information illegally. In other words, it is possible to prevent the meaningful information from being leaked. [Brief description of the drawings]

[0008] [Figure 1] 1 is a block diagram illustrating an example of the configuration of a voice recognition system including a voice information processing device according to an embodiment. [Diagram 2] 1 is a block diagram illustrating an example of a configuration of a voice information processing device according to an embodiment. [Diagram 3] 3A to 3C are diagrams illustrating examples of formats of transmission data transmitted and reception data received by the voice information processing device according to the embodiment. [Figure 4] 4 is a diagram for explaining a process executed before a voice recognition process in the voice information processing device of the embodiment. FIG. [Diagram 5] 4 is a diagram for explaining a process executed after a voice recognition process in the voice information processing device of the embodiment. FIG. [Figure 6] 1 is a diagram for explaining a process executed before a voice recognition process in a case where voice information of a plurality of speakers is recorded on different tracks and is to be processed. [Figure 7] 1A and 1B are diagrams for explaining examples of formats of transmission data and reception data when processing voice information of multiple speakers. [Figure 8] 11 is a diagram for explaining a process executed after a voice recognition process in a case where voice information of a plurality of speakers recorded on different tracks is to be processed. FIG. [Figure 9] 11 is a diagram for explaining a process executed before a voice recognition process in the case where voice information of a plurality of speakers is recorded on the same track and is to be processed. FIG. [Figure 10] 1 is a diagram for explaining a problem that occurs when performing voice recognition using a voice recognition engine provided in a server device on the Internet. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of a voice information processing device according to the present invention will be described with reference to the drawings. The voice information processing device of the embodiment described below has a function of processing voice information before voice recognition and processing text data after voice recognition when performing voice recognition using a voice recognition engine. Note that the voice recognition engine refers to hardware (device) or software that realizes the function of performing voice recognition on voice information and forming text data.

[0010] [Example of voice recognition system configuration] FIG. 1 is a block diagram for explaining a configuration example of a voice recognition system including a voice information processing device 1 of an embodiment. As shown in FIG. 1, the voice information processing device 1 of this embodiment is connected to a LAN (Local Area Network) 2. A voice recognition engine 3 and a network connection device 4 are connected to the LAN 2. The voice recognition engine 3 is a function installed in a computer device connected to the LAN 2, and is realized by software, for example, and is available to the voice information processing device 1. The network connection device 4 is a device such as a router or a gateway, and enables a connection between a wide area network 5 and the LAN 2.

[0011] The wide area network 5 is mainly the Internet, but also includes various networks, such as the public switched telephone network, that connect the network connection device 4 and the wide area network 5. Each of the voice recognition engines 6, 7, and 8 connected to the wide area network 5 is a function installed in a server device operated by a different service entity (company, etc.), and is realized by software, for example. Each of the voice recognition engines 6, 7, and 8 can be used through the wide area network 5 by multiple users who have signed a service contract in advance. Although three voice recognition engines 6, 7, and 8 are shown in FIG. 1, in reality, more voice recognition engines may exist.

[0012] When it is necessary to generate text data by performing voice recognition, for example, the voice information processing device 1 may be provided with a voice recognition engine, or as shown in FIG. 1, the voice information processing device 1 may be provided with a voice recognition engine 3 for exclusive use. However, preparing a voice recognition engine on one's own requires time and expense. In recent years, there has been an increasing need to use voice recognition services, such as converting conference voices into text or converting conversation voices between call center personnel and customers into text. For this reason, multiple voice recognition engines 6, 7, 8, ... that provide subscribers with a service of performing voice recognition with high accuracy and high speed have been provided on the Internet.

[0013] Therefore, the voice information processing device 1 performs voice recognition using voice recognition engines 6, 7, 8, ... provided on the wide area network 5 in addition to the voice recognition engine 3 connected to the LAN 2. In other words, the voice information processing device 1 is connected to the wide area network 5 via the LAN 2 and the network connection device 4, and can access the voice recognition engines 6, 7, 8, ... to use the voice recognition services they provide.

[0014] However, there is a possibility that the transmission paths connecting the voice information processing device 1 and the voice recognition engines 6, 7, 8, ... or the voice recognition engines 6, 7, 8, ... themselves may be attacked, resulting in information leakage. Therefore, the voice information processing device 1 is designed to make it difficult for meaningful information to leak when converting voice information into text data by performing voice recognition using an external voice recognition engine (on the wide area network 5). The configuration and operation of the voice information processing device 1 of this embodiment will be described in detail below.

[0015] [Configuration example of voice information processing device 1] Fig. 2 is a block diagram for explaining a configuration example of the voice information processing device 1 according to the embodiment. Fig. 3 is a diagram for explaining an example of a format of transmission data transmitted by the voice information processing device 1 according to the embodiment and a format of reception data received by the voice information processing device 1 according to the embodiment.

[0016] 2, the connection terminal 101T constitutes a connection end to the LAN 2. The LAN I / F 101 converts data addressed to the device itself transmitted through the LAN 2 into a format that can be processed by the device itself and takes it in, and also converts data to be transmitted from the device itself into a transmission format and transmits it. The control unit 102 is a microprocessor formed by connecting a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a non-volatile memory, etc. through a bus (not shown), and controls each unit of the voice information processing device 1.

[0017] The storage device 103 is a device unit consisting of a recording medium such as an SSD (Solid State Drive) and its driver, and records, reads, modifies, deletes, etc. various data on the recording medium. The storage device 103 stores and holds necessary data and programs, and is also used as a working area for temporarily storing intermediate data generated in various processes.

[0018] The voice information file 111 stores and holds voice data (digital data) to be recognized. The voice data is formed by A / D (Analog / Digital) converting analog voice information to be recognized, which is recorded on a magnetic recording medium (not shown), for example. When the voice recognition target is an analog voice signal, the voice information processing device 1 of this embodiment can perform A / D conversion to process the analog voice signal.

[0019] That is, the voice information processing device 1 can process not only voice data (digital data) but also voice information in general. Therefore, in this specification, voice signals (analog data) and voice data (digital data) are included in the voice to be processed, and voice information in general is referred to as voice information. In the following description, the voice information processing device 1 processes voice data (digital data) formed by A / D conversion.

[0020] The audio information file 111 can record audio data (digital data) in various formats such as WAV, AIFF, MP3, WMA, and AAC. WAV is an abbreviation for "RIFF waveform Audio Format," and AIFF is an abbreviation for "Audio Interchange File Format." MP3 is an abbreviation for "MPEG-1 Audio Layer-3," WMA is an abbreviation for "Windows Media Audio," and AAC is an abbreviation for "Advanced Audio Coding."

[0021] Also, audio data expressed in uncompressed formats such as WAV and AIFF can be processed as is, but audio data expressed in compressed formats such as MP3, WMA, AAC, etc. is processed after decompression. In this way, the audio information file 111 stores and holds audio data (waveform data) expressed in various formats, which are obtained by A / D converting analog audio signals to be recognized.

[0022] The voice information division unit 112 divides the voice data stored in the voice information file 111 using, for example, a voice activity detection (VAD) technique to form divided voice data, which is recorded in the divided voice information file 113. Thus, the divided voice information file 113 stores and holds the divided voice data to be recognized that is divided and formed by the voice information division unit 112. The VAD technique is a technique for distinguishing between sections where voice (speech) is present and other sections from a signal containing voice and noise, and, simply put, is a technique for dividing voice at sections where voice is interrupted (silent periods) such as at the end of a sentence or when there is a hesitation in speech.

[0023] The order information adding unit 114 adds order information indicating the playback order to each of the divided audio data stored and held in the divided audio information file 113, and records the added order information in the order information added file 115 in association with the divided audio data. Therefore, the order information added file 115 stores and holds the divided audio data associated with the added order information. The order information is additional information for restoring the divided audio data to the original order (turn) even if the divided audio data to be recognized is rearranged randomly, and indicates the position of each divided audio data in the series of audio information before being divided into the audio information to be recognized.

[0024] The order information may be, for example, a number (consecutive number) that counts up from "1" as an initial value, or information with a fixed order such as the alphabet or the Japanese syllabary that can indicate an order like the number. Also, various information that can indicate the position (part) of the divided voice data in the original series of voice information to be recognized, such as a combination of a number and information with a fixed order, may be used.

[0025] The audio information rearrangement unit 116 randomly rearranges the divided audio data associated with the order information stored and held in the order information-added file 115, and records the rearranged data in the rearranged file. Therefore, the rearranged file 117 stores and holds the divided audio data (associated with the order information) whose order has been randomly rearranged.

[0026] The transmission control unit 118 forms transmission data including the randomly rearranged divided voice information stored and held in the rearrangement file 117, and controls to distribute and transmit this to multiple voice recognition engines. For example, as shown in Fig. 3(A), the transmission control unit 118 sequentially forms transmission data consisting of a destination ID, a sender ID, order information, and divided voice data, and distributes and transmits the data to multiple voice recognition engines.

[0027] In the transmission data shown in FIG. 3(A), the divided voice data is a part of the voice data to be recognized divided by the voice information dividing unit 112, and the order information is, as described above, added to the divided voice data by the order information adding unit 114. The destination ID is an IP address assigned to a server device equipped with a voice recognition engine to be the destination, and the sender ID is an IP address assigned to the voice information processing device 1. By changing the destination ID, the divided voice data can be distributed and sent to a plurality of voice recognition engines. Necessary information such as the destination ID and sender ID is, for example, information that is registered in advance in the non-volatile memory of the control unit 102.

[0028] The transmission data (FIG. 3(A)) is sent to the LAN 2 via the LAN I / F 101 and the connection terminal 101T, and is then sent from the LAN 2 to the wide area network 5 via the network connection device 4, and is then transmitted to the target speech recognition engine. In this way, a block of speech data to be recognized is divided into a plurality of divided speech data using, for example, the VAD technique, and sequence information is added, the data are randomly rearranged, and the data is distributed and transmitted to a plurality of speech recognition engines together with the sequence information.

[0029] In this way, when the divided voice data is distributed and transmitted to the multiple voice recognition engines in the form of transmission data shown in Fig. 3(A), text data converted by voice recognition in the multiple voice recognition engines is returned to the voice information processing device 1. The text data includes, for example, a destination ID, a sender ID, order information, and text data as shown in Fig. 3(B), and becomes received data received by the voice information processing device 1.

[0030] In the received data shown in FIG. 3(B), the destination ID is an IP address assigned to the voice information processing device 1 which is the sender of the divided voice data. The sender ID is an IP address assigned to a server device equipped with a voice recognition engine which is the sender of the received data (text data). The order information corresponds to the order information of the transmitted data shown in FIG. 3(A), and is additional information for rearranging (restoring) the text data obtained by performing voice recognition on the corresponding divided voice data to the original order (order). The text data is obtained by performing voice recognition on the divided voice data transmitted from the voice information processing device 1. In other words, the text data corresponds to the divided voice data transmitted from the voice information processing device 1.

[0031] The text data (received data shown in FIG. 3(B)) sent from each voice recognition engine to the voice information processing device 1 is provided to the voice information processing device 1 via the wide area network 5, the network connection device 4, and the LAN 2. The voice information processing device 1 receives the received data (FIG. 3(B)) from the multiple voice recognition engines via the connection terminal 101T and the LAN I / F 101, and records it in the received recognition result file 121 under the control of the control unit 102. Therefore, the received recognition result file 121 stores and holds the received data from the multiple voice recognition engines. This received data includes text data corresponding to the divided voice data obtained as a result of voice recognition, as shown in FIG. 3(B).

[0032] The recognition result rearrangement unit 122 rearranges the text data of the received data (FIG. 3(B)) stored in the received recognition result file 121 according to the order information added thereto, thereby restoring a series of text data corresponding to the original voice information of the voice recognition target. The restored series of text data is recorded in the recognition result restoration file 123. That is, the recognition result restoration file 123 stores and holds a series of text data obtained by performing voice recognition on a series of voice information of the voice recognition target.

[0033] In this manner, the voice information processing device 1 of this embodiment can convert the voice information of the voice recognition target into text data by performing voice recognition using multiple voice recognition engines, and obtain a series of text data corresponding to the voice information of the voice recognition target. By using multiple voice recognition engines through processes such as dividing the voice information, adding order information, randomizing the order, and allocating (distributed allocation), meaningful information is prevented from leaking even if the transmission path or the voice recognition engine is subjected to a malicious attack.

[0034] In the voice information processing device 1, the voice information file 111, the divided voice information file 113, the file after adding order information 115, the file after rearrangement 117, the received recognition result file 121, and the recognition result restoration file 123 are formed on a predetermined recording medium. That is, these files are formed on a device section consisting of a recording medium such as an SSD (Solid State Drive) and its driver. Each of these files may be formed on a different recording medium, or two or more files may be formed on the same recording medium.

[0035] [Details of processing performed by voice information processing device 1] Next, the processing performed in the voice information processing device 1 described with reference to Figures 2 and 3 will be specifically described, being divided into processing performed before the voice recognition processing and processing performed after the voice recognition processing. For simplicity of explanation, the following description will be given by taking as an example a case where the multiple voice recognition engines used are voice recognition engines 6, 7, and 8 provided on a wide area network 5.

[0036] <Processing performed before speech recognition processing> 4 is a diagram for explaining a process executed before the voice recognition process in the voice information processing device of the embodiment. As shown at the top of FIG. 4, it is assumed that the voice information file 111 contains voice data obtained by A / D conversion of a series of voice information starting with "This is Suzuki from Sakusa" and ending with "Please inform me of the date and time." In this case, the voice information division unit 112 functions to execute a voice information division process (step S1), form divided voice data, and record this in the divided voice information file 113. As a result, in this example, the series of voice data stored and held in the voice information file 111 is divided into five divided voice data, which are stored and held in the divided voice information file 113.

[0037] Next, the order information adding unit 114 starts to function. In this example, the order information adding unit 114 adds order information to the five divided audio data stored and held in the divided audio information file 113, and records the added order information in the file in association with each divided audio data (step S2). As described above, the order information is additional information for restoring the divided audio data to the original order (sequence) even if the divided audio data to be recognized is rearranged randomly, and indicates the position of each divided audio data in the series of audio information before division of the audio recognition target. In this example, the order information is a number (consecutive number) that counts up by one from "1".

[0038] 4, each piece of divided voice data is stored in file 115 after order information is added, with a serial number from 1 to 5 corresponding to its position in the series of voice information before division of the voice recognition target. After that, voice information rearrangement unit 116 functions to randomly rearrange the divided voice data in file 115 after order information is added, and then transmission control unit 118 functions to distribute and transmit the rearranged divided voice data to voice recognition engines 6, 7, and 8 (step S3).

[0039] 4 are assigned to the divided audio data of 1 to 5 and stored in the post-order-information-assigned file 115, the audio information rearrangement unit 116 randomly rearranges the divided audio data, and records the rearranged data in the post-order-information-assigned file 117. Here, it is assumed that the divided audio data assigned with the order information is rearranged in the order of "4" -> "3" -> "1" -> "5" -> "2" and recorded in the post-order-information file 117.

[0040] After that, the transmission control unit 118 functions, and as shown in FIG. 4, the divided voice data with sequence information "4" is transmitted to the voice recognition engine 6. The divided voice data with sequence information "3" and "1" is transmitted to the voice recognition engine 7. The divided voice data with sequence information "5" and "2" is transmitted to the voice recognition engine 8. In this way, a series of voice information to be recognized is divided and transmitted separately to a plurality of voice recognition engines. As a result, even if the transmission path or each voice recognition engine is attacked and the divided voice data is read, only a part of the series of voice information to be recognized can be recognized, so that leakage of all of the voice information, including important information, can be prevented.

[0041] <Processing performed after speech recognition processing> Fig. 5 is a diagram for explaining the process executed after the voice recognition process in the voice information processing device 1 of the embodiment. That is, the process shown in Fig. 5 is executed after the process executed before the voice recognition process explained with reference to Fig. 4. When the divided voice data with the order information added thereto is transmitted to each of the voice recognition engines 6, 7, and 8 by the process explained with reference to Fig. 4, each of the voice recognition engines 6, 7, and 8 performs voice recognition to form text data, which is then returned. The text data has the order information added thereto, which corresponds to the divided voice data.

[0042] 5, the voice recognition engine 6 to which the divided voice data with the order information "4" has been transmitted returns text data obtained by voice recognition of the divided voice data with the order information "4" added thereto. The voice recognition engine 7 to which the divided voice data with the order information "3" and "1" has been transmitted returns text data obtained by voice recognition of the divided voice data with the order information "3" added thereto and text data with the order information "1" added thereto. The voice recognition engine 8 to which the divided voice data with the order information "5" and "2" has been transmitted returns text data obtained by voice recognition of the divided voice data with the order information "5" added thereto and text data with the order information "2" added thereto.

[0043] 5, received data (text data obtained as a result of voice recognition) from each of the voice recognition engines 6, 7, and 8 is stored in the received recognition result file 121. After receiving text data (received data) corresponding to the transmitted divided voice data (transmitted data) from each of the voice recognition engines 6, 7, and 8 and storing the received data in the received recognition result file 121, the recognition result rearrangement unit 122 starts to function. The recognition result rearrangement unit 122 rearranges the text data of the received data stored and held in the received recognition result file 121 based on the order information of the received data, forms a series of text data, and records the series of text data in the recognition result restoration file 123 (step S11).

[0044] As a result, as shown in Fig. 5, the recognition result restoration file 123 stores a series of text data corresponding to a series of voice data obtained by A / D converting the voice information of the voice recognition target prepared in the voice information file 111 shown in Fig. 4. That is, the voice information file 111 stores and holds the voice data of the voice recognition target. On the other hand, the recognition result restoration file 123 stores and holds text data obtained by performing voice recognition on the voice data stored and held in the voice information file 111.

[0045] In this way, text data obtained as a result of voice recognition is returned from each of the multiple voice recognition engines. Therefore, even if an attack is made on the return path of the text data and the text data is read, only a part of the whole can be recognized, so leakage of the entire text data including important information can be prevented.

[0046] [Speech recognition of speech information from multiple speakers] The speech information to be recognized is not limited to that of a single speaker. There are cases where speech information containing mixed speech information from two or more speakers, such as a conversation between two speakers or a recording of a conference attended by multiple people, is the target of speech recognition. This invention is also applicable to speech recognition of speech information containing mixed speech information from two or more speakers. Note that speech information containing mixed speech information from two or more speakers may be recorded (recorded) on a different track (storage area) for each speaker, or the speech signals of multiple speakers may be recorded (recorded) on the same track. Each of these cases will be described below.

[0047] <When audio information from multiple speakers is recorded on different tracks> <Processing before speech recognition> Fig. 6 is a diagram for explaining a process executed before a voice recognition process when voice information of multiple speakers recorded on different tracks is to be processed. Fig. 7 is a diagram for explaining an example of a format of transmission data and reception data when processing voice information of multiple speakers.

[0048] For example, it is assumed that audio information such as recorded voice of a telephone call or recorded voice of a conference is stored and held in the audio information file 111. In this case, it is assumed that the audio data (audio information that has been A / D converted) of multiple speakers is recorded on different tracks in the audio information file 111. For the sake of simplicity, the following description will be given taking as an example a case where the recorded voice of a conference between three speakers, A, B, and C, is subjected to voice recognition.

[0049] In this case, in order to perform voice recognition on the voice data for each speaker, the voice information processing device 1 is provided with a track separation unit 131. The track separation unit 131 separates the voice data of each speaker A, B, C, recorded on separate tracks in the voice information file 111, into speaker-specific voice information files 111(1), 111(2), and 111(3) (step S0). After this, the processing before the voice recognition described with reference to FIG. 4 is performed on each of the speaker-specific voice information files 111(1), 111(2), and 111(3).

[0050] That is, the voice information division unit 112 functions to perform a voice information division process (step S1A) on the voice data stored in the voice information files 111(1), 111(2), and 111(3) provided for the speakers A, B, and C. As a result, divided voice information files 113(1), 113(2), and 113(3) are formed, which store and hold divided voice data obtained by dividing the voice data for each speaker.

[0051] Next, the order information adding unit 114 functions to add order information to the divided voice data stored in the divided voice information files 113(1), 113(2), and 113(3) provided for the speakers A, B, and C (step S2A). This results in the formation of order information added files 115(1), 115(2), and 115(3) that store and hold information in which order information is associated with the divided voice data for each speaker.

[0052] After that, the divided voice data associated with the order information stored in each of the order information-added files 115(1), 115(2), and 115(3) is randomly rearranged and distributed to a plurality of voice recognition engines (step S3A). The process of step S3A is performed by the voice information rearrangement unit 116 and the transmission control unit 118.

[0053] First, the voice information rearrangement unit 116 operates to randomly rearrange the divided voice data associated with the order information stored and held in the order information-added files 115(1), 115(2), and 115(3) provided for each of the speakers A, B, and C. As a result, the rearranged divided voice data (associated with the order information) is stored and held in the rearranged files 117(1), 117(2), and 117(3) provided for each of the speakers A, B, and C.

[0054] Next, the transmission control unit 118 functions to form transmission data including the divided voice data associated with the order information of the rearranged files 117(1), 117(2), and 117(3), and distributes and transmits the data to the voice recognition engines 6, 7, and 8. As a result, the voice data for each of the speakers A, B, and C is divided, assigned order information, and randomly rearranged, and then distributed and transmitted to the voice recognition engines 6, 7, and 8. Therefore, even when voice information of multiple speakers is recognized, it is possible to prevent meaningful information from being leaked even if the divided voice data is leaked during the process of transmission to the voice recognition engine or in the voice recognition engine due to an attack.

[0055] The transmission data formed by the transmission control unit 118 has a format shown in, for example, FIG. 7(A). Basically, it is the same as the transmission data shown in FIG. 3(A), but a speaker ID for distinguishing the speaker and the start time of each divided voice data are newly added. In this embodiment, Transmitted DataThe start time is a relative time (time elapsed since the start point) with the start point of recording as the reference (start point). It is also possible to express the start time as an absolute time from the start time of recording. However, in this case, it is necessary to obtain the start time (absolute time) from the clock circuit when starting recording.

[0056] <Post-voice recognition processing> Fig. 8 is a diagram for explaining the process executed after the voice recognition process in the case where the voice information of a plurality of speakers is recorded on different tracks as the processing target. The process explained using Fig. 8 is performed after the process executed before the voice recognition explained using Fig. 6 and after the voice recognition is performed by each of the voice recognition engines 6, 7, and 8, and is basically performed in the same manner as the process explained using Fig. 5.

[0057] The text data obtained as a result of the speech recognition from the speech recognition engines 6, 7, and 8 has the format shown in Fig. 7(B). It is basically the same as the received data shown in Fig. 3(B), but includes a speaker ID and a start time corresponding to the transmitted data in Fig. 7(A). That is, the received data (Fig. 7(B)) from the speech recognition engines 6, 7, and 8 has a speaker ID added corresponding to the transmitted data (Fig. 7(A)). Therefore, in the speech information processing device 1, the received data can be sorted and stored in the received recognition result files 121(1), 121(2), and 121(3) provided for each speaker.

[0058] Next, the recognition result rearrangement unit 122 operates to rearrange the text data of the received data in the received recognition result files 121(1), 121(2), and 121(3) for speakers A, B, and C based on the associated order information (step S11A). The text data rearranged in this manner is recorded in the recognition result restoration files 123(1), 123(2), and 123(3) provided for speakers A, B, and C. This provides a series of text data corresponding to a series of speech information to be recognized for each speaker A, B, and C. In the case of the process described with reference to FIG. 5, the process ends here.

[0059] However, when processing speech information of multiple speakers, such as recorded speech of a meeting, the overall content of the meeting (the overall content of the recorded speech) cannot be understood unless each speaker's remarks can be correlated and confirmed. Therefore, speaker IDs and start times are added to the text data stored in recognition result restoration files 123(1), 123(2), and 123(3) provided for speakers A, B, and C. In addition, the speech information processing device 1 is provided with a text data combining unit 132.

[0060] The text data combining unit 132 arranges the text data stored in the recognition result restoration files 123(1), 123(2), and 123(3) provided for speakers A, B, and C based on the start time added to each text data. At this time, it is possible to know which speaker's voice information the text data corresponds to. In other words, it is possible to distinguish the speaker for each text data. For example, it is possible to color-code the text data according to the speaker, or to display the name of the speaker specified by the speaker ID (first name, pet name, nickname), etc., before or after the text data.

[0061] In this way, the text data according to the utterances of each speaker is rearranged according to the timing of the utterances and recorded in the recognition result restoration combined file 124. If the text data recorded in this recognition result restoration combined file 124 is text data obtained by performing voice recognition on the recorded voice of a conference, it becomes the minutes of the conference. Also, in the case of a recorded voice of a telephone call, a call record of two parties (two speakers) can be formed. In this way, even in the process of returning text data resulting from voice recognition of the voice information of multiple speakers, it is possible to prevent meaningful information from being leaked even if the divided voice data is leaked due to an attack.

[0062] <When multiple speakers' voice information is recorded on the same track> FIG. 9 is a diagram for explaining a process executed before a voice recognition process when voice information of multiple speakers recorded on the same track is to be processed. For example, when recording a telephone call, the transmitted voice from the device and the received voice from the other party are often recorded on the same track, rather than being recorded on separate tracks. Also, when recording the voice of a conference, it is often the case that a microphone is not provided for each participant in the conference, and the voice during the conference is picked up and recorded by, for example, one microphone. In such a case, it is sufficient to separate and extract the voices of the multiple speakers recorded on one track.

[0063] For example, it is assumed that audio information such as recorded voice of a telephone call or recorded voice of a conference is stored and held in the audio information file 111. In this case, it is assumed that the audio data (audio information that has been A / D converted) of multiple speakers is recorded on one track in the audio information file 111. Again, for the sake of simplicity, an example will be described in which voice recognition is performed on the recorded voice of a conference between three speakers, A, B, and C.

[0064] In this case, in order to perform voice recognition of the voice data for each speaker, the voice information processing device 1 is provided with a speaker voice separation unit 133. The speaker voice separation unit 133 separates and extracts the voice data of each speaker A, B, and C recorded on the same track in the voice information file 111, for example, by using a frequency analysis technique or other techniques. The separated and extracted voice data for each speaker A, B, and C is recorded separately in speaker-specific voice information files 111(1), 111(2), and 111(3) (step S0A). The subsequent processes are the same as those in steps S1A, S2A, and S3A described using FIG. 6.

[0065] 6, the voice data for each of the speakers A, B, and C is divided, given order information, randomly rearranged, and then distributed and transmitted to the voice recognition engines 6, 7, and 8. Therefore, even when voice information of multiple speakers is recognized, it is possible to prevent meaningful information from being leaked even if the divided voice data is leaked during the process of transmission to the voice recognition engine or in the voice recognition engine due to an attack.

[0066] Furthermore, the divided voice data for each speaker is sent to the voice recognition engines 6, 7, and 8 using Fig. 9 to perform voice recognition, and the processing after the voice recognition is performed by receiving the text data as reply data is performed in the same manner as the processing described using Fig. 8. Therefore, even if the voice information of multiple speakers is recorded on the same track, after the voice data for each speaker is separated and extracted, it is possible to prevent meaningful information from being leaked even if an attack occurs during the process of replying the text data.

[0067] [Effects of the embodiment] According to the voice information processing device of the above-mentioned embodiment, when voice information is converted into text data by performing voice recognition using an external voice recognition engine, it is possible to make it difficult for meaningful information to be illegally obtained. In this case, even when performing voice recognition on voice information dictated by one speaker, or performing voice recognition on multi-speaker voice information including voice information of two or more speakers, it is possible to prevent the contents of the entire voice information from being leaked. This makes it possible to perform voice recognition safely by utilizing an external voice recognition engine.

[0068] In addition, by using multiple voice recognition engines, the load on the voice recognition engines can be distributed, and highly accurate voice recognition can be performed quickly. In addition, by using multiple voice recognition engines, it is possible to reduce the influence of the quirks of each voice recognition engine.

[0069] [Variations] In the voice information processing device 1 of the above-mentioned embodiment, the voice information rearrangement unit 116 randomly rearranges the divided voice data to which order information has been added, and then the transmission control unit 118 distributes and transmits the divided voice data to a plurality of voice recognition engines. However, this is not limited to this. It is not necessarily required to rearrange the data randomly, and even if the divided voice data to which order information has been added is distributed and transmitted to a plurality of voice recognition engines, it is possible to make it difficult for meaningful information to leak. In this case, the transmission control unit 118 controls so that divided voice data with adjacent (consecutive) order information is not transmitted to the same voice recognition engine.

[0070] Of course, leakage of meaningful information can be prevented more reliably by having the transmission control unit 118 distribute and transmit the divided voice data with the order information added to a plurality of voice recognition engines after the voice information rearrangement unit 116 randomly rearranges the divided voice data with the order information added thereto. Even in this case, the transmission control unit 118 controls so that divided voice data with adjacent (consecutive) order information is not transmitted to the same voice recognition engine.

[0071] In addition, while the number of voice recognition engines used is basically better, using two or more voice recognition engines is effective in preventing leakage of meaningful information. Of course, one of the multiple voice recognition engines used may be voice recognition engine 3 connected to LAN 2. In this way, by mixing voice recognition engines that are difficult to attack in the transmission path and in the voice recognition engines themselves, leakage of meaningful information can be effectively prevented.

[0072] Also, it is conceivable that there may be a situation where only one external voice recognition engine is available. Even in such a case, it is possible to prevent leakage of meaningful information by adopting a configuration in which the voice information rearrangement unit 116 randomly rearranges the divided voice data to which the order information has been added, and then transmits the rearranged data to the one voice recognition engine. However, it is preferable to use a plurality of voice recognition engines configured in a plurality of server devices.

[0073] Furthermore, when the voice information processing device 1 can use transmission paths applied by a plurality of different carriers (telephone companies), information may be sent and received between the voice recognition engine and the voice information processing device 1 using the transmission paths of the different carriers. That is, the voice information processing device 1 is provided with a means for selecting the transmission paths of the different carriers. This helps to prevent leakage of meaningful information, and when a failure occurs in the transmission path provided by one carrier, it becomes possible to use the transmission path of another carrier, thereby preventing delays in the voice recognition process.

[0074] In the above embodiment, the voice data to be recognized is described as being stored in a voice information file. In other words, the voice recognition process is performed by so-called batch processing. This is because if there is not a certain amount of voice data to be recognized, it is not possible to effectively divide the voice data to form divided voice data and distribute them to multiple voice recognition engines. However, this is not limited to the above.

[0075] For example, multiple voice information files may be prepared, and conference voice or telephone call voice may be stored in different voice information files for 5 minutes each. When 5 minutes of voice data is stored, voice recognition may be performed on the 5 minutes of voice data. In other words, the present invention may be applied to a case where voice recognition is performed on an ongoing basis for each 5 minutes of voice data. In this way, leakage of meaningful information can be prevented, and voice data can be quickly converted into text. Note that, since a short time for storing voice data may lead to leakage of meaningful information, it is preferable to store the voice data for a certain length of time.

[0076] In addition, in the above embodiment, the voice data is divided using the VAD technique, but the present invention is not limited to this. Various techniques (methods) capable of dividing voice data can be used. By using a technique capable of dividing voice data at hesitations or sentence breaks, such as the VAD technique, voice recognition can be performed more reliably. However, the voice data to be recognized may be divided, for example, in units of time or data amount. In this case, the voice recognition engine is selected to perform voice recognition more faithfully to the spoken voice. In the voice information processing device 1, for example, when the recognition result rearrangement unit 122 rearranges the text data, the concatenated portion of the text data may be changed to a more suitable word than the words before and after it.

[0077] Furthermore, the transmission data and the reception data may of course be encrypted before being transmitted and received. In particular, since the sequence information is important information for restoring meaningful information, it is desirable to transmit and receive at least the sequence information after performing a concealment process to make it difficult to read, such as encrypting the sequence information using various encryption methods. Furthermore, the recording position of the sequence information in the transmission data and the reception data may be changed based on a predetermined rule. That is, various methods can be used for concealing the sequence information. Such processing can be performed by, for example, the transmission control unit 118 or the control unit 102. Of course, the voice information processing device 1 may be provided with a concealment processing unit and a concealment release unit.

[0078] Also, each function of the voice information division unit 112, the order information addition unit 114, the voice information rearrangement unit 116, the transmission control unit 118, and the recognition result rearrangement unit 122 can be realized as a function of the control unit 102 by a program executed by the control unit 102. Also, the voice information file 111, the divided voice information file 113, the order information addition file 115, the rearrangement file 117, the received recognition result file 121, and the recognition result restoration file 123 may be formed in the storage device 103. In particular, the divided voice information file 113, the order information addition file 115, and the rearrangement file 117 hold intermediate data, and therefore may be formed in the work area of ​​the storage device 103. [Explanation of symbols]

[0079] 1...speech information processing device, 101T...connection terminal, 101...LAN I / F, 102...control unit, 103...storage device, 111...speech information file, 112...speech information division unit, 113...divided speech information file, 114...order information assignment unit, 115...file after order information assignment, 116...speech information rearrangement unit, 117...rearranged file, 118...transmission control unit, 121...received recognition result file, 122...recognition result rearrangement unit, 123...recognition result restoration file, 2...LAN, 3...speech recognition engine, 4...network connection device, 5...wide area network, 6...speech recognition engine, 7...speech recognition engine, 8...speech recognition engine

Claims

1. A voice information separation unit that separates recorded voice information of a call or conference containing voice information of multiple speakers into voice information for each of the speakers; a voice information division unit that divides each of the voice information of the voice recognition target that has been separated for each speaker by the voice information separation unit; an order information adding unit that adds order information indicating a reproduction order to each of the plurality of pieces of divided voice information for each speaker, which are divided by the voice information dividing unit; a transmission control unit that assigns speaker identification information for distinguishing the speaker and a start time based on a start time of recording to each of the plurality of divided voice information for each speaker to which the order information is assigned, and distributes and transmits the divided voice information to a plurality of voice recognition units through a transmission unit; a recognition result rearrangement unit that rearranges the recognition results for each speaker, in which the order information, the speaker identification information, and the start time are associated with each other, received from each of the plurality of speech recognition units via a receiving unit, based on the order information to restore a recognition result corresponding to the speech information of a speech recognition target for each speaker; a recognition result combining unit that rearranges the recognition results for each speaker, which have been rearranged by the recognition result rearrangement unit, based on the start time and makes each speaker identifiable based on the speaker identification information; 13. A voice information processing device comprising:

2. 2. The voice information processing device according to claim 1, a voice information rearrangement unit that randomly rearranges the divided voice information to which the order information has been added, The transmission control unit distributes the divided voice information rearranged by the voice information rearrangement unit to the plurality of voice recognition units.

13. A voice information processing apparatus comprising:

3. 3. The voice information processing device according to claim 1, The transmission control unit prevents the divided voice information having adjacent order information from being transmitted to the same voice recognition unit.

13. A voice information processing apparatus comprising:

4. 4. A voice information processing device according to claim 1, 2 or 3, The voice information division unit detects silent periods in which no speech exists, and divides the voice information to be recognized into a plurality of pieces of divided voice information.

13. A voice information processing apparatus comprising:

5. 5. A voice information processing apparatus according to claim 1, 2, 3 or 4, At least the sequence information is transmitted / received in a confidential manner.

13. A voice information processing apparatus comprising:

6. A voice information processing device according to any one of claims 1, 2, 3, 4, or 5, The recorded voice information of a call or conference including voice information of the multiple speakers is recorded on a different track for each speaker, and the voice information separation unit separates the recorded voice information for each speaker on a different track.

13. A voice information processing apparatus comprising:

7. A voice information processing device according to any one of claims 1, 2, 3, 4, or 5, The recorded voice information of a call or conference including voice information of the multiple speakers is recorded on the same track, and the voice information separation unit separates the voice information for each speaker using a predetermined method.

13. A voice information processing apparatus comprising:

Images