Method for converting audio data to text, information processing device, and program

Abstract

Improve the technology for converting audio data to text. [Solution] A method for converting audio data to text performed by an information processing device, comprising: acquiring audio data; generating text data based on the audio data; and correcting error patterns contained in the text data based on an error correction table, wherein the error correction table includes a first error pattern; at least one second error pattern generated based on the audio data from which the first error pattern was generated; and correction patterns corresponding to the first error pattern and the at least one second error pattern.

Description

Technical Field

[0001] This disclosure relates to a method for text conversion of voice data, an information processing apparatus, and a program.

Background Art

[0002] Conventionally, techniques for analyzing the content of business negotiations have been known. For example, Patent Document 1 discloses a dialogue analysis system that checks whether a salesperson explains what should be explained and does not mention what should not be mentioned in a business negotiation with a customer.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Although Patent Document 1 shows a technique for analyzing the content of business negotiations by machine learning, there is no mention of a technique for transcribing voice in business negotiations or the like, that is, a text conversion technique for voice data. On the other hand, for analyzing and providing feedback on the content of business negotiations or the like, it is preferable to improve the text conversion technique for voice data. Thus, there has been room for improvement in the text conversion technique for voice data in business negotiations or the like.

[0005] In view of such circumstances, an object of the present disclosure is to improve the text conversion technique for voice data.

Means for Solving the Problems

[0006] A method for text conversion of voice data according to an embodiment of the present disclosure is a method for text conversion of voice data executed by an information processing apparatus, comprising acquiring voice data, Based on the aforementioned audio data, text data is generated, Based on the error correction table, the error patterns contained in the text data are corrected, The error correction table includes a first error pattern, at least one second error pattern generated based on the audio data from which the first error pattern was generated, and correction patterns corresponding to the first error pattern and the at least one second error pattern. [Effects of the Invention]

[0007] According to one embodiment of this disclosure, the technique for converting speech data to text is improved. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the schematic configuration of the system according to this embodiment. [Figure 2] This is a flowchart showing the operation of an information processing device. [Figure 3] This is a flowchart showing the operation of an information processing device. [Modes for carrying out the invention]

[0009] The embodiments of this disclosure will be described below.

[0010] (Summary of the Embodiment) Referring to Figure 1, the overview and configuration of System 1 according to this embodiment will be described. System 1 according to this embodiment comprises an information processing device 10 and a terminal device 20. The information processing device 10 and the terminal device 20 are communicated with a network 30, which includes, for example, a mobile communication network and the Internet.

[0011] The information processing device 10 is, for example, a server device installed in a data center or the like. For example, the information processing device 10 is a server belonging to a cloud computing system or other computing system. Although Figure 1 shows an example where system 1 has one information processing device 10, it is not limited to this. System 1 may have two or more information processing devices 10.

[0012] The terminal device 20 is any device used by the user. For example, general-purpose electronic devices such as personal computers, smartphones, tablet devices, and wearable devices, or dedicated electronic devices, can be used as the terminal device 20. Although Figure 1 shows an example where System 1 has one terminal device 20, it is not limited to this. System 1 may have two or more terminal devices 20.

[0013] First, an overview of the audio data text conversion technology according to this embodiment will be described, and details will be described later. Here, the audio data may be data relating to a specific domain. For example, the audio data may be audio data from a business negotiation. In this embodiment, the business negotiation is, for example, a business negotiation relating to the sale of a vehicle, and the offering related to the business negotiation is a vehicle, but is not limited to this. For example, the business negotiation may be a meeting aimed at concluding various types of contracts, such as the buying and selling of real estate, the contract of an insurance product, or the sale of a financial product. Furthermore, the offering related to the business negotiation in this embodiment may be goods, services, digital content, licenses, data / information, financial products, real estate, intangible assets, other tradable rights, etc.

[0014] The information processing device 10 acquires audio data. The information processing device 10 also generates text data based on the audio data. Furthermore, the information processing device 10 corrects patterns of misspellings, misrecognized phrases, etc. (hereinafter also referred to as error patterns) contained in the text data based on an error correction table.

[0015] Here, the error correction table includes a certain error pattern (hereinafter also referred to as the first error pattern), at least one other error pattern (hereinafter also referred to as the second error pattern) generated based on the audio data in which the first error pattern was generated, and corrected phrases and the like (hereinafter also referred to as correction patterns) corresponding to the first error pattern and at least one second error pattern.

[0016] As described above, according to this embodiment, the information processing apparatus 10 corrects the error pattern included in the text data based on the error correction table. In particular, the error correction table includes the first error pattern, at least one second error pattern generated based on the audio data in which the first error pattern was generated, and correction patterns corresponding to the first error pattern and at least one second error pattern. Since the error correction table includes a plurality of error patterns in this way, the possibility of correcting incorrect text data is increased, and since the plurality of error patterns are efficiently generated based on the audio data, the text conversion technology of the audio data is improved.

[0017] Next, each configuration of the information processing apparatus 10 and the terminal apparatus 20 will be described in detail.

[0018] (Configuration of Information Processing Apparatus 10) As shown in FIG. 1, the information processing apparatus 10 includes a control unit 11, a storage unit 12, an input unit 13, an output unit 14, and a communication unit 15.

[0019] The control unit 11 includes at least one processor, at least one dedicated circuit, or a combination thereof. The processor is a general-purpose processor such as a CPU (central processing unit) or GPU (graphics processing unit), or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). The control unit 11 controls each part of the information processing device 10 and executes processes related to the operation of the information processing device 10.

[0020] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The semiconductor memory is, for example, a RAM (random access memory) or a ROM (read only memory). The RAM is, for example, a SRAM (static random access memory) or a DRAM (dynamic random access memory). The ROM is, for example, an EEPROM (electrically erasable programmable read only memory). The storage unit 12 functions as, for example, a main memory device, an auxiliary storage device, or a cache memory. The storage unit 12 stores data used in the operation of the information processing device 10 and data obtained by the operation of the information processing device 10. For example, specifically, the storage unit 12 stores a voice recognition engine. The voice recognition engine has a function of converting voice input into text data and plays a role of analyzing the utterance from the user and generating corresponding text information. Also, for example, an error correction table is stored in the storage unit. The error correction table is a table for converting the error pattern of the text recognized by the voice recognition engine into a correction pattern. For example, the vehicle model name RAV4 (registered trademark), etc., can be recognized by the voice recognition engine as an incorrect pattern such as LOVE FOR, etc. The error correction table includes, for example, information associating the error pattern LOVE FOR with the correction pattern RAV4 (registered trademark). By referring to the error correction table, proper nouns such as vehicle model names and function names are corrected appropriately.

[0021] The input unit 13 includes at least one input interface. The input interface may be, for example, a physical key, a capacitive key, a pointing device, or a touchscreen integrated with a display. Alternatively, the input interface may be, for example, a sound sensor that accepts voice input, or a camera that accepts gesture input. The input unit 13 accepts operations to input data used for the operation of the information processing device 10. Instead of being integrated into the information processing device 10, the input unit 13 may be connected to the information processing device 10 as an external input device. Any connection method can be used, for example, USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface), or Bluetooth (registered trademark).

[0022] The output unit 14 includes at least one output interface. The output interface is, for example, a display that outputs information as video, or a speaker that outputs information as sound. The display is, for example, an LCD (liquid crystal display) or an organic EL (electroluminescence) display. The output unit 14 outputs data obtained by the operation of the information processing device 10. Instead of being provided in the information processing device 10, the output unit 14 may be connected to the information processing device 10 as an external output device. Any connection method can be used, for example, USB, HDMI (registered trademark), or Bluetooth (registered trademark).

[0023] The communication unit 15 includes at least one external communication interface. The communication interface may be either a wired or wireless communication interface. In the case of wired communication, the communication interface may be, for example, a LAN (Local Area Network) interface or a USB (Universal Serial Bus) interface. In the case of wireless communication, the communication interface may be, for example, an interface compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th generation), or 5G (5th generation), or an interface compatible with short-range wireless communication such as Bluetooth (registered trademark). The communication unit 15 receives data used for the operation of the information processing device 10 and transmits data obtained by the operation of the information processing device 10.

[0024] The functions of the information processing device 10 are realized by executing the program according to this embodiment on a processor corresponding to the control unit 11. In other words, the functions of the information processing device 10 are realized by software. The program causes the computer to perform the operations of the information processing device 10, thereby causing the computer to function as the information processing device 10. That is, the computer functions as the information processing device 10 by performing the operations of the information processing device 10 according to the program.

[0025] In this embodiment, the program can be recorded on a computer-readable recording medium. The computer-readable recording medium includes non-temporary computer-readable media, such as magnetic recording devices, optical discs, magneto-optical recording media, or semiconductor memory. The program can be distributed, for example, by selling, transferring, or lending portable recording media such as DVDs (digital versatile discs) or CD-ROMs (compact disc read-only memory) on which the program is recorded. Alternatively, the program may be distributed by storing it on the storage of an external server and transmitting it from the external server to other computers. The program may also be provided as a program product.

[0026] Some or all of the functions of the information processing device 10 may be implemented by a dedicated circuit corresponding to the control unit 11. In other words, some or all of the functions of the information processing device 10 may be implemented by hardware.

[0027] (Configuration of terminal device 20) As shown in Figure 1, the terminal device 20 comprises a control unit 21, a storage unit 22, an input unit 23, an output unit 24, and a communication unit 25.

[0028] The control unit 21 includes at least one processor, at least one dedicated circuit, or a combination thereof. The processor is a general-purpose processor such as a CPU (central processing unit) or GPU (graphics processing unit), or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, an FPGA (field-programmable gate array) or an ASIC (application-specific integrated circuit). The control unit 21 controls each part of the terminal device 20 and executes processes related to the operation of the terminal device 20.

[0029] The storage unit 22 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or at least two combinations thereof. The semiconductor memory is, for example, RAM (random access memory) or ROM (read-only memory). The RAM is, for example, SRAM (static random access memory) or DRAM (dynamic random access memory). The ROM is, for example, EEPROM (electrically erasable programmable read-only memory). The storage unit 22 functions, for example, as a main memory, auxiliary memory, or cache memory. The storage unit 22 stores data used for the operation of the terminal device 20 and data obtained by the operation of the terminal device 20.

[0030] The input unit 23 includes at least one input interface. The input interface may be, for example, a physical key, a capacitive key, a pointing device, or a touchscreen integrated with a display. Alternatively, the input interface may be, for example, a sound sensor that accepts voice input, or a camera that accepts gesture input. The input unit 23 accepts operations to input data used for the operation of the terminal device 20. Instead of being provided in the terminal device 20, the input unit 23 may be connected to the terminal device 20 as an external input device. Any connection method can be used, for example, USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface), or Bluetooth (registered trademark).

[0031] The output unit 24 includes at least one output interface. The output interface is, for example, a display that outputs information as video, or a speaker that outputs information as sound. The display is, for example, an LCD (liquid crystal display) or an organic EL (electroluminescence) display. The output unit 24 outputs data obtained by the operation of the terminal device 20. Instead of being provided in the terminal device 20, the output unit 24 may be connected to the terminal device 20 as an external output device. Any connection method can be used, for example, USB, HDMI (registered trademark), or Bluetooth (registered trademark).

[0032] The communication unit 25 includes at least one external communication interface. The communication interface may be either a wired or wireless communication interface. In the case of wired communication, the communication interface may be, for example, a LAN (Local Area Network) interface or a USB (Universal Serial Bus) interface. In the case of wireless communication, the communication interface may be, for example, an interface compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th generation), or 5G (5th generation), or an interface compatible with short-range wireless communication such as Bluetooth (registered trademark). The communication unit 25 receives data used for the operation of the terminal device 20 and transmits data obtained by the operation of the terminal device 20.

[0033] The functions of the terminal device 20 are realized by executing the program according to this embodiment on a processor corresponding to the control unit 21. In other words, the functions of the terminal device 20 are realized by software. The program causes the computer to perform the operations of the terminal device 20, thereby causing the computer to function as the terminal device 20. That is, the computer functions as the terminal device 20 by performing the operations of the terminal device 20 according to the program.

[0034] Some or all of the functions of the terminal device 20 may be implemented by a dedicated circuit corresponding to the control unit 21. In other words, some or all of the functions of the terminal device 20 may be implemented by hardware.

[0035] (Operation of the information processing device 10) Referring to Figure 2, the operation of the information processing device 10 according to this embodiment will be explained. Here, we will mainly explain an example in which the audio data is audio data of a business negotiation related to vehicle sales.

[0036] S10: The control unit 11 of the information processing device 10 acquires audio data.

[0037] Any method can be used for acquiring audio data. For example, the control unit 11 may acquire audio data from external devices, including the terminal device 20, via the communication unit 15 and the network 30. Alternatively, the control unit 11 may acquire audio data via the input unit 13.

[0038] S20: The control unit 11 generates text data based on the audio data acquired in step S10.

[0039] Any method can be used for generating text data based on audio data. For example, the control unit 11 may generate text data corresponding to the audio data by inputting the audio data to the speech recognition engine.

[0040] S30: The control unit 11 corrects error patterns in the text data based on the error correction table. For example, the control unit 11 extracts all words, phrases, and other patterns in the text data that match the error patterns in the error correction table. Then, the control unit 11 changes the extracted patterns to the correction patterns based on the correction patterns corresponding to the error patterns in the error correction table.

[0041] S40: The control unit 11 outputs the corrected text data.

[0042] Any method can be used for outputting text information. For example, the control unit 11 may transmit data to the terminal device 20 via the communication unit 15, and the output unit 24 of the terminal device 20 may output the text data through a user interface that it displays. Alternatively, the control unit 11 may output the text data through a user interface that it displays through the output unit 14.

[0043] Here, the error correction table includes a first error pattern, at least one second error pattern generated based on the audio data from which the first error pattern was generated, and correction patterns corresponding to the first error pattern and at least one second error pattern. Referring to Figure 3, an example of the operation related to the generation of the error correction table of the information processing device 10 according to this embodiment will be described.

[0044] S110: The control unit 11 of the information processing device 10 acquires audio data in which the voice corresponding to the first error pattern is spoken. Specifically, for example, suppose the first error pattern is LOVE for. In this case, the audio data corresponding to the first error pattern is data in which the vehicle name RAV4 (registered trademark) is spoken. The control unit 11 may identify the audio data in which the voice corresponding to the first error pattern is spoken by referring to an error correction table, for example, and acquire the identified audio data.

[0045] Any method can be used for acquiring audio data in S110. For example, the control unit 11 may acquire audio data from external devices, including the terminal device 20, via the communication unit 15 and the network 30. Alternatively, the control unit 11 may acquire audio data via the input unit 13.

[0046] S120: The control unit 11 generates at least one second error pattern based on audio data within a predetermined time range (hereinafter also referred to as target data) that includes the period during which the audio corresponding to the first error pattern was spoken. The predetermined time may be, for example, 10 seconds. In other words, the target data may be audio data for a period that includes 10 seconds before the period during which the audio corresponding to the first error pattern was spoken and 10 seconds after the period during which the audio corresponding to the first error pattern was spoken. Context and contextual information can also be obtained by performing the processing described later, including the periods before and after the period during which the audio corresponding to the first error pattern was spoken. For example, if the first error pattern is LOVE4, the second error pattern may be LAB4, LAVE4, RAB4, REV4, RAF4, RAP4, LAB4, etc.

[0047] Any method can be used to generate at least one second error pattern. For example, at least one second error pattern may be generated by inputting processed data, obtained by processing the target data, into the speech recognition engine. The processed data may be data processed by at least one of the following processes applied to the target data: noise addition, noise removal, frequency change, or volume change. In this way, the control unit 11 may generate at least one second error pattern by inputting processed data, obtained by processing the target data, into the speech recognition engine. By using processed data, the variations in error patterns can be increased. Therefore, the possibility of correcting erroneous text data using an error correction table increases.

[0048] Alternatively, for example, at least one second error pattern may be generated by inputting the target data into a parameterized speech recognition engine. The parameters may include, for example, recognition accuracy, thresholds for confidence, parameters related to the language model, parameters related to the acoustic model, and parameters related to a custom dictionary. In this way, the control unit 11 may generate at least one second error pattern by inputting the target data into a parameterized speech recognition engine without processing it. The variations in error patterns can also be increased by adjusting the parameters of the speech recognition engine. This increases the likelihood that erroneous text data can be corrected by the error correction table.

[0049] S130: The control unit 11 associates at least one second error pattern generated in S120 with a correction pattern and stores them in the error correction table. For example, as described above, if the second error pattern is LAB4, LAVE4, RAB4, REV4, RAF4, RAP4, LAB4, etc., these error patterns are associated with the correction pattern RAV4 (registered trademark) and stored in the error correction table.

[0050] With this configuration, the information processing device 10 corrects error patterns contained in the text data based on an error correction table. In particular, the error correction table includes a first error pattern, at least one second error pattern generated based on the audio data from which the first error pattern was generated, and correction patterns corresponding to the first error pattern and at least one second error pattern. Because the error correction table includes multiple different error patterns in this way, the likelihood of correcting erroneous text data increases, and since multiple error patterns are efficiently generated based on the audio data, the audio data to text conversion technology is improved.

[0051] While this disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art may make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions, etc., included in each component or step can be rearranged in a logically consistent manner, and multiple components or steps can be combined into one or divided into two.

[0052] Furthermore, in the embodiment described above, it is also possible to have an embodiment in which the configuration and operation of the information processing device 10 are distributed among multiple computers that can communicate with each other.

[0053] Some embodiments of the present disclosure are described below. However, it should be noted that the embodiments of the present disclosure are not limited to these. [Note 1] A method for converting audio data to text, performed by an information processing device, Acquiring audio data and, Based on the aforementioned audio data, text data is generated, Based on the error correction table, the error patterns contained in the text data are corrected, A method for converting speech data to text, wherein the error correction table includes a first error pattern, at least one second error pattern generated based on the speech data from which the first error pattern was generated, and correction patterns corresponding to the first error pattern and the at least one second error pattern. [Note 2] The method for converting audio data to text as described in Appendix 1, A method for converting speech data to text, wherein the at least one second error pattern is generated based on target data within a predetermined time range, including the period during which the speech corresponding to the first error pattern was spoken, from the speech data on which the first error pattern was generated. [Note 3] The method for converting audio data to text as described in Appendix 2, The method for converting speech data to text is generated by inputting the processed data obtained by processing the target data into a speech recognition engine, wherein the at least one second error pattern is the target data. [Note 4] The method for converting audio data to text as described in Appendix 3, A method for converting audio data to text, wherein the processed data is data that has been processed by at least one of the following processes applied to the target data: noise addition processing, noise removal processing, frequency modification processing, and volume modification processing. [Note 5] The method for converting audio data to text as described in Appendix 2, The method for converting speech data to text is generated by inputting the target data into a parameter-tuned speech recognition engine, wherein the at least one second error pattern is the target data. [Note 6] An information processing device comprising a control unit, The control unit, Acquire audio data, Based on the aforementioned audio data, text data is generated. Based on the error correction table, the error patterns contained in the text data are corrected. The error correction table includes a first error pattern, at least one second error pattern generated based on the audio data from which the first error pattern was generated, and correction patterns corresponding to the first error pattern and the at least one second error pattern, in an information processing device. [Note 7] On the computer, Acquiring audio data and, Based on the aforementioned audio data, text data is generated, Based on the error correction table, the error patterns contained in the text data are corrected, A program that executes, The error correction table is a program that includes a first error pattern, at least one second error pattern generated based on the audio data from which the first error pattern was generated, and correction patterns corresponding to the first error pattern and the at least one second error pattern. [Explanation of Symbols]

[0054] 1 System 10 Information Processing Devices 11 Control Unit 12 Storage section 13 Input section 14 Output section 15 Communications Department 20 Terminal devices 21 Control Unit 22 Memory section 23 Input section 24 Output section 25 Communications Department 30 Networks

Claims

1. A method for converting audio data to text, performed by an information processing device, Acquiring audio data and, Based on the aforementioned audio data, text data is generated, Based on the error correction table, the error patterns contained in the text data are corrected, A method for converting speech data to text, wherein the error correction table includes a first error pattern, at least one second error pattern generated based on the speech data from which the first error pattern was generated, and correction patterns corresponding to the first error pattern and the at least one second error pattern.

2. A method for converting audio data to text according to claim 1, A method for converting speech data to text, wherein the at least one second error pattern is generated based on target data within a predetermined time range, including the period during which the speech corresponding to the first error pattern was spoken, from the speech data on which the first error pattern was generated.

3. A method for converting audio data to text according to claim 2, The method for converting speech data to text is generated by inputting the processed data obtained by processing the target data into a speech recognition engine, wherein the at least one second error pattern is the target data.

4. A method for converting audio data to text according to claim 3, A method for converting audio data to text, wherein the processed data is data that has been processed by at least one of the following processes applied to the target data: noise addition processing, noise removal processing, frequency modification processing, and volume modification processing.

5. A method for converting audio data to text according to claim 2, The above-mentioned at least one second error pattern is a method for converting speech data to text, which is generated by inputting the target data into a parameter-tuned speech recognition engine.

6. An information processing device comprising a control unit, The control unit, Acquire audio data, Based on the aforementioned audio data, text data is generated. Based on the error correction table, the error patterns contained in the text data are corrected. The error correction table includes a first error pattern, at least one second error pattern generated based on the audio data from which the first error pattern was generated, and correction patterns corresponding to the first error pattern and the at least one second error pattern, respectively, in this information processing device.

7. On the computer, Acquiring audio data and, Based on the aforementioned audio data, text data is generated, Based on the error correction table, the error patterns contained in the text data are corrected, A program that executes, The error correction table is a program that includes a first error pattern, at least one second error pattern generated based on the audio data from which the first error pattern was generated, and correction patterns corresponding to the first error pattern and the at least one second error pattern.

Images