Text conversion method for voice data

Abstract

The invention provides a text conversion method of voice data executed by an information processing device. The text conversion method comprises the following steps: acquiring voice data; detecting a specific expression in the voice data and feature information related to the sound production of the specific expression; and outputting text information related to the voice data in a pattern capable of discriminating a portion in which standard language conversion processing has not been performed when standard language conversion processing for converting the specific expression into a corresponding standard language based on the detected specific expression and the feature information has not been performed.

Description

Technical Field

[0001] The present disclosure relates to a method for text conversion of voice data. Background Art

[0002] Conventionally, techniques for analyzing negotiation content have been known. For example, Japanese Unexamined Patent Application Publication No. 2019-28910 discloses a dialogue analysis system for checking whether a salesperson has stated items that should be stated and has not stated items that cannot be stated when negotiating with a customer. Additionally, for example, in Horimoto et al., "Toyama Dialect Speech Recognition Based on Deep Learning and Its Conversion to Standard Japanese," The 38 th Annual Conference of the Japanese Society for Artificial Intelligence (2024), a speech recognition technology for the Toyama dialect was disclosed. Summary of the Invention

[0003] Although Japanese Unexamined Patent Application Publication No. 2019-28910 discloses a technique for analyzing negotiation content through machine learning, neither Japanese Unexamined Patent Application Publication No. 2019-28910 nor Horimoto et al., "Toyama Dialect Speech Recognition Based on Deep Learning and Its Conversion to Standard Japanese," The 38 th Annual Conference of the Japanese Society for Artificial Intelligence (2024) mentions speech transcription in negotiations or the like, that is, neither mentions the text conversion technology for voice data. In particular, there is still room for improvement in the technology for speech transcription of voice data containing non-standard languages such as dialects and accents. On the other hand, in order to analyze and provide feedback on the content of negotiations or the like, it is desirable to improve the text conversion technology for voice data. Therefore, there is room for improvement in the technology for text conversion of voice data in negotiations or the like.

[0004] An object of the present disclosure completed in view of the above circumstances is to improve the text conversion technology for voice data.

[0005] A method for text conversion of voice data according to an embodiment of the present disclosure is a method for text conversion of voice data executed by an information processing device, including:

[0006] Obtaining voice data;

[0007] Detecting a specific expression in the voice data and feature information related to the pronunciation of the specific expression; and

[0008] In cases where it is impossible to perform standard language conversion processing based on the detected specific expression and the feature information to convert the specific expression into the corresponding standard language, text information related to the speech data is output in a style that can identify the part that failed to undergo standard language conversion processing.

[0009] According to one embodiment of this disclosure, the text conversion technology for voice data is improved. Attached Figure Description

[0010] The features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which the same reference numerals denote the same elements, and wherein:

[0011] Figure 1 This is a block diagram illustrating the general structure of the system according to this embodiment;

[0012] Figure 2 This is a flowchart illustrating the operation of an information processing device. Detailed Implementation

[0013] The embodiments of this disclosure will be described below. (Refer to...) Figure 1 This section describes the overview and structure of System 1 in this embodiment. System 1 in this embodiment includes an information processing device 10 and a terminal device 20. The information processing device 10 is, for example, a server device installed in a data center. The terminal device 20 is any device used by a user. These devices are communicatively connected via a network 30 such as the Internet. Although in Figure 1 The diagram shows one information processing device 10 and one terminal device 20, but system 1 may include multiple such devices.

[0014] First, an outline of the voice data text conversion method of this embodiment will be given, followed by a detailed explanation. Voice data can be, for example, voice data from a negotiation. In this embodiment, the negotiation is, for example, a negotiation involving the sale of a vehicle, and the offering involved in the negotiation is a vehicle, but it is not limited to this. For example, the negotiation can be a meeting aimed at signing various contracts such as the sale of real estate, insurance contracts, and the sale of financial products. Furthermore, in this embodiment, the offering related to the negotiation can be goods, services, digital content, licenses, data / information, financial products, real estate, intangible assets, other tradable rights, etc.

[0015] The information processing device 10 acquires speech data. The information processing device 10 also detects specific expressions in the speech data and feature information related to the pronunciation of those specific expressions. If the information processing device 10 cannot perform standard language conversion processing to convert the specific expression into the corresponding standard language based on the detected specific expression and feature information, it can identify the style of the portion that could not be converted into standard language and output text information related to the speech data.

[0016] Thus, according to this embodiment, the information processing device 10 detects specific expressions in the speech data and feature information related to the pronunciation of those specific expressions. When standard language conversion processing is not possible, it outputs text information related to the speech data by identifying the pattern of the portion that could not be converted to standard language. Therefore, when a specific expression cannot be converted, text information is output by identifying the pattern of that portion, thus improving the text conversion technology for speech data by easily identifying the points where conversion to standard language could not be completed.

[0017] Next, the structures of the information processing device 10 and the terminal device 20 will be described in detail. For example... Figure 1 As shown, the information processing device 10 includes a control unit 11, a storage unit 12, an input unit 13, an output unit 14, and a communication unit 15. The control unit 11 includes at least one processor. The processor is a general-purpose processor such as a CPU, or a dedicated processor for specific processing. The control unit 11 performs processing related to the operation of the information processing device 10 while controlling each part of the information processing device 10. The storage unit 12 includes at least one semiconductor memory, such as RAM or ROM. The storage unit 12 functions as a main storage device or an auxiliary storage device, for example. The storage unit 12 stores data used for the operation of the information processing device 10 and data obtained through the operation of the information processing device 10. The input unit 13 includes at least one input interface. The input interface may be, for example, a physical key, a touchscreen, a voice sensor for receiving voice input, or a camera for receiving gesture input. The input unit 13 handles operations for inputting data used for the operation of the information processing device 10. The output unit 14 includes at least one output interface. The output interface may be, for example, a display for outputting images of information, or a speaker for outputting voice information. The output unit 14 outputs data obtained through the operation of the information processing device 10. The communication unit 15 includes at least one external communication interface. The communication interface can be a wired communication interface or a wireless communication interface. In the case of wired communication, the communication interface is, for example, a LAN or USB. In the case of wireless communication, the communication interface is, for example, an interface corresponding to mobile communication standards such as 5G, or an interface corresponding to short-range wireless communication. The communication unit 15 receives data for the operation of the information processing device 10 and transmits data obtained through the operation of the information processing device 10.

[0018] In addition, such as Figure 1 As shown, the terminal device 20 includes a control unit 21, a storage unit 22, an input unit 23, an output unit 24, and a communication unit 25. The control unit 21 includes at least one processor. The processor is a general-purpose processor such as a CPU, or a dedicated processor for specific processing. The control unit 21 controls each part of the terminal device 20 and performs processing related to the operation of the terminal device 20. The storage unit 22 includes at least one semiconductor memory, such as RAM or ROM. The storage unit 22 functions as a main storage device or an auxiliary storage device, for example. The storage unit 22 stores data used for the operation of the terminal device 20 and data obtained through the operation of the terminal device 20. The input unit 23 includes at least one input interface. The input interface can be, for example, a physical key, a touchscreen, a voice sensor for receiving voice input, or a camera for receiving gesture input. The input unit 23 handles operations for inputting data used for the operation of the terminal device 20. The output unit 24 includes at least one output interface. The output interface can be, for example, a display for outputting images of information, or a speaker for outputting voice information. The output unit 24 outputs data obtained through the operation of the terminal device 20. The communication unit 25 includes at least one external communication interface. The communication interface can be a wired communication interface or a wireless communication interface. In the case of wired communication, the communication interface is, for example, a LAN or USB. In the case of wireless communication, the communication interface is, for example, an interface corresponding to mobile communication standards such as 5G, or an interface corresponding to short-range wireless communication. The communication unit 25 receives data used by the operation of the terminal device 20 and transmits data obtained through the operation of the terminal device 20.

[0019] The functions of the information processing device 10 or the terminal device 20 are implemented by executing a program of the fundamental embodiment in a processor corresponding to the control unit 11 or the control unit 21. That is, the functions of the information processing device 10 or the terminal device 20 are implemented by software. The program causes the computer to perform the actions of the information processing device 10 or the terminal device 20, thereby enabling the computer to function as the information processing device 10 or the terminal device 20. In other words, the computer performs the actions of the information processing device 10 or the terminal device 20 according to the program, thereby enabling it to function as the information processing device 10 or the terminal device 20. In this embodiment, the program can be recorded on a computer-readable recording medium. Computer-readable recording media include non-transitory computer-readable media, such as magnetic recording devices, semiconductor memory, etc. The program is distributed, for example, by selling, transferring, or lending a portable recording medium such as a DVD containing the program. Furthermore, the program can be distributed by storing it in the memory of an external server and sending it from the external server to other computers. Additionally, the program can be provided as a program product. Some or all of the functions of the information processing device 10 or the terminal device 20 can be implemented by a dedicated circuit corresponding to the control unit 11 or the control unit 21. That is, some or all of the functions of the information processing device 10 or the terminal device 20 can also be implemented by hardware.

[0020] Reference Figure 2 The operation of the information processing apparatus 10 according to this embodiment will now be described. Here, we will primarily describe an example of a negotiation related to vehicle sales. First, the control unit 11 of the information processing apparatus 10 acquires voice data (step S10). Any method can be used in acquiring and processing the voice data. For example, the control unit 11 may acquire voice data from an external device, including the terminal device 20, via the communication unit 15 and the network 30. Alternatively, the control unit 11 may acquire voice data via the input unit 13.

[0021] Next, the control unit 11 detects specific expressions in the speech data acquired in step S10 and feature information related to the pronunciation of those specific expressions (step S20). Specific expressions include those used in actual speech, such as dialects, non-standard languages, slang, etc. Any method can be used in the detection processing of specific expressions in the speech data. For example, a speech recognition engine can be used to match keywords, phrases, etc., pre-registered as specific expressions. Alternatively, a machine learning model, such as deep learning, can be used to classify speech patterns and identify predetermined expressions. Feature information related to the pronunciation of specific expressions includes, for example, tone information, rhythm of pronunciation, pitch and tone variations, and pronunciation speed. This feature information typically reflects the speaker's speech habits, emotions, and intentions, and is considered important for capturing in detail how specific expressions are pronounced. Any method can be used in the detection processing of feature information related to the pronunciation of specific expressions. For example, a pitch and tone detection algorithm can be used to analyze tone variations. Alternatively, a method using spectrogram analysis or similar methods to extract time / frequency features of the speech signal can be used. Furthermore, machine learning models such as deep learning can be used to comprehensively capture the features of speech, thereby detecting feature information related to vocalization.

[0022] Next, the control unit 11 determines whether it can perform standard language conversion processing to convert the specific expression into the corresponding standard language based on the detected specific expression and feature information (step S30). This determination process can be performed using any method. For example, the standard language conversion processing is the process of converting the specific expression into standard language based on the detected pairs of specific expressions and feature information, and the conversion rules between non-standard language and standard language. In this case, if a pair of specific expressions and feature information exists in the conversion rules, the control unit 11 determines that standard language conversion processing can be performed. On the other hand, if a pair of specific expressions and feature information does not exist in the conversion rules, the control unit 11 determines that standard language conversion processing cannot be performed.

[0023] When it is determined in step S30 that the standard language conversion process can be performed, the control unit 11 converts the detected specific expression into the corresponding standard language according to the detected specific expression and characteristic information (step S40). Any method can be adopted to perform the conversion process to the standard language. For example, the control unit 11 can perform the process of converting the specific expression into the standard language according to the detected specific expression and characteristic information pair, and the conversion rules between the non-standard language and the standard language. The conversion rules can be stored in the storage unit 12, for example, and the control unit 11 can execute the above conversion process by referring to the conversion rules in the storage unit 12. In this case, the conversion rules can be a conversion table of the pair of the specific expression and characteristic information and the corresponding standard language. The corresponding standard language is the standard language to be converted for each specific expression. In other words, the corresponding standard language is the language expression obtained as a result of the conversion process. Alternatively, the control unit 11 can perform the conversion process to the standard language through a machine learning model for performing the standard language conversion process based on the detected specific expression and characteristic information.

[0024] After step S40, the control unit 11 outputs the text information related to the voice data (step S50). The text information is the information that converts the voice data into text, and is the information that converts the above specific expression in the voice data into the standard language. In other words, the output text information is the standard language sentence obtained as a result of speech recognition, and is the text after detecting the specific expression and converting it into the standard language. For example, for voice data including the Aichi dialect "この道はとーめんこ (Chinese: This road is prohibited from entry)", the control unit 11 outputs the text information "この道は通行止め (Chinese: This road is closed to traffic)" based on the specific expression "とーめんこ" and the tone using the conversion rules. In this way, even if the specific expression is a dialect, diction, etc. used in different regions or cultures, it can be converted into the standard language by using appropriate conversion rules, so as to provide the text information in a form that can be understood by a wider range of users. Any method can be adopted to perform the output process of the text information. For example, the control unit 11 can send data to the terminal device 20 via the communication unit 15, and output the determination result by displaying the output user interface of the terminal device 20. Alternatively, the control unit 11 can also output the determination result by displaying the output user interface through the output unit 14.

[0025] If it is determined in step S30 that standard language conversion processing cannot be performed, the control unit 11 outputs text information related to the voice data in a style that identifies the portion that failed standard language conversion processing (step S60). The output processing of this text information can be performed using any method. For example, the control unit 11 can send data to the terminal device 20 via the communication unit 15, and output the determination result by displaying the output user interface on the output unit 24 of the terminal device 20. Alternatively, the control unit 11 can also output the determination result by displaying the output user interface on the output unit 14. Styles that identify the portion that failed standard language conversion processing include, for example, styles that emphasize the specific expression that failed standard language conversion processing, styles that display underlines, etc.

[0026] According to this configuration, the information processing device 10 detects specific expressions in the speech data and feature information related to the pronunciation of those specific expressions. Furthermore, when standard language conversion is not possible, it outputs text information related to the speech data in a style that identifies the portion that cannot be converted to standard language. Therefore, by outputting text information in a style that identifies the portion when a specific expression cannot be converted, the text conversion technology for speech data is improved by easily identifying the points where conversion to standard language is impossible.

[0027] Although this disclosure has been described with reference to the accompanying drawings and embodiments, it should be noted that those skilled in the art can make various modifications and changes based on this disclosure. Therefore, it should be understood that such modifications and changes are included within the scope of this disclosure. For example, the functions contained in each component or step can be reconfigured in a logically consistent manner, and multiple components or steps can be combined into one or divided.

[0028] For example, the control unit 11 of the information processing device 10 can also perform annotation processing for specific expressions that have failed to be converted into standard language. The annotation processing can employ any method. For example, the control unit 11 can receive user input and perform manual annotation processing for specific expressions. Alternatively, the control unit 11 can automatically perform annotation processing by inputting the specific expression that failed to be converted into standard language into a Large Language Model (LLM) or similar object, estimating the standard language for that specific expression. Thus, the specific expression can be corrected based on the annotation, and textual information related to the speech data can be stored or accumulated. Furthermore, the control unit 11 can update the conversion rules based on the annotation processing and store the updated conversion rules in the storage unit 12. Alternatively, the control unit 11 can relearn the machine learning model involved in the standard language conversion processing based on the annotation processing.

[0029] Furthermore, for example, in the above embodiments, the configuration and operation of the information processing device 10 can be distributed among multiple computers capable of communicating with each other.

Claims

1. A method for converting speech data into text, performed by an information processing device, comprising: Acquire voice data; Detect specific expressions in the speech data and feature information related to the pronunciation of those specific expressions; as well as In cases where it is impossible to perform standard language conversion processing to convert the specific expression into the corresponding standard language based on the detected specific expression and the feature information, text information related to the speech data is output in a style that can identify the part that failed to undergo standard language conversion processing.

2. The method for converting speech data to text according to claim 1, wherein, The standard language conversion process is a process that converts the specific expression into standard language based on the detected specific expression and the feature information, as well as the conversion rules between non-standard language and standard language.

3. The method for converting speech data to text according to claim 1, wherein, The characteristic information related to the vocalization is tone information.

4. The method for converting speech data to text according to claim 1, wherein, The specific expressions mentioned include dialects and non-standard languages.

5. The method for converting speech data to text according to claim 1, wherein, The text conversion method for the speech data also includes: Perform annotations on specific expressions that failed to be translated into standard language.

Images