Pronunciation editing for automatic text-to-speech for long-form text documents

Abstract

Aspects of the present disclosure are directed to a technique for enabling efficient automatic text-to-speech pronunciation editing for long-form text documents. A computing device including a memory and a processor may be configured to execute the technique. The memory may store a text document. The processor may process words in the text document to identify first candidate words predicted to be mispronounced during automatic text-to-speech processing of the text document. The processor may then filter the first candidate words to exclude one or more candidate words of the first candidate words and obtain second candidate words having fewer candidate words than the first candidate words. The processor may then annotate the text document to obtain an annotated text document identifying the second candidate words, and output at least a portion of the annotated text document identifying at least one candidate word of the second candidate words.

Description

[Background technology]

[0001] There are millions of long-form text documents published for the general public, such as books, manuals, research papers, etc. For example, consumers of certain long-form text genres, such as self-help, business, history, biography, health, and religion, may desire narrated versions of the long-form text documents, which may be referred to as audiobooks. However, while there is a demand for audiobooks (particularly the long-form text genres mentioned above), unless the long-form text documents are successful (in terms of sales units), producing an audiobook may be prohibitively expensive. For example, producing a given audiobook may require the hiring of expensive voice actors (or in other words, narrators), who may require many hours to properly recite the long-form text documents in a studio environment (where hourly rates are generally expensive and may exceed $100 per hour).

[0002] As such, many authors choose to rely on automatic text-to-speech algorithms that utilize narration models to recite the text of long-form text documents. While automatic text-to-speech algorithms may provide a quick alternative to human narration that utilizes the repetition of an audiobook experience (in some cases, near real-time, meaning that such automatic text-to-speech algorithms reside on the reader's device to provide this audiobook experience), automatic text-to-speech algorithms may mispronounce words that would be narrated correctly by a human narrator. To overcome these mispronunciations, authors (or other editors) can edit the underlying long-form text document to reduce mispronunciations by the automatic text-to-speech algorithm that would degrade the author's intent when a reader subsequently attempts to read the text version of the long-form text document. Summary of the Invention

[0003] Aspects of the present disclosure are directed to techniques that enable efficient automatic text-to-speech pronunciation editing for long-form text documents. The pronunciation editing may be similar to (and therefore may also be referred to as) a spell check provided in a text editor (also called a word processor). A computing device may receive a long-form text document and perform pronunciation editing to identify candidate words that may be mispronounced during the automatic text-to-speech process. However, rather than providing all candidate words to an editor for consideration, the computing device may filter the candidate words in various ways to reduce the number of candidate words to a more manageable number (e.g., a threshold number of candidate words).

[0004] Filtering the candidate words can reduce time spent on unimportant pronunciation errors and focus attention on words that are difficult to pronounce (e.g., proper nouns, words that are ambiguous in terms of inflection, etc.), intentionally misspelled words, words that are spelled the same but pronounced differently (called homographs), etc. In this way, the editor can focus on candidate words that may significantly impair the audio experience. In some cases, the editor can interact with the computing device to input a verbal pronunciation, which the computing device then uses to modify the automatic text-to-speech pronunciation (e.g., select the pronunciation that best matches the input verbal pronunciation).

[0005] In this manner, various aspects of the present technology can facilitate more efficient execution of computing devices in terms of computing resources (e.g., processor cycles, memory usage, memory bus bandwidth usage, and associated power consumption) while also improving the user experience for editing potential mispronunciations by the automatic text-to-speech algorithm. The present technology can reduce computing resource usage by filtering candidate words to allow editors to focus on words that are difficult to pronounce, resulting in reduced computing resource usage due to less processing during the editing process. Additionally, these techniques can provide a better editing experience by focusing attention on words that are difficult to pronounce while the automatic text-to-speech algorithm improves the pronunciation of candidate words (thus producing a higher quality audiobook experience compared to automatic text-to-speech algorithms that do not facilitate review and adjustment of pronunciations).

[0006] In one example, various aspects of the present technology are directed to a method that includes processing words in a text document to identify a first plurality of candidate words that are predicted to be mispronounced during automatic text-to-speech processing of the text document; filtering the first plurality of candidate words to exclude one or more candidate words of the first plurality of candidate words to obtain a second plurality of candidate words having fewer candidate words than the first plurality of candidate words; annotating the text document to obtain an annotated text document that identifies the second plurality of candidate words; and outputting at least a portion of the annotated text document that identifies at least one candidate word of the second plurality of candidate words.

[0007] In another example, various aspects of the present technology are directed to a computing device including a memory configured to store a text document and one or more processors, the one or more processors configured to: process words in the text document to identify a first plurality of candidate words predicted to be mispronounced during automatic text-to-speech processing of the text document; filter the first plurality of candidate words to exclude one or more candidate words of the first plurality of candidate words to obtain a second plurality of candidate words having fewer candidate words than the first plurality of candidate words; annotate the text document to obtain an annotated text document identifying the second plurality of candidate words; and output at least a portion of the annotated text document identifying at least one candidate word of the second plurality of candidate words.

[0008] In another example, various aspects of the present technology are directed to a non-transitory computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to process words in a text document to identify a first plurality of candidate words predicted to be mispronounced during automatic text-to-speech processing of the text document; filter the first plurality of candidate words to exclude one or more candidate words of the first plurality of candidate words to obtain a second plurality of candidate words having fewer candidate words than the first plurality of candidate words; annotate the text document to obtain an annotated text document identifying the second plurality of candidate words; and output at least a portion of the annotated text document identifying at least one candidate word of the second plurality of candidate words.

[0009] In another example, various aspects of the present technology are directed to an apparatus including: means for processing words in a text document to identify a first plurality of candidate words predicted to be mispronounced during automatic text-to-speech processing of the text document; means for filtering the first plurality of candidate words to exclude one or more candidate words of the first plurality of candidate words to obtain a second plurality of candidate words having fewer candidate words than the first plurality of candidate words; means for annotating the text document to obtain an annotated text document identifying the second plurality of candidate words; and means for outputting at least a portion of the annotated text document identifying at least one candidate word of the second plurality of candidate words. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 illustrates an example computing device configured to perform pronunciation editing of an automatic text-to-speech algorithm in accordance with one or more aspects of the present disclosure. [Figure 2A] 1A-1C illustrate example user interfaces with which a human editor can interact to facilitate efficient pronunciation checking of long-form text documents in accordance with various aspects of the techniques described in this disclosure. [Figure 2B] 1A-1C illustrate example user interfaces with which a human editor can interact to facilitate efficient pronunciation checking of long-form text documents in accordance with various aspects of the techniques described in this disclosure. [Figure 2C] 1A-1C illustrate example user interfaces with which a human editor can interact to facilitate efficient pronunciation checking of long-form text documents in accordance with various aspects of the techniques described in this disclosure. [Figure 3A] 1A-1C are additional diagrams illustrating exemplary user interfaces with which a human editor may interact to facilitate efficient pronunciation checking of long-form text documents, in accordance with various aspects of the techniques described in this disclosure. [Figure 3B]1A-1C are additional diagrams illustrating exemplary user interfaces with which a human editor may interact to facilitate efficient pronunciation checking of long-form text documents, in accordance with various aspects of the techniques described in this disclosure. [Figure 3C] 1A-1C are additional diagrams illustrating exemplary user interfaces with which a human editor may interact to facilitate efficient pronunciation checking of long-form text documents, in accordance with various aspects of the techniques described in this disclosure. [Figure 4] 1 is a flowchart illustrating an example operation of an example computing device configured to perform pronunciation editing of an automatic text-to-speech algorithm in accordance with one or more aspects of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] 1 illustrates an exemplary computing device configured to perform pronunciation editing of an automatic text-to-speech algorithm in accordance with one or more aspects of the present disclosure. Computing device 100 may represent any type of computing device capable of performing automatic text-to-speech processing on long-form text documents. Examples of computing devices 100 may include desktop computers, laptop computers, cellular handsets (including so-called smartphones), workstations, servers, game consoles, personal reading devices (such as dedicated e-book readers), and the like.

[0012] Although described with respect to a single computing device 100, the functions described herein as being performed by computing device 100 may be performed by one or more computing devices. That is, computing device 100 may represent a distributed computing system (e.g., a so-called cloud computing system) in which, as an example, a server may perform some aspects of the techniques described herein to generate a user interface that the server hosts for access over a network (e.g., a public network such as the Internet). In this example, a client computing device may accept the user interface (e.g., via a web browser) and input via the user interface to interact with the server. In this manner, various aspects of the present technology should not be limited to a single computing device 100, but may be applied across a variety of computing devices (e.g., a server and a client device, such that multiple servers may host different aspects of the present technology).

[0013] 1, computing device 100 includes a display 102, processor(s) 104, a storage system 106, input device(s) 108, output device(s) 110, and a communication unit 112. Computing device 100 may include a subset of the components included in exemplary computing device 100 or may include additional components not shown in FIG. 1 for ease of explanation.

[0014] In either case, display 102 represents any type of display capable of serving as an output for the visual presentation of data. Examples of display 102 include liquid crystal displays (LCDs), dot matrix displays, light emitting diode (LED) displays, mini LED displays, micro LED displays, organic LED (OLED) displays, e-ink, or similar monochrome or color displays capable of outputting visual information to a user of computing device 100. In this regard, display 102 may represent one example of output device 110.

[0015] In some examples, the display 102 may represent a presence-aware display (sometimes also commonly referred to as a touch screen, although this is somewhat of a misnomer in that some presence-aware displays can sense input approaching the presence-aware display that does not require physical contact) configured to operate as both an output and an input. In addition to serving as an output, a presence-aware display may also operate as an interface to receive input, such as the selection of an icon or other graphical user interface element, the input of text, gestures (including multi-touch gestures), and the like.

[0016] Although the display 102 is shown as an internal component of the computing device 100, it may also represent an external component that shares a data path with the computing device 100 to transmit and / or receive inputs and outputs. For example, in one example, the display 102 represents an embedded component of the computing device 100 (e.g., a screen on a smartphone or all-in-one computing device) that is disposed within and physically connected to the external packaging of the computing device 100. In another example, the display 102 represents an external component of the computing device 200 that is disposed outside of and physically separated from the packaging of the computing device 100 (e.g., a monitor, projector, etc. that shares a wired and / or wireless data path with a tablet computer).

[0017] Processor 104 may represent any type of processor capable of executing firmware, middleware, software, etc., including instructions that, when executed, cause processor 104 to perform the operations described with respect to processor 104. Examples of processor 104 include central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), display processors, field programmable gate arrays (FPGAs), microprocessors, application specific processors (ASICs, such as artificial intelligence accelerator ASICs), etc.

[0018] In some examples, processor 104 may include one or more cores integrated into a single so-called chip (meaning one or more processing cores packaged together and typically sharing memory, a memory bus, registers, and / or other resources). The multiple cores may include cores dedicated to arithmetic, graphics processing, sensor processing, artificial intelligence processing (e.g., in the form of one or more ASICs), etc. While assumed to represent a SoC, processor 104 may represent any type of processor capable of executing instructions that facilitate implementation of various aspects of the techniques described herein.

[0019] The storage system 106 may store information for processing during operation of the computing device 100 and may represent an example of a computer-readable medium. That is, in some examples, the storage system 106 includes temporary memory, meaning that the primary purpose of the storage system 106 is not long-term storage. The storage system 106 of the computing device 100 may be configured to store information short-term as a volatile memory, and thus does not retain stored content when power is turned off. Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of volatile memory.

[0020] Storage system 106 may also include one or more computer-readable storage media, which may be configured to store a larger amount of information than volatile memory, in some examples. Such computer-readable storage media may be non-transient in nature, meaning that such data is maintained within the computer-readable storage media and is not transitory (e.g., is not a transitory signal traveling on a wire or other conductor). Storage system 106 may further be configured for long-term storage of information as a non-volatile memory space, capable of retaining information after power on / off cycles. Examples of non-volatile memory include magnetic hard disks, optical disks, floppy disks, flash memory, or forms of electrically programmable memory (EPROM) or electrically erasable and programmable (EEPROM) memory. Storage system 106 may store program instructions and / or information (e.g., data) associated with modules 118-224 and associated data, such as long-form text documents (LFTD) 119, synthetic documents (SYN DOCS) 121, and audiobooks 125.

[0021] One or more input devices 108 of computing device 100 may be configured to receive input. Input devices 108 of computing device 100, in one example, include a presence-sensitive display (e.g., display 102), a mouse, a keyboard, a video camera, a microphone, physical buttons and / or switches (or other activators), ports (e.g., power ports, headphone ports, etc.), or any other type of device for detecting input from a human or machine. Input devices 108 may receive input data in the form of an activation state (e.g., of a button, switch, or any other physical interaction object), audio, an image, an image sequence (which may also be referred to as video), and the like.

[0022] One or more output devices 110 of computing device 100 may be configured to generate output. Output device 110 of computing device 100, in one example, includes a presence-aware display (e.g., display 102), an electronic rotating mass actuator (e.g., for generating haptic feedback), a sound card, a video graphics card, a speaker, a cathode ray tube (CRT) display, a liquid crystal display (LCD) display, a light-emitting diode (LED) display, a micro LED display, a mini LED display, an organic LED (OLED) display, a plasma display, or any other type of device for generating output to a human or machine. In some examples, display 102 may include the functionality of input device 108 (e.g., when display 102 represents a presence-aware display) and / or output device 110.

[0023] One or more communication units 112 of computing device 100 may be configured to communicate with external devices over one or more wired and / or wireless networks by transmitting and / or receiving network signals on one or more networks. Examples of communication units 112 include a network interface card (e.g., an Ethernet card, etc.), an optical transceiver, a radio frequency transceiver, a GPS receiver, or any other type of device capable of transmitting and / or receiving information. Other examples of communication units 112 may include a shortwave radio, a cellular data radio, a wireless network radio, and a universal serial bus (USB) controller.

[0024] The communication channel 114 may interconnect (physically, communicatively, and / or operatively) each of the components 102-112 for inter-component communication. In some examples, the communication channel 114 may include a system bus, a network connection, an inter-process communication data structure, or any other method for communicating data.

[0025] As further shown in the example of FIG. 1, the storage system 106 can store LFTDs 119, which can represent text documents having an average word count of more than 1,000 words, or even more than 10,000 words. The LFTDs 119 can represent, by way of example, books (including electronic books, which may also be referred to as e-books), manuals, research papers, and the like. The LFTDs 119 can represent e-books, whose average word count is typically more than 40,000 words, such that e-books range from 40,000 words to about 110,000 words. Thus, the LFTDs 119 can be distinguished from short-form text documents, such as text messages, electronic messages (including electronic mail, also referred to as e-mail), in that the LFTDs 119 include an average word count that is at least an order of magnitude greater than the average word count contained in short-form text documents.

[0026] For example, consumers of certain long-form text genres, such as self-help, business, history, biography, health, and religion, may desire a narrated version of the LFTD 119, which may be referred to as an audiobook. However, while there is a demand for audiobooks (particularly the long-form text genres mentioned above), unless the LFTD 119 is successful (in terms of unit sales), producing an audiobook may be prohibitively expensive. For example, producing a given audiobook may require the hiring of expensive voice actors (or in other words, narrators), who may take many hours to properly recite the LFTD 119 in a studio environment (where hourly rates are generally expensive and may exceed $100 per hour).

[0027] As such, many authors choose to rely on automatic text-to-speech algorithms that utilize narration models to recite the text of the LFTD119. While automatic text-to-speech algorithms can provide a quick alternative to human narration that utilizes the repetition of the audiobook experience (in some cases, near real-time, meaning that such automatic text-to-speech algorithms reside on the reader's device to provide this audiobook experience), these automatic text-to-speech algorithms may mispronounce words that would be narrated correctly by a human narrator. To overcome these mispronunciations, the author (or other editor) can edit the underlying LFTD119 to reduce mispronunciations by the automatic text-to-speech algorithm that would degrade the author's intent when a reader attempts to read a text version of the LFTD119.

[0028] According to various aspects of the techniques described in this disclosure, the computing device 100 can provide efficient automatic text-to-speech pronunciation editing for the LFTD 119. The pronunciation editing can be similar to (and therefore may be referred to as) a spell check provided in a text editor (also called a word processor). The computing device 100 can receive the LFTD 119 (through a user interface) and perform a pronunciation edit to identify candidate words that may be mispronounced during the automatic text-to-speech process. However, rather than providing all candidate words to the editor for consideration, the computing device 100 can filter the candidate words in various ways to reduce the number of candidate words to a more manageable number (e.g., a threshold number of candidate words).

[0029] 1, the storage system 106 may store a user interface (UI) module 118, a preprocessing module 120, a pronunciation module 122, and a text-to-speech (TTS) module 124. As a first matter, when an operation is described as being performed by any one of the modules 118-124, it should be understood that such module 118-224, when executed, causes the processor 104 to perform the operation described with respect to each of the modules 118-124.

[0030] UI module 118 may represent instructions forming a module that, when executed, causes processor 104 to generate and present a UI with which a user (e.g., a human editor) may interact to receive output from and / or provide input to computing device 100. Exemplary GUIs generated and output from UI module 118 are described in further detail below for the example of Figures 2A-3C.

[0031] The preprocessing module 120 may represent instructions forming a module that, when executed, causes the processor 104 to perform preprocessing on the LFTDs 119. Preprocessing may refer to a process in which the LFTDs 119 are reformatted, split, parsed, synthesized, or otherwise processed to convert the LFTDs 119 into manageable data chunks (e.g., the synthesized document 121). Preprocessing may facilitate parallel (or in other words simultaneous) processing of the LFTDs 219 to improve the efficiency (in terms of computing resources) of checking the pronunciation of the LFTDs 119 and the efficiency of generating the audiobook 125 from the LFTDs 119.

[0032] The pronunciation module 122 may represent instructions forming a module that, when executed, causes the processor 104 to perform a pronunciation check on the synthesized document 121 to obtain candidate words (CWs) 123A. The pronunciation module 122 may also include instructions that, when executed, causes the processor 104 to perform filtering of the CWs 123A to remove one or more candidate words from the CWs 123A, thereby generating remaining candidate words (RCWs) 123B.

[0033] The TTS module 124 may represent instructions that, when executed, cause the processor 104 to perform an automated TTS process to form a module that produces an audiobook 125. The automated TTS process may refer to one or more TTS algorithms (e.g., a generative adversarial network (GAN) model) that synthesize the LFTD 119 to produce the audiobook 125. The audiobook 125 may include an annotated version of the LFTD 119 (such annotations facilitating near real-time application of TTS algorithms on the computing device 100 and other computing devices (e-readers, smartphones, etc.)) and, in some cases, audio data output from the TTS module 124 after synthesizing the LFTD 119.

[0034] In operation, the processor 104 may first invoke the UI module 118, which may generate a UI (e.g., a graphical UI (GUI)) to facilitate uploading and / or distribution of the LFTD 119. The UI module 118 may interface with the display 102 to present the GUI via the display 102. In a distributed computing system, the UI module 118 may interface with the communication unit 112 to provide the GUI to a client device (e.g., server, client distributed system). In either case, a human editor may interact with the GUI to provide the LFTD 119 to the computing device 100, which may store the LFTD 119 in the storage system 106.

[0035] In response to receiving the LFTD 119, the processor 104 can then invoke the preprocessing module 120, which can process the LFTD 119 to generate the synthesized document 121. The preprocessing module 120 can share the fragments of the LFTD 119 into manageable sized chunks (e.g., N sentences) and aggregate each chunk with markup text. The preprocessing module 120 can then analyze each aggregated chunk using text normalization to identify spans of words. Text normalization refers to the process of detecting multi-word expressions across large chunks of text. The preprocessing module 120 can then analyze the results of the text normalization to generate contiguous, non-overlapping spans of the input text, which are aggregated chunks in this example. The preprocessing module 120 can then output the contiguous, non-overlapping spans determined for each aggregated chunk as each of the synthesized documents 121.

[0036] In response to obtaining the synthesized document 121, the processor 104 can invoke the pronunciation model 122, which can process the synthesized document 121 to identify CWs 123A that are predicted to be mispronounced during automatic text-to-speech processing of the LFTD 119. The pronunciation model 122 can include one or more different sub-models that perform pronunciation checks for a variety of different scenarios.

[0037] For example, the pronunciation module 122 may include a sub-model that identifies any text spans (specified in the synthesized document 121) that have multiple possible pronunciations, and designates these text spans as CW 123 A. Additionally, the pronunciation module 122 may include another sub-model that identifies any text spans that have a single pronunciation (e.g., for pronounced words) whose pronunciation is out of the lexicon, and designates these text spans as CW 123 A. Additionally, the pronunciation module 122 may include a different sub-model that identifies any text spans that have a single pronunciation, whose pronunciation has multiple pronunciation parts (e.g., for spelled out words and words in symbol classes, such as emoticons, emoticons, logos, trademarks, etc.).

[0038] In some examples, the pronunciation module 122 can calculate or otherwise determine a confidence score. That is, the pronunciation module 122 can apply a learning model to spans of words represented by the synthesized document 121 to determine a confidence score for each word span. The learning model can refer to a machine learning model trained based on past pronunciation editing operations. The pronunciation editing GUI (shown in the examples of Figures 2A-3C) can provide direct feedback to determine the accuracy of the pronunciation for any given word. Using this user feedback (which may be anonymized, disabled by default, and requiring explicit user consent to enable such feedback collection), various machine learning algorithms can train the underlying learning model to provide a confidence score. In any case, the pronunciation module 122 can then determine whether to add each word span to the CW 123A based on the confidence score. For example, for any word span that does not have a confidence score or has a confidence score below a threshold confidence score, the pronunciation module 122 can add the word span to the CW 123A.

[0039] The pronunciation module 122 may then filter the CW 123A to exclude one or more candidate words from the CW 123A to obtain an RCW 123B that has fewer candidate words than the CW 123A. The filtering of the CW 123A may occur iteratively or simultaneously according to several different sub-models.

[0040] For example, the pronunciation module 122 may include a stop word sub-model that identifies one or more candidate words of CW123A that are stop words. Stop words refer to a set of very commonly used words (in terms of average occurrence in any given text) such that these stop words convey little useful information. Examples of stop words include "a", "the", "is", "are", etc. Some of these stop words may have multiple pronunciations, and typically the TTS module 124 selects a given pronunciation (as determined by the selected automatic TTS voice), which generally does not impact the audiobook experience, such that editing of the stop words is justified. As such, the pronunciation module 122 may filter out one or more candidate words of CW123A that are stop words to obtain RCW123B.

[0041] As another example, the pronunciation module 122 may include a frequency count sub-model that identifies a candidate word count for each of the CWs 123A, indicating the number of times each candidate word appears in the CWs 123A. This frequency count sub-model may operate under similar assumptions as those described above with respect to the stop word sub-model, and may convey less information in terms of frequency according to the candidate words of the CWs 123A (those candidate words of the CWs 123A that have a candidate word count greater than the candidate word count threshold). The pronunciation module 122 may then remove from the CWs 123A one or more candidate words that have an associated candidate word count that exceeds the candidate word count threshold.

[0042] As yet another example, the pronunciation module 122 may include a homograph sub-model that identifies one or more candidate words of CW123A that are homographs not specified in the common homograph list. The homograph sub-model may dynamically update the common homograph list (e.g., via machine learning based on subsequent edits, editor / user phonetic pronunciation input, etc.). The term "homograph" refers to words that have the same spelling but different pronunciation (e.g., "I will read this book" vs. "I read this book already," where read is a homograph). The pronunciation module 122 may filter out one or more candidate words of CW123A that are identified as homographs not specified in the common homograph list (such that only a subset of all homographs may require editing, assuming that some homographs are, for example, stop words).

[0043] The pronunciation module 122 can also selectively apply the learning model described above to CW123A to calculate the confidence score. In some examples (to reduce the utilization of computing resources), the pronunciation module 122 selectively applies the learning model to determine a confidence score (not to be understood in a precise mathematical sense including a zero element, but used to indicate less than all of the set) of each subset of CW123A. For example, the pronunciation module 122 can, in some examples, apply the learning model described above to CW123A to determine a confidence score of each candidate word of CW123A that is a homograph. The pronunciation module 122 can then eliminate one or more candidate words of CW123A based on the confidence score of each candidate word of CW123A that is a homograph (e.g., by comparing the confidence score to a threshold confidence score).

[0044] Additionally, the pronunciation module 122 may include a named entity sub-model that identifies one or more candidate words of CW123A that are named entities not specified in the common named entity list. The named entity sub-model may dynamically update the common named entity list (e.g., via machine learning based on subsequent edits, editor / user phonetic pronunciation input, etc.). A named entity refers to a proper noun, such as a name, place, company name, trade name, etc. The pronunciation module 122 may filter out one or more candidate words of CW123A that are identified as named entities not specified in the common homograph list (e.g., because not all named entities are difficult to pronounce).

[0045] The pronunciation module 122 may further include an incomprehensibility sub-model that identifies one or more candidate words of CW123A that have high incomprehensibility (in terms of pronunciation and as measured against an incomprehensibility threshold). That is, the incomprehensibility sub-model may include a language model that determines the incomprehensibility of each candidate word of CW123A. The pronunciation module 122 may eliminate one or more candidate words of CW123A based on the associated incomprehensibility. For example, the pronunciation module 122 may compare each determined incomprehensibility with an incomprehensibility threshold, and may eliminate the associated candidate word of CW123A when the determined incomprehensibility exceeds the incomprehensibility threshold.

[0046] In this regard, the pronunciation module 122 may represent one of several models that apply sequences to several different (possibly overlapping, at least) sub-models to filter the CW 123A to generate the RCW 123B. As noted above, one or more of the sub-models may be adaptive, in that the sub-models may use machine learning to continually adapt to changing text criteria and editor feedback (even to tailor to individual editor preferences). In some examples, the sub-models may include binary classification models.

[0047] After generating RCW 123B through application of the sub-model, the pronunciation module 122 can annotate the LFTD 119 to obtain an annotated text document that identifies RCW 123B, which may be represented by an audiobook 125 in the example of FIG. 1. The pronunciation module 122 can annotate the LFTD 119 by populating each of the occurrences of RCW 123B at the start and end positions of each candidate word in the LFTD 119 and linking each of RCW 123B back to the LFTD 119. The pronunciation module 122 can also invoke the TTS module 124 to provide each of RCW 123B with one or more pronunciation candidates, which may be ordered according to their respective confidence scores. The pronunciation module 122 can also associate each of the one or more pronunciations with a respective location in the LFTD 119. In this manner, the pronunciation module 122 can automatically form an audiobook 125 that includes an annotated version of the LFTD 119.

[0048] The pronunciation module 122 cannot edit the underlying text of the LFTD 119, but merely adds annotations (perhaps via markup text) that facilitate better pronunciation by the automatic TTS algorithm. By avoiding text editing of the LFTD 119, the pronunciation module 122 allows for the reading of the actual text, but the automatic TTS algorithm can utilize the annotations to allow the editor to make an informed decision regarding the correct pronunciation, which may be randomly changed by the programmer employed by the automatic TTS algorithm, if the reader prefers an audiobook experience such as one read by a narrator. Thus, various aspects of the technology enable a guided pronunciation experience on a computing device, where a human editor utilizes a streamlined (through filtering) pronunciation editor to generate annotations, resulting in a guided TTS with generally improved pronunciation compared to unguided TTS synthesis.

[0049] After generating the audiobook 125, the processor 104 can then invoke the UI module 118, which can generate a GUI that outputs at least a portion of the annotated text document contained within the audiobook 125 that identifies at least one candidate word of the RCW 123B. The UI module 118 can output the GUI via the display 102 and / or the communication unit 112 (again, e.g., in the context of a distributed server-client system). Via the GUI, a human editor can interact with the computing device 100 to edit the pronunciation via a visual representation of the pronunciation checker, and possibly input a verbal narration for a preferred pronunciation of the RCW 123B, as described in more detail below.

[0050] In this regard, filtering of candidate words can reduce time spent on unimportant mispronunciations and focus attention on words that are difficult to pronounce (e.g., proper nouns, words that are ambiguous in terms of inflection, etc.), intentionally misspelled words, words that are spelled the same but pronounced differently (also referred to as homographs), etc. In this way, the human editor can focus on candidate words that may significantly impair the audio experience. In some cases, the editor can interact with the computing device 100 to input a verbal pronunciation that the computing device then uses to correct the automatic text-to-speech pronunciation (e.g., select the pronunciation that best matches the input verbal pronunciation).

[0051] In this manner, various aspects of the present technology can facilitate more efficient execution of the computing device 100 in terms of computing resources (e.g., processor cycles, memory usage, memory bus bandwidth usage, and associated power consumption) while also improving the user experience for editing potential mispronunciations by the automatic text-to-speech algorithm. The present technology can reduce the use of computing resources by filtering candidate words to allow the editor to focus on words that are difficult to pronounce, which results in less processing during the editing process, thereby reducing the use of computing resources. Additionally, these techniques can provide a better editing experience by focusing attention on words that are difficult to pronounce, while the automatic text-to-speech algorithm improves the pronunciation of the candidate words (which produces a higher quality audiobook experience compared to an automatic text-to-speech algorithm that does not facilitate review and adjustment of pronunciation).

[0052] 2A-2C are diagrams illustrating example user interfaces with which a human editor may interact to facilitate efficient pronunciation checking of a long-form text document, in accordance with various aspects of the techniques described in this disclosure. In the example of FIG. 2A, user interface 200A may represent an example of an interactive user interface (e.g., an interactive graphical user interface—iGUI) generated and output by UI module 118.

[0053] User interface 200A may represent an interactive user interface with which a user (e.g., a human editor) can interact to edit the pronunciation by the automated narrator. User interface 200A may include an e-book overview pane 202 that includes an audiobook text selector 204, among other selectors, that allows viewing of book information (such as author, publisher, publication year), contents (e.g., table of contents), pricing information, a summary of the e-book (as would appear on the inside paper or back of the book), and an event history. In the example of FIG. 2A, the user has selected audiobook text selector 204.

[0054] In response to selecting the audiobook text selector 204, the user interface 200A presents a section pane 206, an audiobook text pane 208, an auto-narrator voice selector 210, a save button 212, and a publish button 214. The section pane 206 may present a list of sections (similar to a table of contents). In the section pane 206, the user interface 200A presents several lined sections (e.g., cover page, title page, copyright, contents, endpaper photos, paperback introduction...) along with unlined sections (e.g., preface, Becoming Me, Chapter 1, etc.). The UI module 118 may interface with the various modules 120-124 to identify which sections should be excluded from the audiobook 125. The pre-processing module 120 may be configured to identify which sections should be excluded from the audio narration and pass a list of sections to be excluded from the audio narration to the UI module 118. The UI module 118 can then generate the section pane 206 based on the list of excluded sections.

[0055] The audiobook text pane 208 can reproduce the text of the audiobook 125 based on the LFTD 119. The audiobook text pane 208 can include an exclude toggle 216 and a play button 218. The exclude toggle 216 can toggle the exclusion of a section or subsection within the audiobook 125 (i.e., in the example of FIG. 2A, the Chapter 1 subsection of the Becoming Me section). The play button 218 can initiate playback of the audiobook text shown in the audiobook text pane 208 using a selected auto-narrator voice. The user can interact with the auto-narrator voice selector 210 to select between different auto-narrator voices (which may have specific genders and accents). The user can interact with the audiobook text pane 208 to edit the underlying audiobook text and select a save button 212 to save the edits or select a publish button 214 to publish the audiobook 125 (e.g., to an online store, such as an online audiobook store) when editing is complete.

[0056] 2B, user interface 200B may represent user interface 200A after a user selects word 220 (e.g., the user clicks the right mouse button while hovering over word 220, a so-called "right click" of the mouse). Word 220 may represent an example of RCW 123B, which is a suitable entity identified by the appropriate entity sub-model of pronunciation module 122. In response to selecting word 220, UI module 118 may update user interface 200A to include edit pane 230 (thereby transitioning user interface 200A to user interface 200B).

[0057] The edit pane 230 includes an edit pronunciation button 232 and a play word button 234. The edit pronunciation button 232 can allow a user to edit the pronunciation of the word 220, while the play word button 234 can cause the UI module 118 to present the audio data provided by the TTS module 124 for the word 220 in the audiobook 125 (using the selected automatic narrator voice).

[0058] 2C, user interface 200C depicts an example of user interface 200B after a user selects edit pronunciation button 232. User interface 200C still displays audiobook text pane 208, but displays edit pronunciation pane 240 that includes play / pause button 242 for initiating playback of individual RCWs 123B shown in list 244.

[0059] 2C, the user has selected entry 246 in list 244 of RCW 123B that contains the word "mavjee," which represents a non-lexicon word. The user also selects play / pause button 242, causing the UI module 118 to play the word span in which entry 246 appears. The UI module 118 may highlight the word span in the audiobook text pane 208, shown as highlighting 260.

[0060] In this regard, the UI module 118 can receive input (e.g., selection of the play / pause button 242) via the interactive user interface 200C selecting at least one candidate word (represented by entry 246) of the RCW 123B. In response to this input, the UI module 118 can obtain pronunciation audio data representing an oral pronunciation of the at least one candidate word (provided by the TTS module 124 and linked by the pronunciation module 122, as described above) of the RCW 123B. The UI module 118 can then output the pronunciation audio data for playback via a speaker (loudspeaker, internal speaker, headphone speaker, etc.).

[0061] In either case, the user may listen to the pronunciation of the entry 246 in the context of the highlighted word span as highlighting 260 to determine whether the pronunciation by the auto-narrator is appropriate in the context of the highlighted word span. If the user determines that the pronunciation is not appropriate, the user may edit the pronunciation. That is, the entry 246 includes an edit button 248 and a status 250A with the number of occurrences of the word "mavjee" in the LFTD 119 (as determined by the pronunciation module 122). The user may select the edit button 248 to edit the pronunciation (e.g., by providing a phonetic spelling of the word "mavjee," providing a verbal pronunciation, etc.). The status 250A may indicate that the entry 246 is currently playing. Once edited, the status 250A may shift to a status 250B (although shown with respect to another entry, such a status 250B is illustrative of the shift in status that may occur with respect to the status 250A of the entry 246).

[0062] 3A-3C are additional diagrams illustrating example user interfaces with which a human editor may interact to facilitate efficient pronunciation checking of a long-form text document, according to various aspects of the techniques described in this disclosure. In the example of FIG. 3A, user interface 300A may represent another example of an interactive user interface generated and output by UI module 118. User interface 300A may be similar to user interface 200C shown in the example of FIG. 2C in that user interface 300A includes a section pane 306, an audiobook text pane 308, an auto-narrator voice selector 310, a save button 312, and a pronunciation editing pane 340A.

[0063] Panes 306, 308, and 340A may be similar, if not substantially similar, to their respective panels 206, 208, and 240. However, section pane 306 does not include lined sections, but rather omits sections that are not subject to automatic TTS narration. Audiobook text pane 308 is substantially similar to audiobook text pane 208, in that such audiobook text pane 308 also includes an exclude toggle 316 that functions similarly to exclude toggle 216, and a play button 318 that functions similarly to play button 218 (shown in the example of FIG. 2A).

[0064] The pronunciation edit pane 340A is similar to the pronunciation edit pane 240, but differs in that the pronunciation edit pane 340A includes a review of all potential pronunciation errors. The pronunciation edit pane 340A may, as a result, be referred to as the pronunciation review pane 340A. The pronunciation review pane 340A includes a review entry 370 that specifies the number of RCWs 123B that are predicted to require review (e.g., based on the confidence score being less than a threshold confidence score).

[0065] 3A , user interface 300A includes an auto narrator voice selector 310 that is not identical to, but functions similarly to, auto narrator voice selector 210. User interface 300A also includes a save button 312 that functions similarly to save button 212. User interface 300A further includes a create audio file button 314, which may be similar to publish button 314 in that an audiobook 125 is created, but differs in that the audiobook 125 is not immediately published in an online store. Instead, selecting the create audio file button 314 may result in the audiobook 125 being generated without being immediately published in an online store.

[0066] 3B, the pronunciation review pane 340B represents the result of a user selecting a review entry 370 in the pronunciation review pane 340A. As such, the UI module 118 can transition from the pronunciation review pane 340A to the pronunciation review pane 340B upon receiving the selection of the review entry 370.

[0067] The pronunciation review pane 340B may include tabs 380A and 380B. A line under tab 380A indicates whether the user selected tab 380A or the UI module 118 defaulted to the selection of tab 380A. Tab 380B, when selected, may cause the review pane 340B of the UI module 118 to be populated with all specific review entries previously reviewed by the user. In response to the selection of tab 380A (default or otherwise), the UI module 118 populates the pronunciation review pane 340B with specific review entries 382A-382D ("specific review entries 382"). Each of the specific review entries 382 may represent a particular pronunciation error and display a number of instances of each error, a play button 384, a confirm all button 386, and a review button 388 (each of which is shown only for the particular review entry 382A for ease of illustration).

[0068] The play button 384 can be configured to configure the UI module 118 to provide audio data associated with one or more instances of the particular review entry 382A for playback via the speaker. The confirm all button 386 can be configured to confirm the current pronunciation (e.g., existing audio data) for each of the instances of the particular review entry 382A (i.e., all 12 instances in the example of the particular review entry 382A). The review button 388 can be configured to transition from the pronunciation review pane 340B to a pronunciation review pane 340C (shown in the example of FIG. 3C) that facilitates review of the pronunciation of the associated particular review entry 382. In this regard, the review button 388 can enable individual review (or in other words, case-by-case review) of particular instances of the mispronounced word.

[0069] In the example of Figure 3C, user interface 300C is similar to user interface 300A, except that user interface 300C includes a pronunciation review pane 340C that shows the results of a user selecting a particular review entry 382A shown in pronunciation review pane 340B of Figure 3B. Pronunciation review pane 340C includes a back button 390, a play button 384, a confirm button 391, a phonetic text entry field 392, a record button 393, a play button 394, an apply to instance button 395, an apply to all instances button 396, and an instance selector 398.

[0070] The back button 390 can be configured to cause the UI module 118 to return from the pronunciation review pane 340C to the pronunciation review pane 340B shown in the example of FIG. 3B. The play button 384 can be configured to cause the UI module 118 to output audio data associated with a particular instance in the review for playback through the speaker. The apply instance button 395 can be configured to cause the UI module 118 to apply the currently selected audio data to a particular instance in the review. The phonetic text input field 392 allows the user to type a phonetic text phrase that will result in different pronunciations, which the user can hear by selecting the play button 394. Thus, the phonetic text input field 392 can be configured to cause the UI module 118 to provide any entered text to the TTS module 124 upon selection of the play button 394, which the TTS module 124 can synthesize into audio data. The UI module 118 can then output the synthesized audio data for playback through the speaker.

[0071] Similarly, a record button 393 allows a user to speak the pronunciation of a particular instance, thereby providing pronunciation audio data to the UI module 118. The UI module 118 can provide the pronunciation audio data to the pronunciation module 122, which can use the pronunciation audio data to select one of the available pronunciations synthesized by the TTS module 124 and associated with any one of the pronunciations for any of the instances. Selecting the play button 394 after providing the pronunciation audio data can be configured to cause the UI module 118 to obtain the selected pronunciation based on the pronunciation audio data and output the pronunciation for playback by the speaker.

[0072] In this regard, the UI module 118 can receive input via the interactive user interface selecting at least one candidate word of the RCW 123B (i.e., "Reeaallyy" in this example), selected via input of a selection of a particular review entry 382A. The UI module 118 can also receive an oral pronunciation of the candidate word of the RCW 123B (in response to a selection of the record button 393). The UI module 118, interfacing with the pronunciation module 122, can identify a potential pronunciation from a number of different potential pronunciations based on the oral pronunciation. The pronunciation module 122 can then associate the potential pronunciation with the at least one candidate word of the RCW 123B.

[0073] The apply button 395 can be configured to cause the UI module 118 to associate the current pronunciation with a particular instance of the candidate word. Apply to all instances 396 can be configured to cause the UI module 118 to associate the current pronunciation of the selected instance with all instances of the candidate word (again, "Reeaallyy" in this example). The instance selector 398 can be configured to cause the UI module 118 to switch between different instances of the candidate word, thereby allowing the user to select a different instance for review.

[0074] 4 is a flowchart illustrating an example operation of an example computing device configured to perform pronunciation editing of an automatic text-to-speech algorithm, according to one or more aspects of the present disclosure. The processor 104 can first invoke the UI module 118, which can generate a UI (e.g., a graphical UI (GUI)) to facilitate uploading and / or distribution of the LFTD 119. The UI module 118 can interface with the display 102 to present the GUI via the display 102. In a distributed computing system, the UI module 118 can interface with the communication unit 112 to provide the GUI to a client device (e.g., server, client distributed system). In either case, a human editor can interact with the GUI to provide the LFTD 119 to the computing device 100, which can receive and store the LFTD 119 in the storage system 106 (400).

[0075] In response to receiving the LFTD 119, the processor 104 can then invoke the preprocessing module 120, which can process the LFTD 119 to generate a synthesized document 121 (402). The preprocessing module 120 can share the fragments of the LFTD 119 into manageable sized chunks (e.g., N sentences) and aggregate each chunk with markup text. The preprocessing module 120 can then analyze each aggregated chunk using text normalization to identify spans of words. Text normalization refers to the process of detecting multi-word expressions across large chunks of text. The preprocessing module 120 can then analyze the results of the text normalization to generate contiguous, non-overlapping spans of the input text, which are aggregated chunks in this example. The preprocessing module 120 can then output the contiguous, non-overlapping spans determined for each aggregated chunk as each of the synthesized documents 121.

[0076] In response to obtaining the synthesized document 121, the processor 104 may then invoke the pronunciation model 122, which may process the synthesized document 121 to identify CWs 123A (representing a first plurality of candidate words) predicted to be mispronounced during automatic text-to-speech processing of the LFTD 119 (404). The pronunciation module 122 may then filter the CWs 123A to exclude one or more candidate words from the CWs 123A to obtain RCWs 123B (representing a second plurality of candidate words) having fewer candidate words than the CWs 123A (406). After generating the RCWs 123B through application of the above-mentioned sub-models, the pronunciation module 122 may annotate the LFTD 119 to obtain an annotated text document identifying the RCWs 123B, which may be represented by an audiobook 125 in the example of FIG. 1 (408).

[0077] Upon generating the audiobook 125, the processor 104 can then invoke the UI module 118, which can generate a GUI that outputs at least a portion of the annotated text document contained within the audiobook 125 that identifies at least one candidate word of the RCW 123B (410). The UI module 118 can output the GUI via the display 102 and / or the communication unit 112 (again, e.g., in the context of a distributed server-client system). Via the GUI, a human editor can interact with the computing device 100 to edit the pronunciation via a visual representation of the pronunciation checker, and possibly input a verbal narration for a preferred pronunciation of the RCW 123B, as described in more detail below.

[0078] In this regard, various aspects of the technology enable the following examples.

[0079] Example 1. A method comprising: processing words in a text document to identify a first plurality of candidate words predicted to be mispronounced during automatic text-to-speech processing of the text document; filtering the first plurality of candidate words to exclude one or more candidate words of the first plurality of candidate words to obtain a second plurality of candidate words having fewer candidate words than the first plurality of candidate words; annotating the text document to obtain an annotated text document identifying the second plurality of candidate words; and outputting at least a portion of the annotated text document identifying at least one candidate word of the second plurality of candidate words.

[0080] Example 2. The method of Example 1, wherein filtering the first plurality of candidate words includes identifying one or more candidate words of the first plurality of candidate words that are stop words, and excluding the one or more candidate words of the first plurality of candidate words that are stop words to obtain the second plurality of candidate words.

[0081] Example 3. The method of any combination of Examples 1 and 2, wherein filtering the first plurality of candidate words includes determining a candidate word count for the first plurality of candidate words indicating the number of times each candidate word appears in the first plurality of candidate words, and removing the one or more candidate words from the first plurality of candidate words having a candidate word count above a threshold.

[0082] Example 4. The method of any combination of Examples 1-3, wherein filtering the first plurality of candidate words includes identifying the one or more candidate words of the first plurality of candidate words that are homographs not specified in a common homograph list, and excluding the one or more candidate words of the first plurality of candidate words identified as homographs not specified in the common homograph list.

[0083] Example 5. The method of any combination of Examples 1-4, wherein filtering the first plurality of candidate words includes identifying one or more of the candidate words of the first plurality of candidate words that are named entities specified in a common named entity list, and filtering out the one or more candidate words of the first plurality of candidate words that are identified as named entities specified in the common named entity list.

[0084] Example 6. A method according to any combination of Examples 1-5, wherein filtering the first plurality of candidate words includes applying a language model to the first plurality of candidate words to determine an intelligibility of each candidate word of the first plurality of candidate words, and eliminating the one or more candidate words of the first plurality of candidate words based on the intelligibility of each candidate word of the first plurality of candidate words.

[0085] Example 7. A method according to any combination of Examples 1-6, wherein filtering the first plurality of candidate words includes applying a learning model to the first plurality of candidate words to determine a confidence score for each candidate word of the first plurality of candidate words that is a homograph, and eliminating the one or more candidate words of the first plurality of candidate words based on the confidence score for each candidate word of the first plurality of candidate words that is a homograph.

[0086] Example 8. The method according to any combination of Examples 1-7, wherein outputting at least the portion of the annotated text document includes displaying at least the portion of the annotated text document via an interactive user interface, the method further including receiving input via the interactive user interface selecting the at least one candidate word of the second plurality of candidate words, and in response to receiving the input, obtaining pronunciation audio data representing an oral pronunciation of the at least one candidate word of the second plurality of candidate words, and outputting the pronunciation audio data for playback via a speaker.

[0087] Example 9. The method according to any combination of Examples 1-8, wherein outputting at least the portion of the annotated text document includes displaying at least the portion of the annotated text document via an interactive user interface, the method further including receiving an input via the interactive user interface selecting the at least one candidate word of the second plurality of candidate words, receiving an oral pronunciation of the at least one candidate word of the second plurality of candidate words, identifying a potential pronunciation from a plurality of potential pronunciations based on the oral pronunciation, and associating the potential pronunciation with the at least one candidate word of the second plurality of candidate words.

[0088] Example 10. A computing device including a memory configured to store a text document and one or more processors, the one or more processors configured to: process words in the text document to identify a first plurality of candidate words predicted to be mispronounced during automatic text-to-speech processing of the text document; filter the first plurality of candidate words to exclude one or more candidate words of the first plurality of candidate words to obtain a second plurality of candidate words having fewer candidate words than the first plurality of candidate words; annotate the text document to obtain an annotated text document identifying the second plurality of candidate words; and output at least a portion of the annotated text document identifying at least one candidate word of the second plurality of candidate words.

[0089] Example 11. The computing device of Example 10, wherein the one or more processors, when configured to filter the first plurality of candidate words, are configured to: identify the one or more candidate words of the first plurality of candidate words that are stop words; and exclude the one or more candidate words of the first plurality of candidate words that are stop words to obtain the second plurality of candidate words.

[0090] Example 12. The computing device of any combination of Examples 10 and 11, wherein the one or more processors, when configured to filter the first plurality of candidate words, are configured to: determine a candidate word count for the first plurality of candidate words indicating the number of times each candidate word appears in the first plurality of candidate words; and exclude the one or more candidate words from the first plurality of candidate words that have the candidate word count exceeding a threshold.

[0091] Example 13. The computing device of any combination of Examples 10-12, wherein the one or more processors, when configured to filter the first plurality of candidate words, are configured to: identify the one or more candidate words of the first plurality of candidate words that are homographs not specified in a common homograph list; and exclude the one or more candidate words of the first plurality of candidate words identified as homographs not specified in the common homograph list.

[0092] Example 14. The computing device of any combination of Examples 10-13, wherein the one or more processors, when configured to filter the first plurality of candidate words, are configured to: identify the one or more candidate words of the first plurality of candidate words that are named entities specified in a common named entity list; and exclude the one or more candidate words of the first plurality of candidate words that are identified as named entities specified in the common named entity list.

[0093] Example 15. A computing device described in any combination of Examples 10 to 14, wherein the one or more processors, when configured to filter the first plurality of candidate words, are configured to: apply a language model to the first plurality of candidate words to determine an obfuscation level for each candidate word of the first plurality of candidate words; and eliminate the one or more candidate words of the first plurality of candidate words based on the obfuscation level for each candidate word of the first plurality of candidate words.

[0094] Example 16. A computing device described in any combination of Examples 10 to 15, wherein the one or more processors, when configured to filter the first plurality of candidate words, are configured to: apply a learning model to the first plurality of candidate words to determine a confidence score for each candidate word of the first plurality of candidate words that is a homograph; and eliminate the one or more candidate words of the first plurality of candidate words based on the confidence score for each candidate word of the first plurality of candidate words that is a homograph.

[0095] Example 17. The computing device of any combination of Examples 10-16, wherein the one or more processors are configured to display at least the portion of the annotated text document via an interactive user interface when configured to output at least the portion of the annotated text document, and the one or more processors are further configured to: receive input via the interactive user interface selecting the at least one candidate word of the second plurality of candidate words; and, in response to receiving the input, obtain pronunciation audio data representing an oral pronunciation of the at least one candidate word of the second plurality of candidate words; and output the pronunciation audio data for playback via a speaker.

[0096] Example 18. The computing device of any combination of Examples 10-17, wherein the one or more processors are configured to display at least the portion of the annotated text document via an interactive user interface when configured to output at least the portion of the annotated text document, and the one or more processors are further configured to: receive input via the interactive user interface selecting the at least one candidate word of the second plurality of candidate words; receive an oral pronunciation of the at least one candidate word of the second plurality of candidate words; identify a potential pronunciation from a plurality of potential pronunciations based on the oral pronunciation; and associate the potential pronunciation with the at least one candidate word of the second plurality of candidate words.

[0097] Example 19. A non-transitory computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to process words in a text document to identify a first plurality of candidate words predicted to be mispronounced during an automatic text-to-speech process of the text document; filter the first plurality of candidate words to exclude one or more candidate words of the first plurality of candidate words to obtain a second plurality of candidate words having fewer candidate words than the first plurality of candidate words; annotate the text document to obtain an annotated text document identifying the second plurality of candidate words; and output at least a portion of the annotated text document identifying at least one candidate word of the second plurality of candidate words.

[0098] Example 20. The non-transitory computer-readable storage medium of Example 19, wherein the instructions that, when executed, cause the one or more processors to filter the plurality of candidate words include instructions that, when executed, cause the one or more processors to identify the one or more candidate words of the first plurality of candidate words that are homographs not specified in a common homograph list, and exclude the one or more candidate words of the first plurality of candidate words identified as homographs not specified in the common homograph list.

[0099] Example 21. A non-transitory computer-readable storage medium having instructions stored thereon that, when executed, cause one or more processors to perform a method described by any combination of Examples 1-9.

[0100] Example 22. An apparatus comprising means for performing each step of the method described by any combination of Examples 1-9.

[0101] In one or more examples, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over a computer-readable medium as one or more instructions or code and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium, including any medium that facilitates transfer of a computer program from one place to another, for example, according to a communication protocol. As such, a computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to read instructions, code, and / or data structures for implementing the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0102] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM, or other optical, magnetic, or other magnetic disk storage, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Any connection is also properly termed a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory, tangible storage media. As used herein, disk and disc include compact discs (CDs), laser discs, optical disks, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically while discs reproduce data optically using lasers. Combinations of the above should also be included within the scope of computer readable media.

[0103] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor" as used herein may refer to any of the foregoing structures, or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided in dedicated hardware and / or software modules. Also, the techniques may be implemented entirely in one or more circuits or logic elements.

[0104] The techniques of the present disclosure may be implemented in a variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chipset). Various components, modules, or units are described in the present disclosure to highlight functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined within a hardware unit or may be provided by a collection of operating hardware units, including one or more processors as described above, in conjunction with appropriate software and / or firmware.

[0105] Various examples of the present disclosure have been described. Any combination of the described systems, operations, or functions is contemplated. These and other examples are within the scope of the following claims.

Claims

1. processing words in a text document to identify a first plurality of candidate words predicted to be mispronounced during an automatic text-to-speech process of the text document; filtering the first plurality of candidate words to exclude one or more candidate words of the first plurality of candidate words to obtain a second plurality of candidate words having fewer candidate words than the first plurality of candidate words; annotating the text document to obtain an annotated text document identifying the second plurality of candidate words; and outputting at least a portion of the annotated text document identifying at least one candidate word of the second plurality of candidate words.

2. Filtering the first plurality of candidate words includes: identifying one or more candidate words of the first plurality of candidate words that are stop words; and eliminating the one or more candidate words of the first plurality of candidate words that are the stop words to obtain the second plurality of candidate words.

3. Filtering the first plurality of candidate words includes: determining a candidate word count for the first plurality of candidate words indicating a number of times each candidate word appears in the first plurality of candidate words; and removing the one or more candidate words from the first plurality of candidate words having the candidate word count above a threshold.

4. Filtering the first plurality of candidate words includes: identifying one or more candidate words of the first plurality of candidate words that are homographs not specified in a common homograph list; and removing the one or more candidate words of the first plurality of candidate words identified as a homograph not specified in the common homograph list.

5. Filtering the first plurality of candidate words includes: identifying one or more candidate words of the first plurality of candidate words that are named entities specified in a common named entity list; and removing the one or more candidate words of the first plurality of candidate words that are identified as named entities specified in the common named entity list.

6. Filtering the first plurality of candidate words includes: applying a language model to the first plurality of candidate words to determine an obfuscation level for each candidate word of the first plurality of candidate words; and eliminating one or more candidate words of the first plurality of candidate words based on the obscurity of each candidate word of the first plurality of candidate words.

7. Filtering the first plurality of candidate words includes: applying a learning model to the first plurality of candidate words to determine a confidence score for each candidate word of the first plurality of candidate words that is a homograph; and eliminating one or more candidate words of the first plurality of candidate words that are homographs based on the confidence score for each candidate word of the first plurality of candidate words that are homographs.

8. Outputting at least the portion of the annotated text document includes displaying at least the portion of the annotated text document via an interactive user interface, the method comprising: receiving an input via the interactive user interface selecting the at least one candidate word of the second plurality of candidate words; obtaining pronunciation audio data representative of an oral pronunciation of the at least one candidate word of the second plurality of candidate words in response to receiving the input; 2. The method of claim 1, further comprising: outputting the pronunciation audio data for playback through a speaker.

9. Outputting at least the portion of the annotated text document includes displaying at least the portion of the annotated text document via an interactive user interface, the method comprising: receiving an input via the interactive user interface selecting the at least one candidate word of the second plurality of candidate words; receiving a spoken pronunciation of the at least one candidate word of the second plurality of candidate words; identifying a potential pronunciation from a plurality of potential pronunciations based on the oral pronunciation; and associating the potential pronunciation with the at least one candidate word of the second plurality of candidate words.

10. a memory configured to store a text document; one or more processors, the one or more processors comprising: processing words in the text document to identify a first plurality of candidate words predicted to be mispronounced during an automatic text-to-speech process of the text document; filtering the first plurality of candidate words to exclude one or more candidate words of the first plurality of candidate words to obtain a second plurality of candidate words having fewer candidate words than the first plurality of candidate words; annotating the text document to obtain an annotated text document identifying the second plurality of candidate words; and outputting at least a portion of the annotated text document identifying at least one candidate word of the second plurality of candidate words.

11. When the one or more processors are configured to filter the first plurality of candidate words, identifying one or more candidate words of the first plurality of candidate words that are stop words; and filtering out the one or more candidate words of the first plurality of candidate words that are stop words to obtain the second plurality of candidate words.

12. When the one or more processors are configured to filter the first plurality of candidate words, determining a candidate word count for the first plurality of candidate words indicating a number of times each candidate word appears in the first plurality of candidate words; and removing the one or more candidate words from the first plurality of candidate words having the candidate word count above a threshold.

13. When the one or more processors are configured to filter the first plurality of candidate words, identifying one or more candidate words of the first plurality of candidate words that are homographs not specified in a common homograph list; and removing the one or more candidate words of the first plurality of candidate words identified as a homograph not specified in the common homograph list.

14. When the one or more processors are configured to filter the first plurality of candidate words, identifying one or more candidate words of the first plurality of candidate words that are named entities specified in a common named entity list; and filtering out the one or more candidate words of the first plurality of candidate words that are identified as named entities specified in the common named entity list.

15. When the one or more processors are configured to filter the first plurality of candidate words, applying a language model to the first plurality of candidate words to determine an obfuscation level for each candidate word of the first plurality of candidate words; and eliminating one or more candidate words of the first plurality of candidate words based on the obscurity of each candidate word of the first plurality of candidate words.

16. When the one or more processors are configured to filter the first plurality of candidate words, applying a learning model to the first plurality of candidate words to determine a confidence score for each candidate word of the first plurality of candidate words that is a homograph; and eliminating one or more candidate words of the first plurality of candidate words that are homographs based on the confidence score for each candidate word of the first plurality of candidate words that are homographs.

17. when the one or more processors are configured to output at least the portion of the annotated text document, the one or more processors are configured to display at least the portion of the annotated text document via an interactive user interface; the one or more processors: receiving an input via the interactive user interface selecting the at least one candidate word of the second plurality of candidate words; obtaining pronunciation audio data representative of an oral pronunciation of the at least one candidate word of the second plurality of candidate words in response to receiving the input; 11. The computing device of claim 10, further configured to: output the phonetic audio data for playback through a speaker.

18. when the one or more processors are configured to output at least the portion of the annotated text document, the one or more processors are configured to display at least the portion of the annotated text document via an interactive user interface; the one or more processors: receiving an input via the interactive user interface selecting the at least one candidate word of the second plurality of candidate words; receiving a spoken pronunciation of the at least one candidate word of the second plurality of candidate words; identifying a potential pronunciation from a plurality of potential pronunciations based on the oral pronunciation; The computing device of claim 10 , further configured to: associate the potential pronunciation with the at least one candidate word of the second plurality of candidate words.

19. A non-transitory computer-readable storage medium having instructions stored thereon, the instructions, when executed, causing one or more processors to: processing words in a text document to identify a first plurality of candidate words predicted to be mispronounced during an automatic text-to-speech process of the text document; filtering the first plurality of candidate words to exclude one or more candidate words of the first plurality of candidate words to obtain a second plurality of candidate words having fewer candidate words than the first plurality of candidate words; annotating the text document to obtain an annotated text document identifying the second plurality of candidate words; and outputting at least a portion of the annotated text document identifying at least one candidate word of the second plurality of candidate words.

20. The instructions, which when executed, cause the one or more processors to filter the plurality of candidate words, may, when executed, cause the one or more processors to: identifying one or more candidate words of the first plurality of candidate words that are homographs not specified in a common homograph list; and eliminating the one or more candidate words of the first plurality of candidate words identified as a homograph not specified in the common homograph list.

Images