Linguistically driven automated text formatting

Abstract

To provide a system and a technique for linguistically-driven automated text formatting.SOLUTION: Data representing a linguistic structure of an input text is received from Natural Language Processing (NLP) services, including but not limited to constituents, dependencies, and coreference relations. A text model of the input text may be built using linguistic components and linguistic relations. Cascade rules may be applied to the text model to generate a cascaded text data structure. Cascaded data may be displayed on various media, including phones, tablets, laptops, monitors, and VR / AR devices. The cascaded data may be presented in dual screen formats to promote further accurate and efficient reading comprehension, greater ease in teaching grammatical structures of native languages and foreign languages, and tools for improving reading-related disabilities.SELECTED DRAWING: Figure 10

Description

[Technical Field]

[0001] Embodiments described herein relate generally to machine-automated text processing driven by large-scale natural language processing (NLP) techniques derived from theoretical linguistics, and in some embodiments more specifically to component and dependency analysis to produce cascaded text for the purpose of improving reading comprehension. [Background technology]

[0002] Standard text formatting involves presenting language in blocks with little formatting beyond basic punctuation and line breaks or indentations to indicate paragraphs. The alternative text format described herein presents text so that linguistic relationships are emphasized, providing support for the comprehension process, which can increase accuracy or reduce reading time. Summary of the Invention [Problem to be solved by the invention]

[0003] Provides cascading text formatting. [Means for solving the problem]

[0004] Cascaded text formatting transforms traditional block-shaped text into a cascading pattern with the goal of helping readers identify grammatical structures and related content. Text cascades make the syntax of a sentence visible. Syntactic units are the building blocks of sentences. When parsing natural language, the reader's mind must do more than simply "roughly slice and dice" a sentence into smaller strings of units. Rather, the reader's mind must identify dependencies between phrases and recognize how each phrase relates to larger phrases that contain it. Cascaded text formatting helps the reader identify these relationships within a sentence. The human mind's ability to construct sentences through the process of embedding linguistic units within other units allows language to express a myriad of meanings. Therefore, cascaded parsing patterns are intended to enable readers, when looking at a particular phrase, to immediately notice how that phrase relates to the phrases before and after it.

[0005] In the drawings, which are not necessarily drawn to scale, like numerals may describe like components in different figures. Like numerals with different letter suffixes may represent different instances of like components. The drawings illustrate generally, by way of example, but not by way of limitation, various embodiments discussed in this document. [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 illustrates an example of a parse tree that defines components to be used for linguistically driven automated text formatting, according to one embodiment. [Figure 2] FIG. 1 is a block diagram of an example of dependency analysis and cascaded text output for linguistically driven automated text formatting, according to one embodiment. [Figure 3]FIG. 1 is a block diagram of an example of dependency analysis and cascaded text output for linguistically driven automated text formatting, according to one embodiment. [Figure 4] FIG. 1 illustrates an example of cascaded text output for linguistically driven automated text formatting, according to one embodiment. [Figure 5] FIG. 1 illustrates an example of cascaded text output extended to identify denotatively linked phrases for linguistically driven automated text formatting, according to one embodiment. [Figure 6A] 1 illustrates a portion of text that has been segmented into multiple dependent segments based on grammatical information identified by a constituency and dependency parser, according to one embodiment. [Figure 6B] FIG. 1 illustrates a portion of text displayed with hierarchical placement specified for each segment via linguistically driven automated text formatting, according to one embodiment. [Figure 7] FIG. 1 is a block diagram of an example environment and system for linguistically driven automated text formatting, according to one embodiment. [Figure 8] FIG. 1 illustrates an example environment for receiving linguistic corrections to a cascade format for linguistically driven automated text formatting, according to one embodiment. [Figure 9] FIG. 1 illustrates an example of a system for cross-sentence coreference tracking for linguistically driven automated text formatting, according to one embodiment. [Figure 10] FIG. 1 is a data flow diagram for an example of a system for linguistically driven automated text formatting, according to one embodiment. [Figure 11] FIG. 1 illustrates an example of a method for linguistically driven automated text formatting, according to one embodiment. [Figure 12]FIG. 1 illustrates an example of a method for linguistically driven automated text formatting, according to one embodiment. [Figure 13] FIG. 1 illustrates an example of a method for linguistically driven automated text formatting, according to one embodiment. [Figure 14] FIG. 1 illustrates an example of a method for cascading text using machine learning classifiers for linguistically driven automated text formatting, according to one embodiment. [Figure 15] FIG. 1 illustrates an example of a method for training machine learning classifiers to cascade text for linguistically driven automated text formatting, according to one embodiment. [Figure 16] FIG. 10 illustrates an example of text transformations for displaying sentences in a cascaded format for linguistically driven automated text formatting, according to one embodiment. [Figure 17] FIG. 1 illustrates an example of a parsing structure for HyperText Markup Language (HTML) code tagged for cascaded text using natural language processing for linguistically driven automated text formatting, according to one embodiment. [Figure 18] FIG. 10 illustrates an example of generating cascaded text from a captured image using natural language processing for linguistically driven automated text formatting, according to one embodiment. [Figure 19] FIG. 1 illustrates an example of a method for generating cascaded text from captured images using natural language processing for linguistically driven automated text formatting, according to one embodiment. [Figure 20] FIG. 10 illustrates an example of converting text from a first display format to a second display format in an eyewear device using natural language processing for linguistically driven automated text formatting, according to one embodiment. [Figure 21]FIG. 1 illustrates an example of a method for converting text from a first display format to a second display format in an eyewear device using natural language processing for linguistically driven automated text formatting, according to one embodiment. [Figure 22] FIG. 10 illustrates an example of generating cascaded text using natural language processing as text is authored for linguistically driven automated text formatting, according to one embodiment. [Figure 23] FIG. 1 illustrates an example architecture for cascaded text user and publisher preference management that uses natural language processing based on feedback input for linguistically driven automated text formatting, according to one embodiment. [Figure 24] FIG. 1 illustrates an example of a method for cascaded text personalization using natural language processing based on feedback input for linguistically driven automated text formatting, according to one embodiment. [Figure 25] FIG. 10 illustrates an example of a dual display of cascaded text using natural language processing based on feedback input for linguistically driven automated text formatting, according to one embodiment. [Figure 26] FIG. 1 illustrates an example of a system for translating input text in a first language into cascaded output in a second language for linguistically driven automated text formatting, according to one embodiment. [Figure 27] FIG. 1 illustrates an example of a method for linguistically driven automated text formatting, according to one embodiment. [Figure 28] 1 is a block diagram illustrating an example of a machine in which one or more embodiments may be implemented. DETAILED DESCRIPTION OF THE INVENTION

[0007] The systems and methods discussed herein utilize linguistic analysis derived from linguistic theory to identify cascades. Such analysis is state-of-the-art in automated natural language processing (NLP), which allows the systems and methods discussed herein to leverage input provided by NLP technologies (hereinafter NLP services).

[0008] The systems and methods discussed herein use NLP services (e.g., constituency parsers, dependency parsers, coreference parsers, etc.) to parse incoming text into displays that highlight its underlying linguistic features. Display rules, including cascading rules, are then applied to these displays to make linguistic relationships more visible to the reader.

[0009] A linguistic constituent is a word or group of words that fulfills a particular function in a sentence. For example, in the sentence "John believed X," X can be replaced by a single word ("Mary") or ("facts"), or by a phrase ("the girl") or ("the girls with curls") or ("the girl who shouted loudly"), or by a complete clause ("the story was true."). In this case, all of these are equivalent to "John believed X." "The story was" is a constituent that fills the role of the direct object of "John believed." In particular, constituents have the property of completeness. "The story was" is not a constituent because it cannot stand alone as a grammatical unit. Similarly, "the girl who" or "the" are not constituents. In addition, constituents may be embedded within other constituents. For example, the phrase "the girls with curls" is a constituent, but so are "the girls" and "with curls." However, the phrase "girls with" is not a constituent because it cannot stand alone as a grammatical unit. Consequently, "girls with" is not capable of fulfilling either grammatical function, while the constituent phrases "the girls" or "with curls" are both eligible to fulfill the necessary grammatical functions in the sentence. Parts of speech are categories of word syntactic functions (e.g., noun, verb, preposition, etc.). Unlike parts of speech, which describe the function of a single word, constituencies describe sets of words that function as units to fulfill a particular grammatical role in a sentence (e.g., subject, direct object, etc.). The concept of "constituencies" therefore provides more information about how groups of words are related to each other within a sentence.

[0010] The systems and methods discussed herein implement component cascading, in which components are displayed according to a set of rules specifying various levels of indentation. The rules are jointly based on information from a constituency parser and a dependency parser. The constituency parser is an NLP service that uses a theory of phrase structure (e.g., X-bar theory) to identify components as just described. The dependency parser is an NLP service that provides labeled syntactic dependencies for each word in a sentence to describe the syntactic function held by that word (and the component it leads). The set of syntactic dependencies is enumerated by the Universal Dependencies Initiative (UD, http: / / universaldependencies.org), which aims to provide a consistent syntactic annotation standard across languages. In addition to English, syntactic analysis can support a variety of additional languages, including, by way of example and not limitation, Chinese (Simplified), Chinese (Traditional), French, German, Italian, Japanese, Korean, Portuguese, Russian, and Spanish.

[0011] Through implementing the process of text cascading, the systems and methods discussed herein provide visual clues to the underlying linguistic structure in the text. These clues serve a didactic function, and numerous embodiments are presented that utilize these clues to promote more accurate and efficient reading comprehension, greater ease in teaching grammatical structures, and tools for remediation of reading-related disabilities.

[0012] In one example, cascades are formed using line breaks and indentations based on constituency and dependency data obtained from the parsing operation. Cascade rules are applied so that precedence is placed on components that remain entirely on a single line or are presented as consecutive units in situations where device display limitations may prevent display on a single line. This facilitates easy identification of which groups of words play a role together in linguistic functions, allowing components to be more easily identified. Accurate language comprehension requires the ability to identify relationships between entities or concepts presented in text. A prerequisite for this is the ability to parse components (i.e., units of text that perform distinct grammatical functions). Evidence suggests that poor comprehenders have significant difficulty identifying syntactic boundaries that define components during both reading and oral production (e.g., Breen et al., 2006; Miller and Schwanenflugel, 2008). Moreover, boundary recognition is particularly important for complex syntactic structures of the kind found in expository texts (i.e., textbooks, newspapers, etc.). These facts suggest that the ability to identify syntactic boundaries in text is particularly important for reading comprehension, and that methods of signaling these boundaries can serve as an important aid for struggling readers. However, standard text presentation methods (i.e., presenting text in left-justified blocks) do not explicitly identify linguistic components or provide any means to support the process of doing so. The systems and methods discussed herein offer means of explicitly signaling syntactic boundaries and dependencies through visual cues such as line breaks (e.g., carriage returns, line feeds, etc.), indentation, color highlighting, italics, underlining, etc.

[0013] Figure 1 shows an example parse tree 100 that defines building blocks to be used for linguistically driven automated text formatting, according to one embodiment. Linguistic theory provides established diagnostic tests for linguistic building blocks (also called phrases) and formalisms for expressing relationships between them. These tests include, by way of example and not limitation, i) do-so / one substitution, ii) coordination, iii) topicalization, iv) ellipsis, v) cleft / pseudo-cleft sentence formation, vi) passivization, vii) wh-foregrounding, and viiii) right-branch node raising, ix) pronoun substitution, x) question-answering, xi) omission, and xii) adverbial intrusion.

[0014] An influential theory known as X'-theory (pronounced "X-bar theory") describes how phrases are created (Chomsky, 1970; Jackendoff, 1977). This theory abstracts across specific parts of speech (e.g., nouns, verbs, etc.) and asserts that all types of phrases, described as XPs or X-phrases (e.g., if X = noun, it is a noun phrase; if X = verb, it is a verb phrase, etc.), are created via three general binary branching rewrite rules. First, a phrase ("XP") consists of optional "designators" and required "X-bars," in any order. Second, an X-bar can be optionally composed of an X-bar and any type of adjunct allowed to modify an X-bar. Third, the X-bar consists of a required head of a phrase (e.g., a word of any part of speech) and, optionally, any number of complement phrases permitted by that head, occurring in any linear order.

[0015] These rules can be used to create parse tree 100 such that specifiers and X's, and heads and complements, are sister relationships, with XP at the highest position in the hierarchy and dominating X's, which in turn dominate X's. For example, the verb phrase "saw the girl" can be represented as a VP with a V-head and an NP-complement, and the NP-complement is represented as an N' with "the" in the specifier position, which is represented as a projection from the N-head, as shown in parse tree 105. Note that other adjunct phrases (such as prepositional phrases) can be further added to the N'-phrase of the NP, as in sentence 110, "saw the girl with the curls." Similarly, specifier phrases, such as the adverb "recently," can be added to V's, as shown in parse tree 110. Although the specifier instances are simplified in Figure 1, they are also understood to be XP-phrases.

[0016] The systems and methods discussed herein utilize these phrase structure rules to define the first part of a two-part process of constituent cueing. Part 1 defines how sentences are divided into smaller pieces for display. That is, sentences are broken into constituent pieces such that words are not separated from their governing XP phrases (e.g., as shown in FIG. 6A, element 1110 in FIG. 11, element 1205 in FIG. 12, etc.). For example, a break between "with" and "curl" cannot occur because "curl" is governed by the PP prefixed by "with." Similarly, "the" cannot appear on a separate line from "girl" because they are both governed by the NP prefixed by "girl." Therefore, for purposes of the cascade generator, segments are defined as XP-governed phrases that will stay together on a presentation line. A split between "saw" and "the" is possible because "the" is governed by a different XP phrase than "saw" (i.e., "saw" is governed by the VP and "the" is governed by the NP). Similarly, a split between "girl" and "with" is possible because "with" is governed by the PP and "girl" is governed by the NP. A line break occurs at the presence of an XP because this marks a new constituent. If the line length is too short to break between XPs, a line break occurs at the X level with accompanying visual formatting to indicate constituent continuation (including but not limited to parentheses, flush indentation, color coding, italic matching, etc.).

[0017] Exemplary embodiments of Part 1 may utilize other phrase structure theories (e.g., bare phrase structure) as representative language for delimiting constituents. Some aspects of the systems and methods discussed herein are that line breaks occur at constituent boundaries, and that constituent boundaries are established based on established linguistic diagnostic tests (e.g., substitutions, displacements, cleft sentence formation, questions, etc.). Constituent boundaries are constant regardless of the specific phrase structure representation.

[0018] In an exemplary embodiment, an automated constituent parser is used to provide constituents for use by the cascade generator, but the cascade generator is not dependent on the use of one or any particular automated parser, which may be viewed as an NLP service (e.g., constituent parser 1025 of NLP service 1020 discussed in FIG. 10). It should be appreciated that a variety of parsing techniques and services may be used to identify the constituents to be next processed by the cascade generator.

[0019] The second part of the constituent cascading operation specifies an indentation scheme to provide clues to linguistic structure (e.g., as shown in Figure 6B). These clues are based on the output of an automated dependency parser, which specifies the specific linguistic function of each word relative to each other word (e.g., as shown in element 1115 of Figure 11 or element 1210 of Figure 12). This function is used to determine the horizontal displacement (indentation) of the cascade. The clues can be represented by the horizontal displacement (indentation) of constituent phrases in the cascade, so that it is the phrase on a given line that is indented. The cascade refers to the entire result, including phrase boundaries, line breaks, and horizontal displacements.

[0020] Dependency analysis alone cannot identify linguistic components because the dependency parser assigns linguistic functions to words, not components. A system for linguistically driven automated text formatting creates a language model (1050 in FIG. 10) that links complete components with specific dependencies associated with the beginning of the component (e.g., FIG. 6A). For example, the beginning of the component XP is X. That is, the beginning of a noun phrase (NP) is a noun (N), and the beginning of a prepositional phrase (PP) is a preposition (P). Dependencies associated with the beginning of core arguments and non-core subordinate words are used to trigger line breaks and horizontal displacements. The dependency parser 625 (e.g., dependency parser 1035 described in FIG. 10) generates dependency output 635 (e.g., dependency data 1040 described in FIG. 10) that provides the dependencies of words in the text. Core arguments and non-core dependent selector 640 receives constituent output 630 (e.g., constituent data 1030 described in FIG. 10) from constituency parser 620 (e.g., constituent parser 1025 described in FIG. 10) and dependency output 635, and selects the dependencies associated with the constituents to generate linguistic structure 615 (e.g., model 1050 described in FIG. 10). As a result, a linguistically driven automated text formatting system requires both constituency information and dependency information.

[0021] Alternative embodiments may include only a dependency parser, but also require additional rules (e.g., phrase structure rules) that define linguistic components. Such rules may not include a computationally implemented parser, but may include any collection of rules based on linguistic theory that describe components. This includes, but is not limited to, heuristics for identifying components based on keywords (e.g., prepositions), subordinate clause markers (i.e., that, which, who), clause conjunctions (i.e., either, but, and), and other single-word indicators of phrase structure.

[0022] Exemplary embodiments of the constituent cascading operation are based on characterization of constituents and dependencies within a sentence, as well as other linguistic features provided by the listed NLP-Services, including, by way of example and not limitation, coreference, sentiment analysis, named entity recognition, and topic tracking. In addition to cascading, the output from these parsers can be used to decorate text by highlighting, coloring, underlining, accompanying audio information, etc., to provide cognitive cues and reduce the user's cognitive load.

[0023] While cascading is used as an example, the systems and methods discussed herein are applicable to a variety of visual, audible, and tactile outputs that provide a user with reduced cognitive load when engaging with text. In another exemplary embodiment, other formatting may be used to provide cuing for a user to reduce cognitive load. In an exemplary embodiment, cuing may be achieved through text formatting and / or emphasis modifications, such as using color, italics, or providing video, vibration, or audio outputs (e.g., tones) using analysis outputs such as constituency and dependency data.

[0024] FIG. 2 illustrates a block diagram of an example of dependency analysis 205 and cascaded text output 210 for linguistically driven automated text formatting, according to one embodiment.

[0025] The dependency parser can employ a variety of labeling conventions or dependency sets for specifying linguistic features. The systems and methods discussed herein incorporate the use of any dependency set for specifying inter-word linguistic features, including those based on syntax, semantics, or prosody. In an exemplary embodiment, a dependency set from the Universal Dependency (UD) initiative, a collaborative open-source international project for developing cross-linguistically valid dependency sets, is employed. The set of relationships is available at https: / / universaldependencies.org / u / dep / index.html.

[0026] UD dependency sets are divided into core arguments of noun phrases and clauses and other types of subordinates, including non-core subordinates (i.e., oblique arguments, adverbial clauses, and relative clauses) and noun modifiers (i.e., adjectives, noun attributes, and clause modifiers). The component cascading process specifies that core arguments and non-core subordinates should be mandatorily indented under their heads. This indentation provides a visual cue to the core relationship within a sentence. Thus, direct or indirect objects (labeled "obj" or "iobj" in the dependency analysis 205) are indented under the verbs they modify (often labeled "root"), as shown in the cascaded output 210. In one example, subordinates of noun phrases may also be indented under their heads. These include various sets of noun modifiers and adjective modifiers (i.e., possessive phrases, reduced relative clauses, numerical modifiers, and appositive phrases). The cascaded text output 210 includes indentation based on dependency analysis 205. The amount of indentation may be specified in system preferences, as described below. In an exemplary embodiment, subordinate noun phrases are treated separately, depending on line length or component type, with the goal of minimizing line breaks whenever possible. Different indentation amounts may optionally be applied to different subordinate word types to visually distinguish them.

[0027] FIG. 3 illustrates a block diagram of an example of dependency analysis 305 and cascaded text output 310 for linguistically driven automated text formatting, according to one embodiment.

[0028] Indentation rules are applied to the components. When components are embedded within other components (e.g., relative clauses, complements), unindentation rules are applied to signal the completion of the component, as shown in the cascaded output 310. Unindentation results in the horizontal displacement being restored to the beginning of the embedded phrase. This generates a cascading pattern that provides clear clues about the structure of the embedding and the relationship between each verb and its arguments. The cascaded text output 310 includes indentation based on dependency analysis 305 according to the cascading rules specified in the cascade generator. Additional processing may be used to provide additional clues in cascaded output displayed on display-limited devices. For example, additional characters or other signals may be inserted to indicate that the component wraps onto an additional line. In these cases, the horizontal displacement remains consistent for the wrapped component (e.g., if the component begins at position 40, the wrapped segment also begins at position 40 and has visual markings (e.g., parentheses, shading, etc.) to indicate that it is a continuation).

[0029] FIG. 4 illustrates an example of cascaded text output 400 for linguistically driven automated text formatting, according to one embodiment. Horizontal positions indicate the components to which they should be associated. For example, positioning the verb "left" indicates that its subject is "the janitor" (rather than, e.g., "the principal"), as shown in cascade 405. Similarly, positioning the phrase "every night" indicates that it accompanies the verb "cleaned," rather than, e.g., "noticed." Note that the vertical lines shown in FIG. 4 are for illustrative purposes and are not part of the text output.

[0030] Horizontal displacement is similarly used to signal pre-positioned subordinate clauses by indenting the first clause relative to the matrix clause, as shown in cascade 410. This technique provides clear cues regarding the central information of the sentence and the subordinate status of the first clause.

[0031] In another exemplary embodiment, the cascading process determines indentation using specifier and complement positions from the X-bar theory analysis, without reference to the specific syntactic dependencies provided by the dependency parser. This takes advantage of the fact that specifier and complement positions themselves specify general dependencies between sentence components. However, in an embodiment limited to two types of dependencies (e.g., specifiers and complements), the information available for cuing linguistic structure within a sentence is more limited.

[0032] In another exemplary embodiment, the cascading process determines indentation according to a list of specific dependencies, which may be supplied by the user, including the associated indentation amount. For example, a user may prefer that direct objects be indented four spaces but indirect objects be indented only two spaces. In one embodiment, these user specifications are made by a teacher or tutor who may wish to emphasize certain grammatical relationships as part of a comprehensive lesson plan. Such specifications may be made individually or for categories of dependency types. For example, a user may specify that core arguments should be indented more than non-core modifiers. Note that the component cascading operation determines whether a component is indented based on its dependency type, and user preferences determine how much indentation is reflected in the formatting. User preferences may additionally affect the display attributes of the cascade, such as font type, font size, font color, and line length.

[0033] In an exemplary embodiment, the cascade automatically adjusts to fit the constraints of the display device. For example, a computer screen can tolerate longer line lengths than a tablet or phone display. If the line length is too short to allow for a line break between XPs, the line break occurs at the X' level, and there is no additional horizontal displacement. Thus, visual cues related to horizontal displacement are reserved to signal the start of a new language dependency. Additional cues (e.g., parentheses, font styles, colors) may be added to maintain easy identification of components.

[0034] FIG. 5 illustrates an example 500 of a cascaded text output for linguistically driven automated text formatting that has been extended to identify referentially linked phrases, according to one embodiment. The component cascading operation defines the cascade for syntactic and semantic relationships between components. In an exemplary embodiment, another NLP service that parses text into references and antecedents is used to extend linguistically driven automated text formatting to include referential cueing, whereby visual cues (e.g., color, etc.) identify the referential relationship between words or components (e.g., as shown in FIG. 8). This involves presenting the antecedent with the same font characteristics as the pronoun reference (shown underlined in FIG. 8). Referential cueing helps prevent confusion in sentences such as those in FIGS. 5 and 8 regarding whether the person "occupied herself," shown in cascade 505, is "the performer" or "the soloist." An example of referential cueing at the constituent level (e.g., the entire phrase "measure the room" is the antecedent of "it") is shown in cascade 510. The referential cueing operation is performed based on the output of an automated coreference resolution parser (NLP service), such as, by way of example and not limitation, Stanford CoreNLP or AllenNLP. Coreference resolution finds expressions that refer to the same entity in text. This information is incorporated into a language model that serves as input to the cascade generator. Coreference information can also be represented by hand-coded rules or formatting, such as those produced by trained language engineers, or by automated processing by a non-rule-based probabilistic inference engine.

[0035] Augmenting cascades to indicate coreference cues illustrates, by way of example and not limitation, how linguistic features other than those based on constituent or dependency relationships can be informed in cascades. For example, a coreference parser may be replaced with an alternative NLP service that produces analysis indicating various linguistic relationships in text. Examples include, but are not limited to, named entity recognition, sentiment analysis, semantic role labeling, textual entailment, topic tracking, and prosodic analysis. These may be implemented as rule-based or probabilistic reasoning systems. The output of these NLP services yields information that can be used to modify the display features of the cascade in a way that emphasizes linguistic relationships. These modifications may occur within or across sentences and serve as cues that help readers or learners maintain coherence as they read.

[0036] As described herein, various embodiments of systems and methods for generating cascaded text displays are provided. In embodiment 600 shown in FIG. 6A, a text portion 605 is segmented into multiple dependent segments based on grammatical information determined from the text portion 605 based on output from an NLP service 610. Each of these is a complete constituent defined by a constituency parser 620 and has a dependency role determined by a dependency parser 625 associated with the beginning of the constituent. In one embodiment, a constituent is identified by its maximal projection (XP) and assigned to the dependency role associated with the beginning of its projection (X). For example, the constituent "the principal" has two dependencies associated with it (det and nsubj), but the complete NP constituent is assigned as nsubj for interpretation by the cascade generator.

[0037] FIG. 6B illustrates a text portion 605 with indentations within a cascade 630 according to the hierarchical position specified for each segment by linguistically driven automated text formatting provided by a cascade generator 635, according to one embodiment.

[0038] FIG. 7 is a block diagram of an example environment 700 and system 705 for linguistically driven automated text formatting, according to one embodiment. FIG. 7 may present features shown in FIGS. 1-5, 6A, and 6B. The environment may include system 705, which may be a cloud-based delivery system (or other computing platform (e.g., virtual computing infrastructure, software-as-a-service (SaaS), Internet of Things (IoT) network, etc.)). The system may be distributed among various backend systems 710 that provide infrastructure services, such as computing and storage capacity, for a cloud service provider hosting system 705. System 705 may be communicatively coupled to a network 720 (e.g., the Internet, a private network, a public network, etc.) (e.g., via a wired network, a wireless network, a cellular network, a shared bus, etc.). An end-user computing device 715 may be communicatively connected to the network and may establish a connection to system 705. An end-user device may communicate with the system 705 via a web browser, a download application, an on-demand application, etc. In one example, components of the system may be prepared for delivery to an end-user computing device 715 via an installation application that provides offline access to the features of the system 705.

[0039] The system 705 can provide online connectivity directly through the end-user computing device 715 and deliver a set of packaged services to end-user applications on the end-user computing device 715, which can operate offline without an Internet connection and as a hybrid with the end-user applications connecting (e.g., via a plug-in) to cloud services (or other computing platforms) through the Internet. The hybrid mode allows users to read in cascading format regardless of connectivity, while still providing data to improve the system 705. End-user applications can be distributed in several forms. For example, browser plug-ins and extensions can allow users to change the formatting of text read on the web and in applications using cascading formatting. In another example, an end-user application may be integrated into a menu bar, clipboard, or text editor, so that when a user highlights text using a mouse or hotkeys, a window can be presented with the selected text rendered using cascading formatting. In another example, the end-user application may be a portable document file (PDF) reader that can accept a PDF file as an input source and output a cascaded format for display to a user. In yet another example, the end-user application may be an augmented image enhancement that converts a live view from a camera and applies optical character recognition (OCR) to convert images to text in real time and render the layout in a cascaded format.The version control service 755 can track application versions and can also periodically update portable components provided to applications running on the end-user computing device 715 when connected to the Internet.

[0040] According to an exemplary embodiment, the end-user computing device 715 includes OCR capabilities, which allow a user to capture an image of text with a camera (e.g., on their cell phone) and instantly convert it into cascaded-formatted text (e.g., as shown in FIG. 18). According to one embodiment, the end-user computing device 715 includes or is fitted with a user-worn device, such as smart glasses or smart contact lenses, where text input viewed by the user is converted into a cascaded format for improved comprehension. In this manner, text can be converted in real time by the user's personal visual device. According to another exemplary embodiment, the end-user computing device 715 provides cascaded-formatted augmented video (AV), augmented reality (AR), and virtual reality (VR), where application of cascaded formatting can be completed within a user-worn visual display device, including AV and VR headsets, eyeglasses, contact lenses, or implantable lenses, to enable the user to view text in cascaded format.

[0041] The systems and methods discussed herein are applicable to various environments in which text is rendered on a device by processing the text and converting it into cascaded formatting. Displaying text on a screen requires rendering instructions, and a set of cascaded instructions can be inserted into the command sequence. This may apply to certain document types (e.g., PDF) and systems with embedded rendering engines, where calls to the rendering engine can be intercepted and cascaded formatting instructions can be inserted. In one example, a user can scan a barcode, quick response (QR) code, or other mechanism to access content (e.g., on a menu, product label, etc.), and the content can be returned in cascaded formatting.

[0042] The system 705 can include various service components that can execute in whole or in part on various computing devices of the backend system 710, including a cascade generator 725, a natural language processing (NLP) service 730, a machine learning service 735, an analytics service 740, a user profile service 745, an access control service 750, and a version control service 755. The cascade generator 725, the NLP service 730, the machine learning service 735, the analytics service 740, the user profile service 745, the access control service 750, and the version control service 755 can include instructions, including application programming interface (API) instructions, that can input and output data to and from external systems and other services.

[0043] System 705 can operate in a variety of modes: an end user (e.g., a reader) can use a local client with a copy of the offline components for generating cascaded text to convert text on the local client; an end user can submit text to system 705 to convert standard text into cascaded text; a publisher can submit text to system 705 to convert the text into cascaded format; a publisher can use a set of offline components in system 705 to convert the text into cascaded format; and a publisher can use system 705 to publish text in traditional block formatting or cascaded formatting.

[0044] The cascade generator 725 can receive text input and send the text to an NLP service 730 parser to generate linguistic data. Linguistic data can include, by way of example and not limitation, parts of speech, word lemmas, component parse trees, charts of discrete components, lists of named entities, dependency graphs, lists of dependencies, linked coreference tables, linked topic lists, lists of named entities, the output of sentiment analysis, semantic role labels, and entailment confidence statistics. Thus, for a given text, linguistic analysis can return a word breakdown that includes a rich set of linguistic information about each token. This information can include a list of relationships between words or components that appear in separate sentences or separate paragraphs.

[0045] The cascade generator 725 can apply cascade formatting rules and algorithms to the language model generated by the machine learning service 735 created using the constituency data and dependency data to generate a probabilistic cascade output.

[0046] FIG. 8 illustrates an example environment 800 for receiving linguistic corrections to a cascaded format for linguistically driven automated text formatting, according to one embodiment. A user interface 805 and a plug-in 810 can provide the features described in FIGS. 1A, 1B, and 2-7. The user interface 805 and the plug-in 810 can be connected to the system 705 described in FIG. 7. The system 705 can include a coreference parser 810 that can analyze text and identify related terms within the text. For example, "The performer" in a cascaded sentence displayed in the user interface 805 can be referred to by a pronoun in other parts of the sentence. "The performer" and "herself" can be highlighted (e.g., underlined, highlighted in color, displayed in a contrasting color with other text, italicized, etc.) to indicate that they refer to the same entity. In one example, the coreference parser 810 may be replaced with an alternative NLP service that produces an analysis that indicates linguistic relationships in the text. Mode selection buttons 815 may be provided that allow the user to turn features on or off. For example, when cascade mode is enabled, text may be displayed in a cascade format, when coreference tracking is enabled, related terms may be highlighted, etc.

[0047] 9 illustrates an example system 900 for cross-sentence coreference tracking for linguistically driven automated text formatting, according to one embodiment. The example 900 can provide the features described in FIGS. 1-5, 6A, 6B, and 7-8.

[0048] The system 900 can include a cloud-based delivery system 705, which can include a coreference parser 910, a constituency parser 915, a dependency parser 920, and a topic tracker 925. The coreference parser 910, the constituency parser 915, the dependency parser 920, and the topic tracker 925 can provide indications of relationships between particular pieces of text. These systems can work in conjunction with a cascade generator (such as the cascade generator 725 described in FIG. 7) to output cascaded text to the user interface 905 with topic and pronoun referents indicated by visual cues (e.g., using color, etc.).

[0049] Figure 10 shows a data flow diagram of a system 1000 for linguistically driven automated text formatting, according to one embodiment. System 1000 can provide the features described in Figures 1-5, 6A, 6B, and 9.

[0050] A user can provide input 1005 including text. In one example, the text can be provided as typed text, text captured using OCR processing, text captured via an application or browser plug-in, Hypertext Markup Language (HTML), etc. The text can also include visual content, including figures, tables, graphs, photographs, and visually enhanced text, including headings with emphasized font size or font style. Input can also come directly from a user, such as a user typing the text string "The patient who the girl liked was coming today." The input processor 1010 can process the text content to remove formatting and special characters and, in the case of paragraphs, to break the text into individual sentences.

[0051] Processed input 1015, including processed text, may be sent by the input processor 1010 to a set of NLP services 1020. According to one embodiment, two primary NLP services are a constituency parser 1025 and a dependency parser 1035. Sentence constituents are linguistically defined parts of a sentence and may correspond to words, phrases, or clauses. Constituents are organized hierarchically and defined by phrase structure rules (e.g., as shown in FIG. 1). The constituency parser 1025 processes the input text and generates constituent data 1030, including a parse tree (e.g., a constituency parse), a chart of discrete constituents, parts of speech, word lemmas, and metadata, which may be sent to the data processor 1045.

[0052] For example, the constituency path 1025 may generate constituency data 1030 for "The patient who the girl liked was coming today." as shown in Table 1.

[0053] [Table 1]

[0054] The processed input 1015, including the processed text, may be sent by the input processor to a dependency parser 1035 of the NLP service 1020. The dependency parser 1035 may process the input text and generate dependency data 1040, which provides data about dependencies between words, and send it to the data processor 1045. The dependency data 1040 may include a parse tree or directed graph that describes the dependents embedded under a root node along with additional hierarchical embeddings (e.g., FIG. 3), tokens, dependency labels, and metadata. For example, the dependency parser 1035 may generate dependency data 1040 for "The patient who the girl liked was coming today." as shown in Table 3.

[0055] [Table 2]

[0056] In another example, the dependency analyzer 1035 can generate dependency data 1040 as shown in Table 2 for "The patient who the girl liked was coming today."

[0057] [Table 3-1] [Table 3-2]

[0058] The data processor 1045 can use the constituency data 1030 and dependency data 1040, as well as information from any other NLP services, to generate a model 1050. The model 1050 can include parts of speech, word lemmas, component parse trees, charts of discrete components, lists of named entities, dependency graphs, lists of dependencies, linked coreference tables, linked topic lists, output of sentiment analysis, semantic role labels, and entailment reference confidence statistics.

[0059] In one example, the data processor 1045 may generate a model 1050 for "The patient who the girl liked was coming today." as shown in Table 4.

[0060] [Table 4-1] [Table 4-2]

[0061] The model 1050 may be processed by a cascade generator 1055. The cascade generator 1055 may apply cascade rules to the model 1050 to generate a cascade output 1060. A cascade rule is an operation that is triggered for execution based on the detection of a particular component or dependency in the analysis output. The operations performed by a cascade rule may include determining whether line breaks should be inserted in the output, the indentation level of lines of text, and other output-generating operations that create cues for the user. In one example, the cascade rules may be used to generate a text model that includes data identifying the indentation and line break placement of text output to be displayed on a display device. The cascade generator 1055 may return the cascaded text and metadata in the cascaded output 1060. For example, the cascade generator 1055 can generate the cascaded output 1060 for the sentence "the patient who the girl liked was coming." using the cascade rules shown in Table 5.

[0062] [Table 5-1] [Table 5-2]

[0063] In another example, the cascade generator 1055 may generate the cascaded output 1060 using the cascading rules shown in Table 6.

[0064] [Table 6-1] [Table 6-2]

[0065] Further examples of cascaded text are shown in Table 7.

[0066] [Table 7]

[0067] Figure 11 illustrates an example of a method 1100 for linguistically driven automated text formatting, according to one embodiment. Method 1100 can provide the features described in Figures 1-5, 6A, 6B, 7, and 10.

[0068] In operation 1105, the text portion may be obtained from an interface. In one example, the interface may be a physical keyboard, a soft keyboard, a text-to-speech dictation interface, a network interface, or a disk controller interface. In one example, the text portion may be a string of text in a common format selected from the group consisting of rich text, plain text, HyperText Markup Language, Extensible Markup Language, or American Standard Code for Information Interchange.

[0069] In operation 1110, the text portion may be segmented into multiple dependent segments. The segmentation may be based on evaluation of the text portion using a constituency parser and a dependency parser. In one example, the constituency parser identifies complete segments that hold a particular dependency role identified by the dependency parser.

[0070] In operation 1115, the multiple dependent segments may be encoded according to queuing rules that describe the hierarchical position of each segment. In one example, a text model for the text portion may be constructed using the output of the constituency parser and the dependency parser, as well as other NLP services, and cascading rules may be applied to the text model to generate the encoded segments. In one example, the text model may be a data structure that includes parts of speech, lemmas, a constituency chart, a parse tree, and a dependency list for each word in the text. The encoded segments may include text and metadata that defines the hierarchical position of the dependent segments. In one example, the dependent segments may be segments from a sentence. In one example, the hierarchical position may correspond to the offset of the encoded segment relative to another one of the dependent segments in a user interface. In one example, the encoded segments may include line break data and indentation data. In one example, segmenting a text portion may include appending the text portion to another text portion, modifying the indentation of the text portion, or inserting a line break before the text portion.

[0071] In operation 1120, the encoded dependent segments may be displayed in a user interface according to user preferences. In one example, the dependent segments may be encoded using JavaScript® Object Notation, Extensible Markup Language, or American Standard Code for Information Interchange. In one example, encoding the dependent segments may include concatenating the dependent segments of several sentences to create a text composition. In one example, the combined sentences may be written to a file, transmitted via a cloud protocol, or displayed directly on an output device.

[0072] In one example, encoded segments may be received. The encoded segments may be parsed to extract respective text and the hierarchical location of the encoded segments, and the text may be displayed according to its location. In one example, displaying the text may include modifying an offset of the portion of the text and adjusting a character height of the portion of the text according to the location. In one example, the offset may be from the left for left-to-right languages ​​and from the right for right-to-left languages.

[0073] In one example, displaying text may include adding, modifying indents, and modifying line breaks depending on the position without affecting the positional placement of the text based on linguistic structure. Figure 12 illustrates an example of a method 1200 for linguistically driven automated text formatting, according to one embodiment. Method 1200 can provide the features described in Figures 1-5, 6A, 6B, and 7-11.

[0074] In operation 1205, data representing one or more constituents of an input sentence may be received from a constituency parser (such as, for example, constituency parser 1025 depicted in FIG. 10). The data representing the one or more constituents may be generated based on evaluating the input sentence using the constituency parser. In one example, the constituency parser may identify constituents of the sentence. In one example, a constituent may be a word or a group of words that function as a single unit within a hierarchical structure.

[0075] In operation 1210, data representing relationships between words of the input sentence may be received from a dependency parser (such as, for example, dependency parser 1035 described in FIG. 10). The relationships may be based on sentence structure and may be derived based on evaluating the input sentence using the dependency parser. In one example, the dependencies may be a one-to-one correspondence, such that for one element of the input sentence, there is exactly one node in the structure of the input sentence that corresponds to that element.

[0076] In operation 1215, a text model may be constructed (e.g., by input processor 1015 described in FIG. 10) using the components and dependencies (e.g., as shown in FIG. 6A). In one example, the text model may be further refined with linguistic features produced by additional NLP services, including, by way of example and not limitation, coreference information, sentiment tracking, named entity lists, topic tracking, probabilistic inference evaluation, prosodic contours, and semantic analysis.

[0077] In operation 1220, the cascading rules may be applied to the text model (e.g., by the cascade generator 1055 described in FIG. 10 ) to generate a cascaded text data structure. In one example, the cascaded text data structure comprises text and metadata specifying display parameters for the text. In one example, the cascaded text data structure comprises a file (e.g., an Extensible Markup Language (XML) file) organized according to a schema (e.g., an XML schema). In one example, the schema comprises specifications for components and component placement within the file. In one example, the text model may be a data structure that describes the constituents and dependencies of words in an input sentence. In one example, the text model may be a data structure that includes a text parse tree, parts of speech, tokens, a constituent chart, and dependencies. In one example, the cascading rules comprise formatting rules that create line breaks and indentations defined corresponding to the constituents and dependencies.

[0078] In one example, metadata associated with the cascaded text may be generated. In another example, the cascaded text comprises a set of formatted text segments, including line breaks and indentations, for display on a display device. In some examples, the input sentence of the text may be received from a source specified by a user.

[0079] In the example of a paragraph or collection of sentences, the text can be processed to divide the text into a list of sentences before it is provided to the constituency parser or dependency parser. Each sentence may be processed individually by the constituency parser 1205 and the dependency parser 1210. In one example, the method 1200 is applied to each sentence in the text. For example, sentences may be displayed sequentially, each cascaded separately, but depending on user preferences. In some examples, sentences may be grouped into paragraphs by visual cues other than indentation (e.g., background shading, dedicated markers, etc.).

[0080] Figure 13 illustrates an example of a method 1300 for linguistically driven automated text formatting, according to one embodiment. The method 1300 can provide the features described in Figures 1-5, 6A, 6B, and 7-12.

[0081] A model of the text may be constructed (e.g., in operation 1305) from dependency data and constituency data obtained by parsing the text (e.g., by input processor 1015 described in FIG. 10 ). A cascade data structure may be generated (e.g., in operation 1310) according to cascade rules applied (e.g., by cascade generator 1055 described in FIG. 10 ) to the model of the text. Sentences of the text may be displayed (e.g., in operation 1315) according to the cascaded text data structure. The data structure may specify horizontal and vertical alignment of the text for display.

[0082] 14 illustrates an example method 1400 for cascading text using machine learning classifiers for linguistically driven automated text formatting, according to one embodiment. Method 1400 can provide the features described in FIGS. 1-5, 6A, 6B, and 7-13.

[0083] In operation 1405, a text portion is obtained from an interface. For example, the text may be entered by a user via an input device, may be obtained from a local or remote text source (e.g., a file, a publisher data source, etc.), etc. In operation 1410, the text portion is processed through an NLP service (e.g., NLP service 730 described in FIG. 7 ) to obtain a linguistic encoding. For example, the text may be parsed using various parsers, which may include, by way of example and not limitation, a constituent parser, a dependency parser, a coreference parser, etc., to identify constituents, dependencies, coreferences, etc. In operation 1415, a machine learning (ML) classifier is applied to the linguistic encoding to determine a cascade (e.g., by cascade generator 725 described in FIG. 7 ). For example, the machine learning classifier may use information identified by the parser (e.g., a portion of text encoded with linguistic information identified by the parser) to classify the portion of text for formatting (e.g., line breaks, indentation, etc.). In operation 1420, the cascade is displayed in a user interface according to user preferences. For example, the cascade may be displayed in an application window, a web browser, a text editor, etc. on the screen of a computing device (e.g., a standalone computer, a mobile device, a tablet, etc.).

[0084] 15 illustrates an example method 1500 for training machine learning classifiers to cascade text for linguistically driven automated text formatting, according to one embodiment. Method 1500 can provide the features described in FIGS. 1-5, 6A, 6B, and 7-14.

[0085] A machine learning service, such as machine learning service 735 shown in FIG. 7, is trained using cascade examples to cascade outputs (such as, for example, output 1060 described in FIG. 10) as an alternative to using logic and rules defined manually by human effort or by a cascade generator (such as, for example, cascade generator 1055 discussed in FIG. 10).

[0086] In operation 1505, a corpus of cascaded text may be obtained. The corpus may be separated into a training set and a test set. In operation 1510, the corpus may be divided into subsets. For example, a large portion of the corpus may be designated for training and the remaining portion for validation. In operation 1515, a probabilistic machine learning method (e.g., a support vector machine with recursive feature reduction) may be applied to generate a set of pattern classifiers for a portion of the subset (e.g., the training set). A cross-validation procedure may be performed to evaluate the set of pattern classifiers by applying the pattern classifiers to uncascaded examples of sentences in the test set. In operation 1520, the set of pattern classifiers may be applied to an uncascaded version of the corpus of cascaded text for the remaining portion of the subset to generate a new set of cascaded text. The validity of the cascades generated by the set of classifiers may be evaluated with respect to known cascades. In operation 1525, the validity of the new set of cascaded text can be evaluated against known cascades in a test set according to accuracy, sensitivity, and specificity. For example, a corpus of cascaded text with marked constituents and dependencies can serve as a training set to produce a classifier function that can be used to generate an appropriate cascade for a particular new sentence (not in the training set) based on its linguistic attributes. By way of example and not limitation, classification can be performed using a linear kernel support vector machine with recursive feature reduction (SVM-RFE; Guyon et al., 2002). The SVM classification algorithm (Vapnik, 1995, 1999) has been used in a wide range of applications and has produced better accuracy than other methods (e.g., Asri et al., 2016; Huang et al., 2002; Black et al., 2015).SVMs divide data into classes (e.g., cascade patterns) by identifying the optimal separation point (hyperplane) between two classes in a high-dimensional feature space, maximizing the margin width around the hyperplane and minimizing misclassification error. The cases closest to the hyperplane are called support vectors, and they serve as important discriminators for distinguishing classes. Cross-validation (CV) techniques are used to evaluate the generalizability of classification models (e.g., Arlot and Celisse, 2010; James, Witten, Hastie, and Tibshirani, 2013). This involves dividing the data into subsets or folds (10 is used according to a convention called 10-fold CV), with 9 folds used to train the classifier and a rejection set used to validate the resulting classifier.

[0087] Cross-validation is performed to validate the generalizability of our classifiers across i) cases (sentences), ii) features (e.g., syntactic categories or dependencies), and iii) tuning parameters (optimization) using a multilevel approach. This method prevents overfitting and avoids biased classification accuracy estimates that can result from using the same CV subset to simultaneously evaluate multiple aspects of a classifier. The results of each CV step are evaluated using the measures specificity = TN / (TN + FP), sensitivity = TP / (TP + FN), and accuracy = (sensitivity + specificity) / 2, where TN is the number of true negatives, FP is the number of false positives, TP is the number of true positives, and FN is the number of false negatives.

[0088] It may be understood that a variety of machine learning techniques may be used to train a classifier to recognize cue insertion points and cue formatting using labeled or unlabeled data. Machine learning techniques consistent with observing and learning from labeled data or learning from intrinsic coding based on the positional structure of a training cascade corpus may be used to facilitate training of the classifier. Thus, while an SVM is used as an example to further inform the training process, it should be understood that alternative machine learning techniques with similar functionality may also be used.

[0089] This process can be applied using training sets generated by alternative means and does not rely on a cascade generator, for example, hand-coded training data may be used to train an ML model to generate cascaded text.

[0090] In one example, the NLP services referred to herein may use pre-trained AI models (e.g., AMAZON® Comprehend or Stanford Parser (https: / / nlp.stanford.edu / software / lex-parser.shtml), GOOGLE® Natural Language, or MICROSOFT® Text Analytics, AllenNLP, Stanza, etc.) that can use RNNs for text analysis. Given larger amounts of data, the RNNs can learn mappings from free text inputs to create outputs such as predicted entities, key phrases, parts of speech, constituent charts, or dependencies that may be present in the free text. In one example, additional machine learning models can be trained using key phrase formatting rules, part-of-speech formatting rules, entity formatting rule pairs, constituency data, dependency data, etc. as training data to learn to identify various parts of speech, key phrases, entities, constituents, dependencies, etc. that can be used in future parsing operations. In another example, user preferences and parts of speech, key phrases, entity pairs, constituencies, dependencies, etc. can be used to train a machine learning model to identify user preferences based on various parts of speech, key phrases, and entities. Various machine learning models can provide outputs based on the statistical likelihood that a given input is associated with a selected output. For example, recurrent neural networks including various threshold layers can be used to generate models for filtering outputs to improve the accuracy of output selection.

[0091] The cascaded output may be presented to the user in a variety of media: for example, a side-by-side display of the original text and the cascaded text may be displayed on a display device, and the original text may be replaced or modified by the cascaded output.

[0092] In one example, machine learning can be used to evaluate a corpus of cascaded text to train a cascade pattern classifier for linguistically driven automated text formatting. The pattern classifier specifies actual visual cues (e.g., cascading, indentation, line breaks, color, etc.) that signal linguistic attributes contained in a text segment depending on the style of the cascade rule. In one example, the classifier can evaluate the words, parts of speech, constituent groups, or dependency labels of the text segment and produce formatting structures that match the visual cues present in the cascade training set. In one example, the classifier can directly evaluate the shape and display characteristics of the cascades in the training set and produce formatting structures that match the visual cues present in the training set.

[0093] FIG. 16 illustrates an example of a text transformation 1600 for displaying a sentence of text in a cascaded format for linguistically driven automated text formatting, according to one embodiment. The text transformation 1600 can provide the functionality described in FIGS. 1-5, 6A, 6B, and 7-15. A browser window 1605 can include a plug-in 1610 connected to the system 1605 and a cascade mode button 1615. When cascade mode is off, standard text 1620 can be displayed when a user browses the text of a website. When cascade mode is on, the text can be formatted and displayed in a cascade format 1625.

[0094] FIG. 17 illustrates an example 1700 of tagged HyperText Markup Language (HTML) code for cascaded text for linguistically driven automated text formatting, according to one embodiment. The example 1700 can provide the features described in FIGS. 1-5, 6A, 6B, and 7-16. The source text 1705 can be processed by the NLP service 730 described in FIG. 7 to yield a text model 1050 (e.g., parts of speech, key phrases, constituency data, dependency data, etc.). While this example shows the source text 1705 as HTML code, the text may be in ASCII, XML, or various other machine-readable formats. The cascade generator 725 described in FIG. 7 generates line breaks (e.g., punctuation marks in HTML code) inserted into the text based on the text model 1050. A tagged output text 1715 may be rendered indicating the tag (such as a tag) and indentation (e.g., using the text-indent: tag in HTML).

[0095] The cascade generator 725 may make several calls to the NLP service 730 for functions as described herein regarding components and dependencies or other linguistic data. In one embodiment, each of these is a separate call / function that, when combined, provides a set of tags to be used to generate the cascaded text. Other embodiments Returning to the description of FIG. 7 , according to one exemplary embodiment, key phrases, constituents, dependencies, etc. may be determined probabilistically by the NLP service 730 using computational linguistics. The cascading text generated by the cascade generator 725 can improve reading comprehension. According to one exemplary embodiment, constituents describing the “who,” “when,” “where,” and “what” information in a sentence are separated by line breaks and / or indentation to achieve cascaded text formatting. The cascade generator 725 determines the “who,” “when,” “where,” and “what” of a sentence using constituents linked by dependencies, etc. For example, the indentation inserted by the cascade generator 725 is based on the sentence constituency hierarchy informed by the NLP service 730.

[0096] Continuous improvement of cascade formatting can be achieved through the use of artificial intelligence, including machine learning techniques, by the machine learning service 735. Input / output to and from the machine learning service 735 can be processed actively and passively. For example, the machine learning service 735 can operate under an active machine learning approach in an instance where two users may be reading the same material with slightly different cascade formats between Person A and Person B. The users' comprehension of the material can be tracked through ratings, and the machine learning service 735 can generate output that can be used by the cascade generator 725 to modify display characteristics over time. This approach allows the system 705 to improve with scale and usage. The machine learning service 735 can operate in a passive mode to measure attributes such as reading behavior without explicit ratings. For example, the analytics service 740 can collect data, without the user receiving ratings, that may be used (alone and when combined) to generate output indicative of reader comprehension. Some of the data that may be evaluated to improve cascade formatting may be, by way of example and not limitation, the time spent reading a given portion of text (e.g., less time spent reading may indicate greater comprehension, etc.), eye tracking using a camera (e.g., embedded camera, external camera, etc.) to assess the efficiency of eye movements between different cascade formats of the same content to provide machine learning input, and user modifications to assess the degree of personal modification for user preferences (e.g., more spacing, sentence length, font, color highlighting of key phrases or parts of speech, constituents, subordinate words, etc.).

[0097] The analytics service 740 can log and store user analytics, including measures of reading comprehension and user preferences reflected in system modifications (e.g., scroll speed, spacing, fragment length, dark mode, key phrase highlighting, etc.). Modifications can be evaluated for various cascading formats to determine how various modifications affect comprehension performance and can be provided to the machine learning service 735 as input for improving default display formats. The user profile service 745 can store user profile data, including anonymized identifiers for privacy and privacy compliance (e.g., HIPAA compliance), and track performance metrics and progress. For example, modifications made by a user to display parameters over time can be learned by the machine learning service 735 based on input from the analytics service 740 and used to generate user-specific display characteristics. The access control service 750 can provide access to stored content, including metrics on frequency, duration of use, and text attributes (e.g., fiction, non-fiction, author, research, poetry, etc.). According to another personalization, users can personalize display parameters such as text color, font, font size, etc.

[0098] In one example, a reading score may be developed for a user. A scoring matrix may be created, which may be based on the student's age and grade level, and this scoring matrix may be the result of the student's reading of specified content and then the associated student comprehension (e.g., a reading comprehension score (RCS)). The RCS approach is similar to a grade point average (GPA) in that it establishes a relative scoring metric among similar populations. This metric may be based on an assessment of the user's consumption of text using cascade formatting. For example, the assessment may be tracked (actively or passively) over time to determine the user's speed and efficiency of comprehension of text. A score may be calculated for the user that increases with the speed of comprehension. Input received prior to running a reading assessment may be used to generate a baseline quota for the user.

[0099] Cognitive training libraries can be developed using cascading formatting. For example, various published content, such as books, magazines, and articles, can be translated using cascading formatting, and users can be made to access the libraries for the purpose of training their brains to improve comprehension and retention. Tests and quizzes can be used to assess users' comprehension performance, providing users with tangible evidence of cognitive improvement.

[0100] Cascade formatting may be applied to process various languages. NLP services may be used to provide linguistic features of other language text; rules can be created and applied to the NLP output (e.g., Spanish text) to cascade the foreign language text accordingly. In addition to linguistic adaptations, adaptations may be based on the syntax of other languages; therefore, cascaded text in different languages ​​can be cascaded differently using different formatting rules to account for language-specific syntactic variations. Constituency and dependency NLP services are available in multiple languages. Because cascade rules are based on a universal set of dependencies, they are likely to transfer to other languages, although the output may look different (e.g., some languages ​​read right-to-left rather than left-to-right, top-to-bottom). Some have words of different lengths that may require different types of line breaks (e.g., agglutinative words add grammar by adding morphemes, resulting in very long words). Language-adaptive display characteristics are applied, but the process of component cueing discussed herein remains constant.

[0101] In one example, spoken language may be combined with a visual display of cascading formatted text. Audio may be integrated so that as a person reads the cascading text, a soundtrack reading the same text is also played at a speed that can be controlled to match the individual reader's speed. For example, text-to-speech conversion may be used to provide audio / speech output in real time and at a reading pace, comprehension may be assessed, and the speed of the text-to-speech conversion output may be altered (e.g., speed up, slow down, pause, resume, etc.) to maintain synchronization between the audio and the user's reading. Cascades can be accompanied by speech, allowing readers to visually recognize prosodic cues associated with line breaks. This deductive system can be used to coach users to imitate this prosody in their own reading aloud, supporting increased knowledge of syntactic structure.

[0102] FIG. 18 illustrates an example 1800 of generating cascaded text from a captured image for linguistically driven automated text formatting, according to one embodiment. The example 1800 can provide the features described in FIGS. 1-5, 6A, 6B, and 7-17. An imaging device 1805 can capture an image including text 1810. The image can be processed to extract the text 1810 from the image (e.g., by optical character recognition, etc.). The text 1810 can be evaluated by the system 1000 to identify text models in the text (e.g., based on constituencies, dependencies, etc.). Cascaded formatting rules can be applied to the text 1810 based on linguistic attributes assigned by the system 1000. The text 1810 can be transformed by the cascaded formatting rules into cascaded output text and displayed for consumption by a user.

[0103] FIG. 19 illustrates an example method 1900 for generating cascaded text from a captured image for linguistically-driven automated text formatting, according to one embodiment. Method 1900 can provide the features described in FIGS. 1-5, 6A, 6B, and 7-18. In operation 1905, an image of text may be captured within a field of view of an imaging device. In operation 1910, text may be recognized within the image to generate a machine-readable text string. In operation 1915, the machine-readable text string may be processed by an NLP service to identify linguistic attributes (e.g., parts of speech, lemmas, constituency data, dependency data, coreference data, sentiment analysis, topic tracking, probabilistic inference, and prosodic structure, etc.) to generate a text model (e.g., as shown by element 1050 in FIG. 10). In operation 1920, a text cascade may be generated, including line breaks and indentation based on rules applied using the text model. In one example, a text string may include a sentence of text, to which line breaks and indentation may be applied, hi one example, a text string may include multiple sentences grouped into paragraphs by visual cues (e.g., background color, explicit markers, etc.).

[0104] FIG. 20 illustrates an example 2000 of converting text from a first display format to a second display format in an eyewear device using natural language processing for linguistically driven automated text formatting, according to one embodiment. The example 2000 can provide the features described in FIGS. 1-5, 6A, 6B, and 7-19. A user 2005 may be wearing smart glasses including an imaging device 2010. The imaging device 2010 can capture an image including text 2015. The image can be processed to extract the text 2015 from the image (e.g., by optical character recognition, etc.). The text 2015 can be evaluated by the system 705 to identify linguistic attributes in the text (e.g., parts of speech, key phrases, constituency data, dependency data, etc.) (e.g., to form a model 1050, etc., described in FIG. 10). Cascaded formatting rules may be applied to text 2015 using language attributes (e.g., model 1050 described in FIG. 10 ) identified by cloud-based distribution system 705. Text 2015 may be converted into cascaded output text 2020 and displayed on a display device of the smart glasses including imaging device 2010 for consumption by the user.

[0105] FIG. 21 illustrates an example of a method 2100 for converting text from a first display format to a second display format in an eyewear device for linguistically-driven automated text formatting, according to one embodiment. Method 2100 may provide the features described in FIGS. 1-5, 6A, 6B, and 7-20. In operation 2105, an image of text may be captured within a field of view of an imaging component of the eyewear device or may be captured from another source of text input. In one example, the eyewear device may have memory where text is saved or downloaded as a file, may have wireless capabilities, such as where text is wirelessly acquired and then converted. In operation 2110, text may be recognized in the image to generate a machine-readable text string. In operation 2115, the machine-readable text string may be processed by an NLP service 1020 to identify linguistic attributes (e.g., parts of speech, lemmas, constituency data, dependency data, coreference data, sentiment analysis, topic tracking, probabilistic inference, and prosodic structure, etc.) to generate a text model (e.g., model 1050 described in FIG. 10 ). In operation 2120, a text cascade may be generated, including line breaks and indentation based on the rules applied using the text model 1050. In one example, the text string may include a sentence of text, and line breaks and indentation may be applied to the sentence. In one example, the text string may include multiple sentences grouped into paragraphs by visual cues (e.g., background color, explicit markers, etc.). In operation 2125, the text cascade may be displayed to the user via a display component of the eyewear device.

[0106] FIG. 22 illustrates an example 2200 of generating cascaded text for linguistically driven automated text formatting, according to one embodiment. The example 2200 can provide the functionality described in FIGS. 1-5, 6A, 6B, and 7-21. The text authoring tool 2205 can receive text input 2210 from a user (e.g., via a keyboard, voice command, etc.). Once the text input 2210 is received, it is processed by the system 705 (via an online connection, a locally available component, etc.) so that linguistic attributes of the text input 2210 (e.g., part-of-speech data, constituency data, dependency data, etc.) are identified. In one example, the system 705 (or components of the system 705) can execute on a remote computing platform or on the device on which the text authoring tool 2205 executes. Cascading formatting rules may be applied to the input text 2210 using the language attributes identified in the input text 2210. The displayed output text 2215 may be displayed as cascaded output text as the user enters text to provide the user with real-time cascading formatting as the text is entered.

[0107] FIG. 23 illustrates an example architecture 2300 for user and publisher preference management of cascaded text using natural language processing based on feedback input for linguistically driven automated text formatting, according to one embodiment. The architecture 2300 can provide the features described in FIGS. 1-5, 6A, 6B, and 7-19. The user interface 2305 and plug-ins 2310 can connect to the system 705 as described in FIG. 7. The system 705 can include a user registry 2310 that allows users and publishers to register to publish and consume text through the system 705. The user registry can maintain user and publisher profiles, including display preferences to be applied when viewing cascaded text. User preferences do not change the formation of the cascaded text, but can provide user- or publisher-defined display modifications, such as text color, font size, and line spacing. A cascade mode button 2315 can be provided that allows the user to turn edit mode on or off. The text may be displayed in a cascading format, and when cascading editing mode is enabled, changes to the cascading display properties may be tracked and evaluated to generate personalized cascading display properties for the user. The tracked changes may be used as feedback information stored in a profile that may be used to learn user preferences over time. Feedback may also be direct input provided by a user or publisher indicating preferences for content display properties.

[0108] In one example, publishers can register with the user registry 2310 and provide published content for consumption by users of the system 705. Publisher data 2320 can include content including books, magazines, manuals, exams, etc., that can be entered into the system 705. Linguistic attributes (e.g., parts of speech, constituency data, dependency data, coreferences, etc.) can be identified by an NLP service for each sentence in the publisher data 2320, and the NLP service classifies words or word combinations as named entities, parts of speech, having constituency relationships, having dependencies, coreferences, topic links, etc. Publisher data 2320, including linguistic analysis, can be received by the system 705.

[0109] The identified language attributes of the publisher data 2320 can be evaluated against a set of cascading formatting rules, which may include rules for inserting line breaks, indentations, and other formatting elements. The cascading formatting rules may be triggered by classes or parts of speech, key phrases, component or dependency types, etc. identified in the text. The cascaded publisher data may be returned to the publisher for inclusion in the publisher data 2320 or may be consumed by the user via the user interface 2305.

[0110] Additional formatting may be used to provide further cues to the user. For example, highlighting of component structures may be performed to reduce the cognitive load of reading. This allows the systems and methods discussed herein to provide a system that simplifies the task of reading. Various modifications, including modifying text with color, underlining, highlighting, etc., may be applied to help reduce the cognitive load on the user.

[0111] FIG. 24 illustrates an example method 2400 for cascaded text personalization using natural language processing based on feedback input for linguistically driven automated text formatting, according to one embodiment. Method 2400 can provide the features described in FIGS. 1-5, 6A, 6B, and 7-23. The systems and methods discussed herein can be used to develop personalized algorithms specific to individuals. People process text information differently, and personal reading algorithms (PRAs) can be created that are optimized for each person. When text is displayed in cascaded format and cascaded editing mode is enabled, changes to the display characteristics of the cascaded formatted output can be tracked and analyzed to recognize medical conditions, cognitive functions, learning disabilities, and / or to generate cascaded formatting rules personalized for the user. By fine-tuning various attributes and evaluating comprehension and readability, personalized settings can be developed that are used to customize display characteristics. This can be created by changing display characteristic parameters made by the user to suit their reading preferences. User adjustments to display characteristics that can be tracked can include modifying the amount of indentation, highlighting or coloring of key components, scroll speed, etc. The cascading logic remains based on language information, but the user's profile can be used to highlight, italicize, color, increase spacing, add line breaks, etc. in the display characteristics of the cascaded text.

[0112] Like the difference between two eyeglass prescriptions, different eyes process information slightly differently, which can result in different display characteristics for two users. Cognitive diversity (e.g., different ways in which people encounter and process text) can also be addressed by optimizing display characteristics between individuals or groups that share similar cognitive profiles, such as ADHD, dyslexia, autism, etc.

[0113] For example, reading profiles can be created (alone or in combination) that modify presentation parameters based on reader attributes. Contextual profiles can be informed by user input or through computer-automated assessments about specific medical and cognitive conditions (e.g., dyslexia, attention-deficit hyperactivity disorder (ADHD), attention-deficit disorder (ADD), etc.), autism, native language (e.g., a native Chinese speaker may benefit from a different format than a native Spanish speaker), the nature of the content being read (e.g., fiction, non-fiction, news, poetry, etc.), or screen mode (e.g., phone, tablet, laptop, monitor, AR / VR device, etc.). Cascaded output display characteristic optimization can also be used in diagnostic and therapeutic modalities to address injuries and disabilities. A key symptom of people suffering from a concussion or traumatic brain injury (TBI) is changes in eye movement and gaze tracking. As one example, an athlete may be seen on the sideline being asked to track a finger moving horizontally in front of their eye. Reading places a heavy burden on a concussed brain. People with TBI need to take frequent breaks to allow their brains to rest and recover. In the military and many sports, participants are required to perform baseline cognitive testing to assess future injuries. In one example, the systems and techniques discussed herein can be used in conjunction with an augmented reality headset / glasses to measure a comprehension level baseline using cascaded text and adjusted display characteristics for the participant. This baseline can include eye-tracking information in addition to comprehension assessments and user-reported comfort / strain.

[0114] After that baseline is established, eye tracking and comprehension can be measured using cascades and the degree to which changes in display characteristics from the baseline settings provide relief. For example, if the baseline measurement was that a participant read with a three-letter indent and scored at a comprehension level of X, it can be determined how the participant would fare on that same measure after a TBI and how well the participant might do with a display enhanced with emphasized indents, colors, etc. Display characteristic changes necessary to support the injured participant can provide information for the participant's diagnosis and treatment plan. In one example, a user can set their own personal parameters and output format (e.g., length, spacing, highlighting, indentation, line breaks, etc.), font, color, size, background color.

[0115] In operation 2405, content (e.g., text, images, etc.) may be retrieved from a cloud-connected source, including text passages. In operation 2410, a registry of individuals and personalized text formatting parameters for the individuals may be maintained. In one example, the text formatting parameters may include rules for generating a personalized cascaded text display format for the individuals. In operation 2415, display control instructions may be generated that enable display of the cascade with user-specified attributes based on the individuals' personalized text formatting parameters. For example, a text passage may be received from a publisher associated with the individuals from the registry, and display control instructions may be generated to control display of the passage on the client device. In operation 2420, the cascade may be displayed by applying user-specified attributes to modify display characteristics of the cascade output on the display device. The display control instructions may use the individuals' personalized text formatting parameters, and the instructions may enable display of the passage in a personalized cascaded format for the individuals. In one example, a cloud-connected source may be a browser application, which may supply passages of text to a text processing engine.

[0116] FIG. 25 illustrates an example 2500 of a dual display of cascaded text based on feedback input (e.g., user modifications of displayed text) for linguistically driven automated text formatting, according to one embodiment. Example 2500 can provide the features described in FIGS. 1-5, 6A, 6B, and 7-24. Text can be received via a user interface 2505. The text 2510 can be processed by the system 705 to identify parsing structure (e.g., parts of speech, constituency data, dependency data, etc.). The text 2510 (e.g., uncascaded, source, cascaded, etc.), or source text, can be displayed in a first window of the user interface 2505. While block text is used as an example, the raw text can have HTML emphasis, including titles, headings, figure captions, etc. The modified text 2515 can be displayed in a second window of the user interface 2505. The qualified text 2515 may be text that has been cascade formatted, translated, formatted in another cascade format that may include user preferences, etc. In one example, the qualified text 2515 may be presented in a cascade format with line breaks and indentation according to cascade formatting rules applied using the identified language attributes. The qualified text 2515 may be presented with parts of speech, constituency data, dependency data, etc. highlighted (e.g., underlined, bolded, colored, shaded, etc.) to allow the user to identify the structure of the text 2510.

[0117] Thus, parallel displays of source text and cascaded formatted text can be presented on two screens or in a split-screen parallel presentation. For example, the original source text can be displayed on one screen, and the cascaded text can be displayed in parallel on a second screen, and the two screens can be synchronized so that they remain synchronized as the user scrolls through either display. This can be used for educational purposes to highlight syntax, key phrases, parts of speech, constituency data, dependency data, etc., or to present a "highlighted comprehension view" similar to a text X-ray that displays the underlying structure of the text. For example, underlying linguistic features can be highlighted, such as constituents, coreferences, and key phrases being bolded or not, verbs being colored or bolded, etc.

[0118] 26 illustrates an example system 2600 for translating input text in a first language 2610 into a cascaded output in a second language 2615 for linguistically driven automated text formatting, according to one embodiment. A translation engine 2620 can receive input 2610 in the first language and translate the input into the second language. The translation engine can be running locally on a device used to view a user interface 2605 displaying the input and cascaded output in the second language 2615, or can be running remotely, such as from a web service, cloud-based service, remote server, etc. The translation engine 2620 can include translation logic that, when run on input text, converts the text from the input language to the second language.

[0119] The output from the translation engine 2620 may be processed by the cloud-based system 705 to produce the cascaded second language output 2615, as described above. For example, a cascaded-formatted passage may be displayed in one window of the screen, and a translation of the passage in another language may be presented in a cascaded format in a second window of the screen. In another example, block text may be displayed on one side of a dual display, and cascaded-formatted text may be displayed on the other side. In yet another example, a split screen may be provided with block and cascaded scrolling, and / or languages ​​may be displayed on separate sides of the screen or separate displays in a dual display configuration. The dual screens may allow the user to refer to the other view to aid in understanding a difficult passage.

[0120] Figure 27 illustrates an example of a method 2700 for linguistically driven automated text formatting, according to one embodiment. Method 2700 can provide the features described in Figures 1-5, 6A, 6B, and 7-26.

[0121] Text input may be received from an input device in operation 2705. In operation 2710, the input text may be formatted using cues to convey identified linguistic relationships from the constituency and dependency analysis operations of the natural language processing service.

[0122] In operation 2715, cascading rules may be applied to prioritize the placement of constituents of the text input as consecutive units of display output. The cascading rules may determine the horizontal displacement of one of the constituents based on information output from the automated dependency parser and may group the constituent with other constituents based on horizontal positioning to emphasize dependencies. De-indentation may indicate the completion of a constituency group. In one example, the cascading rules may further identify core arguments and non-core subordinate words in the input text using rules that link dependencies with constituents. These rules may indent core arguments and non-core subordinate words below the beginning of linguistic phrases in the input text.

[0123] An output including indentation and line feeds may be generated based on the application of the cascading rules in operation 2720. In one example, the output may be augmented with additional linguistic feature cues provided by natural language processing services, including coreference information, sentiment analysis, named entity recognition, semantic role labeling, text alignment, topic tracking, or prosody analysis.

[0124] In operation 2725, the output may be displayed on a display of an output device. In one example, anonymized usage data or user-specified preferences may be received, and a custom display profile may be generated that includes output display characteristics based on the anonymized usage data or the user-specified preferences. In one example, the output may be tailored using the output display characteristics of the custom display profile, which may modify the display characteristics of the output without modifying the shape of the output. In one example, the output may be generated for display on a phone, tablet, laptop, monitor, virtual reality device, or augmented reality device. In one example, the output may be generated for display in a dual-screen format that displays a side-by-side text format, a format-while-edit format, or a cascade-and-translate format.

[0125] FIG. 28 shows a block diagram of an example machine 2800 upon which any one or more of the techniques (e.g., methodologies) discussed herein may be implemented. In alternative embodiments, machine 2800 may operate as a standalone device or may be connected (e.g., networked) with other machines. In a networked deployment, machine 2800 may operate in the capacity of a server machine, a client machine, or both in a server-client network environment. In one example, machine 2800 may operate as a peer machine in a peer-to-peer (P2P) (or other distributed) network environment. Machine 2800 may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile phone, a web appliance, a network router, switch, or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by the machine. Additionally, although only a single machine is shown, the term "machine" is also intended to include any collection of machines that individually or collectively execute a set (or sets) of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, software as a service (SaaS), other computer cluster configurations, etc.

[0126] The examples described herein may include or operate with logic or several components or mechanisms. A circuit set is a collection of circuits implemented in a tangible entity including hardware (e.g., simple circuits, gates, logic, etc.). Circuit set membership may be adaptive over time and to underlying hardware variations. A circuit set includes elements that, when operational, can perform specified operations, either singly or in combination. In one example, the hardware of a circuit set may be invariably designed (e.g., hardwired) to perform specific operations. In one example, the hardware of a circuit set may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including computer-readable media physically modified (e.g., magnetically, electrically movable arrangements of invariant mass particles, etc.) to encode specific operational instructions. When connecting the physical components, the underlying electrical properties of the hardware components are changed, for example, from insulator to conductor or vice versa. The instructions enable embedded hardware (e.g., an execution unit or a loading mechanism) to variably connect and create a member of a circuit set in the hardware to perform a portion of a specific operation when in operation. Thus, the computer-readable medium is communicatively coupled to other components of the circuit set member when the device is operating. In one example, any of the physical components can be used for multiple members of multiple circuit sets. For example, during operation, an execution unit can be used by a first circuit of a first circuit set at one time and reused by a second circuit of the first circuit set or a third circuit of the second circuit set at another time.

[0127] The machine (e.g., computer system) 2800 may include a hardware processor 2802 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), a main memory 2804, and a static memory 2806, some or all of which may communicate with each other via an interlink (e.g., a bus) 2808. The machine 2800 may further include a display unit 2810, an alphanumeric input device 2812 (e.g., a keyboard), and a user interface (UI) navigation device 2814 (e.g., a mouse). In one example, the display unit 2810, the input device 2812, and the UI navigation device 2814 may be touch-screen displays. Machine 2800 may additionally include a storage device (e.g., a drive unit) 2816, a signal generating device 2818 (e.g., a speaker), a network interface device 2820, and one or more sensors 2821, such as a global positioning system (GPS) sensor, a compass, an accelerometer, or other sensor. Machine 2800 may include an output controller 2828, such as a serial (e.g., universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection, for communicating with or controlling one or more peripheral devices (e.g., a printer, a card reader, etc.).

[0128] Storage device 2816 may include machine-readable medium 2822 on which one or more sets of data structures or instructions 2824 (e.g., software) are stored that embody or are utilized by any one or more of the techniques or functions described herein. Instructions 2824 may also reside, completely or at least partially, within main memory 2804, within static memory 2806, or within hardware processor 2802 during execution thereof by machine 2800. In one example, one or any combination of hardware processor 2802, main memory 2804, static memory 2806, or storage device 2816 may constitute a machine-readable medium.

[0129] Although machine-readable medium 2822 is shown as a single medium, the term “machine-readable medium” may include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store one or more instructions 2824.

[0130] The term "machine-readable medium" may include any medium capable of storing, encoding, or retaining instructions for execution by machine 2800 and causing machine 2800 to perform any one or more of the techniques of this disclosure, or capable of storing, encoding, or retaining data structures used by or associated with such instructions. Non-limiting examples of machine-readable media may include solid-state memory, as well as optical and magnetic media. In one example, machine-readable media may exclude transient, propagating signals (e.g., non-transitory machine-readable storage media). Specific examples of non-transitory machine-readable storage media may include non-volatile memory, such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0131] The instructions 2824 may further be transmitted or received over a communications network 2826 using a transmission medium via a network interface device 2820 utilizing any one of several transport protocols (e.g., frame relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). Exemplary communication networks may include, among others, local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), plain old telephone (POTS) networks, and wireless data networks (e.g., the IEEE 802.11 family of standards, known as Wi-Fi®, LoRa® / LoRaWAN® LPWAN standards, etc.), the IEEE 802.15.4 family of standards, peer-to-peer (P2P) networks, 3rd Generation Partnership Project (3GPP®) standards for 4G and 5G wireless communications, including the 3GPP Long Term Evolution (LTE) family of standards, the 3GPP LTE Advanced family of standards, the 3GPP LTE Advanced Pro family of standards, the 3GPP New Radio (NR) family of standards, and the like. In one example, network interface device 2820 may include one or more physical jacks (e.g., Ethernet jacks, coaxial jacks, or telephone jacks) or one or more antennas for connecting to communications network 2826. In one example, network interface device 2820 may include multiple antennas for communicating wirelessly using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) techniques. The term “transmission medium” is intended to include any intangible medium capable of storing, encoding, or carrying instructions for execution by machine 2800, including digital or analog communications signals or other intangible media for facilitating the communication of such software. Additional notes The above detailed description includes references to the accompanying drawings, which form a part of the detailed description. The drawings illustratively show specific embodiments that may be practiced. These embodiments are also referred to herein as "examples." Such examples may include elements in addition to those shown or described. However, the inventors also contemplate examples in which only the elements shown or described are provided. Furthermore, the inventors also contemplate examples that use any combination or permutation of those elements (or one or more aspects thereof) shown or described with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0132] All publications, patents, and patent documents mentioned herein are incorporated herein in their entirety as if individually incorporated by reference. In the event of a conflicting usage between this specification and those documents so incorporated, the usage of the incorporated documents should be considered supplementary to the usage of this specification, and in the event of any irreconcilable conflict, the usage of this specification will control.

[0133] As is common in patent documents, the terms "a" or "an" are used herein to include one or more, regardless of any other instance or usage of "at least one" or "one or more." The term "or" is used herein to refer to a non-exclusive "or," unless otherwise stated, such that "A or B" includes "A but not B," "B but not A," or "A and B." In the appended claims, the terms "including" and "in which" are used as the plain-English equivalents of the respective terms "comprising" and "wherein." Also, in the following claims, the terms "including" and "comprising" are open-ended, i.e., systems, devices, articles, or processes that include elements in addition to those recited after such terms in a claim are still deemed to be within the scope of that claim. Moreover, in the following claims, the terms "first," "second," and "third," etc., are used merely as labels and do not impose numerical requirements on their objects.

[0134] The above description is illustrative, and not limiting. For example, the above examples (or one or more aspects thereof) may be used in combination with each other. Other embodiments may be utilized by those of ordinary skill in the art upon reviewing the above description. The Abstract is submitted to enable the reader to quickly ascertain the nature of the technical disclosure, and with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Also, in the above Detailed Description, various features may be grouped together to streamline the disclosure. This should not be construed as indicating that an unclaimed disclosed feature is essential to any claim. Rather, inventive subject matter may lie in less than all features of a particular disclosed embodiment. Accordingly, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separate embodiment. The scope of the embodiments should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

1. 1. A method for generating a display of text including linguistic cues from input text, comprising: receiving, by at least one processor of a computing device, the input text from an input device; the at least one processor obtaining dependency information for the input text; the at least one processor determining constituency information for the input text; generating, by the at least one processor, a language model from the constituency information and the dependency information, the language model being a data structure including a parse tree representing dependencies between components of the input text and coded data representing relationships between the components and dependencies of the components of the input text; the at least one processor applying cascade rules to the components of the input text using the language model, the cascade rules comprising: instructions for determining a vertical placement of each component, each component comprising a word or group of words that function as an individual grammatical unit in a hierarchy of a parse tree of the generated language model, the vertical placement placing components of the input text within respective lines or consecutive units; and instructions for determining a horizontal placement of each component; one or more dependencies and the beginning of each component are determined from the encoding data of the language model, and the horizontal alignment provides the same horizontal displacement for components of the input text that have dependencies on the same beginning or on each other; generating output by said at least one processor applying said vertical alignment and said horizontal alignment according to said cascading rules using line feeds and indents; and causing the at least one processor to display a representation of the output on a display device.

2. The cascade rule: further comprising identifying core arguments and non-core subordinate words in the components of the input text using rules linking subordinate words with respective components containing the subordinate words; said horizontal arrangement indenting said core arguments and said non-core subordinate words with respect to said respective constituents below the beginning of a linguistic phrase of said input text; The method of claim 1 , wherein the horizontal arrangement de-indents the respective components to indicate completion and aligns subordinate components with each other.

3. 10. The method of claim 1, wherein the output is augmented with linguistic feature cues provided by a natural language processing service that generates linguistic feature cues by performing coreference analysis, sentiment analysis, named entity recognition, semantic role labeling, text alignment, topic tracking, or prosodic analysis on the input text.

4. receiving, by the at least one processor, anonymized usage data or user-specified preferences; The method of claim 1 , further comprising: generating a custom display profile including output display characteristics based on the anonymized usage data or the user-specified preferences.

5. 5. The method of claim 4, further comprising: the at least one processor adjusting the output using the output display characteristics of the custom display profile, the output display characteristics modifying display characteristics of the output without modifying the shape of the output.

6. The method of claim 1 , wherein the output is generated for display on a phone, tablet, laptop, monitor, virtual reality device, or augmented reality device.

7. 10. The method of claim 1, wherein the output is generated for display in a dual screen format providing a side-by-side text display, a format-while-edit display, or a cascade-and-translate display.

8. 10. The method of claim 1, wherein the constituent information is generated from one or more diagnostic tests to determine respective constituents, the one or more diagnostic tests involving at least one of: i) do-so / one substitution, ii) coordination structure, iii) topicalization, iv) ellipsis, v) cleft or pseudo-cleft sentence formation, vi) passivization, vii) wh-initiation, viii) right-branch node raising, ix) pronoun substitution, x) question answering, xi) omission, and xii) adverbial intrusion.

9. The method of claim 1 , wherein the input text is presented in the output in at least one contiguous unit using multiple lines for the display of the output on the display device.

10. 10. At least one machine-readable medium comprising instructions for generating a display of text including linguistic cues from input text, the instructions, when executed by the at least one processor of the computing device, causing the at least one processor to perform the method of any one of claims 1 to 9.

11. 1. A system for generating a display of text including linguistic cues from input text, comprising: at least one processor; and a memory containing instructions that, when executed by said at least one processor, cause said at least one processor to perform the operations of the method of any one of claims 1 to 9.

Images