Reading support in a browser environment

Abstract

An assistive reading tool implemented in a browser application facilitates the presentation of text content in a manner and format that addresses the reading and / or comprehension abilities of individual users while maintaining the context of the original content. The assistive reading tool outputs a reader view pane along with a main content pane. The original content is presented in the main content pane, and simplified / reformatted content corresponding to text content excerpted from the original content is presented in the reader view pane. Scrolling of the excerpted text content in the reader view pane is synchronized with scrolling of the original content in the main content pane, so that context is maintained as the user navigates through the content. The simultaneous presentation of the original content and the excerpted text content allows the user to more easily consume the excerpted text content while maintaining the originally intended context.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application is a continuation of and claims priority to U.S. Application No. 17 / 931,283, "READING ASSISTANT IN A BROWSER ENVIRONMENT," filed September 12, 2022, the disclosure of which is incorporated herein by reference in its entirety.

[0002] This description relates generally to assistive reading tools for computing devices, and more particularly to assistive reading tools that facilitate consumption of content by a user while the user maintains the context of the original content. [Background technology]

[0003] Some users face challenges with reading and / or comprehension and / or learning and / or language. For example, conditions such as dyslexia, attention deficit disorder, and other such conditions can affect a user's ability to concentrate on, process, and / or understand content, such as content available via web-based applications, web-based sources, etc. Existing reading modes replace content with simplified content, for example, providing simplified output or a text-centric presentation of the content. Summary of the Invention

[0004] Systems and methods according to embodiments described herein provide assistive reading tools that improve the reading experience and comprehension of web-based content. The assistive reading tool provides output of text content in a manner and format that addresses the reading and / or comprehension abilities of individual users while maintaining the context of the original content. The assistive reading tool may be implemented in a browser application, including a browser-based operating system. This may make the assistive reading tool available for virtually any content output by the browser application / browser-based operating system. The availability of the assistive reading tool via the browser application / browser-based operating system may reduce or substantially eliminate dependency on third-party applications to output content in this manner. The use of browser-level / operating system-level assistive reading tools may improve security compared to third-party applications or programs because the content accessed by the user and the user's interaction with the content are separated within the system. The assistive reading tool may output a reader view that presents a modified or simplified version of the original content. The assistive reading tool outputs a reader view pane alongside, i.e., beside / next to, the main content pane, with the original content presented in the main content pane and the modified or simplified content presented in the reader view pane. The modified or abbreviated content may include text content excerpted from the original content, and the output of the excerpted text content may be customized, modified, or reformatted according to user settings or preferences to address particular user needs.The simultaneous output of the original content in the main content pane and the excerpted text content in the reader view pane allows a user to more easily consume the excerpted text content while maintaining context as originally intended by the simultaneous output (in one example, side-by-side) of the original content. Scrolling of the excerpted text content may be synchronized with or anchored to scrolling of the original content, thereby maintaining this continuity and context as the user navigates through the content.

[0005] In some aspects, the techniques described herein relate to a computer-implemented method, the method including: excerpting, by a text excerpting module of a browser application executing on a computing device, text content included in browser content displayed by the browser application; generating corrected text content based on the excerpted text content; and rendering, by the browser application, a reader view for displaying the corrected text content together with the browser content, wherein rendering the reader view includes rendering a first content pane that displays the browser content and rendering a second content pane adjacent to the first content pane that displays the corrected text content, and the computer-implemented method further includes providing a reader view including the second content pane adjacent to the first content pane for display on a display device of the computing device.

[0006] In some aspects, the techniques described herein relate to a computer-implemented method, wherein generating the corrected text content includes at least one of modifying a format of the excerpted text content according to pre-stored user preferences for displaying the corrected text content in a second content pane, or modifying a format of the excerpted text content in response to user input detected in a reading assistance toolbar of the browser application.

[0007] In some aspects, the techniques described herein relate to a computer-implemented method, wherein modifying the format of the excerpted text content includes modifying at least one of a font type, a font size, a font color, a background color, a pitch, and line spacing of the excerpted text content.

[0008] In some aspects, the techniques described herein relate to a computer-implemented method as set forth in claim 3, further including rendering, by a browser application, browser content; analyzing, by a reading assistance module of the browser application, the browser content; and determining, in response to detecting excerptable text content in the browser content, that the browser content is associated with an extractable page.

[0009] In some aspects, the techniques described herein relate to a computer-implemented method, further comprising launching a reading assistance tool, wherein launching the reading assistance tool comprises at least one of launching the reading assistance tool in response to determining that browser content is associated with an extractable page or launching the reading assistance tool in response to user input selecting the reading assistance tool from a browser application menu.

[0010] In some aspects, the techniques described herein relate to a computer-implemented method, further including determining that browser content is associated with a non-extractable page, detecting user input selecting a section of text content from the browser content, and launching the assistive reading tool in response to user input selecting the assistive reading tool from a browser application menu.

[0011] In some aspects, the techniques described herein relate to a computer-implemented method, wherein generating the modified text content includes generating the modified text content based on a section of the text content selected by user input.

[0012] In some aspects, the techniques described herein relate to the computer-implemented method of any preceding claim, further including detecting a scroll input for scrolling the modified text content displayed in the second content pane, scrolling the modified text content displayed in the second content pane in response to the scroll input, and scrolling the browser content displayed in the first content pane in response to the scroll input.

[0013] In some aspects, the techniques described herein relate to a computer-implemented method, further including detecting a scroll input for scrolling browser content displayed in a first content pane, scrolling the browser content displayed in the first content pane in response to the scroll input, and scrolling the modified text content displayed in a second content pane in response to the scroll input.

[0014] In some aspects, the techniques described herein relate to a computer-implemented method, wherein scrolling browser content displayed in a first content pane includes synchronizing scrolling of the browser content displayed in the first content pane with scrolling of modified text content displayed in a second content pane.

[0015] In some aspects, the techniques described herein relate to a computer-implemented method, in which synchronizing scrolling of browser content displayed in a first content pane with scrolling of modified text content displayed in a second content pane includes associating first bounding boxes assigned to text elements of the browser content displayed in the first content pane with second bounding boxes assigned to corresponding text elements of the modified text content displayed in the second content pane, and performing the scrolling synchronization based on the association of the first bounding boxes and the second bounding boxes.

[0016] In some aspects, the techniques described herein relate to a computer-implemented method, wherein generating modified text content includes extracting text content from browser content and omitting from the modified text content at least one of image content, table content, links, input fields, or browser control devices included in the browser content.

[0017] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium storing instructions that, when executed by at least one processor of a computing device, cause the at least one processor to: excerpt, by a text excerpting module of a browser application running on the computing device, text content included in browser content displayed by the browser application; generate corrected text content based on the excerpted text content; and render, by the browser application, a reader view for displaying the browser content together with the corrected text content, wherein rendering the reader view includes rendering a first content pane that displays the browser content and rendering a second content pane beside the first content pane that displays the corrected text content, and the instructions further cause the at least one processor to provide a reader view including the first content pane and the second content pane for display on a display device of the computing device.

[0018] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium having instructions that cause at least one processor to detect pre-stored user settings related to displaying modified text content in a second content pane, modify a format of the excerpted text content according to the pre-stored user settings, detect user input in a reading assistance toolbar of the browser application, and modify a format of the excerpted text content in response to the user input detected in the reading assistance toolbar.

[0019] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium or claim 14, wherein the instructions cause at least one processor to render browser content by a browser application; analyze the browser content by a reading assistance module of the browser application; and, in response to detecting excerptable text content in the browser content, determine that the browser content is associated with an extractable page.

[0020] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium, wherein instructions cause at least one processor to launch a reading assistance tool, the launching of the reading assistance tool including at least one of launching the reading assistance tool in response to determining that browser content is associated with an extractable page or launching the reading assistance tool in response to user input selecting the reading assistance tool from a browser application menu.

[0021] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium having instructions that cause at least one processor to determine that browser content is associated with a non-extractable page, detect user input selecting a section of textual content from the browser content, and launch the assistive reading tool in response to user input selecting the assistive reading tool from a browser application menu.

[0022] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium having instructions that cause at least one processor to generate modified text content based on a section of the text content selected by user input.

[0023] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium as set forth in claim 18, wherein instructions cause at least one processor to detect a scroll input for scrolling modified text content displayed in a second content pane, scroll the modified text content displayed in the second content pane in response to the scroll input, and scroll browser content displayed in the first content pane in response to the scroll input.

[0024] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium, wherein instructions cause at least one processor to synchronize scrolling of browser content displayed in a first content pane with scrolling of modified text content displayed in a second content pane, wherein the synchronizing includes associating first bounding boxes assigned to text elements of the browser content displayed in the first content pane with second bounding boxes assigned to corresponding text elements of the modified text content displayed in the second content pane, and synchronizing the scrolling based on the association of the first bounding boxes and the second bounding boxes.

[0025] In some aspects, the technology described herein relates to a non-transitory computer-readable medium as set forth in claims 13-20, wherein the browser content includes text content and at least one of image content, table content, links, input fields, or browser control devices, and the instructions cause at least one processor to modify at least one of a font type, a font size, a font color, a background color, a pitch, or line spacing of the excerpted text content.

[0026] In some aspects, the techniques described herein relate to a non-transitory computer-readable medium as set forth in claim 21, wherein instructions cause at least one processor to generate modified text content, wherein generating the modified text content includes extracting text content from browser content and omitting at least one of image content, table content, links, input fields, or browser control devices included in the browser content from the modified text content.

[0027] In some aspects, the techniques described herein relate to a computing system comprising a processor, a display, and a memory that stores instructions that, when executed by the processor, cause a browser application running on the computing system to perform operations including: excerpting text content included in browser content displayed on the display by a text excerpt module of the browser application; generating corrected text content based on the excerpted text content; and rendering a reader view for displaying the browser content together with the corrected text content, wherein rendering the reader view includes rendering a first content pane that displays the browser content and rendering a second content pane beside the first content pane that displays the corrected text content, and wherein the operations further include providing a reader view including the first content pane and the second content pane for display on the display.

[0028] In some aspects, the techniques described herein relate to a computing system that includes instructions that, when executed by a processor, cause a browser application to generate modified text content, where generating the modified text content includes extracting text content from the browser content and omitting at least one of image content, table content, links, input fields, or browser control devices included in the browser content from the modified text content.

[0029] The details of one or more implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings. [Brief explanation of the drawings]

[0030] [Figure 1A] 1 illustrates an exemplary browser application user interface according to implementations described herein. [Figure 1B] 1 illustrates an exemplary browser application user interface according to implementations described herein. [Figure 2] FIG. 1 is a block diagram of a system according to embodiments described herein. [Figure 3A] 1 illustrates an exemplary browser application user interface with content displayed in a browser content window. [Figure 3B] 3B illustrates an alternative example for launching an assistive reading tool in the exemplary browser application user interface shown in FIG. 3A. [Figure 3C] 3B illustrates an alternative example for launching an assistive reading tool in the exemplary browser application user interface shown in FIG. 3A. [Figure 3D] 3B illustrates an alternative example for launching an assistive reading tool in the exemplary browser application user interface shown in FIG. 3A. [Figure 3E]3B illustrates an alternative example for launching an assistive reading tool in the exemplary browser application user interface shown in FIG. 3A. [Figure 3F] 3B illustrates features of a reader view generated by an assistive reading tool and presented in a browser content window of the exemplary browser application user interface shown in FIG. 3A. [Figure 3G] 3B illustrates features of a reader view generated by an assistive reading tool and presented in a browser content window of the exemplary browser application user interface shown in FIG. 3A. [Figure 3H] 3B illustrates features of a reader view generated by an assistive reading tool and presented in a browser content window of the exemplary browser application user interface shown in FIG. 3A. [Figure 4A] 1 illustrates an exemplary browser application user interface with content displayed in a browser content window. [Figure 4B] 1 illustrates an exemplary browser application user interface with content displayed in a browser content window. [Figure 4C] 4C illustrates the selection of an assistive reading tool in the exemplary browser application user interface shown in FIGS. 4A and 4B. [Figure 4D] 4C illustrates features of a reader view generated by an assistive reading tool and presented in a browser content window of the exemplary browser application user interface shown in FIGS. 4A and 4B. [Figure 5] 1 is a flowchart of an exemplary method according to embodiments described herein. DETAILED DESCRIPTION OF THE INVENTION

[0031] Web-based content may include numerous content items (i.e., text content, image content, promotional / advertising content, links, browser control devices, text input fields, etc.). The arrangement of different types of content items can make consuming web-based content difficult, especially for users with reading and / or comprehension challenges. Existing reading modes may be somewhat limited in their ability to extract content output in formats other than typical text content structures (i.e., articles, blogs, etc.), i.e., text content. Furthermore, existing reading modes often lose connection with the original content in the process of extracting text content into a reading view. Thus, some of the context and level of understanding of the content originally intended by the author / publisher may be lost by some users. Readily available assistive reading tools that are effective for extracting content in a variety of different formats and allow users to maintain context with the original content while consuming the extracted content can improve the content consumption experience, especially for users with reading and / or comprehension and / or learning and / or language challenges. Systems and methods according to embodiments described herein relate to tools that provide improved reading and comprehension of content output by a computing device. These tools can provide customization of how content is presented or output to accommodate a particular user's unique reading and / or comprehension and / or learning and / or language abilities, etc. In some examples, customization of content can include, for example, curating content to eliminate redundant content items that are not relevant to the reading and understanding of the content. In some examples, customization can include modifying the appearance of textual content, including, for example, font type, font size, font color, background color, pitch, spacing, and other such elements that can facilitate consumption and understanding of the textual content by a user.Systems and methods according to embodiments described herein can improve accessibility to a wide range of content for a wide range of users by accommodating a wide range of users' reading and / or comprehension and / or learning and / or language abilities for a wide range of content. Accessibility can be further improved by making assistive reading tools easily and readily available to users via their browsers, thereby enabling the use of assistive reading tools across a wide range of resources.

[0032] In some examples, the original, unmodified content can be output in a first pane, and customized content, including reformatted text content or having a modified appearance to facilitate consumption by a particular user, can be output in a second pane displayed adjacent to the first pane. Outputting the customized and / or reformatted text content in the second pane alongside the first pane displaying the original, unmodified content can provide and / or maintain context for the customized and / or reformatted text content. For example, content items, such as images, that were excluded during the reorganization of the original content can be displayed simultaneously with the customized and / or reformatted text content to facilitate user consumption. Scrolling of the customized and / or reformatted text content output in the second pane can be synchronized with scrolling of the original, unmodified content output in the first pane so that context is maintained as the user scrolls through the content. In some situations, the original content can be output in a format that cannot be easily extracted. Thus, in some examples, systems and methods according to embodiments described herein may provide for selection of portions of the original content to be customized or reformatted, as described above, and outputting the customized and / or reformatted text content in a second pane, as described above.

[0033] Presenting the original, unmodified content alongside the modified text content not only helps the user retain context, but also helps ensure that the publisher's intent is not lost when modifying the text content. Customizing content output in this manner based on the user's reading and / or learning and / or language ability can increase user concentration and / or facilitate the user's understanding of the content, and can improve accessibility to the content for a wide range of users with different levels of reading and / or comprehension and / or language skills.

[0034] Systems and methods according to embodiments described herein can be incorporated into the operating system of a computing device, thereby making the ability to customize content output in this manner available to users of the computing device across a wide range of web-based interfaces / applications in multiple separate browser processes without the use of third-party applications. Availability of assistive reading tools through a browser application / browser-based operating system can reduce or substantially eliminate dependency on third-party applications, and can improve security compared to third-party applications or programs because user interaction with the content is isolated within the system when the content is accessed by the user.

[0035] In some examples, users may be provided with controls that allow them to make choices about both if and when the systems, programs, or features described herein may enable the collection of user information. Such user information may include, for example, information related to the user's reading and / or learning and / or comprehension and / or language skills in connection with use of the systems and methods, information related to the user's learner / professional status, information related to the user's social network, social actions, or activities, the user's current location, and other such information. In some examples, user information may include information related to the source of content (e.g., the content to be modified or other content) and / or communications, such as content or communications from a server. Additionally, some data may be processed in one or more ways to remove personally identifiable information before being stored or used. For example, a user's identifying information may be processed so that personally identifiable information about the user cannot be determined, or, if location information is obtained, the user's geographic location may be generalized (e.g., to the city, zip code, or state level) so that the user's specific location cannot be determined. In some examples, personal information, including settings and / or preferences related to modifying content, related to a user's reading and / or comprehension and / or learning and / or language skills, may be processed before the personal information is stored or used so that such information cannot be associated with a particular user. Thus, users can control what information is collected about them, how that information is used, and what information is provided to them.

[0036] FIG. 1A illustrates an exemplary browser interface (UI) 100 according to implementations described herein. In some examples, the UI 100 is generated and rendered by a browser running on an operating system of a computing device. The exemplary UI 100 includes a tab strip 102 associated with a browser application. In this example, the UI 100 includes a browser tab 104, but this is for purposes of explanation and illustration only. The UI 100 may include any number of browser tabs and / or tab groups that can be opened by a browser application. Content 110 (e.g., web content, web pages, online resources, web applications, etc.) may be presented in a corresponding browser content window 120 of the browser application. In some examples, the content 110 may be rendered by a browser application, which may be at least partially generated or served by and / or from an external resource, such as, for example, a website, a server including a server hosting a web application, or other external resource. In some examples, content, as used herein, may include resources associated with a resource identifier, including, for example, a URL (Universal Resource Locator), a URI (Universal Resource Identifier), etc. Content may refer to the web page (resource) currently being rendered in browser content window 120, as well as other data (e.g., metadata) used in the rendering (e.g., markup and scripts that are not actually displayed), models generated to render content 110 (e.g., document object model, accessibility model, etc.).

[0037] The exemplary UI 100 includes an address bar area 106. The address of a web page displayed in a browser content window 120 may be displayed in the address bar area 106. A resource identifier for desired content may be entered (e.g., by a user) in an address entry area 108 of the address bar area 106. Other controls, icons, etc. may be included in the address bar area 106. For example, a user icon 122 may be displayed in the address bar area 106 to provide an indication of a user account or user profile associated with the browser session. This user icon 122 may be an image, text, or some other representation of the user account. Other controls and / or icons may include, for example, a forward control, a back control, a refresh control, a home control, an expand control, a save location control, a utility menu control icon 124, and other such controls and / or icons. In some examples, a scroll control icon 129 may be provided along one or more peripheral portions of the browser content window 120. User manipulation of the scroll control icon 129 may cause the content 110 presented in the browser content window 120 to scroll in a direction corresponding to the user manipulation of the scroll control icon 129. In some examples, an assistive reading control icon 126 may be included in the address bar area 106. In some examples, the assistive reading control icon 126 may be present in the address input area 108 or another portion of the address bar area 106. The assistive reading control icon 126, when selected by a user, may launch an assistive reading tool. The assistive reading control icon 126 is only one way in which an assistive reading tool may be launched. In some examples, a side panel control icon 128 may be included in the address bar area 106. When selected by a user, the side panel control icon 128 may present a side panel or a second display pane in the browser content window 120, as shown in FIG. 1B .

[0038] As shown in FIG. 1B , in some examples, selection of an assistive reading control icon 126 may launch an assistive reading tool, causing a first pane 130 and a second pane 140 to be displayed in the browser content window 120. In some examples, the original, unmodified content 110 may be displayed in the first pane 130. In some examples, modified content 115 generated by the assistive reading tool based on the original, unmodified content 110 may be displayed in the second pane 140. In some examples, the modified content 115 may be generated based on user settings and / or preferences. The user settings and / or preferences may be based on various factors, such as the user's learning style, reading / comprehension ability, language skills, etc. As described above, modifying the original content 110 to generate the modified content 115 for presentation in the second pane 140 may include, for example, cleanup, such as removal of images, etc., so that the remaining text content is presented in the modified content 115. Modifications to the remaining text content may include modifications to font size and / or style and / or color, spacing, pitch, background color, etc. In some examples, scrolling of the modified content 115 in the second pane 140 may be synchronized with scrolling of the original content 110 in the first pane 130. Placing the modified content 115 in the second pane 140 alongside the original content 110 in the first pane 130 and scrolling the modified content 115 and original content 110 in synchronization may allow a user to maintain context with the original content 110 while consuming / reading the modified content 115. The modified content 115 has been modified to fit the user's particular capabilities.

[0039] 1B , selection of the side panel control icon 128 may cause a first pane 130 and a second pane 140 to be displayed in the browser content window 120. In some examples, an assistive reading control icon 126 may be available to the user in one or both of the first pane 130 or the second pane 140. For example, user selection of the assistive reading control icon 126 from the second pane 140 may launch an assistive reading tool. In this situation, the original, unmodified content 110 may be presented in the first pane 130, and modified content generated by the assistive reading tool as described above may be presented in the second pane 140.

[0040] 2 is a block diagram of an exemplary system 200 for providing a reading assistance tool in a browser application, according to an embodiment. The system 200 includes a computing device 210 that includes an operating system 220. The computing system 210 may also be referred to as a client computing device. In some examples, the computing device 210 may include, for example, a personal desktop computing device, a personal laptop computing device, a tablet computing device, a smartphone, a wearable computing device (e.g., smart glasses, smart watches, etc.), and other such computing devices with image output capabilities. The computing device 210 may include one or more output device(s) 212 that enable a user to view visual content and / or receive audio content. Exemplary output devices 212 may include, for example, a display device (e.g., a monitor, a touchscreen, etc.) that enables a user to view and / or interact with displayed content, an audio output device (e.g., a speaker, etc.), etc. Computing device 210 may include one or more input devices 214 that enable a user to, for example, interact with content output by computing device 210, input commands for execution by computing device 210, etc. Exemplary input devices 214 may include, for example, a keyboard, a mouse, a touch-sensitive display, a trackpad, a trackball, etc. In some examples, input device(s) 214 may include an audio input device (e.g., a microphone) that detects audio signals for processing by computing device 210 as user input. In some examples, one or more sensing devices of sensing system 216 may function as input device(s) 214.For example, one or more image sensor(s) may detect user gesture input for processing as user input by computing device 210. One or more position sensor(s) may detect user gesture input (i.e., changes in the position and / or orientation of computing device 210) for processing as user input by computing device 210.

[0041] The computing device 210 may include one or more processors 218. The one or more processors 218 may be implemented as a CPU and / or GPU formed on a substrate configured to execute one or more machine-executable instructions, or software, firmware, or a combination thereof. The one or more processor(s) 218 ​​may be semiconductor-based processors including semiconductor materials capable of executing digital logic. The computing system may include one or more communications modules 219 that provide communications between the computing device 210 and other external systems and resources. The computing system 210 may include one or more memory devices 215. The memory device(s) 215 may include a main memory that stores information in a format that can be read and / or executed by the one or more processor(s) 218. The memory device 215 may store applications or modules, such as, for example, an operating system 220 and one or more applications 230, including a browser application 232, that perform certain operations when executed by the one or more processor(s) 218.

[0042] Operating system 220 may be implemented in system-level software that manages the hardware and software resources of a computing system and provides common services to computing programs executed by computing device 210. In some examples, operating system 220 is operable to run on a personal computing device, such as a laptop computing device, a desktop computing device, a convertible computing device, etc. In some examples, operating system 220 is operable to run on a mobile computing device, such as a tablet computing device, a handheld computing device (e.g., a smartphone), or a wearable computing device (e.g., smart glasses, a smart watch, etc.). Operating system 220 may include multiple modules that can provide common services and manage the resources of computing device 210.

[0043] As described above, the computing device 210 may include one or more applications 230, which represent programmed software that performs different functions. One of the applications 230 may be a browser application 232. The browser application 232 may display web pages, run web applications, and the like. In some examples, the browser application 232 may include additional functionality in the form of extensions. In some examples, the browser application 232 may also be the operating system 220 of the computing device 210. The browser application may generate and output a user interface, such as the example UI 100 shown in FIGS. 1A and 1B. The browser application 232 may include one or more modules that can facilitate the functionality of the browser application 232. In the example shown in FIG. 2, the exemplary system 200 includes an assistive reading tool, and the browser application 232 may include, for example, an assistive reading module 233, a text excerpt module 235, a formatting module 237, a synchronization module 239, and other such modules that can facilitate the operation of the assistive reading tool via the browser application 232. The incorporation of the reading assistance module 233, text excerpt module 235, formatting module 237, synchronization module 239, etc. into the browser application reduces or substantially eliminates dependency on third party applications and improves security compared to third party applications or programs as user interaction with content is isolated within the system when the content is accessed by the user.

[0044] In response to invoking the reading assistance module 233, to analyze content output by the computing device 210. , TeThe computing device 210 may access a text excerpt module 235. The text excerpt module 235 may be accessed by the computing device 210 and may include one or more models, e.g., machine learning models, trained and configured to analyze content (e.g., web pages, documents, or other such content) that is output by one of the output devices 212. In some examples, the text excerpt module 235 analyzes the content to detect attributes associated with the text content, such as, for example, headings, paragraphs, and other such attributes associated with the text content, and to generate a text excerpt for the text content output by the computing device 210. Tsu The formatting module 237 may be configured to excerpt text content. The formatting module 237 may organize the text content excerpted by the text excerpt module into a modified format that is more easily consumed by the user. For example, the formatting module 237 may apply user settings and / or preferences to the excerpted text content to make the text content more easily consumed by the user. The user settings and / or preferences may include, for example, a selected font type and / or size and / or color, pitch, spacing that are applied to generate the modified text content output by the computing device 210. In some examples, a user may choose to save preferences for formatting modifications to the text content in a user profile or account accessible by the formatting module 237, thereby eliminating the need to select the user's preferences each time the text content is excerpted and modified.

[0045] As described above, the modified text content may be output by the computing device 210 alongside the original content, including in the first and second panes 130 and 140, as in the example shown in FIG. 1B. The synchronization module 239 may control scrolling of the original, unmodified content and the modified content presented alongside it, such that scrolling is synchronized between the original, unmodified content and the modified content. This may allow a user to maintain a connection between the original, unmodified content and the modified content, for example, to preserve context with the original, unmodified content as intended by the publisher. Preserving context in this manner may improve a user's understanding of the content.

[0046] In some examples, content may be provided to computing device 210 from additional resources 250 accessible to computing device 210. In some examples, additional resources 250 are accessible to computing device 210 via network 240. In some examples, additional resources are available locally on computing device 210. In some examples, some of additional resources 250 are available to computing device 210 via network 240, and some of the additional resources are available locally on computing device 210. In some examples, resources associated with one or more of reading assistance module 233, text excerpt module 235, formatting module 237, and / or synchronization module 239 may be available on additional resources 250. In some examples, additional resources 250 may include, for example, one or more server computing systems 252, one or more processors 254, one or more memory devices 256, and one or more databases 258. One or more server computing systems 252 may include one or more computing devices, including, for example, a standard server, a group of such servers, a rack server system, etc. In some examples, the content may be accessible to computing device 210 from one or more of server computing systems 252 available, for example, via additional resources 250. The content may include web-based content, including, for example, content available via a website or the like.

[0047] 3A illustrates an exemplary browser interface (UI) 300 according to embodiments described herein. The exemplary UI 300 may be generated and rendered by a browser application (e.g., the browser application 232 described above with respect to FIG. 2 or another such browser application) running on an operating system (e.g., the operating system 220 described above with respect to FIG. 2 or another such operating system) of a computing device (e.g., the computing device 210 described above with respect to FIG. 2 or another computing device). While the systems and methods are described below with respect to the system 200 including the exemplary computing device 210 described above with respect to FIG. 2, this is for purposes of explanation and illustration only. The systems and methods described herein may be implemented by other computing systems having similar components arranged similarly or differently to those described above with respect to FIG. 2.

[0048] In this example, UI 300 includes one browser tab 304, but this is for purposes of explanation and illustration only. In this example, browser content 310 in the form of an article provided on a web page is displayed in the browser tab 304. 33A includes an address bar area 306, and the address (i.e., URL) of the web page displayed in the browser content window 320 is displayed in an address input area 308 of the address bar area 306. In the example shown in FIG. 3A, a user icon 322 is displayed in the address bar area 306. The user icon 322 may identify a user account or user profile associated with the current browser session. A utility menu control icon 324 can provide access to secondary actions and tools. In some examples, a scroll control icon 329 may be provided along one or more peripheral portions of the browser content window 320. User manipulation of the scroll control icon 329 may cause the content 310 presented in the browser content window 320 to scroll in a direction corresponding to the user manipulation of the scroll control icon 329.

[0049] In some examples, the reading assistance control icon 326 may be included in the address bar area 306, such as the address input area 308, or other portion of the address bar area 306. Selection of the reading assistance control icon 326 is one way in which an assistive reading tool can be launched. In some examples, a side panel control icon 328 may be included in the address bar area 306. Selection of the side panel control icon 328 may cause a side panel, or a second display pane, to be presented in the browser content window 320. In some examples, an assistive reading tool can be launched by selection of the side panel control icon 328. That is, in some examples, the reading assistance control icon 326 may be included in a side panel presented in response to selection of the side panel control icon 328, such that the assistive reading tool can be launched from within the side panel.

[0050] In the example shown in FIG. 3A , exemplary browser content 310 is presented in browser content window 320 in the form of an article presented on a web page. The exemplary content 310 includes text content 312, image content 315, and table content 318 arranged in browser content window 320. In this example, image content 315 includes an image presented within content 310, which is located near associated text content 312. Table content 318 includes menu items or links that provide access to additional information related to content 310 presented in browser content window 320. Users, particularly those experiencing reading and / or comprehension and / or learning and / or language challenges, may find content 310 difficult to consume in the format presented in exemplary browser content window 320 shown in FIG. 3A . Text content 312 is arranged in a section or paragraph format, with headings identifying the start of sections. The section of text preceded by the heading may be detectable by a text excerpt module, such as, for example, text excerpt module 235 described above with respect to FIG. 2, which identifies at least that portion of content 310 as text content 312. This may allow text excerpt module 235 to relatively easily separate text content 312 from image content 315, table content 318, icons representing browser control devices, links, text entry fields, and any other content that may be presented in browser content window 320 that is text content that is not clearly / easily excerptable.

[0051] Generally, browser content may include text content, image content, table content, links, input fields, browser control devices, and other types of content. In some examples, text-intensive sections of web-based content may be identified as main content intended for consumption by a user. The text excerpt module 235 may identify main content based on text characters arranged in headings, paragraphs, sentences, etc., for example. The text content excerpted from these text-intensive sections (main content) by the text excerpt module 235 may be processed by an assistive reading tool to generate modified or simplified text content that is more easily consumed by a user. For example, a web page containing primarily text-intensive content, where the majority of the page's content is text-intensive content, may be considered an extractable page. In contrast, a web page containing content such as table content 318, icons, browser control devices, text input fields, etc. may be considered a non-extractable page. Even if some of these content items may contain some text (e.g., characters, words, short phrases, etc. that identify the content item), extracting and presenting that text may not produce useful results for the user. In some implementations, a machine learning model may be used to determine whether a web page is considered an extractable page. The machine learning model may be trained, for example using supervised or semi-supervised training, to receive as input an accessibility tree (e.g., the nodes of the tree) and provide as output an indication of which nodes, if any, may be considered main content and / or a prediction of whether the web page is an extractable page.

[0052] As described above, a user may want to consume content, such as the exemplary content 310 shown in FIG. 3A , in a format tailored or customized to the user's preferences, viewing abilities, reading abilities, and comprehension abilities so that the content 310 can be more easily consumed and understood. Accordingly, the user may select to activate an assistive reading tool to extract the content 310. In some examples, a page type associated with the web page content presented in the exemplary web browser UI 300 may be determined to determine that the exemplary content 310 contains extractable content. The web page presented in the exemplary browser UI 300 may be determined to be an extractable page based on identification of sections of the text content 312. Detection may be determined by a machine learning model. A user may activate the assistive reading tool from within the browser in several different ways. In some examples, the assistive reading tool may be activated in response to a user selection of a reading assistance control icon 326. As shown in FIG. 3B , selection of the reading assistance control icon 326 may launch the assistive reading tool in a side panel located beside or alongside the main panel in which the content 310 is displayed. In other words, the original content 310 may be displayed in a first content pane 330 presented in a browser content window 320, and a side panel may represent a second pane in which other content, including modified content generated based on the original content 310, may be presented. In some examples, the assistive reading tool may be launched in response to a user selection of a side panel control icon 328 and a user selection of an assistive reading control icon 326 from a menu 342 available in the resulting side panel, as shown in FIG. 3C. In some examples, the assistive reading tool may be launched in response to a user selection of the assistive reading tool from a menu 344 displayed in response to input from an input device (i.e., a right-click of a mouse, a tap of a trackpad, a long hold on a touchscreen or touchpad, etc.), as shown in FIG. 3D.In some examples, a user may wish to consume only a portion of the textual content 312 using an assistive reading tool. As shown in FIG. 3E , the assistive reading tool may be launched in response to a user selection (e.g., highlighting) of a portion of the textual content 312 and a menu 346 that is displayed in response to input from an input device, such as selecting the assistive reading tool from a browser application menu. In some examples, the assistive reading tool may be launched in response to a user selection of the assistive reading tool from a browser application menu in the form of a drop-down menu of a utility menu control icon 324. In some examples, the assistive reading tool may be launched in response to a user selection of a reading assistance icon or key provided on an input device, such as a keyboard or touchscreen interface.

[0053] In some cases, reading aids toolThe assistive reading tool may be launched according to pre-stored user settings or according to set user preferences. For example, the assistive reading tool may be launched automatically, i.e., without additional user action or intervention. For example, in some implementations, the assistive reading tool may be launched automatically, such as in response to navigation to a particular designated website, for any / all websites containing content extractable by the assistive reading tool, in response to detection of a particular type of content, etc. Such preferences may be pre-set, stored, and associated with a user account or profile accessible via user icon 322. In some examples, such settings may include instructions to launch the assistive reading tool for a particular website specified by the user (e.g., by URL, name, etc.), for all websites having content extractable by the assistive reading tool, for a particular type of content specified by the user, for any explicitly extractable content presented in browser content window 320, etc. In some examples, preferences regarding how to customize or modify content can be selected by the user after the assistive reading tool is launched, e.g., via the assistive reading toolbar 355 presented along with the content to be modified. In some examples, preferences related to how content is modified or customized may be pre-set and stored by the user and may be saved in association with a user account or user profile accessible, for example, via user icon 322.

[0054] 3F-3H illustrate exemplary reader views generated by an assistive reading tool of the exemplary computing device 210. For example, the reader view may be generated by the assistive reading module 233 of the browser application 232 and output or displayed by one of the output devices 212 of the computing device 210. As shown in FIG. 3F, in some examples, upon launching the assistive reading tool, a first content pane 330 and a second content pane 340 may be displayed in the browser content window 320. In some examples, the original content 310 may be displayed in the first content pane 330. In some examples, the revised content 350 may be displayed in the second content pane 340. The revised content 350 may be a customized or revised presentation of the original content 310. As described above, the revised content 350 may be generated according to user settings and / or preferences. The user settings and / or preferences may be based on various factors, such as, for example, the user's learning style, the user's reading / comprehension ability, language skills, etc. 3F and 3G, in generating the modified content 350, the original content 310 is organized to remove the image content 315, such that only the modified text content 352 is included in the modified content 350 presented in the second content pane 340. That is, in the example shown in Figures 3F and 3G, the image content 315 (e.g., the first image content item 315A, the second image content item 315B, and the third image content item 315C) and the table content 318 are excluded in generating the modified content 350, such that the modified content 350 includes only the modified text content 352, presented in a format customized based on the user's preferences.

[0055] As described above, in some examples, the modification of the original text content 312 to generate the modified text content 352 may be performed according to pre-established and stored user preferences. The pre-stored user preferences may include, for example, pre-established selections for font type and / or size and / or color, background color, line spacing, etc. In some examples, preferences for how the original, unmodified content 310 is customized or modified may be selected by the user after the assistive reading tool is launched, for example, via a reading assistance toolbar 355 included in the second content pane 340. As shown in FIGS. 3F and 3G , the exemplary reading assistance toolbar 355 includes various icons that provide adjustment and selection of various characteristics to be applied in the modification of the original text content 312. In some examples, the icons may include, for example, a font type selector 355A, font size selectors 355B and 355C, a font color and / or background color selector 355D, line spacing selector 355E, a pitch selector 355F, etc. In some examples, the secondary content pane 340 includes an adjustment bar that includes an adjustment selector 357. The adjustment selector 357 may provide user adjustment of the size of the secondary content pane 340. For example, a user may manipulate the adjustment selector 357 (e.g., via an input device such as a mouse, or via touch, such as a touchscreen or touchpad) to adjust the width of the secondary content pane 340 (and the corresponding width of the first content pane 330) when moving an input device.

[0056] In some examples, an assistive reading tool (e.g., through operation of an assistive reading module of the browser application 232) may detect and excerpt (e.g., through operation of a text excerpt module 235 of the browser application 232) text content from the original content 310. The excerpted text may be processed according to the established accessibility tree to generate modified text content 352. In some examples, the accessibility tree includes semantic roles and layout information for each tree node. Thus, the accessibility tree may form the basis of a data model for excerpting text content from the original content 310. A Document Object Model (DOM) may be, for example, While the accessibility tree includes some semantic information, such as the identification of paragraph nodes defined by tags, the accessibility tree includes more extensive semantic information, providing more effective excerpting and extraction of text content. Examples of semantic information included in the accessibility tree include Accessible Rich Internet Application (ARIA) landmarks, such as headers (known as ARIA banners) and footers (known as ARIA content info), and the main text. Furthermore, the DOM does not include layout information that defines the boundaries of each element placed on a web page. These types of elements, which can be provided by the accessibility tree, are useful for informing machine learning models of where nodes are related to each other in the excerpt. Modified text content 352 may be generated according to user preferences (e.g., font type / size / color, background color / spacing, pitch, etc., implemented via the formatting module 237 of the browser application 232) and presented in the second content pane 340 of the browser content window 320.

[0057] 3F and 3G, the modified content 350 presented in the second content pane 340 is positioned next to or side-by-side with the original, unmodified content 310 in the first content pane 330. This side-by-side positioning of the first and second content panes 330 and 340 may help a user maintain context between the modified content 350 and the original, unmodified content 310. As noted above, in some examples, scrolling through the modified content 350 in the second content pane 340 may be synchronized with scrolling through the original, unmodified content 310 in the first content pane 330. Synchronizing or anchoring the modified content 350 with the original, unmodified content 310, even when scrolling the content 310, 350, may further allow a user to maintain context with the original, unmodified content 310 while scrolling and consuming the modified content 350.

[0058] In some examples, scrolling of the modified content 350 in the second content pane 340 may be performed in conjunction with scrolling of the original content 310 in the first content pane 330. Scrolling Synchronizing with the browser application 232 may be performed, for example, via a synchronization module 239 of the browser application 232. In some examples, bounding boxes are assigned to text elements. Text elements may include, for example, text characters, words, sections of content, which are primarily characters used in the language of the content. Text elements may include sections of content including individual characters, words, paragraphs, etc. In some examples, bounding boxes are assigned to or associated with text characters and / or words and / or sections of text in the original text content 312 of the original content 310 presented in the first content pane 330, for example, as part of the rendering process. The bounding boxes may be represented as nodes in a Document Object Model (DOM) tree or an accessibility tree. The text of these bounding boxes may be assigned to or associated with text characters and / or words and / or sections of text in the modified text content 352 presented in the second content pane 340. Thus, the modified textual content 352 may be anchored to (assigned to, associated with) one or more bounding boxes associated with corresponding portions of the original textual content 312. With the textual content 312 and the modified textual content 352 of the original content 310 anchored in this manner, user manipulation of the scroll control icons 329 causes scrolling of the modified textual content 352 and corresponding synchronized scrolling of the original content 310. The corresponding synchronized scrolling of the original content 310 includes scrolling of the textual content 312 and scrolling of the image content 315 / image content items 315A, 315B, 315C. In some examples, scrolling may be performed through user manipulation of other types of input devices, such as, for example, directional keys on a keyboard, touch-and-drag input on a touchpad or touchscreen, or manipulation of a scroll wheel on a mouse.Similarly, synchronized scrolling of content can begin with the original content 310 presented in the first content pane 330, and the modified text content 351 presented in the second content pane 340 can scroll together with the original content 310 in the first content pane 330.

[0059] In the exemplary arrangement shown in Figure 3H, the user manipulated the scroll control icon 329 from a first position shown in Figures 3E and 3F to a second position shown in Figure 3H. User manipulation of the scroll control icon 329 in this manner caused scrolling of the modified textual content 352 presented in the second content pane 340 and corresponding synchronized scrolling of the original content 310 in the first content pane 330. In the scrolled arrangement shown in Figure 3H, both the textual content 312 and the image content 315 of the original content 310 are scrolled in coordination with the scrolling of the modified textual content 352, such that the first image content item 315A is no longer visible in the first content pane 330. In this example, the promotional content item 313, along with an additional portion of the original textual content 312, has become visible in the original content 310 presented in the first content pane 330. The additional portions of the original text content 312 are modified, reformatted, and included in modified text content 352 presented in the second content pane 340. However, the promotional content items 313 are not included in the modified content 350 presented in the second content pane 340, which matches the preselected and stored user preferences.

[0060] In the examples described above with respect to Figures 3A-3H, the web pages include content in the form of main text content that can be identified and analyzed, for example, using the accessibility tree described above. For example, web pages presenting content such as articles, papers, blogs, etc. can be identified by analyzing the accessibility tree information. These types of extractable web pages presenting this type of content can be extracted by an assistive reading tool and presented in a reader view as described above. Some web pages do not include this type of main text content. For example, some web pages include content that can be identified as tabular content or tabular information, either alone or with other types of content that are not identified as main text content (e.g., email interfaces, spreadsheet applications, calendar or schedule pages, etc.). These types of non-extractable web pages may be processed differently by an assistive reading tool, so that the results presented to the user in the reader view address the user's needs and allow the user to consume the content more easily.

[0061] FIG. 4A illustrates an exemplary browser interface (UI) 400 representing a web page of an email application. Similar to the exemplary browser UI 300 described above, the exemplary UI 400 may be generated and rendered by a browser application (e.g., browser application 232 described above with respect to FIG. 2 or another such browser application) running on an operating system (e.g., operating system 220 described above with respect to FIG. 2 or another such operating system) of a computing device (e.g., computing device 210 described above with respect to FIG. 2 or another computing device). The exemplary UI 400 includes one browser tab 404, with browser content 410 in the form of an email inbox presented in a browser content window 420 associated with the browser tab 404, for ease of explanation and illustration only. The browser content 410 may be provided, for example, by additional resource 250 described above with respect to FIG. 2 (e.g., a server hosting a website or web application) and rendered by the browser application for output by the computing device. The example UI 400 shown in FIG. 4A includes an address bar region 406, where a web page address (i.e., URL) is displayed in an address entry area 408 of the address bar region 406. A user icon 422 is displayed in the address bar region 406 to identify a user account or user profile associated with the current browser session. A utility menu control icon 424 can provide access to secondary actions and tools. A scroll control icon 429 can be provided along one or more peripheral portions of the browser content window 420 for a user to scroll the content 410 presented in the browser content window 420. In some examples, a reading assistance control icon 426 can be included in the address bar region 406.In some examples, reading assistance control icon 426 may be omitted / not displayed in response to detecting unextractable content output in browser content window 420. In some examples, side panel control icon 428 may be selected to cause a side panel, or second display pane, to be presented in browser content window 420. The functionality of the icons / controls provided in association with exemplary UI 400 may be similar to the functionality described above with respect to Figures 3A-3G, and therefore, redundant detailed description will be omitted.

[0062] In the example shown in FIG. 4A , the exemplary content 410 represents an email interface containing a list of email communications in a user's inbox. In some examples, a page type associated with the web page content presented in the exemplary web browser UI 400 may be determined to determine that the exemplary content 410 contains unextractable content. The web page presented in the exemplary browser UI 400 may be determined to be an unextractable page based on the absence of text-intensive sections of text content that can be extracted into simplified content for consumption by the user. Existing reading modes may be unavailable or may not be usable for extracting this type of content and, therefore, may not be able to provide the user with a simplified, more easily consumable presentation of the content. According to implementations described herein, the assistive reading tool may further provide the user with an option to extract content contained in the web page deemed to be an unextractable page.

[0063] In response to selecting one of the content items within the content presented in browser content window 420 (e.g., a link to the first email item in a list of email items in an inbox), exemplary content 412 is displayed in browser content window 420 in the form of the content of the selected email item, as shown in FIG. 4B. Analysis of exemplary content 410 shown in FIG. 4A and exemplary content 412 shown in FIG. 4B using an accessibility tree, for example, via text excerpt module 235 of browser application 232, may determine that content 410 of a web page (i.e., an email interface) and content 412 including the selected email item, as well as various other links, input fields, control items, etc. shown in FIG. 4B, cannot be easily extracted. For example, due to the tabular format of the text, lists of links, input fields, control items, etc., a standard extraction of content 410 shown in FIG. 4A and / or content 412 shown in FIG. 4B is unlikely to produce modified text that is easily consumable by a user. In some examples, in response to determining that the web page displayed in browser content window 420 is not readily extractable, assistive reading control icon 426 may be omitted / not displayed in address bar area 406 of browser tab 404. Omission of assistive reading control icon 426 in address bar area 406 of browser tab 404 may provide an indication to the user that the web page and associated content presented in browser content window 420 is not extractable. While assistive reading control icon 426 is included in FIG. 4A , this is solely for purposes of explanation and illustration.

[0064] As noted above, users, particularly those experiencing reading and / or comprehension and / or learning and / or language challenges, may find it difficult to consume content 410 shown in FIG. 4A and / or content 412 shown in FIG. 4B in the format presented in exemplary browser content window 420. Users can benefit from the ability to consume content 410 and / or content 412 in a format tailored or customized to the user's preferences and abilities, so that content 410 and / or content 412 can be more easily consumed and understood. In situations where an assistive reading tool cannot easily isolate and / or excerpt the text and / or extract the content into a format that is easy for the user to consume, the user can still apply the assistive reading tool to the relevant portions of the content. For example, as shown in FIG. 4C, an assistive reading tool may be activated in response to a user selection (e.g., highlighting) of a portion of content 412, e.g., a portion of content 412 that is of particular interest to the user. A user may select an assistive reading tool from a menu 446, e.g., a browser application menu, that is displayed in response to input from an input device (i.e., right-clicking a mouse, tapping on a trackpad, long-holding on a touchscreen or touchpad, entering a shortcut key on a keyboard, etc.). In this example, a reader view, such as that shown in FIG. 4D , may be presented in the browser content window 420 in response to activating the assistive reading tool. The reader view may be generated by the assistive reading module 233 of the browser application 232 and may be output or displayed by one of the output devices 212 of the computing device 210. In some examples, activating the assistive reading tool and generating the reader view may cause a first content pane 430 and a second content pane 440 to be presented side-by-side in the browser content window 420. The original, unmodified content 412 may be presented in the first content pane 430, and the customized or modified content 414 may be presented in the second content pane 440.In this example, modified content 414 represents a customized or modified version of the exemplary portion of original content 412 selected by the user as shown in FIG. 4C.

[0065] As noted above, in some examples, preferences for how to customize or modify the original content 412 to generate the modified content 414 may be selected by the user after the assistive reading tool is launched, for example, via the assistive reading toolbar 455 presented alongside the content to be modified in the second content pane 440. In some examples, preferences for how the content is modified or customized may be pre-set by the user. Such preferences may be saved in association with a user account or user profile accessible, for example, via the user icon 422. User preferences may be based on various factors, such as the user's learning style, the user's reading and / or comprehension ability, language skills, etc. Pre-stored user preferences may include, for example, pre-stored selections for font type and / or size and / or color, background color, line spacing, etc. In the example shown in FIG. 4D , in generating the modified content 414, the selected portion of the original text content 312 is presented in a customized format based on the user's preferences, without additional input fields, control icons, links, etc., which may be distracting or negatively affect the user's consumption of the content. In some examples, the second content pane 440 includes an adjustment bar that includes an adjustment selector 457 for a user to adjust the size of the second content pane 440. For example, a user may manipulate the adjustment selector 457 (e.g., via an input device such as a mouse, or via touch, such as a touchscreen or touchpad) to adjust the width of the second content pane 440 (and the corresponding width of the first content pane 330) when moving an input device.

[0066] FIG. 5 is a flowchart of an exemplary method 500 according to embodiments described herein. Method 500 may be performed by a browser application, such as browser application 232 described above with respect to FIG. 2. Method 500 may provide for launching a reading assistance tool and outputting content via a display device of a computing device, such as computing device 210 described above with respect to FIG. 2, in a reader view presented in a browser user interface, for example. The reader view may present both the original, unmodified content and the excerpted text content in side-by-side reading panes, allowing a user to maintain the originally intended context when consuming the excerpted text content. The appearance of the excerpted text content may be modified according to user preferences to facilitate the user's reading and understanding of the excerpted text content. Scrolling of the excerpted text content may be synchronized or anchored with scrolling of the original content in its original form to maintain the originally intended context even as the content is scrolled. Excerpting text content from the original content and modifying the appearance of the excerpted text content may reduce visual noise and allow users to view the excerpted text content in a format that accommodates the user's particular abilities. Presenting the original, unmodified content and the excerpted text content side-by-side in a format, i.e., with a modified appearance based on the user's preferences, and synchronized scrolling of the original, unmodified content and the excerpted / reformatted text content, may facilitate the user's reading and comprehension of the text while maintaining improved comprehension through the connection with the original, unmodified content.

[0067] The browser application may render content for output to the user, for example, via a display device of the computing device (block 510). The content may be a presentation of a web page, including text content, image content, links, input fields, control devices, etc. The rendered content may be output to a browser content window of a browser user interface generated by the browser application. The browser application may include an assistive reading tool implemented by an assistive reading module of the browser application. The assistive reading tool may be activated in many different ways. In some examples, the assistive reading tool may be actively activated for any web page output by a particular user's browser application, or for specified web pages, or for specified types of content, without specific user input or intervention. In some examples, the user may initiate activation of the assistive reading tool, for example, through selection of a reading assistance control icon available in a side panel application within the browser user interface and / or browser content window, and / or through selection from within an input device control menu and a browser control menu.

[0068] In response to invoking the assistive reading tool (block 520), the content of the browser content window output by the browser application may be analyzed, and text content may be extracted from the content (block 530). The content may include, for example, image content, links, input fields, control devices, and other such items in addition to text content. In some examples, the content may be analyzed using, for example, an accessibility tree, to determine the type of web page presented in the browser user interface and the extent to which the associated content is extractable in its current form. The text content may be extracted by a text excerpting module, such as, for example, text excerpting module 235 of browser application 232 described above with reference to FIG. 2.

[0069] The excerpted text may be customized or modified according to a particular user's preferences to accommodate the user's reading and / or comprehension and / or learning and / or language abilities. The excerpted text may be formatted by a formatting module, such as the exemplary formatting module 237 of the browser application described above with reference to FIG. 2, to incorporate preferences for font type and / or size and / or color, background color, spacing, pitch, etc., that facilitate user consumption of the excerpted text. In some examples, these preferences may be pre-stored and associated with a user account or user profile.

[0070] The formatting module 237 may generate modified excerpted text content in response to detecting pre-stored preferences for modifying the excerpted text content (block 540) (block 550). The modified excerpted text content may be displayed in a reader view within the browser content window along with the original, unmodified content (block 565). In particular, the original, unmodified content may be displayed in a first content pane presented within the browser content window, and the modified excerpted text content may be displayed in a second content pane presented within the browser content window. The second content pane may be positioned beside the first content pane in the browser content window, such that the modified excerpted text content can be viewed alongside or together with the original content.

[0071] If no user preferences for formatting / modifying the excerpted text content are pre-stored (block 540), the excerpted text content may be displayed in a reader view within the browser content window along with the original content (block 545). In the reader view presented within the browser content window, the original content may be displayed in a first content pane, and the excerpted text content may be displayed in a second content pane positioned alongside the first content pane. In some examples, a reading assistance toolbar, such as the reading assistance toolbar 355 described above with respect to FIGS. 3A-3G and / or the reading assistance toolbar 455 described above with respect to FIGS. 4A-4D, may be provided by the browser application. The reading assistance toolbar may enable formatting or reformatting of the excerpted text content to suit the needs of a particular user. In response to the correction input detected via the reading assistance toolbar (block 555), the formatting module 237 may generate the modified excerpted text content (block 560). In a reader view presented within a browser content window, the modified excerpted text content may be displayed in a first content pane; original The content may be displayed in a second content pane positioned alongside the first content pane (block 565), so that the modified excerpted text content can be viewed alongside or together with the original content.

[0072] In response to a scroll input causing scrolling of the excerpted text content (modified or unmodified) displayed in the second pane (block 570), the system may anchor the (modified or unmodified) text content in the second content pane together with the original, unmodified content of the first content pane (block 580). Similarly, in response to a scroll input causing scrolling of the original, unmodified content of the first content pane, the system may anchor the unmodified content of the first content pane to the (modified or unmodified) text content of the second content pane. In some examples, a synchronization module, such as synchronization module 239 of browser application 232 described above with respect to FIG. 2, may anchor bounding boxes assigned to or associated with text characters and / or words and / or sections of text of the original, unmodified content of the first pane to bounding boxes associated with corresponding text characters and / or words and / or sections of text of the excerpted text content (modified or unmodified) of the second pane of the reader view. With the original, unmodified content and the excerpted text content (modified or unmodified) anchored in this manner, user manipulation of a scrolling device (e.g., scroll control icons displayed in a browser user interface, directional keys on a keyboard, a mouse wheel, drag input on a touchpad or touchscreen, etc.) causes the system to perform synchronous scrolling of the excerpted text content (modified or unmodified) and the original, unmodified content (block 590).

[0073] Thus, an assistive reading tool is implemented in a browser application to facilitate the presentation of text content in a manner and format that addresses an individual user's reading and / or comprehension abilities while maintaining the context of the original content. The assistive reading tool outputs a reader view pane along with a main content pane. The original content is presented in the main content pane, and simplified / reformatted content corresponding to text content excerpted from the original content is presented in the reader view pane. Scrolling of the excerpted text content in the reader view pane is synchronized with scrolling of the original content in the main content pane, so that context is maintained as the user navigates through the content. The simultaneous presentation of the original content and the excerpted text content allows the user to more easily consume the excerpted text content while maintaining the originally intended context.

[0074] In one general aspect, a computer-implemented method includes: excerpting, by a text excerpting module of a browser application executing on a computing device, text content included in browser content displayed by the browser application; generating corrected text content based on the excerpted text content; and rendering, by the browser application, a reader view for displaying the corrected text content together with the browser content. The rendering may include rendering a first content pane that displays the browser content and rendering a second content pane adjacent to the first content pane that displays the corrected text content. The computer-implemented method may also include providing a reader view including the second content pane adjacent to the first content pane for display on a display device of the computing device.

[0075] In some implementations, generating the corrected text content includes at least one of modifying a format of the excerpted text content according to pre-stored user preferences for displaying the corrected text content in the second content pane or modifying a format of the excerpted text content in response to user input detected in a reading assistance toolbar of the browser application. Modifying the format of the excerpted text content may include modifying at least one of a font type, a font size, a font color, a background color, a pitch, or line spacing of the excerpted text content.

[0076] In some embodiments, a computer-implemented method includes rendering browser content by a browser application, analyzing the browser content by an assistive reading module of the browser application, and determining a page type associated with the browser content based on the analyzing, where determining includes determining that the browser content is associated with an extractable page in response to detecting excerptable text content in the browser content. In some embodiments, the computer-implemented method includes launching an assistive reading tool, where launching the assistive reading tool includes at least one of launching the assistive reading tool in response to determining that the browser content is associated with an extractable page or launching the assistive reading tool in response to user input selecting the assistive reading tool from a browser application menu. In some embodiments, the computer-implemented method includes determining that the browser content is associated with a non-extractable page, detecting user input selecting a section of text content from the browser content, and launching the assistive reading tool in response to user input selecting the assistive reading tool from the browser application menu. In some embodiments, generating corrected text content includes generating corrected text content based on the section of text content selected by the user input.

[0077] In some embodiments, a computer-implemented method includes detecting a scroll input for scrolling modified text content displayed in a second content pane, scrolling the modified text content displayed in the second content pane in response to the scroll input, and scrolling browser content displayed in the first content pane in response to the scroll input.

[0078] In some implementations, a computer-implemented method includes detecting a scroll input for scrolling browser content displayed in a first content pane, scrolling the browser content displayed in the first content pane in response to the scroll input, and scrolling corrected text content displayed in a second content pane in response to the scroll input. Scrolling the browser content displayed in the first content pane may include synchronizing the scrolling of the browser content displayed in the first content pane with scrolling of the corrected text content displayed in the second content pane. Synchronizing the scrolling of the browser content displayed in the first content pane with scrolling of the corrected text content displayed in the second content pane may include associating first bounding boxes assigned to text elements of the browser content displayed in the first content pane with second bounding boxes assigned to corresponding text elements of the corrected text content displayed in the second content pane, and performing the scrolling synchronization based on the association of the first bounding boxes and the second bounding boxes.

[0079] In another general aspect, a non-transitory computer-readable medium stores instructions that, when executed by at least one processor of a computing device, cause the at least one processor to: excerpt, by a text excerpting module of a browser application executing on the computing device, text content included in browser content displayed by the browser application; generate modified text content based on the excerpted text content; and render, by the browser application, a reader view for displaying the browser content together with the modified text content, wherein rendering the reader view includes rendering a first content pane that displays the browser content and rendering a second content pane beside the first content pane that displays the modified text content. The instructions also cause the at least one processor to provide a reader view including the first content pane and the second content pane for display on a display device of the computing device.

[0080] In some embodiments, the instructions cause the at least one processor to detect pre-stored user settings associated with displaying the modified text content in the second content pane, modify a format of the excerpted text content according to the pre-stored user settings, detect user input in a reading assistance toolbar of the browser application, and modify the format of the excerpted text content in response to the user input detected in the reading assistance toolbar. In some embodiments, the instructions cause the at least one processor to render, by the browser application, the browser content, analyze, by a reading assistance module of the browser application, the browser content, and determine a page type associated with the browser content, wherein determining the page type includes determining that the browser content is associated with an extractable page in response to detecting the excerptable text content in the browser content.

[0081] In some embodiments, the instructions cause the at least one processor to launch an assistive reading tool, where launching the assistive reading tool includes at least one of: launching the assistive reading tool in response to determining that the browser content is associated with an extractable page, or launching the assistive reading tool in response to user input selecting the assistive reading tool from a browser application menu. In some embodiments, the instructions cause the at least one processor to determine that the browser content is associated with a non-extractable page, detect user input selecting a section of text content from the browser content, and launch the assistive reading tool in response to user input selecting the assistive reading tool from the browser application menu. In some embodiments, the instructions cause the at least one processor to generate modified text content based on the section of text content selected by the user input.

[0082] In some implementations, the instructions cause the at least one processor to detect a scroll input for scrolling the modified text content displayed in the second content pane, scroll the modified text content displayed in the second content pane in response to the scroll input, and scroll the browser content displayed in the first content pane in response to the scroll input. In some implementations, the instructions cause the at least one processor to synchronize scrolling of the browser content displayed in the first content pane with scrolling of the modified text content displayed in the second content pane, where synchronizing includes associating first bounding boxes assigned to text elements of the browser content displayed in the first content pane with second bounding boxes assigned to corresponding text elements of the modified text content displayed in the second content pane, and synchronizing the scrolling based on the association of the first bounding boxes and the second bounding boxes.

[0083] In some implementations, the browser content includes text content and at least one of image content, table content, links, input fields, or browser control devices, and the instructions cause the at least one processor to modify at least one of a font type, font size, font color, background color, pitch, or line spacing of the excerpted text content.

[0084] In another general aspect, a computing system includes a processor, a display, and a memory including instructions that, when executed by the processor, cause a browser application running on the computing system to perform operations. The operations may include: excerpting, by a text excerpt module of the browser application, text content included in browser content displayed on the display; generating revised text content based on the excerpted text content; and rendering a reader view to display the browser content together with the revised text content, wherein rendering the reader view includes rendering a first content pane that displays the browser content and rendering a second content pane beside the first content pane that displays the revised text content, and the operations further include providing the reader view including the first content pane and the second content pane for display on the display.

[0085] In addition to the above, users may be provided with controls that allow them to make choices about whether and when the systems, programs, or features described herein enable the collection of user information (e.g., information about the user's social networks, social actions or activities, occupation, user preferences, or the user's current location), as well as whether content or communications are sent from the server to the user. Furthermore, certain data may be processed in one or more ways to remove personally identifiable information before it is stored or used. For example, a user's identifying information may be processed so that personally identifiable information about the user cannot be determined, or, if location information is obtained, the user's geographic location may be generalized (such as to the city, zip code, or state level) so that the user's specific location cannot be determined. Thus, users can control what information is collected about them, how that information is used, and what information is provided to them.

[0086] Various implementations of the systems and techniques described herein may be realized in digital electronic circuitry, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementation in one or more computer programs executable and / or interpretable by a programmable system including at least one programmable processor, which may be special or general-purpose, coupled to receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0087] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural and / or object-oriented programming language, and / or assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., magnetic disk, optical disk, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0088] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and pointing device (e.g., a mouse or trackball) by which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form, including acoustic, verbal, or tactile input.

[0089] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or that includes middleware components (e.g., an application server), or that includes front-end components (e.g., a client computer having a graphical user interface or web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), and the Internet.

[0090] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0091] As used herein and in the appended claims, the singular forms "a," "an," and "the" do not exclude plural references unless the context clearly dictates otherwise. Furthermore, conjunctions such as "and," "or," and "and / or" are inclusive unless the context clearly dictates otherwise. For example, "A and / or B" includes A alone, B alone, and A and B. Furthermore, connecting lines or connectors shown in the various figures presented are intended to represent exemplary functional relationships and / or physical or logical couplings between the various elements. Many alternative or additional functional relationships, physical connections, or logical connections may be present in a practical device. Furthermore, unless an element is specifically described as "essential" or "critical," no item or component is essential to the practice of the embodiments disclosed herein.

[0092] Terms such as, but not limited to, about, substantially, generally, etc. are used herein to indicate that an exact value or range of values ​​is not required or need not be specified. As used herein, the above terms have meanings that can be easily and immediately understood by one of ordinary skill in the art.

[0093] Furthermore, use of terms such as up, down, top, bottom, side, end, front, back, etc. herein are used with reference to the orientation currently considered or illustrated, and it is understood that when they are considered with respect to other orientations, such terms will need to be modified accordingly.

[0094] Although certain exemplary computer-implemented methods, apparatus, and articles of manufacture have been described herein, the scope of coverage of this patent is not limited thereto. It is to be understood that the terminology used herein is for the purpose of describing particular aspects and is not intended to be limiting. On the contrary, this patent covers all methods, apparatus, and articles of manufacture fairly falling within the scope of the claims of this patent.

[0095] In one aspect, a non-transitory computer-readable medium includes instructions that, when executed by a processor at a receiving computing device, cause the receiving computing device to perform any of the methods disclosed herein.

[0096] In one aspect, a computing device may be configured with at least one processor and a memory that stores instructions that, when executed by the at least one processor, perform any of the methods disclosed herein.

Claims

1. 1. A computer-implemented method comprising: extracting, by a text excerpting module of a browser application executing on a computing device, text content included in browser content displayed by the browser application, the browser content including the text content and additional content, the additional content including at least one of image content, table content, link, input field, or browser control, the method further comprising: generating modified text content based on the excerpted text content; and rendering, by the browser application, a reader view, wherein rendering the reader view includes: rendering a first content pane that displays the browser content including the text content and the additional content; and rendering a second content pane next to the first content pane that displays the modified text content and omits the additional content, the method further comprising: synchronizing a display of the browser content, including the text content and the additional content, in the first content pane with the modified text content in the second content pane, wherein the synchronizing includes: Associating first bounding boxes assigned to text elements of the browser content and the additional content displayed in the first content pane with second bounding boxes assigned to corresponding text elements of the modified text content displayed in the second content pane; Anchoring the second bounding box to the first bounding box; and synchronizing the display such that the textual content and the additional content displayed in the first content pane remain anchored to the corresponding modified textual content displayed in the second content pane based on the anchoring of the second bounding box to the first bounding box. A computer-implemented method.

2. generating the modified text content includes: modifying the format of the excerpted text content according to pre-stored user preferences associated with displaying the modified text content in the second content pane; or modifying the format of the excerpted text content in response to user input detected on a reading assistance toolbar of the browser application; 10. The computer-implemented method of claim 1, comprising at least one of:

3. 3. The computer-implemented method of claim 2, wherein modifying the format of the excerpted text content includes modifying at least one of a font type, a font size, a font color, a background color, a pitch, or line spacing of the excerpted text content.

4. rendering, by the browser application, the browser content; analyzing the browser content by a reading assistance module of the browser application; determining, in response to detecting excerptable text content within the browser content, that the browser content is associated with an extractable page; The computer-implemented method of any one of claims 1 to 3, further comprising:

5. The method further includes starting a reading assistance tool, wherein starting the reading assistance tool includes: In response to determining that the browser content is associated with the extractable page, launching the assistive reading tool; or launching the reading assistance tool in response to a user input selecting the reading assistance tool from a browser application menu; 5. The computer-implemented method of claim 4, comprising at least one of:

6. determining that the browser content is associated with a non-extractable page; detecting user input selecting a section of text content from the browser content; launching the reading assistance tool in response to a user input selecting the reading assistance tool from a browser application menu; The computer-implemented method of claim 4 further comprising:

7. 7. The computer-implemented method of claim 6, wherein generating the modified textual content comprises generating the modified textual content based on the section of textual content selected by the user input.

8. detecting a scroll input for scrolling the modified text content displayed in the second content pane; scrolling the modified text content displayed in the second content pane in response to the scroll input; scrolling the browser content displayed in the first content pane in response to the scroll input; The computer-implemented method of any one of claims 1 to 3, further comprising:

9. detecting a scroll input for scrolling the browser content displayed in the first content pane; scrolling the browser content displayed in the first content pane in response to the scroll input; scrolling the modified text content displayed in the second content pane in response to the scroll input; The computer-implemented method of any one of claims 1 to 3, further comprising:

10. instructions that, when executed by at least one processor of a computing device, cause the at least one processor to: and causing a text excerpting module of a browser application executing on the computing device to excerpt text content included in browser content displayed by the browser application, the browser content including the text content and additional content, the additional content including at least one of image content, table content, link, input field, or browser control, the instructions further causing the at least one processor to: generating modified text content based on the excerpted text content; and rendering a reader view, wherein rendering the reader view comprises: rendering a first content pane that displays the browser content including the text content and the additional content; and rendering a second content pane next to the first content pane that displays the modified text content and omits the additional content, wherein the instructions further direct the at least one processor to: and synchronizing a display of the browser content, including the text content and the additional content, in the first content pane with the modified text content in the second content pane, the synchronizing comprising: Associating first bounding boxes assigned to text elements and the additional content included in the browser content displayed in the first content pane with second bounding boxes assigned to corresponding text elements of the modified text content displayed in the second content pane; Anchoring the second bounding box to the first bounding box; and synchronizing the display such that the textual content and the additional content displayed in the first content pane remain anchored to the corresponding modified textual content displayed in the second content pane based on the anchoring of the second bounding box to the first bounding box. program.

11. The instructions cause the at least one processor to: Detecting a pre-stored user preference associated with displaying the modified text content in the second content pane; and modifying a format of the excerpted text content according to the pre-stored user preferences; Detecting user input on a reading assist toolbar of the browser application; modifying the format of the excerpted text content in response to the user input detected on the reading assistance toolbar; The program according to claim 10, which causes the program to perform the following.

12. The instructions cause the at least one processor to: rendering, by the browser application, the browser content; analyzing the browser content by a reading assistance module of the browser application; determining, in response to detecting excerptable text content in the browser content, that the browser content is associated with an extractable page; The program according to claim 10 or 11, which causes the program to perform the following.

13. The instructions cause the at least one processor to launch a reading assistance tool, and launching the reading assistance tool includes: In response to determining that the browser content is associated with the extractable page, launching the assistive reading tool; or launching the reading assistance tool in response to a user input selecting the reading assistance tool from a browser application menu; The program according to claim 12, comprising at least one of:

14. The instructions cause the at least one processor to: determining that the browser content is associated with a non-extractable page; detecting user input selecting a section of text content from the browser content; launching the assistive reading tool in response to a user input selecting the assistive reading tool from a browser application menu; The program according to claim 12, which causes the program to perform the following.

15. The program of claim 14 , wherein the instructions cause the at least one processor to generate the modified text content based on the section of text content selected by the user input.

16. The instructions cause the at least one processor to: detecting a scroll input for scrolling the modified text content displayed in the second content pane; scrolling the modified text content displayed in the second content pane in response to the scroll input; scrolling the browser content displayed in the first content pane in response to the scroll input; The program according to claim 10 or 11, which causes the program to perform the following.

17. 1. A computing system comprising: a processor; The display and and a memory storing instructions that, when executed by the processor, cause a browser application running on the computing system to perform operations, including: extracting text content included in browser content displayed on the display by a text excerpting module of the browser application, the browser content including the text content and additional content, the additional content including at least one of image content, table content, link, input field, or browser control, and the operation further comprising: generating modified text content based on the excerpted text content; and rendering a reader view, wherein the rendering of the reader view comprises: rendering a first content pane that displays the browser content including the text content and the additional content; and rendering a second content pane next to the first content pane that displays the modified text content and omits the additional content, the operations further comprising: synchronizing a display of the browser content, including the text content and the additional content, in the first content pane with the modified text content in the second content pane, wherein the synchronizing includes: Associating first bounding boxes assigned to text elements of the browser content and the additional content displayed in the first content pane with second bounding boxes assigned to corresponding text elements of the modified text content displayed in the second content pane; Anchoring the second bounding box to the first bounding box; and synchronizing the display such that the textual content and the additional content displayed in the first content pane remain anchored to the corresponding modified textual content displayed in the second content pane based on the anchoring of the second bounding box to the first bounding box. Computing system.

Images