Rendering digital content on multiple displays

Abstract

A method for rendering digital content, the method comprising: Executing a first content browser (180) on a first computing device (100), wherein the content browser is configured to render a first browser window on a first display device (170) that is (a) operatively coupled to the first computing device and (b) forms part of a composite display (170'); executing a second content browser on a second computing device, wherein the second content browser is configured to render a second browser window on a second display device that also forms part of the composite display (170); receiving, by the first content browser, a first display configuration defining a first sub-region of the composite display corresponding to the first display device; Receiving, by the second content browser, a second display configuration defining a second sub-area of the composite display; Receiving, by the first content browser, a content item; Receiving, by the second content browser, the content item; rendering a first portion of the content item in the first browser window, wherein the first portion of the content item rendered corresponds to the first sub-region of the composite display; rendering a second portion of the content element in the second browser window, wherein the second portion of the content element corresponds to the second sub-region of the composite display; Detecting, in the first browser window, a modification of one of the elements that make up the content item after the part of the content item has been rendered; Generating a mutation notification that identifies the modification and the element to which the modification applies; Sending the mutation notification to the second computing device; and Modifying the appearance of the content item in the second browser window in response to processing the mutation notification.

Description

Area of Revelation

[0001] The present disclosure relates generally to the visual rendering of digital content, and more particularly to methods for segmenting digital content into a plurality of tiles that can be displayed across multiple application windows. background

[0002] Large electronic displays are used in a wide variety of applications to display a correspondingly diverse range of digital content. For example, stadiums and arenas are often equipped with giant screens used to display video images, photographs, and other information to tens of thousands of spectators. Screens are often part of advertising displays installed in store windows, convention centers, and other locations where consumers may be present. Screens are also often incorporated into information kiosks, interactive terminals, and similar devices installed in airports, train stations, amusement parks, or other places where people tend to congregate.Large screens are particularly useful in applications such as these because they can be easily viewed by crowds, are able to command attention over long distances, and can display large amounts of information. A wide range of techniques have been used to implement such screen displays, including projection screens and flat-panel arrays of liquid crystal (LCD) or light-emitting diode (LED) elements. For example, a large screen can be created by using a computer or other electronic device to drive multiple individual screens physically arranged in a matrix, such as a 3 x 3 matrix of LED televisions placed side by side. This technique can be replicated on a smaller scale by arranging a plurality of smaller devices, such as tablet computers, in a matrix.Regardless of the technology used to implement this, such composite display systems are often used with a driver that contains software and electronic circuitry capable of simultaneously driving all of the displays in a coordinated manner.

[0003] Furthermore, KR 10 1370 714 B1 relates to a method and system for splitting a web page screen for use with a multi-screen device. The method for splitting a web page into multiple screens comprises the following steps: receiving a web page whose tag file contains primary information and secondary information from a server; and rendering the web page. The primary information is information about a main screen containing an image and a figure, and the secondary information is information about an additional function of the main screen.

[0004] Furthermore, US 2014 / 0 132 833 A1 relates to a method that combines multiple screens of multiple mobile computing devices when playing a video.

[0005] CN 1 03 942 023 A relates to a display processing method for increasing page loading speed. The display processing method comprises the steps of dividing a page to be displayed by a terminal device into K subpages, where K is a positive integer greater than or equal to two; the terminal device receiving a first user instruction; the terminal device generating the K subpages according to the first user instruction and connecting the K subpages to a WebSocket server page; the terminal device receiving a second user instruction; the terminal device launching a browser at the position corresponding to each subpage on a spliced wall according to the second user instruction and allowing each browser to load the corresponding subpage. Short description of the drawings Fig. Figure 1A is a screenshot showing digital content displayed in an application window that is conceptually segmented into multiple tiles. Fig. 1B is a screenshot showing the digital content of the Fig. 1A shows this content being displayed across multiple application windows. Fig. 2 is a block diagram schematically illustrating selected components of a networked computer system that may be used to implement certain of the embodiments disclosed herein. Fig. 3 is a block diagram schematically illustrating an exemplary embodiment of a composite display device matrix including a plurality of touch-sensitive display devices. Fig. 4 is a flowchart illustrating an example method for segmenting a digital content item into a plurality of tiles displayed across multiple application windows. Fig. 5 is a data flow diagram schematically illustrating example data flows that occur when a digital content item is segmented into a plurality of tiles displayed across multiple application windows. Fig. Figure 6A shows an example cascading style sheet mutation notification message transmitted from a sensing browser to one or more responsive browsers. Fig. Figure 6B shows an example Document Object Model mutation notification message transmitted from a detecting browser to one or more responding browsers. Detailed description

[0006] Large electronic displays often comprise a plurality of smaller flat panel displays arranged side by side, for example, to form a matrix. When multiple displays are arranged in this manner, the resulting larger display matrix is often referred to as a composite display or a multi-panel display. With the improvement of display technology, the decrease in individual bezel sizes, and the decrease in display costs, composite displays have become increasingly popular. For example, the increasing versatility of tablet computers, handheld computers, and other portable devices has made it possible to realize composite displays on a smaller scale using such devices.Despite their popularity, existing composite displays still suffer from a number of shortcomings, partly because such displays often rely on a proprietary drive system designed for use with a specific display configuration and / or display hardware. This makes it difficult for users to assemble their own custom displays that have a unique size or shape, or use devices for which a composite display drive system is not available. More fundamentally, many existing composite display drive systems rely on screen-based logic, which relies on dividing a larger screen area into smaller segments. This approach cannot easily be applied to page-based digital content, such as web pages, which have become ubiquitous in current computing environments.For example, existing display segmentation systems lack the ability to robustly extend the interactive nature of web content to a composite display environment. These deficiencies translate into a substantial handicap for developers seeking to extend the functionality and market penetration of composite display systems.

[0007] For this reason, and in accordance with certain of the embodiments disclosed herein, techniques are provided for segmenting digital content into a plurality of tiles that can be displayed across multiple application windows. For example, in a specific implementation, the methods disclosed herein can be applied to segment a web page into a plurality of tiles, wherein each of the tiles can be displayed in a separate web browser displaying a portion of the original web page. Generally, each content browser identifies which tile to display based on a configuration that can be stored locally by the browser.To maintain the interactivity provided, for example, by active content, each browser includes a mutation observer configured to detect changes in a document object model (DOM) and / or a cascading style sheet (CSS) associated with the displayed content. Detected changes are recorded in notification messages that can be broadcast to other content browsers via a WebSocket connection or via a direct peer-to-peer network. This approach advantageously allows page-based content to be displayed using a composite display without using a proprietary triggering system, making it easy to manipulate the size and shape of the composite display and to implement the display using a wide variety of different display devices.

[0008] As used herein, the term "content," in addition to its ordinary meaning, refers to information intended for direct or indirect consumption by a user. For example, the term content includes information that is directly consumed by a user, such as when displayed directly on a display device, rendered using a content browser, or printed on a piece of paper. The term content also includes information that is not specifically intended to be displayed and therefore also includes elements such as software, executable instructions, scripts, hyperlinks, addresses, pointers, metadata, and formatting information. For example, a web page can be understood as a content element consisting of a plurality of other content elements such as images, text, hyperlinks, formatting information, and metadata.The use of the term content is (a) independent of how the content is presented to the user for consumption, and (b) independent of the software application used to create the content or used to render the content. Content may be encoded using one or more compression algorithms designed to reduce the bandwidth required to transmit the content over a network. Content may also be characterized as comprising "content items" or "assets." The terms "digital content" and "digital assets" refer to content encoded in binary numbers (e.g., zeros and ones). Therefore, in the context of applications involving digital computing, the terms "content," "digital content," "asset," and "digital assets" are often used interchangeably.

[0009] As used herein, the terms "browser" and "content browser," in addition to their usual meanings, refer to software applications configured to retrieve and render digital content obtained from a storage resource. Content to be browsed may be identified by a network address entered by a user or selected from previously rendered content in the form of a hyperlink. A browser may include both user interface elements that provide access to functionality associated with consuming content (such as scroll bars, menu bars, toolbars, and the like) and a browser window in which the rendered content appears. It will be understood that a browser window may or may not have a visible border.The browser may be configured to render content stored according to a wide variety of file formats and protocols. The functionality provided by a content browser may be extended through the use of plug-ins and / or extensions, which may be configured to establish, for example, a WebSocket connection, allowing content to be actively pushed between connected devices. A "web browser" is a class of browser specifically configured to retrieve and render digital content accessible via the web (World Wide Web, WWW), although many web browsers are also capable of accessing information provided by content servers on private networks and by file servers on a variety of different file systems.

[0010] As used herein, the term "display," in addition to its usual meaning, refers to a device configured to visually render content. Display devices may be implemented using a wide range of technologies, including projection displays, arrays of flat panel displays with LCD or LED elements, and plasma flat panel displays. Displays may also be implemented using a touch-sensitive surface, resulting in a display that can function as both an input device and an output device. A "composite" or "multi-panel" display is fabricated by arranging a plurality of display devices side by side, resulting in a larger display having a custom dimension and / or shape.While composite displays often have a rectangular shape, it will be understood that, in general, substantially any shape or size can be used for a composite device, and therefore, in certain embodiments, a composite display may have an irregular and / or asymmetrical shape, such as an L-shape or an O-shape with a hollow central region. A composite display may include any suitable number of display devices. In some cases, a composite display may include displays having different sizes and / or shapes, such that, for example, a relatively wider display spans the combined widths of two or more relatively narrower displays.

[0011] As used herein, the term "tile," in addition to its usual meaning, refers to a conceptual sub-area of a content item or a sub-area of a composite display in which at least a portion of the content item is rendered. For example, a content browser may be understood as including a browser window that can be conceptually divided into a left tile and a right tile. Portions of rendered content appearing in the left tile may be configured to appear on a first (left-side) screen of a composite display, while portions of rendered content appearing in the right tile may be configured to appear on a second (right-side) screen of a composite display.It will be understood that tiles are generally conceptual elements that can be configured to have a custom shape and size. For example, . Fig. 1A a screenshot showing digital content displayed in a browser window 3000 that is conceptually segmented into multiple tiles, while the Fig. 1B shows the same content displayed across a first browser window 3100 corresponding to the first tile and a second browser window 3200 corresponding to the second tile. A tile may also be referred to as a "viewport." System architecture

[0012] Fig. 2 is a block diagram schematically illustrating selected components of a networked computer system 1000 that may be used to implement certain of the embodiments disclosed herein. As shown, the networked computer system 1000 includes one or more computing devices 100, and may also include a content server 200 and / or a configuration manager 300. The computing device 100, and optionally the content server 200 and / or the configuration manager 300, may be configured to communicate with each other over a network 500. The architecture and functionality of the various components and subcomponents that make up the networked computer system 1000 will be described in turn. In general, however, it will be understood that such embodiments provide techniques for segmenting digital content into a plurality of tiles that may be displayed across multiple application windows.However, because the particular resources, components, and functionality provided in an implementation may be tailored specifically to the requirements of a particular application, this disclosure is not intended to be limited to the provision of any particular resources, components, or functionality, or to exclude any resources, components, or functionality.

[0013] Computing device 100 includes, for example, a device selected from a desktop computer, a laptop computer, a workstation computer, a tablet computer, a smartphone, a handheld computer, a set-top box, or other suitable computing device. In certain embodiments, a combination of different types of computing devices may be used. Computing device 100 is associated with a display device 170, such as a touch-sensitive display device. In general, display device 170 may be implemented using a wide range of technologies, including projection displays, flat panel arrays of LCD or LED elements, and plasma flat panel displays. Display device 170 may, but need not, include a touch-sensitive surface.In embodiments where the networked computer system 1000 includes a plurality of computing devices 100, as shown in FIG. Fig. 2, where the networked computer system 1000 includes a plurality of additional computing devices 100', a corresponding plurality of display devices 170 may be implemented using different technologies. In some embodiments, a single computing device may be associated with multiple display devices, in other embodiments, each of a plurality of computing devices is associated with a single display device, and in still other embodiments, a combination of the foregoing configurations is used. Each display device 170 is associated with a unique display identifier 172. A plurality of display devices 170 may be arranged to form a composite display device matrix 170' having a user-defined size and shape, as disclosed herein and as described in Fig. 3 shown.

[0014] With reference again to the Fig. 2, the computing device 100 includes, among other things, a processor 110, a memory 120, a communication module 140, an operating system 150, a user interface module 160, and a content browser 180. A bus and / or interconnect 190 is also provided to enable communications with other devices and within the device, for example, using the communication module 140. Other components and functionality not shown in the block diagram of Fig. 2 will be readily apparent in light of this disclosure, and it will be understood that it is not intended to limit the claimed invention to any specific architecture or hardware configuration. For example, in embodiments including a plurality of additional computing devices 100', one or more of the additional computing devices 100' may have a different system architecture than that described in the Fig. 2 for the computing device 100. Therefore, if exemplary subcomponents associated with the computing device 100 are shown in the Fig. 2, the plurality of additional computing devices 100' are depicted generically, assuming that such devices may have a different architecture

[0015] Processor 110 may be implemented using any suitable processor and may include one or more coprocessors or controllers, such as a graphics processing unit or an audio processor, to assist in control and processing operations associated with computer system 100. Similarly, memory 120 may be implemented using any suitable type of digital storage, such as one or more of a hard disk, a USB drive, flash memory, random access memory, or a suitable combination of the foregoing. Memory 120 may be used, for example, to store display configuration information and / or notification messages used to control the appearance of content on display device 170.The communications module 140 may be implemented using a suitable network chip or network chipset that enables a wired or wireless connection to the network 500 and / or to other computing devices and resources. The network 500 may be a local area network (such as a home network or an office network), a wide area network (such as the Internet), or a combination of such networks, whether public, private, or both. In some cases, access to resources on a given network or computing system may require credentials such as usernames, passwords, or some other security mechanism.

[0016] The operating system 150 may comprise any suitable operating system, such as Google Android (Google Inc., Mountain View, CA), Microsoft Windows (Microsoft Corp., Redmond, WA), or Apple OS X (Apple Inc., Cupertino, CA). As will be understood in light of the present disclosure, the techniques provided herein may be implemented regardless of the particular operating system provided with the computing device 100 and, therefore, may also be implemented using any existing or yet-to-be-developed platform. In one embodiment, the operating system 150 manages one or more device drivers used to control the operation of hardware, such as the display device 170.The operating system 150 may also be configured to provide functionality and services that support, for example, the user interface module 160, the content browser 180, and other applications executing on the computing device 100.

[0017] The user interface module 160 provides a user interface that can be used to configure various subcomponents associated with the networked computing system 1000, such as one or more content browsers 180 and one or more display devices 170. In particular, the user interface module 160 can be used to configure how the content browser 180 renders received content, and in particular, to define which spatial portion of the received content should be displayed in a browser window generated by the content browser 180. This configuration may depend, for example, on the position of the corresponding display device 170 in the composite display device matrix 170'. In one embodiment, a particular computing device 100 is capable of configuring only subcomponents associated with that particular device.However, in an alternative embodiment, the particular computing device 100 is capable of also configuring subcomponents associated with the plurality of additional computing devices 100'.

[0018] In general, content browser 180 can be understood to comprise a software application configured to retrieve and render digital content obtained from a storage resource, such as a content archive 220 hosted by content server 200. In one particular embodiment, content browser 180 is a web browser configured to render content encoded and transmitted using Hypertext Markup Language (HTML) protocols. Examples of such protocols include HTML5, WebSockets, and Web Real-Time Communication (Web RTC). In another embodiment, content browser 180 is selected from a group consisting of a web browser, an email reader, a news reader, a word processing application, and an e-commerce storefront application.In other embodiments, other types of software applications may be used to retrieve and render content. It will be understood that the embodiments disclosed herein are configured to work with a wide variety of existing and yet-to-be-developed content browsers. For example, while . Fig. 1 indicates that computing device 100 executes a single content browser, in alternative embodiments, multiple content browsers may be executed simultaneously on the computing device. Such an implementation may be advantageous where computing device 100 is also associated with multiple display devices, as each of the multiple display devices may be associated with one of the multiple content browsers.

[0019] In certain embodiments, the content browser 180 includes configuration settings 182 stored in a local storage resource 184 managed by the content browser 180. In alternative embodiments, the configuration settings 182 may be stored elsewhere, such as in a memory 120. Regardless of where they are stored, the configuration settings 182 may be understood to define a conceptual sub-area—often referred to as a viewport or a tile—of a received digital content item, wherein the conceptual sub-area is to be displayed in a browser window generated by the content browser 180. The definition of the viewport may depend, for example, on the position of an associated display device 170 in the composite display device matrix 170'.In one embodiment, the configuration settings 182 define a single tile to be displayed in the corresponding browser window, while in other embodiments, the configuration settings 182 define multiple tiles that can be displayed simultaneously in different areas of a single browser window, such as in different browser window frames.

[0020] The Fig. 1A and Fig. 1B conceptually illustrate how the configuration settings 182 can be used to define a conceptual sub-area of a digital content item, such as a web page. In particular, Fig. 1A a screenshot showing a web page displayed in browser window 3000, which is conceptually segmented into a left tile and a right tile. Fig. 1B is a screenshot showing the website of the Fig. 1A shows, wherein a first portion of the web page is displayed in the first browser window 3100 corresponding to browser 1, and wherein a second portion of the web page is displayed in the second browser window 3200 corresponding to browser 2. In such an implementation, browser 1 has first configuration settings that define the size and position of the left-side tile displayed in the Fig. 1A, and which therefore define which part of the web page will appear in browser 1. Similarly, browser 2 has second configuration settings which define the size and position of the right-hand tile shown in Fig. 1A, and which therefore define which part of the web page will appear in the browser 2. In certain embodiments, the size and position specified in the configuration settings 182 depends on the size and position of a display device on which the corresponding browser window is to be displayed. While in the Fig. 1A and Fig. 1B only two tiles are shown, in general, any number of configuration settings 182 may be established for corresponding content browsers displayed using corresponding display devices arranged into a composite display.

[0021] In certain embodiments, the content browser 180 further includes a mutation observer 186. The mutation observer 186 is configured to detect changes in the digital content rendered by the content browser 180. In some cases, such changes may occur in response to a user's interaction with the digital content, such as when a user scrolls the displayed content or interacts with active content. In other cases, such changes may occur independent of any user action, such as due to the passage of time, the playing of animation sequences, or the detection of an external event. Regardless of the cause of the changes, in such embodiments, the mutation observer 186 is configured to detect changes in a DOM and / or CSS associated with the digital content.The mutation observer 186 may also be configured to record the detected changes in one or more notification messages, which may be sent to other content browsers via a web socket connection or a direct peer-to-peer network. Changes detected by one content browser may therefore be communicated to other content browsers that may not have detected such changes in order to partially or fully synchronize the appearance on the display.

[0022] With further reference to the Fig. 2, the networked computer system 1000 optionally includes a content server 200. The content server 200 hosts a content archive 220 in which one or more content items 222 are stored. In such embodiments, the content server 200 may be understood as encompassing one or more devices intended for use in enterprise environments that are configured to host content and deliver the hosted content to one or more computing devices 100. While in the Fig. 2, it will be understood that, in general, tens, hundreds, thousands, or more networked content servers 200 may provide the functionality of hosting and delivering content. In still other embodiments, the content server 200 is omitted entirely, and the content items are stored locally in the memory 120 provided by the computing device 100 and are transmitted to additional computing devices 100' via a peer-to-peer network.

[0023] The networked computer system 1000 optionally also includes a configuration manager 300 that may be configured to generate and / or manage a plurality of configuration settings 182 associated with a corresponding plurality of content browsers 180 and display devices 170. In such embodiments, the configuration manager 300 provides a user interface module 360 that may be used to define the characteristics of the composite display device matrix 170', and in particular, the various sizes and positions of the individual display devices 170 that form the matrix 170'. Based on input received via the user interface module 360, the configuration manager 300 generates configuration settings 182 that are distributed to a plurality of computing devices 100, and in particular, to a plurality of content browsers 180.The configuration settings 182 are optionally also stored in a configuration archive 320 hosted by the configuration manager 300. While each content browser 180 can be configured individually, use of the configuration manager 300 facilitates setup of the matrix 170' by allowing all or nearly all of the configuration to be performed using a single user interface.

[0024] In one embodiment, the functionality associated with a configuration manager 300 may be provided by a computing device that is also used to drive one of the display devices used to render digital content. In an alternative embodiment, the configuration functionality is provided by a dedicated computing device that does not drive any of the display devices used to render digital content.For example, in such an alternative embodiment, the configuration manager 300 comprises a desktop computer, a laptop computer, a workstation computer, a tablet computer, a smartphone, a handheld computer, a set-top box, a server device for use in enterprise environments, or any other suitable computing device configured to provide the functionality described herein as associated with the configuration manager 300. The configuration manager 300 may optionally also include a broadcast module 340 configured to broadcast the aforementioned notification messages generated by the mutation observer 186 to additional computing devices 100'.

[0025] The embodiments disclosed herein may be implemented in various forms of hardware, software, firmware, or special-purpose processors. For example, in one embodiment, a non-transitory computer-readable medium has instructions encoded thereon that, when executed by one or more processors, implement one or more of the digital content rendering methodologies disclosed herein. The instructions may be encoded using one or more suitable programming languages, such as C, C++, object-oriented C, JavaScript, Visual Basic .NET, BASIC, or alternatively, using custom or proprietary instruction sets.Such instructions may be provided in the form of one or more software applications or applets embodied in tangible form on one or more storage devices, and which may be executed by a computer having a suitable architecture. In one embodiment, the system may be hosted on a given website and implemented using JavaScript or other suitable browser-based technology.

[0026] The functionality disclosed herein may optionally be incorporated into a wide variety of software applications, such as operating systems, device configuration managers, presentation applications, web browsers, and other applications configured to render HTML content. For example, a web browser may be configured to have a multi-screen mode in which rendered content is segmented and displayed in multiple windows appearing on different display screens. Or, in another embodiment, an operating system is configured to provide the functionality associated with configuration manager 300 so that when a web browser is invoked, or when HTML content is rendered, such content may be segmented based on a user-defined configuration associated with a composite display.Therefore, in some implementations, the functionality described herein may be provided on an individual application basis, while in other implementations, such functionality may be provided by an operating system to a range of applications that utilize resources and / or device configurations provided by the operating system. The computer systems disclosed herein may include a number of different modules, sub-modules, or other components of various functionality, and may provide and receive information to and from yet further components and services. These modules may be used, for example, to communicate with additional computing devices 100', content servers 200, configuration managers 300, or other external components.Other components and functionality not reflected in the drawings will be apparent in light of this disclosure, and it will be understood that the present disclosure is not intended to be limited to any particular hardware or software configuration. Therefore, in other embodiments, the components and functionality shown in . Fig. 2 may include additional, fewer, or alternative subcomponents.

[0027] The aforementioned non-transitory computer-readable medium may be any suitable medium for storing digital information, such as a hard disk, a server, flash memory, or random access memory. In alternative embodiments, the computer and modules disclosed herein may be implemented using software, include gate-level logic such as a programmable gate array (FPGA), or alternatively, include a purpose-built semiconductor such as an application-specific integrated circuit (ASIC). Still further embodiments may be implemented with a microcontroller having a number of input / output ports for receiving and outputting data and including a number of embedded routines for performing the various functionalities disclosed herein.It will be appreciated that any suitable combination of hardware, software, and firmware may be used, and that the present disclosure is not intended to be limited to any particular system architecture. methodology

[0028] Fig. 4 is a flowchart illustrating an exemplary method for rendering digital content 2000 capable of segmenting digital content into a plurality of tiles displayed across multiple application windows. Fig. 5 is a data flow diagram schematically illustrating exemplary data flows that occur when digital content is segmented into a plurality of tiles displayed across multiple application windows. As can be seen, the method for rendering digital content 2000 includes a number of phases and sub-processes, the order of which may vary from one embodiment to another. However, when viewed collectively, these phases and sub-processes form a complete process for rendering digital content that is responsive to user input, in accordance with certain of the embodiments disclosed herein. These methodologies may be implemented, for example, using the Fig. 2. However, other system architectures may be used in other embodiments, as will become apparent in light of this disclosure. For this purpose, the correlation of the various functionalities described in the Fig. 4 and Fig. 5, to the specific components used in Fig. 2 are not intended to imply any structural limitations and / or limitations in use. Rather, other embodiments may have varying degrees of integration, wherein multiple functionalities are performed by one system or by multiple systems. For example, in an alternative embodiment, the functionality associated with configuration manager 300 may be incorporated into computing device 100. Other embodiments may therefore have fewer or more modules and sub-modules, depending on the granularity of the implementation. Numerous variations and alternative configurations will be apparent in light of this disclosure.

[0029] With further reference to the Fig. 4 and Fig. 5, the exemplary method for rendering digital content 2000 begins with each of a plurality of content browsers receiving display configuration settings. See reference numeral 2110 in Fig. 4. The display configuration settings may be received from one or more of a number of different sources, such as from a memory resource local to the computing device on which the content browser is executing, from user input received via a user interface module provided by the computing device on which the content browser is executing, or from an external configuration manager. Fig. 5 shows an exemplary embodiment in which the configuration manager 300 sends different display configuration settings to other computing devices. The configuration manager 300 may receive the configuration settings via a user interface module 360 and cache the configuration settings in a configuration archive 320, as shown in Fig. 2. The configuration manager 300 transmits first configuration settings 182a to a first computing device 100a in a first transmission 10a, and transmits second configuration settings 182b to a second computing device 100b in a second transmission 10b. In such embodiments, the configuration repository 320 may correlate each collection of configuration settings 182a, 182b with a display identifier that identifies the particular display device (and thus the computing device 100a, 100b) to which the configuration settings are to be sent. As described herein, the configuration settings may generally be understood to define a viewport or tile that can be applied to a digital content item to be displayed by a particular content browser.In such embodiments, the configuration of the tile depends on the size and location of a particular display device that forms part of a composite display.

[0030] The method for rendering digital content 2000 additionally or alternatively begins with each of the plurality of content browsers receiving a common content item. See reference numeral 2120 in Fig. 4. In one embodiment, the content item is transmitted from a content server, while in alternative embodiments, the content item is received from one or more other sources, or retrieved from a storage resource local to the computing device on which the content browser is executed. Fig. 5 shows an exemplary embodiment in which the content server 200 transmits a content item 222 to a first computing device 100a in a first transmission 20a and transmits the same content item 222 to a second computing device 100b in a second transmission 20b. In an alternative embodiment, the content server 200 transmits the content item 222 to the first and second computing devices 100a, 100b simultaneously via a broadcast transmission. While the Fig. 4 indicates that in certain embodiments the display configuration settings are received before the content item is received, it will be understood that in general the order of such data transfers is not critical and that in other embodiments they may occur in different orders or simultaneously.

[0031] Once a plurality of content browsers have received the common content item and the browser-specific configuration settings, the content item can be rendered by each browser according to the respective configuration settings. See reference numeral 2130 in Fig. 4. For example, in the Fig. 5, computing device 100a would render content item 222 according to configuration settings 182a, while computing device 100b would render content item 222 according to configuration settings 182b. In particular, when a content browser loads HTML content, the viewport of the browser window may be adjusted to match the viewport defined in the received configuration settings. This may be achieved, for example, using CSS translations. As a result, although the content browser loads the entire content item, only a portion of the loaded content item is visible in the browser window.Because the configuration settings define a tile that has a size and position tailored to a particular display device that forms part of a composite display, the result is generally a unified appearance of the content item across multiple browser windows, optionally displayed on multiple displays, for example as in . Fig. 1B shown.

[0032] It is possible that the content item will be modified after it has been rendered in the multiple content browser windows. Such modifications may occur in response to a user's interaction with the digital content, for example, when a user scrolls or otherwise interacts with active content. This is particularly likely where the display devices comprising a composite display have a touch-sensitive surface. However, even without touch-sensitive displays, content modifications may occur independent of any user action, such as due to the passage of time, the presence of animated objects, or the detection of an external event. Regardless of the cause, such modifications may only be detected in a few of the content browsers.For example, if a user attempts to scroll through a web page by performing a swipe gesture on one of multiple touch-sensitive displays forming a composite display, that gesture may only be recognized on the one display with which the user interacted. However, the modified appearance of the content as a result of scrolling should likely be reflected on multiple displays. As a result, certain of the embodiments disclosed herein include techniques for detecting content modifications and propagating such modifications from a content browser where such a modification is detected (referred to herein as the "detecting browser") to other content browsers where such changes should be reflected (referred to herein as the "responsive browser"). This allows synchronous access to the content item to be simulated.

[0033] In accordance with the foregoing, in certain embodiments, each content browser includes a mutation observer configured to detect modifications in the content rendered by the content browser. See reference numeral 2140 in Fig. 4. For example, the mutation observer may be configured to monitor the DOM that defines the structure of the content element. Elements in the DOM that have been modified may be annotated using data attributes that uniquely identify the modified element, for example, by using a uniform resource locator (URL) associated with the modified element. Therefore, if a DOM element is changed or removed as a result of user interaction with the content element, the mutation observer may be configured to identify the changed or removed element. The mutation observer may also be configured to detect changes in the computed style of DOM elements, and in such embodiments, is therefore capable of responding to CSS changes in the rendered content.An example of an event that can cause changes in the computed style is a user interaction with a scroll box object, which would not necessarily change its dependent elements, but may result in modified computed style attributes. Where the change in the style attributes is a transition from one state to another, a loop can be executed that requests and renders multiple animation frames until the animation ends. This allows the multiple content browsers to render intermediate states of the animation.

[0034] Once a content modification has been identified, the mutation observer may be further configured to generate a notification message to be sent by the detecting browser to one or more responding browsers. See reference numeral 2150 in Fig. 4. In one embodiment, such notification identifies one or more changed items and defines one or more corresponding modifications. Fig. Figure 6A shows an exemplary CSS mutation notification 188 transmitted from a sensing browser to one or more responsive browsers. As shown, the CSS mutation notification 188 includes a first name-value pair 188a identifying the changed element and a second name-value pair 188b defining the modification to be applied to the element. Fig. Figure 6B shows an exemplary DOM mutation notification 189 transmitted from a sensing browser to one or more responsive browsers. As shown, the DOM mutation notification 189 includes a first name-value pair 189a identifying the changed element and a second name-value pair 189b defining the modification to be applied to the element.

[0035] Generally, mutation notifications are sent to the responsive browsers over a network connection that is fast enough to allow updates made to the responsive browsers to appear substantially simultaneously with each other and substantially simultaneously with updates to the sensing browser. Synchronous or asynchronous calls tend to be too slow for this purpose, and therefore, in one embodiment, the communications module 140 is configured to send one or more mutation notifications to a distribution server over a WebSockets connection. See reference numeral 2160 in Fig. 4. The distribution server sends the notifications as a broadcast to one or more responsive browsers. See reference numeral 2162 in Fig. 4. In one particular implementation, a broadcast module 340 installed on the configuration manager 300 acts as a distribution server for the mutation notifications, while in other implementations, a dedicated distribution server is used in this regard. Such an embodiment of notification via WebSockets broadcasts 30 is conceptually described in Fig. 5, wherein a WebSockets mutation notification 188' is initially transmitted from a computing device 100a (associated with a recognizing browser) to a configuration manager 300 in a first WebSockets transmission 30a, and thereafter transmitted from the configuration manager 300 to a computing device 100b (associated with a responsive browser) in a second WebSockets transmission 30b.

[0036] In an alternative embodiment, the communication module 140 is configured to send one or more mutation notifications from the detecting browser to one or more responding browsers by establishing a peer-to-peer connection or a socket connection between the browsers. See reference numeral 2170 in Fig. 4. Such an embodiment 40 with peer-to-peer broadcast notification is also conceptually possible in Fig. 5, wherein a peer-to-peer mutation notification 188" is transmitted from computing device 100a (associated with a sensing browser) to computing device 100b (associated with a responsive browser) via a peer-to-peer connection. Regardless of how the mutation notifications are transmitted from the sensing browser to the one or more responsive browsers, the responsive browsers may be configured to modify the appearance of the content item based on the received mutation notification. See reference numeral 2180 in Fig. 4. conclusion

[0037] Numerous variations and configurations will be apparent in light of the disclosure. For example, an exemplary embodiment provides a method for rendering digital content. The method comprises executing a content browser on a computing device. The content browser is configured to render a browser window on a display device that is (a) operatively coupled to the computing device and (b) forms part of a composite display. The method further comprises receiving, by the content browser, a display configuration that defines a sub-region of the composite display corresponding to the display device. The method further comprises receiving, by the content browser, a content item. The method further comprises rendering a portion of the content item in the browser window. The portion of the content item that is rendered corresponds to the sub-region of the composite display.In some cases, the display configuration is received via user input provided in the browser window. In some cases, the display configuration is received from a configuration manager over a network connection. In some cases, the method further comprises executing a second content browser on the computing device. The second content browser is configured to render a second browser window on a second display device that forms part of the composite display.

[0038] In some cases, the content browser is a web browser and the content item is a web page encoded using a hypertext markup language (HTML). In some cases, the method further comprises (a) detecting a modification of an element comprising the content item after the portion of the content item has been rendered; and (b) generating a mutation notification characterizing the modification and identifying the element to which the modification applies. In some cases, the method further comprises (a) detecting a modification to a document object model defining a structure of the content item; and (b) generating a mutation notification characterizing the modification.In some cases, the method further comprises (a) executing a second content browser on a second computing device, wherein the second content browser is configured to render a second browser window on a second display device that also forms part of the composite display; (b) receiving, by the second content browser, a second display configuration that forms a second sub-region of the composite display; (c) receiving, by the second content browser, the content item; and (d) rendering a second portion of the content item in the second browser window, wherein the second portion of the content item corresponds to the second sub-region of the composite display.

[0039] Another exemplary embodiment provides a system for rendering digital content. The system includes a first content browser configured to render a first browser window on a first display device that forms part of a composite display. The first browser window displays a first tile of a content item that includes one or more elements. The system further includes a second content browser configured to render a second browser window on a second display device that forms part of the composite display. The second browser window displays a second tile of the content item. The system further includes a mutation observer configured to detect a modification in the first tile of the content item and generate a mutation notification characterizing the modification and identifying an element to which the modification is to be applied.The first content browser is further configured to send the mutation notification to the second content browser. The second content browser is further configured to modify an appearance of the second tile as displayed on the second display device based on the mutation notification. In some cases, (a) the content item is a web page encoded using a Hypertext Markup Language; and (b) the modification in the first tile of the content item is a modification to a Document Object Model that defines a structure of the content item. In some cases, (a) the content item is a web page; and (b) the first and second content browsers are configured to receive the web page from a content server.In some cases, (a) the first tile of the content item is defined by a display configuration; and (b) the first content browser is further configured to receive the display configuration from a configuration manager. In some cases, the system further comprises a configuration manager configured to receive first and second display configurations defining the first and second tiles, respectively. In some cases, (a) the content item is a web page encoded with a hypertext markup language; and (b) the mutation notification defines a cascading style sheet (CSS) translation.

[0040] Another exemplary embodiment provides a non-transitory computer-readable medium comprising instructions recorded thereon that, when executed by one or more processors, cause a process for rendering digital content to be performed. The process comprises rendering a first browser window on a first display device that forms part of a composite display. The first browser window displays a first tile of a content item that includes one or more elements. The process further comprises rendering a second browser window on a second display device that forms part of the composite display, wherein the second browser window displays a second tile of the content item. The process further comprises detecting, at the first browser window, a modification of one of the elements that form the content item.The process further comprises generating, by a mutation observer, a mutation notification characterizing the modification and identifying the element to which the modification applies. The process further comprises modifying the appearance of the content element in the second browser window in response to processing the mutation notification. In some cases, the process for rendering digital content further comprises (a) receiving, via the first browser window, a first display configuration defining the first tile and a second display configuration defining the second tile; and (b) sending the second display configuration to a content browser that renders the second browser window.In some cases, the first display device has a touch-sensitive surface; and (b) detecting the modification further comprises detecting a scrolling gesture performed using the touch-sensitive surface. In some cases, the process of rendering digital content further comprises sending the mutation notification from a first computing device executing a first content browser that renders the first browser window to a second computing device executing a second content browser that renders the second browser window.In some cases, (a) the process of rendering digital content further comprises sending the mutation notification from a first computing device executing a first content browser that renders the first browser window to a second computing device executing a second content browser that renders the second browser window; and (b) the mutation notification is sent over a peer-to-peer connection. In some cases, (a) the process of rendering digital content further comprises sending the mutation notification from a first computing device executing a first content browser that renders the first browser window to a second computing device executing a second content browser that renders the second browser window; (b) the mutation notification is sent from the first computing device to a server; and (c) the mutation notification is sent from the server to the second computing device.

[0041] The foregoing detailed description has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form described herein. Many modifications and variations are possible in light of this disclosure. It is intended that the scope of the invention be limited not by this detailed description, but by the appended claims.

Claims

[1] A method for rendering digital content, the method comprising: Executing a first content browser (180) on a first computing device (100), wherein the content browser is configured to render a first browser window on a first display device (170) that is (a) operatively coupled to the first computing device and (b) forms part of a composite display (170'); executing a second content browser on a second computing device, wherein the second content browser is configured to render a second browser window on a second display device that also forms part of the composite display (170); receiving, by the first content browser, a first display configuration defining a first sub-region of the composite display corresponding to the first display device; Receiving, by the second content browser, a second display configuration defining a second sub-area of the composite display; Receiving, by the first content browser, a content item; Receiving, by the second content browser, the content item; rendering a first portion of the content item in the first browser window, wherein the first portion of the content item rendered corresponds to the first sub-region of the composite display; rendering a second portion of the content element in the second browser window, wherein the second portion of the content element corresponds to the second sub-region of the composite display; Detecting, in the first browser window, a modification of one of the elements that make up the content item after the part of the content item has been rendered; Generating a mutation notification that identifies the modification and the element to which the modification applies; Sending the mutation notification to the second computing device; and Modifying the appearance of the content item in the second browser window in response to processing the mutation notification. [2] The method of claim 1, wherein the display configuration is received via user input provided in the first and / or second browser window, or wherein the display configuration is received from a configuration manager (300) via a network connection. [3] A method according to any one of claims 1 to 2, wherein the content browser is a web browser and the content item is a web page encoded using a Hypertext Markup Language (HTML). [4] A system for rendering digital content, the system comprising: a first content browser (180) configured: render a first browser window on a first display device forming part of a composite display, wherein the first browser window displays a first tile of a content item comprising one or more elements; receive a first display configuration defining a first sub-region of the composite display corresponding to the first display device; to receive the content item; and render a first portion of the content item in the first browser window, wherein the first portion of the content item that is rendered corresponds to the first sub-region of the composite display; a second content browser (180) configured: render a second browser window on a second display device forming part of the composite display, wherein the second browser window displays a second tile of the content item; receive a second display configuration defining a second sub-region of the composite display corresponding to the second display device; to receive the content item; and render a second portion of the content item in the second browser window, wherein the second portion of the content item that is rendered corresponds to the second sub-region of the composite display; and a mutation observer configured to detect a modification in the first tile of the content item and generate a mutation notification indicating the modification and identifying an element to which the modification is to be applied; wherein the first content browser is further configured to send the mutation notification to the second content browser; and wherein the second content browser is further configured to modify an appearance of the second tile displayed on the second display device based on the mutation notification. [5] System according to claim 4, wherein: the content item is a web page encoded using a Hypertext Markup Language; and the modification in the first tile of the content item is a modification of a document object model that defines a structure of the content item. [6] System according to claim 4, wherein: the content element is a web page; and the first and second content browsers are configured to receive the web page from a content server. [7] System according to one of claims 4 to 6, wherein: the first tile of the content item is defined by the first display configuration; and the first content browser is further configured to receive the first display configuration from a configuration manager (300). [8] The system of any one of claims 4 to 7, further comprising a configuration manager (300) configured to receive the first and second display configurations defining the first tile and the second tile, respectively. [9] System according to claim 4, wherein: the content element is a web page encoded using a Hypertext Markup Language; and the mutation notification defines a cascading style sheet (CSS) translation. [10] A computer program product comprising instructions which, when executed by a computing device or system, cause the computing device or system to carry out the method according to any one of claims 1 to 3.

Images